Convert HashVocab Values to upper case when parsing

[Danko90]

Hi,
I had to poll data from a STIX repository but I got an error because it has the hash information of SSDEEP only in uppercase. I added the line
TERM_SSDEEP_LOWER = 'SSDeep'
in the file vocabs.py

Everything worked well.
Could you please let me know if I did it correctly or not?

Thank you
Danilo

[gtback]

Thanks, @Danko90.

The CybOX XML schemas define SSDEEP as being in upper-case only. If you're getting XML with the mixed case SSDeep, you should bring that up with the producer of the data.

There might be something we do to work around this in python-cybox. For example, since all of the values in HashNameEnum-1.0 are upper-case, we can call upper() during parsing. I would not recommend adding a separate vocabulary value, since then "SSDeep" and "SSDEEP" will be treated as different values, which is probably not what you want.

Can you paste the exact error you're getting (without adding the TERM_SSDEEP_LOWER = 'SSDeep' line)? That will help us track down the best way to solve this.

[Danko90]

Hi @gtback,
Thanks for replying!
Yes, below the traceback:

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/misp_stix_converter-0.2.9-py3.4.egg/misp_stix_converter/converters/convert.py", line 156, in manage_stix
    stix_package = STIXPackage.from_xml(stix)
  File "/usr/local/lib/python3.4/dist-packages/stix/core/stix_package.py", line 249, in from_xml
    return entity_parser.parse_xml(xml_file, encoding=encoding)
  File "/usr/local/lib/python3.4/dist-packages/mixbox/parser.py", line 187, in parse_xml
    entity = self.get_entity_class(xml_root_node.tag).from_obj(entity_obj)
  File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 361, in from_obj
    val = transformer.from_obj(val)
  File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 359, in from_obj
    val = [transformer.from_obj(x) for x in val]
  File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 359, in <listcomp>
    val = [transformer.from_obj(x) for x in val]
  File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 361, in from_obj
    val = transformer.from_obj(val)
  File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 361, in from_obj
    val = transformer.from_obj(val)
  File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 184, in from_obj
    return klass.from_obj(cls_obj)
  File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 361, in from_obj
    val = transformer.from_obj(val)
  File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 359, in from_obj
    val = [transformer.from_obj(x) for x in val]
  File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 359, in <listcomp>
    val = [transformer.from_obj(x) for x in val]
  File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 361, in from_obj
    val = transformer.from_obj(val)
  File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 184, in from_obj
    return klass.from_obj(cls_obj)
  File "/usr/local/lib/python3.4/dist-packages/cybox/common/vocabs.py", line 142, in from_obj
    obj = super(VocabString, cls).from_obj(cls_obj)
  File "/usr/local/lib/python3.4/dist-packages/cybox/common/attribute_groups.py", line 109, in from_obj
    obj = super(PatternFieldGroup, cls).from_obj(cls_obj)
  File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 363, in from_obj
    field.__set__(entity, val)
  File "/usr/local/lib/python3.4/dist-packages/mixbox/fields.py", line 213, in __set__
    self.preset_hook(instance, value)
  File "/usr/local/lib/python3.4/dist-packages/cybox/common/vocabs.py", line 29, in validate_value
    raise ValueError(error)
ValueError: Value for vocab <class 'cybox.common.vocabs.HashName'> must be one of ('MD5', 'SHA1', 'SHA384', 'SHA256', 'SHA224', 'MD6', 'SSDEEP', 'SHA512'). Received 'SSDeep'

[gtback]

Thanks @Danko90. The error you're getting is what I expected. I'll have to look around a bit to see the best place to fix it, since we don't want to blindly upper-case all vocabularies, just the HashName ones.

[Danko90]

Thanks @gtback ! I'll be waiting for your updates!

[gtback]

@Danko90, in addition to fixing this in python-cybox, I would encourage you to tell the producer of the data that they are producing invalid CybOX. The fix for it here is really just a workaround to "do the right thing" in a case where we're pretty sure what the producer meant to do.

[Danko90]

@gtback, Thank you so much.
I have already advised the producer and been waiting for them answer.
I'll let you know, thanks again!