Structure lists given at the uppermost level break pyhcl

[saj]

Hashicorp's HCL parser for Go will accept input like the the following:

foo {
    key = 7
}
foo {
    key = 12
}

pyhcl 0.2.1 barfs when given the same input:

>>> s = """foo {
...     key = 7
... }
... foo {
...     key = 12
... }"""
>>> hcl.loads(s)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/private/var/folders/hm/lxv5j0v12sj17drqtymbxcyh0000gn/T/tmpbOqahU/.deps/pyhcl-0.2.1-py2-none-any.whl/hcl/api.py", line 62, in loads
    return HclParser().parse(s)
  File "/private/var/folders/hm/lxv5j0v12sj17drqtymbxcyh0000gn/T/tmpbOqahU/.deps/pyhcl-0.2.1-py2-none-any.whl/hcl/parser.py", line 293, in parse
    return self.yacc.parse(s, lexer=Lexer())
  File "/private/var/folders/hm/lxv5j0v12sj17drqtymbxcyh0000gn/T/tmpbOqahU/.deps/ply-3.4-py2-none-any.whl/ply/yacc.py", line 265, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
  File "/private/var/folders/hm/lxv5j0v12sj17drqtymbxcyh0000gn/T/tmpbOqahU/.deps/ply-3.4-py2-none-any.whl/ply/yacc.py", line 971, in parseopt_notrack
    p.callable(pslice)
  File "/private/var/folders/hm/lxv5j0v12sj17drqtymbxcyh0000gn/T/tmpbOqahU/.deps/pyhcl-0.2.1-py2-none-any.whl/hcl/parser.py", line 89, in p_top
    p[0] = self.objectlist_flat(p[1],True)
  File "/private/var/folders/hm/lxv5j0v12sj17drqtymbxcyh0000gn/T/tmpbOqahU/.deps/pyhcl-0.2.1-py2-none-any.whl/hcl/parser.py", line 76, in objectlist_flat
    for k2, v2 in iteritems(vv):
  File "/private/var/folders/hm/lxv5j0v12sj17drqtymbxcyh0000gn/T/tmpbOqahU/.deps/pyhcl-0.2.1-py2-none-any.whl/hcl/parser.py", line 27, in iteritems
    return iter(d.iteritems())
AttributeError: 'int' object has no attribute 'iteritems'

I noticed the unit test for this case is currently commented out.

[virtuald]

Originally the golang hcl parser was inconsistent on how it converted structures like that. Feel free to fix the issue and submit a pull request.

[link2xt]

Shouldn't this issue be closed? This one is resolved.

[saj]

This one is resolved.

Not quite. I can confirm that for the original test case only, pyhcl now provides expected results:

>>> hcl.version.__version__
'0.3.3'
>>> s = """foo {
...     key = 7
... }
... foo {
...     key = 12
... }"""
>>> d = hcl.loads(s)
>>> d.keys()
[u'foo']
>>> len(d['foo'])
2
>>> d['foo'][0]
{u'key': 7}
>>> d['foo'][1]
{u'key': 12}

Things begin to fall apart when the objects have multiple keys. I have provided two new test cases below. For the sake of comparison, each case will be accompanied with output from hashicorp's reference HCL parser. (Golang output was generated by Decode-ing the same input string into a map and Print-ing the resulting map.)

Case 1: Inadvertent merging of object lists

>>> s = """bar {
...     a = "alpha"
...     b = "bravo"
...     c = "charlie"
... }
... bar {
...     x = "x-ray"
...     y = "yankee"
...     z = "zulu"
... }
... """
>>> d = hcl.loads(s)
>>> d.keys()
[u'bar']
>>> len(d["bar"])
6
>>> d["bar"]
{u'a': u'alpha', u'c': u'charlie', u'b': u'bravo', u'y': u'yankee', u'x': u'x-ray', u'z': u'zulu'}

The value of the bar key is a dict that contains six keys.

Reference parser gives:

map[bar:[map[a:alpha b:bravo c:charlie] map[y:yankee z:zulu x:x-ray]]]

The value of the bar key is a list that contains two items. Each item is a map that contains three keys.

Case 2: Inadvertent loss of object keys

>>> s = """bar {
...     a = "alpha"
...     b = "bravo"
... }
... bar {
...     a = "alpha"
...     b = "bravo"
... }
... """
>>> d = hcl.loads(s)
>>> d.keys()
[u'bar']
>>> len(d['bar'])
2
>>> d['bar'][0]
{u'a': u'alpha', u'b': u'bravo'}
>>> d['bar'][1]
{u'b': u'bravo'}

Reference parser gives:

map[bar:[map[a:alpha b:bravo] map[a:alpha b:bravo]]]

[link2xt]

@saj If we consider all those cases to be part of this issue, then #22 is also related

[saj]

Ah, indeed. Thanks. I did search for new issues before posting my previous update but I missed that PR. I opted to update this bug because these (new) symptoms still only affect structure/object lists at the uppermost level, which I felt was consistent with the title of the original bug report.