preprocess.py doesn't produce expected output

[walid-shalaby]

When I tried to validate the output from preprocess.py against text8 as input, I found mismatch between input word sequence and the index encoded sequence which is written to text8-train.pb2.
To reproduce, please add below two lines to preprocess.py after calling build_string_index()
print('{}'.format(words[:100]))
print('{}'.format(index[word_indices[:100]]))
Here is the output I get:
['anarchism' 'originated' 'as' 'a' 'term' 'of' 'abuse' 'first' 'used'
'against' 'early' 'working' 'class' 'radicals' 'including' 'the' 'diggers'
'of' 'the' 'english' 'revolution' 'and' 'the' 'sans' 'culottes' 'of' 'the'
'french' 'revolution' 'whilst' 'the' 'term' 'is' 'still' 'used' 'in' 'a'
'pejorative' 'way' 'to' 'describe' 'any' 'act' 'that' 'used' 'violent'
'means' 'to' 'destroy' 'the' 'organization' 'of' 'society' 'it' 'has'
'also' 'been' 'taken' 'up' 'as' 'a' 'positive' 'label' 'by' 'self'
'defined' 'anarchists' 'the' 'word' 'anarchism' 'is' 'derived' 'from'
'the' 'greek' 'without' 'archons' 'ruler' 'chief' 'king' 'anarchism' 'as'
'a' 'political' 'philosophy' 'is' 'the' 'belief' 'that' 'rulers' 'are'
'unnecessary' 'and' 'should' 'be' 'abolished' 'although' 'there' 'are'
'differing']
['instance' 'dating' 'as' 'a' 'term' 'of' 'distances' 'first' 'used'
'against' 'early' 'working' 'class' 'squid' 'including' 'the' 'hanoi' 'of'
'the' 'english' 'treaty' 'and' 'the' 'malinowski' 'UNK' 'of' 'the'
'french' 'treaty' 'afro' 'the' 'term' 'is' 'still' 'used' 'in' 'a' 'buddy'
'way' 'to' 'islam' 'any' 'act' 'that' 'used' 'zeus' 'lincoln' 'to'
'vector' 'the' 'car' 'of' 'society' 'it' 'has' 'also' 'been' 'latin' 'up'
'as' 'a' 'failed' 'eddington' 'by' 'self' 'command' 'anarchists' 'the'
'word' 'instance' 'is' 'treaty' 'from' 'the' 'born' 'without' 'mml'
'progress' 'coast' 'king' 'instance' 'as' 'a' 'political' 'culture' 'is'
'the' 'me' 'that' 'dating' 'are' 'squid' 'and' 'public' 'be' 'acceptable'
'although' 'there' 'are' 'absent']