Add checksum to header

[jeromekelleher]

There is currently no verification checks that a store has not been corrupted. It would be fairly straightforward to add a checksum field to the header, and to store the checksum of the whole file in this field. The only tricky bit would be to ensure that the checksum itself was zero'd out when computing the value, but we could do this easily enough by taking a copy of the header, zero'ing out these bytes and computing the initial checksum value from this before running though the rest of the bytes in the file.

We would need to increment the file version number, and to make sure that this check is only done on files with at least this version.

See here for a related discussion

[jeromekelleher]

To implement, we could make pack_items return the checksum of all the key/value pairs in turn. On reflection, there's no point in checksumming the header as well, we just want to know if the data integrity is OK. We don't care if someone writes random bytes into the padding areas, and if there's any corruption to the header or item descriptors this will be detected in other ways.

So, the problem is easy enough: compute the checksum of all the keys and values in turn, and store this value in the header in the data_checksum field.