Bug: create_last_snapshot returns invalid orderbook

[wellimbharath]

When you generate snapshot through elapse with a fixed set of bid and ask lenghts i.e while hbt.elapse(elapse_in_ns) == 0: the snapshots looks correct, but when you generate snapshot from create_last_snapshot(data_npz, tick_size=.01, lot_size=0.00001, initial_snapshot=None), its gives an incorrect orderbook at the end.

Sample npz file converted to csv: sample.csv

Tick Size : 0.01, Lot Size : 0.00001

The problem is the ask prices are much lower than bid prices on the final snapshot and they are still in the final snapshot for some reason, it should never happen, its only happening when doing the create_last_snapshot but not when its done over elapse. There must be some issue here which I am unable to figure out.

[nkaz001]

It seems that some depth update messages are missing in the original data (specifically, the updates that set the opposite depth level to zero when the best bid or ask changes), which often happens in crypto exchanges. hftbacktest has the logic updating the best bid and ask when the opposite best is updated and crossed, even if the corresponding zero-update is not received. But, the crossed level data remains in the depth, which means, currently, the snapshot can still reflect a crossed state.

Example:
Current depth:
1 @ 13
2 @ 12
3 @ 11 (Best Ask)
1 @ 10 (Best Bid)

Update:
Bid update: 1 @ 11 (without ask update 0 @ 11)

Resulting depth:
1 @ 13
2 @ 12 (Best Ask)
1 @ 11 (Best Bid) (while 3 @ 11 still exists on the ask depth)
1 @ 10

I'll update the fix.

[SaratAngajalaoffl]

Hey @nkaz001 , thanks for the quick fix. Any ETA on the release with this commit included?

[nkaz001]

You can use the development version. Clone this repo and run a command maturin develop in py-hftbacktest directory. (you need to install maturin first. pip install maturin)

[SaratAngajalaoffl]

Thanks for the suggestion — I’ll try the development version in the meantime.
Do you have an approximate timeline for the next release? I’d like to plan the update for our service accordingly.

[nkaz001]

That would be around the end of this month.