forked from firedancer-io/radiance
-
Notifications
You must be signed in to change notification settings - Fork 20
Open
Labels
Description
in rpc fetch mode, mithril has a thread that fetches blocks from an rpc and writes them to disk.
the execution threads picks these up from disk.
the main benefit is that we can pre-fetch blocks so execution doesn't have to wait.
problems with this
- disk i/o
- unbounded growth on disk
proposed solution
- use a channel/queue with backpressure for the rpc fetch thread to put blocks into (configurable with a default of 200 blocks) ~200-300MB in memory
- execution thread reads from the channel
- disk flush is a separate and optional feature. we need blocks for the getBlock rpc call, but this should be handled entirely by overcast. if we keep the disk flush, it should be async, optional and should include a clean up process (which could add complications). if this gets complicated, handle 1 and 2 and create another issue for 3