Skip to content

mithril blocks written to disk #140

@dubbelosix

Description

@dubbelosix

in rpc fetch mode, mithril has a thread that fetches blocks from an rpc and writes them to disk.
the execution threads picks these up from disk.
the main benefit is that we can pre-fetch blocks so execution doesn't have to wait.

problems with this

  1. disk i/o
  2. unbounded growth on disk

proposed solution

  1. use a channel/queue with backpressure for the rpc fetch thread to put blocks into (configurable with a default of 200 blocks) ~200-300MB in memory
  2. execution thread reads from the channel
  3. disk flush is a separate and optional feature. we need blocks for the getBlock rpc call, but this should be handled entirely by overcast. if we keep the disk flush, it should be async, optional and should include a clean up process (which could add complications). if this gets complicated, handle 1 and 2 and create another issue for 3

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions