feat(depot-client): wire sqlite optimization flags

[NathanFlurry]

Description

Please include a summary of the changes and the related issue. Please also include relevant motivation and context.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[NathanFlurry]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• chore(rivetkit): note duplicate admission logic #4948
• test(rivetkit): cover bypass fetch header forwarding #4947
• feat(kitchen-sink): add actor probe controls #4946
• fix(gateway): split actor response wait errors #4945
• fix(guard): handle bypass preflight before query parse #4944
• feat(kitchen-sink): add mock agent event log copy #4943
• fix(kitchen-sink): default mock agent to local endpoint #4942
• test(rivetkit): enable driver lifecycle regressions #4941
• test(rivetkit): verify cached websocket db refs exceed grace #4940
• fix(rivetkit-core): keep sleeping actors reachable during grace #4939
• test(rivetkit): cover async websocket close db access #4938
• fix(rivetkit): keep actors awake until keepAwake work finishes #4937
• fix(rivetkit): persist native onSleep state on errors #4936
• fix(rivetkit-core): allow dispatch during sleep grace #4935
• fix(rivetkit): make duplicate sleep requests idempotent #4934
• fix(rivetkit): align logs with engine logfmt #4933
• fix(ci): standardize publish rust toolchain #4931
• SPLIT UP: agentic loop test #4932
• chore(rivetkit): doc skip ready wait #4927
• fix(guard): parse gateway bypass query booleans #4926
• feat(depot-client): add vfs staging cache ttl #4930
• feat(depot-client): wire sqlite optimization flags #4923 👈 (View in Graphite)
• feat(rivetkit): support gateway bypass client overrides #4928
• test(depot-client): stabilize fault harness #4924
• chore(depot-client): split sqlite vfs transports #4922
• feat(rivetkit): support gateway bypass probes #4915
• feat(kitchen-sink): add mock agentic lifecycle loop #4914
• feat(kitchen-sink): add mock agentic loop #4891
• feat(rivetkit): support gateway bypass client requests #4917
• fix(pegboard): restore hibernating request ids #4898 : 2 other dependent PRs (#4899 , #4916 )
• chore(guard): configure local timeout knobs #4913
• docs(rivetkit): raise serverless start payload limit #4911
• test(kitchen-sink): add sqlite memory soak harness #4910
• feat(rivetkit): expose runtime diagnostics #4909
• chore(depot-client): migrate to worker thread #4907 : 2 other dependent PRs (#4912 , #4918 )
• test(driver): select matrix cells with vitest filters #4906
• fix(pegboard): restore hibernating requests on serverful start #4904 : 1 other dependent PR (#4905 )
• fix(rivetkit): drain shutdown work before sleep #4902 : 1 other dependent PR (#4903 )
• feat(sqlite): wire remote sqlite execute handlers #4897
• test(serverless): avoid pinning header validation order #4896
• chore(driver-test-runner): add native/wasm runtime matrix to skill #4884 : 3 other dependent PRs (#4892 , #4894 , #4901 )
• chroe: rename rivetkit-sqlite to depot-client #4883
• fix(rivetkit-wasm): fix mem leaks #4880
• chore(rivetkit): wasm support #4860
• feat(sqlite): add cold read benchmarks and simplify optimizations #4857
• feat(sqlite): benchmark cold reads #4848 : 1 other dependent PR (#4864 )
• docs(actors): document manual lifecycle controls #4847
• test(sqlite): expand db stress coverage #4846 : 1 other dependent PR (#4849 )
• test(raw-websocket): add serverless smoke coverage #4845
• feat(serverless): configure drain grace period #4844
• fix(publish): enable frontend build for engine target and use turbo build instead of turbo build:engine #4843
• feat(rivetkit): expose conn handle in onWebSocket context #4842
• fix(rivetkit): normalize waitUntil/keepAwake promises to null to avoid serde undefined error #4841
• fix(rivetkit): remove requirement for token #4839 : 1 other dependent PR (#4840 )
• fix(pegboard-envoy): allow sqlite requests across lifecycle states #4838
• fix(pegboard): route scoped actor key reads by runner dc #4837
• fix(sqlite-storage): let sqlite open take over stale owners #4834
• fix(engine-tests): parallelize multi-DC startup + enable 6 multi-DC tests #4833
• fix(api): honor cursor in actor_ids list path #4832
• fix(api): silently filter invalid IDs from actor_ids query instead of 400 #4831
• fix(api-peer): make actors_delete idempotent for already-destroyed actors #4830
• fix(pegboard): use Iterator streaming mode in list_names so limit param is honored #4829
• chore(engine-tests): convert envoy tests to Sleep crash policy + triage ignored #4828
• feat(engine-tests): wire envoy actor names through prepopulate_actor_names #4827
• chore(depot): fault injection tests #4868
• feat(sqlite): pitr & forking #4859 : 1 other dependent PR (#4863 )
• chore(sqlite): stateless storage refactor #4856
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[claude]

Code Review: feat(depot-client): wire sqlite optimization flags

Overview

This PR wires the existing SqliteOptimizationFlags into the VFS layer to control page caching behavior more granularly. Key changes:

• Introduces SqliteVfsPageCacheMode (Off / Target / Startup / Prefetch / All) to classify which pages are cached
• Adds a protected_page_cache: HashMap<u32, Vec<u8>> alongside the moka LRU cache for eviction-immune pages (low-numbered pages)
• Extends startup preload from single-page (page 1 only) to multi-page (startup_preload_first_page_count pages)
• Adds PageCacheInsertKind (Target / Prefetch / Startup) to classify inserts
• Strips read-pool metrics from the benchmark script (read pool feature deferred)

---

Issues

1. Possible logic bug in fetch_initial_pages_for_registration (vfs.rs)

The early-return guard falls back to fetching only page 1 if !config.preload_hints_on_open:

if !config.startup_preload_first_pages
    || !config.preload_hints_on_open   // suspicious
    || !config.page_cache_mode.caches_startup_preloaded_pages()
    || config.startup_preload_max_bytes < DEFAULT_PAGE_SIZE
{
    return fetch_initial_main_page_for_registration(...)

startup_preload_first_pages and preload_hints_on_open control independent features. Multi-page startup preloading warms the VFS cache directly and is useful regardless of whether actor-state preload hints are applied on open. Gating them together means disabling preload_hints_on_open silently also disables the multi-page cold-start fetch, which is non-obvious. Consider removing the !config.preload_hints_on_open arm or explicitly documenting why it belongs here.

2. Write lock held during page-cache insertion after transport call (vfs.rs)

Previously:

let page_cache = { self.state.read().page_cache.clone() };  // clone arc, drop lock
for fetched in ok.pages {
    page_cache.insert(...);  // moka: &self, no exclusive lock needed
}

Now:

let mut state = self.state.write();  // exclusive lock held for all inserts
for fetched in ok.pages { state.cache_page(...); }

The write lock is necessary because protected_page_cache is a plain HashMap. However, this serializes all concurrent VFS callers that were previously able to insert into moka without contention. Per the CLAUDE.md performance guidelines ("Never use Mutex... Use scc::HashMap"), consider whether protected_page_cache could be a scc::HashMap to restore concurrent inserts. The protected cache pages are a small, bounded set (default 512 entries), so a concurrent map is a natural fit.

3. page_cache_weighted_size metric adds entry count for protected cache (vfs.rs)

page_cache_weighted_size: state
    .page_cache
    .weighted_size()
    .saturating_add(state.protected_page_cache.len() as u64),

Without a custom moka weigher, page_cache.weighted_size() returns entry count (weight 1 per entry), so .len() is consistent. However, the metric name implies bytes and a future reader is likely to misinterpret the combined value. Consider renaming the metric to page_cache_entry_count, or adding a custom weigher that assigns actual page-byte weights so the metric reflects real memory pressure.

4. O(n^2) deduplication for early-page hints (vfs.rs)

for pgno in 1..=early_page_count {
    if !snapshot.ranges.iter().any(|range| range.contains(pgno))
        && !snapshot.pgnos.contains(&pgno)   // Vec::contains is O(n)
    {
        snapshot.pgnos.push(pgno);
    }
}
snapshot.pgnos.sort_unstable();

snapshot.pgnos.contains is O(existing hints) per iteration. With MAX_STARTUP_PRELOAD_FIRST_PAGE_COUNT = 256 and typical small hint sets this is fine in practice. A HashSet membership check or collecting into a BTreeSet before the loop would be cleaner and avoid the separate sort pass.

5. Minor: drain(..) in fetch_initial_main_page (vfs.rs)

.map(|mut pages| pages.drain(..).find(|(pgno, _)| *pgno == 1).map(|(_, bytes)| bytes))

Prefer .into_iter().find(...). drain(..) requires mut binding and is unconventional here.

---

Positive observations

• Splitting the page cache into eviction-immune (protected_page_cache) and LRU (page_cache) tiers is the right architecture for frequently-accessed low-numbered pages.
• Constructing VfsConfig before calling fetch_initial_pages_for_registration so the config drives the fetch is a clean improvement.
• PageCacheInsertKind gives a clear audit trail for why a page entered the cache.
• invalidate_page_cache() correctly clears both caches together, preventing the pre-existing bug where only the moka cache was invalidated on truncate.
• should_cache_page always returning true for page 1 regardless of mode is correct, and can_read_cached_page is consistent with it.
• The benchmark cleanup (removing read-pool metrics/scenarios) is clean.
• Test coverage for VfsConfig::from_optimization_flags flag wiring is solid.

---

Summary

The two concerns worth resolving before merge are: (1) the spurious !config.preload_hints_on_open gate that silently disables an independent optimization; and (2) the HashMap behind a write lock for protected_page_cache, which conflicts with the project's concurrent-map guidelines and reduces write-path concurrency. The remaining items are low-priority.

[NathanFlurry]

Landed in main via stack-merge fast-forward push. Commits are in main; closing to match.