|
| 1 | +# TEST-NEEDS.md — nextgen-databases |
| 2 | + |
| 3 | +> Generated 2026-03-29 by punishing audit. |
| 4 | +
|
| 5 | +## Current State |
| 6 | + |
| 7 | +| Category | Count | Notes | |
| 8 | +|-------------|-------|-------| |
| 9 | +| Unit tests | ~40 | VeriSimDB Elixir: consensus (kraft_node, kraft_wal, kraft_recovery, kraft_transport), federation adapters (mongodb, redis, duckdb, clickhouse, surrealdb, sqlite, neo4j, vector_db, influxdb, object_storage), resolver, adapter + base tests | |
| 10 | +| Integration | ~12 | Federation adapter integration tests (mongodb, redis, neo4j, clickhouse, surrealdb, influxdb) | |
| 11 | +| E2E | 0 | None | |
| 12 | +| Benchmarks | 2 | verisimdb/benches/modality_benchmarks.rs (Rust), lithoglyph core-factor benchmarks.factor | |
| 13 | + |
| 14 | +**Source modules:** ~833 across 2 major subsystems. verisimdb: ~248 files (Rust core, Elixir orchestration, Gleam, Idris2 ABI, Zig FFI, ReScript). lithoglyph: ~212 files (Gleam, Rust, Factor). |
| 15 | + |
| 16 | +## What's Missing |
| 17 | + |
| 18 | +### P2P (Property-Based) Tests |
| 19 | +- [ ] Kraft consensus: property tests for leader election, log replication, partition tolerance |
| 20 | +- [ ] CRDT convergence: property tests for VeriSimDB's CRDT operations |
| 21 | +- [ ] VQL query parsing: arbitrary query fuzzing |
| 22 | +- [ ] Federation: property tests for data consistency across adapters |
| 23 | +- [ ] lithoglyph: data structure invariant tests |
| 24 | + |
| 25 | +### E2E Tests |
| 26 | +- [ ] VeriSimDB: full write -> replicate -> read across nodes |
| 27 | +- [ ] Federation: write through adapter -> verify in external DB -> read back |
| 28 | +- [ ] Kraft consensus: cluster formation -> leader election -> write -> node failure -> recovery |
| 29 | +- [ ] lithoglyph: full lifecycle (create -> write -> query -> archive) |
| 30 | +- [ ] VQL: complex query execution with joins/aggregations |
| 31 | + |
| 32 | +### Aspect Tests |
| 33 | +- **Security:** No tests for authentication bypass, unauthorized federation access, injection through VQL, data exfiltration across adapters |
| 34 | +- **Performance:** Rust modality benchmark exists. Missing: Elixir orchestration throughput, Kraft consensus latency, federation adapter comparison benchmarks |
| 35 | +- **Concurrency:** No tests for concurrent writes across Kraft nodes, federation adapter connection pooling, VQL query contention |
| 36 | +- **Error handling:** No tests for adapter connection failure, Kraft split-brain recovery, malformed VQL, storage corruption |
| 37 | + |
| 38 | +### Build & Execution |
| 39 | +- [ ] `mix test` for VeriSimDB Elixir |
| 40 | +- [ ] `cargo test` for VeriSimDB Rust |
| 41 | +- [ ] `gleam test` for lithoglyph |
| 42 | +- [ ] Zig FFI tests |
| 43 | +- [ ] Container-based multi-node tests |
| 44 | + |
| 45 | +### Benchmarks Needed |
| 46 | +- [ ] Write throughput (single node, cluster) |
| 47 | +- [ ] Read latency (hot, cold, cache miss) |
| 48 | +- [ ] Kraft consensus round-trip time |
| 49 | +- [ ] Federation adapter roundtrip per backend |
| 50 | +- [ ] VQL query execution time by complexity |
| 51 | +- [ ] lithoglyph query performance |
| 52 | +- [ ] Replication lag measurement |
| 53 | + |
| 54 | +### Self-Tests |
| 55 | +- [ ] Cluster health self-check |
| 56 | +- [ ] Federation adapter connectivity verification |
| 57 | +- [ ] Data integrity checksums |
| 58 | +- [ ] WAL consistency validation |
| 59 | + |
| 60 | +## Priority |
| 61 | + |
| 62 | +**CRITICAL.** Two database systems with 833 source files and ~52 tests (6.2%). The consensus layer (Kraft) has 4 tests for a distributed consensus protocol — that is dangerously low. Federation adapters have decent unit coverage but zero E2E. lithoglyph appears to have no dedicated tests at all. A database with no concurrency tests is a ticking time bomb. |
| 63 | + |
| 64 | +## FAKE-FUZZ ALERT |
| 65 | + |
| 66 | +- `tests/fuzz/placeholder.txt` is a scorecard placeholder inherited from rsr-template-repo — it does NOT provide real fuzz testing |
| 67 | +- Replace with an actual fuzz harness (see rsr-template-repo/tests/fuzz/README.adoc) or remove the file |
| 68 | +- Priority: P2 — creates false impression of fuzz coverage |
0 commit comments