fix: auto-compression afterquakes in CommonDao

[mrnagydavid]

Summary

• Fixed index exclusion handling with compression by injecting 'data' into the "global" configuration list, not just the "local" config of save() calls
• Fixed auto-compression architecture: DBM is now consistently the logical (uncompressed) database model. Compression/decompression happens transparently at the storage boundary, not in bmToDBM/dbmToBM
• Added public storage conversion API for direct DB access (e.g., cross-environment data copy):
  • dbmToStorageRow(dbm) → unknown - compress DBM to storage format
  • dbmsToStorageRows(dbms) → unknown[] - batch version
  • storageRowToDBM(row) → DBM - decompress storage row to DBM
  • storageRowsToDBMs(rows) → DBM[] - batch version
• Fixed saveAsDBM(readAsDBM()) round-trip which was broken before
• Comprehensive tests for all read/write APIs with compression enabled

Potential future improvements

Type safety for storage rows could be improved via:

• Branded/opaque StorageRow type to document intent
• Change CommonDB.saveBatch signature to accept any[] (acknowledges DB doesn't validate types)
• Generic constraint saveBatch<T extends object>(table, rows: T[])

Test plan

• All db-lib tests pass (303 tests)
• Tested in NCBackend3-alpha: 4923 tests passed

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR fixes the auto-compression architecture in CommonDao by introducing a proper storage layer separation. Previously, compression/decompression was incorrectly handled in the bmToDBM/dbmToBM conversion layer, which broke the saveAsDBM(readAsDBM()) round-trip pattern. Now, DBM consistently represents the logical (uncompressed) database model, and compression/decompression happens transparently at the storage boundary.

Changes:

• Moved compression/decompression from model conversion layer to storage boundary layer
• Added public storage conversion APIs (dbmToStorageRow, storageRowToDBM, and batch variants) for direct DB access scenarios
• Updated all read/write APIs to use the new storage layer (queries, streams, batch operations, multiGet/multiSave)
• Enhanced index exclusion handling to automatically include the 'data' property when compression is enabled
• Added comprehensive test coverage for all read/write APIs with compression enabled

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
packages/db-lib/src/commondao/common.dao.ts	Refactored compression architecture with storage layer separation; added public storage conversion APIs; updated all read/write methods to use storage layer; enhanced index exclusion logic
packages/db-lib/src/commondao/common.dao.test.ts	Added comprehensive tests for round-trip scenarios, all read/write APIs with compression, and new storage conversion APIs

Comments suppressed due to low confidence (1)

packages/db-lib/src/commondao/common.dao.ts:555

• The prepareSaveOptions method mutates the excludeFromIndexes array when compression is enabled. If a caller passes an excludeFromIndexes array in the opt parameter, that array will be mutated by the .push() operation on line 553.

This is a side effect that could affect the caller's data structure. The method should create a new array instead of mutating the input. Consider using spread syntax to create a copy before mutating:

if (this.cfg.compress?.keys) {
  excludeFromIndexes = excludeFromIndexes ? [...excludeFromIndexes] : []
  if (!excludeFromIndexes.includes('data' as any)) {
    excludeFromIndexes.push('data' as any)
  }
}

    if (this.cfg.compress?.keys) {
      excludeFromIndexes ??= []
      if (!excludeFromIndexes.includes('data' as any)) {
        excludeFromIndexes.push('data' as any)
      }
    }

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

🎉 This PR is included in version @naturalcycles/db-lib-v10.42.1 🎉

The release is available on:

• pnpm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀