perf(drivers): MongoDB + PostgreSQL optimizations (3/3 of #991)

[SamTV12345]

perf(drivers): MongoDB + PostgreSQL driver optimizations

Part 3 of 3 from splitting #991. Touches databases/mongodb_db.ts and databases/postgres_db.ts only.

MongoDB (mongodb_db.ts)

• Drop the per-operation schedulePing(). Every get/set/findKeys/remove/doBulk did a clearInterval + setInterval pair. The mongodb driver already maintains its own server heartbeat, so this was two timer mutations per op for nothing. The interval field and clearPing/schedulePing helpers are removed.
• Fix findKeys regex. Previously stripped * instead of converting it to .*, and had no ^…$ anchor — producing unanchored substring matches that bypassed the _id index and could return false positives. Now escapes regex metacharacters and converts wildcards correctly.
• doBulk uses initializeUnorderedBulkOp() instead of ordered — our bulk ops are keyed per-op so ordering is unnecessary, and unordered parallelizes server-side.
• Tighten types: a UeberDoc document shape + Filter<UeberDoc> / Collection<UeberDoc> narrowing replace the any casts the changes above would otherwise have introduced.

PostgreSQL (postgres_db.ts)

• doBulk collapses N round-trips into one on the native-upsert path via a single multi-row INSERT … VALUES ($1,$2),($3,$4),… ON CONFLICT DO UPDATE. The function-based fallback (old PG / CockroachDB without native upsert) is preserved for single-row and multi-row cases.
• Named prepared statements on the three hottest single-row queries (get, set, remove) so PG parses + plans them once per connection.

Verification

• tsc --noEmit passes.
• vitest run test/mock test/memory → 173 passing. (Live mongo/postgres suites require running databases — CI exercises those.)

Merge order

These two files are also reformatted by the formatting PR, so merge the formatting PR first, then I'll rebase this one on top (it'll resolve to a clean logic-only diff). Independent of the cache PR.

🤖 Generated with Claude Code

[qodo-code-review]

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

[qodo-free-for-open-source-projects]

Review Summary by Qodo

MongoDB and PostgreSQL driver performance optimizations

✨ Enhancement

Walkthroughs

Description

• Remove redundant per-operation MongoDB ping scheduling (clearInterval/setInterval pairs)
• Fix MongoDB findKeys regex to properly escape metacharacters and anchor matches
• Switch MongoDB bulk operations from ordered to unordered for server-side parallelization
• Collapse PostgreSQL bulk upserts into single multi-row INSERT with ON CONFLICT
• Add named prepared statements for PostgreSQL hottest queries (get, set, remove)
• Tighten TypeScript types with UeberDoc document shape and Filter narrowing

Diagram

flowchart LR
  A["MongoDB Operations"] -->|Remove schedulePing| B["Eliminate Timer Overhead"]
  A -->|Fix findKeys Regex| C["Correct Pattern Matching"]
  A -->|Unordered Bulk Ops| D["Enable Server Parallelization"]
  E["PostgreSQL Operations"] -->|Multi-row Upsert| F["Reduce Round-trips"]
  E -->|Named Prepared Statements| G["Cache Query Plans"]
  H["Type Improvements"] -->|UeberDoc + Filter| I["Remove any Casts"]

Loading

File Changes

1. databases/mongodb_db.ts ✨ Enhancement +46/-70

MongoDB driver optimizations and type safety

• Removed interval field and clearPing()/schedulePing() methods that redundantly cleared/set
 intervals on every operation
• Fixed findKeys() regex to properly escape metacharacters and convert wildcards to .* with
 ^...$ anchors for correct pattern matching
• Changed doBulk() from initializeOrderedBulkOp() to initializeUnorderedBulkOp() for
 server-side parallelization
• Added UeberDoc type definition and Filter typing to eliminate any casts and narrow MongoDB's
 generic types

databases/mongodb_db.ts

---

2. databases/postgres_db.ts ✨ Enhancement +62/-39

PostgreSQL bulk operations and prepared statement caching

• Added named prepared statements (ueberdb_get, ueberdb_set_native, ueberdb_set_function,
 ueberdb_remove) to cache query plans per connection
• Refactored doBulk() to collapse multiple round-trips into single multi-row INSERT with ON
 CONFLICT for native upsert path
• Preserved function-based fallback for old PostgreSQL versions and single-row operations
• Improved error handling with AsyncTaskCb type wrapper for async.parallel() callback
 compatibility

databases/postgres_db.ts

---

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (0)

1. doBulk may never resolve 🐞 Bug ☼ Reliability

Description

postgres_db.doBulk() returns early when upsertStatement is unset without invoking the callback,
which can leave the promisified wrapper Promise pending forever. This can stall CacheAndBufferLayer
flush/write paths that await doBulk().

Code

databases/postgres_db.ts[R207-210]

Evidence

The new early-return path in postgres_db.doBulk() does not call the provided callback.
CacheAndBufferLayer wraps non-async drivers with util.promisify and then awaits doBulk();
promisified Promises require the callback to be invoked to settle.

databases/postgres_db.ts[207-210]
lib/CacheAndBufferLayer.ts[171-185]
lib/CacheAndBufferLayer.ts[585-603]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`databases/postgres_db.ts#doBulk()` returns early when `this.upsertStatement` is falsy without invoking `callback`. In the legacy-driver path, `CacheAndBufferLayer` wraps driver methods with `util.promisify()`, so skipping the callback can result in a Promise that never settles (hang).

## Issue Context
- `CacheAndBufferLayer` promisifies legacy drivers via `promisify(f.bind(wrappedDB))` and later `await`s `wrappedDB.doBulk(ops)`.
- If `doBulk()` returns without calling its callback, the promisified Promise remains pending.

## Fix Focus Areas
- databases/postgres_db.ts[207-210]

### Suggested change
Call `callback(new Error('...'))` (or `callback(null)` if you prefer a no-op) before returning when `upsertStatement` is unset, so the promisified wrapper can resolve/reject deterministically.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Untyped Mongo collection field 🐞 Bug ⚙ Maintainability

Description

mongodb_db.ts introduces UeberDoc/Filter<UeberDoc> but keeps this.collection typed as an
unparameterized Collection, forcing non-null assertions and casts in findKeys(). This reduces type
safety and makes future edits more error-prone.

Code

databases/mongodb_db.ts[R27-30]

Evidence

The file introduces UeberDoc and a typed selector (Filter<UeberDoc>), but the stored collection is
not parameterized, leading to a cast at the usage site.

databases/mongodb_db.ts[20-30]
databases/mongodb_db.ts[41-47]
databases/mongodb_db.ts[68-83]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`mongodb_db.ts` now defines `UeberDoc` and uses `Filter<UeberDoc>`, but `this.collection` remains `Collection | undefined`, so code must cast `(this.collection as Collection<UeberDoc> | undefined)!` to use typed filters.

## Issue Context
Keeping the collection untyped makes it easier to accidentally use the wrong document shape in other methods and pushes type assertions into call sites.

## Fix Focus Areas
- databases/mongodb_db.ts[27-30]
- databases/mongodb_db.ts[41-47]
- databases/mongodb_db.ts[68-83]

### Suggested change
- Change the field to `public collection: Collection<UeberDoc> | undefined;`
- Initialize with generics: `this.collection = this.database.collection<UeberDoc>(...)`
- Remove the cast in `findKeys()` and use `this.collection!.find(selector)` directly.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---