feat(store): non-blocking validateStoragePath async background init (Issue #795)

[jlin53882]

Summary

Issue: #795 — plugin init 時 validateStoragePath 的 5 個 sync I/O (lstatSync, realpathSync, existsSync, mkdirSync, accessSync) block event loop，影響高併發 startup 效能。

Fix (方案 B — 真正替換)：移除 _initPluginState 中的 sync validateStoragePath() 呼叫，改用 setImmediate + validateStoragePathAsync()。這樣 5 個 sync I/O 完全不在 critical path 上。

---

Changes

src/store.ts (+74 行)

• 新增 pathExistsAsync(path: string): Promise<boolean> — fs/promises.access 的 async 包裝
• 新增 validateStoragePathAsync(dbPath: string): Promise<string> — 完整 async 版本
  • lstat → lstat
  • realpathSync → realpath
  • mkdirSync → mkdir (recursive + 0o755)
  • accessSync → access
• validateStoragePath (sync 原版) 保留，給其他需要 sync 的 caller 用

index.ts (+18/-2 行)

• import 新增 validateStoragePathAsync
• _initPluginState 內的 validateStoragePath(resolvedDbPath) 改為：

// Defer path validation to after _initPluginState returns but before hooks register.
// Using setImmediate moves the 5 sync I/O (lstatSync/realpathSync/existsSync/mkdirSync/accessSync)
// off the critical path. validateStoragePath failure logs a warning and allows the plugin to
// continue starting — it does not throw, so deferral is safe.
setImmediate(async () => {
  try {
    await validateStoragePathAsync(resolvedDbPath);
  } catch (err) {
    api.logger.warn(
      `memory-lancedb-pro: storage path validation failed — ${String(err)}\n` +
      `  The plugin will still attempt to start, but writes may fail.`,
    );
  }
});

為什麼不用 _startAsyncValidation wrapper：維護者指出 fire-and-forget 方案只是 additive，不是真正替換。方案 B 直接在 _initPluginState 內用 setImmediate 取代 sync 呼叫。

---

Test Coverage

test/validate-storage-path-async.test.mjs — 5 個單元測試，全數通過：

• ✅ resolves and returns absolute path unchanged
• ✅ creates directory when it doesn't exist
• ✅ rejects when parent directory is not writable
• ✅ resolves symlink to real path
• ✅ returns resolved path when directory already exists

---

Maintenance Comment Response

「PR starts with a lot of async infrastructure but the sync call is still there — the sync validateStoragePath() call at line X is still present in _initPluginState」

已修復：validateStoragePath sync 呼叫已從 _initPluginState 移除，替換為 setImmediate(async () => { await validateStoragePathAsync(...) })。5 個 sync I/O 現在完全在 register() critical path 之外執行。

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 930b3d4934

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Remove sync validation from the registration path

When the first register() call runs, this background validation is immediately followed by _initPluginState(api) at line 2200, and _initPluginState still calls validateStoragePath(resolvedDbPath) at line 1963. That means the same lstatSync/realpathSync/existsSync/mkdirSync/accessSync calls still block plugin registration, and the new async path validation just runs a second time later. For the high-concurrency startup scenario this change is meant to fix, the blocking I/O remains on the critical path.

Useful? React with 👍 / 👎.

[OKmiao88888888]

Plugin init is getting blocked by sync I/O in our environment. This async fix would be really helpful. Looking forward to the merge!

[jlin53882]

You can manually download this PR and apply it locally. After testing, it has proven to be very effective in solving the problem. OKmiao88888888 ***@***.***> 於 2026年5月11日 週一 上午11:38 寫道：

…

*OKmiao88888888* left a comment (CortexReach/memory-lancedb-pro#797) <#797 (comment)> Plugin init is getting blocked by sync I/O in our environment. This async fix would be really helpful. Looking forward to the merge! — Reply to this email directly, view it on GitHub <#797 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASID5C75X4KG5K6ELY4YQD342FDMLAVCNFSM6AAAAACYYNRGC2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DIMJXGQYTKNRTGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[TurboTheTurtle]

Review finding:

P1: async validation does not remove the sync startup I/O

This PR starts validateStoragePathAsync() in the background, but _initPluginState() still immediately calls the original sync validateStoragePath() during registration (index.ts:1962-1964). The new register() code then parses the config and starts async validation separately (index.ts:2194-2196), so startup still pays the same sync filesystem cost and then duplicates the validation later.

Because the issue is specifically plugin init blocking on lstatSync / realpathSync / existsSync / mkdirSync / accessSync, the fix needs to remove or defer the sync validateStoragePath() call from the registration critical path. As written, the async validator is additive rather than replacing the blocking path.

[jlin53882]

✅ P2 Fix Applied — Sync validateStoragePath Removed from _initPluginState

The P2 concern was correct: the original PR only added async infrastructure but the sync validateStoragePath(resolvedDbPath) call at line 1963 was still present in _initPluginState, meaning the 5 sync I/O calls (lstatSync/realpathSync/existsSync/mkdirSync/accessSync) still blocked the registration critical path.

This has been fixed in the latest commits. The sync call is now fully replaced.

---

Before → After

Before (sync — still in _initPluginState):

// _initPluginState() — line ~1963
validateStoragePath(resolvedDbPath);  // ← BLOCKING, 5 sync I/O on critical path

After (setImmediate deferral):

// _initPluginState() — line ~1943
try {
  // Defer path validation to after _initPluginState returns but before hooks register.
  // Using setImmediate moves the 5 sync I/O off the critical path.
  setImmediate(async () => {
    try {
      await validateStoragePathAsync(resolvedDbPath);  // ← async, runs in background
    } catch (err) {
      api.logger.warn(
        `memory-lancedb-pro: storage path validation failed — ${String(err)}\n` +
        `  The plugin will still attempt to start, but writes may fail.`,
      );
    }
  });
} catch (err) {
  api.logger.warn(`memory-lancedb-pro: storage path issue — ${String(err)}...`);
}

---

Verification

• ✅ validateStoragePath (sync) removed from _initPluginState — confirmed via grep: grep -n "validateStoragePath(" index.ts returns zero matches (only validateStoragePathAsync remains)
• ✅ validateStoragePathAsync imported at line 23 and called at line 1945
• ✅ npm test — 28/28 existing tests pass
• ✅ Timing test confirms register() returns in <5ms (no sync I/O blocking)

---

Test Coverage

Test	Status
npm test (28 existing tests)	✅ 28/28
validateStoragePathAsync — rejects when path is file	✅
validateStoragePathAsync — succeeds with existing directory	✅
validateStoragePathAsync — creates missing intermediate dirs recursively	✅
register() — returns within 5ms (no sync I/O)	✅
register() — does NOT throw with invalid path (validation deferred)	✅

---

PR diff summary: +179/-2 across 4 files (index.ts, src/store.ts, test/validate-storage-path-async.test.mjs, scripts/ci-test-manifest.mjs)

[app3apps]

Closing this PR for now.

The startup performance issue is narrow, while this changes storage path validation from synchronous startup failure into deferred background warning behavior. That tradeoff needs stronger evidence and a safer design before merge. If we revisit this, please bring benchmarks and preserve clear startup failure semantics where possible.