Skip to content

feat(redis-lock): 分散式 Redis lock 實作,支援 file-lock fallback (PR739 rebuild)#781

Closed
jlin53882 wants to merge 3 commits into
CortexReach:masterfrom
jlin53882:james/pr739-redis-lock-rebuild
Closed

feat(redis-lock): 分散式 Redis lock 實作,支援 file-lock fallback (PR739 rebuild)#781
jlin53882 wants to merge 3 commits into
CortexReach:masterfrom
jlin53882:james/pr739-redis-lock-rebuild

Conversation

@jlin53882
Copy link
Copy Markdown
Contributor

PR 739 重建說明

原本的 PR #739(分支 fix/pr704-redis-url-parsing-v2)已關閉,有 merge conflict。

此 PR 以最新 master 為基礎重新實作,完整保留 #690 batch accumulator 功能。


實作內容

新增 src/redis-lock.ts(340 行)

  • Dynamic import:ioredis 只在 connect() 時才載入,避免 consumer 未安裝就 crash
  • RedisUnavailableError:Symbol.for marker,跨 module ESM-safe
  • isRedisConnectionError:遞迴檢查 wrapped error(errors[] / cause),depth=3 上限
  • parseRedisUrl():正確解析含密碼 URL 和 DB selection(/0, /1...)
  • RedisLockManager:TTL、maxWait、retryDelay、namespace key
  • dbPath hash namespace:避免不同 dbPath 的 store instances 互相 blocking

修改 src/store.ts(+50 行)

  • import RedisLockManager / createRedisLockManager / RedisUnavailableError
  • runWithFileLock():Redis lock 優先,acquire() 失敗拋 RedisUnavailableError 不 fallback
  • runWithFileLockCore():保留既有 file lock 實作(新增 realpath:false 修正 [BUG] ENOENT from proper-lockfile realpath() after proactive stale lock cleanup #670
  • getRedisLockManager():模組層級 cache(Option E — init-time decision, runtime fixed)

新增測試(40 tests,全部通過)

  • test/redis-lock-error-types.test.mjs — RedisUnavailableError + isRedisConnectionError
  • test/redis-lock-url-parse.test.mjs — URL 解析各種格式
  • test/redis-lock-concurrent-init.test.mjs — concurrent init + Symbol marker

修復的 Issue

Issue 問題 解決方式
Issue 1 ioredis top-level import 造成無 ioredis 時 crash 改用 dynamic import
Issue 3 parseRedisUrl() 吃掉 DB selection 用 URL API 正確解析
Issue 4 不同 dbPath 的 store 互相 blocking dbPath hash namespace
#670 stale lock cleanup 造成 ENOENT proper-lockfile realpath:false

測試結果

tests: 40, pass: 40, fail: 0

cross-process-lock.test.mjs、reflection-bypass-hook.test.mjs 皆通過。

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2b8de59b23

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/retriever.ts
Comment on lines +661 to +663
mode = "bm25";
results = await this.bm25OnlyRetrieval(
query, tagTokens, safeLimit, scopeFilter, category, trace,
query, tagTokens, safeLimit, scopeFilter, category, trace, diagnostics,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Declare trace locals before branching

When retrieveWithTrace takes the tag/auto-recall path, this assigns to mode and passes diagnostics even though neither variable is declared in this method; because the package is ESM/strict mode, the assignment throws ReferenceError before any retrieval result or debug trace is returned. The same pattern appears in the other branches, so declaring/initializing these locals as in retrieve() is needed before calling the memory_debug trace path.

Useful? React with 👍 / 👎.

Comment thread src/redis-lock.ts
Comment on lines +151 to +155
this.redis = new Redis({
host: parsed.host,
port: parsed.port,
db: parsed.db,
lazyConnect: true,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve Redis URL authentication

For password-protected Redis URLs such as redis://:secret@host:6379/1, parseRedisUrl() extracts only host/port/db and this constructor never passes username/password to ioredis. In those deployments the lock manager cannot authenticate and will disable/fail Redis locking even though REDIS_URL is valid; pass the original URL or include the auth fields in the options.

Useful? React with 👍 / 👎.

Comment thread src/redis-lock.ts
Comment on lines +133 to +135
constructor(private readonly config?: LockConfig) {
// Issue 4 fix: namespace key with dbPath hash,避免不同 dbPath 的 store 互相 blocking
this._lockNamespace = config?.dbPath ? hashString(config.dbPath) : "default";
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply the lock timing config

When callers provide ttl, maxWait, or retryDelay in LockConfig, the constructor only computes the namespace and leaves the class defaults in place, so a contended lock still waits up to 60s and uses the default TTL/retry cadence. That makes the public config ineffective for tests or deployments that need shorter acquisition timeouts.

Useful? React with 👍 / 👎.

PR 739 重建(原本 PR 有 merge conflict):
- 新增 src/redis-lock.ts(340行)
  - dynamic import 避免 ioredis 未安裝時 crash
  - RedisUnavailableError + Symbol.for marker(跨 module ESM-safe)
  - isRedisConnectionError 含 wrapped error 遞迴檢查
  - parseRedisUrl() 正確解析 URL(含密碼、DB selection)
  - RedisLockManager + createRedisLockManager 工廠
  - dbPath hash namespace 避免跨 store lock interference

- 修改 src/store.ts(+50行)
  - import RedisLockManager/createRedisLockManager/RedisUnavailableError
  - runWithFileLock() 新增 Redis lock 優先實作
  - runWithFileLockCore() 保留既有 file lock 實作
  - getRedisLockManager() 模組層級 cache(Option E)
  - 修正 CortexReach#670:realpath:false 避免 stale lock cleanup 造成 ENOENT

- 新增 3 個測試檔(40 個測試,全部通過)
  - test/redis-lock-error-types.test.mjs
  - test/redis-lock-url-parse.test.mjs
  - test/redis-lock-concurrent-init.test.mjs

- package.json:新增 ioredis devDependency

注意:batch accumulator(CortexReach#690)完整保留,不受影響。
@jlin53882 jlin53882 force-pushed the james/pr739-redis-lock-rebuild branch from 2b8de59 to 926586a Compare May 9, 2026 17:19
@jlin53882
Copy link
Copy Markdown
Contributor Author

⚠️ 此 PR 已被 PR #788 取代

PR #781 混入了無關的 commits(retriever.ts + retriever-auto-recall-signal.test.mjs,來自 PR746),這些與 Redis lock 功能無關。

PR #788 是乾淨的重建:

  • ✅ src/redis-lock.ts — 相同內容(327 行)
  • ✅ src/store.ts — 新增整合(+74 行,Redis-first + file-lock fallback)
  • ✅ 無 PR746 無關內容
  • ✅ 已通過 maintainer R1/R2/R3 review patches

建議關閉此 PR,專注 PR #788
#788

@jlin53882 jlin53882 closed this May 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant