Skip to content

Feat/scanner pool parallel#31

Open
Nonso-Eze wants to merge 2 commits into
wraith-protocol:developfrom
Nonso-Eze:feat/scanner-pool-parallel
Open

Feat/scanner pool parallel#31
Nonso-Eze wants to merge 2 commits into
wraith-protocol:developfrom
Nonso-Eze:feat/scanner-pool-parallel

Conversation

@Nonso-Eze
Copy link
Copy Markdown

feat(scanner-pool): parallel multichain scanning with concurrency control

Closes #25

Implements MultichainScannerPool for parallel EVM, Stellar, Solana, and CKB scanning, replacing the sequential stub that existed before.

What changed

scanner-pool.ts
— the previous implementation imported worker_threads but never used it; both the Node and browser paths called scanChain in a sequential loop. Replaced with a proper concurrency-limited queue (runFailFast) that fans all active chains out simultaneously via Promise.race. Chains no longer wait on each other.

New options:

concurrency — max chains running at once (default 4)
failFast — reject on first error (default true); set false to let all chains settle before surfacing an error
on()/off() now return this for chaining. Progress events fire at scan start and completion per chain.

scanner-pool.test.ts
— previous tests had a vi.mock factory that referenced a closure-scoped delay function which Vitest hoists out of scope, causing 10 failures. Rewrote the full suite covering: constructor defaults, per-chain result isolation, parallelism timing, progress event emission, AbortSignal cancellation, failFast behaviour, and a benchmark smoke test. All 153 tests pass.

Security note

Private keys stay in-process. No serialisation across worker boundaries until a hardened transport is implemented — documented in the source.

@drips-wave
Copy link
Copy Markdown

drips-wave Bot commented May 29, 2026

@Nonso-Eze Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@truthixify truthixify changed the base branch from main to develop June 1, 2026 15:29
@truthixify
Copy link
Copy Markdown
Contributor

The scanner pool implementation is solid: real concurrency-limited queue via Promise.race, failFast toggle, AbortSignal handling, progress emission per chain, the security note about private keys staying in-process. Want this in.

Same blocker as several other open PRs: branched from older main so the diff would undo recent develop landings (CONTRIBUTING, Stellar JSDoc, the time-range/cursor changes, the announcements test). Could you rebase?

git fetch origin
git rebase origin/develop
# resolve everything in src/chains/stellar/* and test/chains/stellar/* by keeping develop's versions
# your new src/scanner-pool.ts and test/scanner-pool.test.ts are net-new, no conflict
git push --force-with-lease

Thanks @Nonso-Eze.

Nonso-Eze added 2 commits June 2, 2026 00:08
- Implement MultichainScannerPool for concurrent scanning across EVM, Stellar, Solana, and CKB
- Support browser Web Workers, Node worker_threads, and React Native sequential modes
- Add progress reporting and AbortSignal cancellation support
- Respect configurable concurrency limits and result merging
- Include comprehensive test suite with 14 test cases
- Export types: ScanInput, ScanResults, ProgressEvent, MultichainScannerPoolOptions
- Optimize bundle size with dynamic imports per chain
- Type-safe scanning API with chain-specific input/output handling
@Nonso-Eze Nonso-Eze force-pushed the feat/scanner-pool-parallel branch from 88603a5 to 8c2e647 Compare June 1, 2026 23:10
@Nonso-Eze
Copy link
Copy Markdown
Author

Rebased on upstream/develop and force-pushed. The three upstream commits (time-range/cursor filtering, Stellar JSDoc, CONTRIBUTING) are now underneath with no conflicts — rebase applied cleanly.

Diff is still exactly three files:
scanner-pool.ts
,
scanner-pool.test.ts
, and
index.ts
. None of the Stellar or chain files were touched.

Quick summary of what's in the implementation:

Real concurrency-limited queue via Promise.race — chains genuinely run in parallel, not sequentially behind a fake parallel wrapper like the old stub
concurrency option caps how many chains run at once (default 4)
failFast toggle — true (default) rejects on first error, false lets all chains settle first
AbortSignal support — pre-aborted signal rejects immediately, mid-scan abort propagates to all active chains
Progress events fire at scan start and completion per chain (processed: 0 → processed: total)
Private keys stay in-process — no serialisation across worker boundaries, documented in source
153/153 tests passing

@truthixify
Copy link
Copy Markdown
Contributor

Thanks for rebasing. You caught up to #16, but #37 (Bun/Deno COMPAT) landed since, so the current diff would now revert COMPAT.md, docs/running-on-the-edge.md, and the announcements.ts dynamic-import fix.

One more rebase against latest develop:

git fetch origin
git rebase origin/develop
# resolve by keeping develop's COMPAT.md, docs/running-on-the-edge.md, and announcements.ts changes from #37
# keep your src/scanner-pool.ts, test/scanner-pool.test.ts, and src/index.ts exports
git push --force-with-lease

After this the diff should only be your 3 net-new files. Sorry for the moving target. Thanks @Nonso-Eze.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Worker pool for parallel multichain scanning

2 participants