Skip to content

feat: add fetchOrderBooks for multi-market order book retrieval#130

Merged
realfishsam merged 15 commits into
pmxt-dev:mainfrom
psyberck:feat/fetch-multi-order-books
May 22, 2026
Merged

feat: add fetchOrderBooks for multi-market order book retrieval#130
realfishsam merged 15 commits into
pmxt-dev:mainfrom
psyberck:feat/fetch-multi-order-books

Conversation

@psyberck
Copy link
Copy Markdown
Contributor

@psyberck psyberck commented May 11, 2026

Summary (Issue #126)

  • Adds new fetchOrderBooks(outcomeIds) API for batch order book retrieval, implemented for Polymarket and Kalshi
  • Adds Kalshi GetMarketOrderbooks implicit endpoint
  • Renames the ids parameter to outcomeIds across the API surface for consistency with existing unified types.
  • Add paramsSerializer so array query params are encoded correctly.

Files of note

  • core/src/BaseExchange.ts, core/src/exchanges/kalshi/{index,api,fetcher}.ts, core/src/exchanges/polymarket/{index,fetcher}.ts — new method + refactor
  • core/specs/kalshi/Kalshi.yaml, core/src/server/openapi.yaml, docs/api-reference/openapi.json — spec updates
  • sdks/{typescript,python}/... — client helper + regenerated reference docs

Side Effects

  • Rather than copying the entire openapi.yaml into Kalshi.yaml, only the GetMarketOrderbooks endpoint was added. Upgrading to Kalshi API v3.16.0 introduces extensive interface changes, which would require modifying a large number of existing function signatures.
  • The createImplicitMethod function signature was updated to accept parameters of type Record<string, any> | any[] for greater flexibility. A more robust approach exists, but implementing it would necessitate widespread changes across related function calls.
  • Aligned indentation and quote style in kalshi/index.ts with project conventions.

Testing

Ploymarket valid outcome Ids (Python)

Screenshot from 2026-05-11 02-00-46

Ploymarket invalid outcome Ids (Python)

Screenshot from 2026-05-11 01-59-48

Ploymarket valid outcome Ids (TypeScript)

Screenshot from 2026-05-11 02-12-03

Kalshi valid outcome Ids (Python)

Screenshot from 2026-05-11 02-53-43

@realfishsam
Copy link
Copy Markdown
Contributor

Awesome. Will merge asap.

For next time: please copy and paste the code input/output instead of using screenshots! Makes things easier on my end!

@psyberck
Copy link
Copy Markdown
Contributor Author

Awesome. Will merge asap.

For next time: please copy and paste the code input/output instead of using screenshots! Makes things easier on my end!

For sure, thanks for pointing out. I will leave my testing scripts in the PR as well in the future. :)

@psyberck psyberck force-pushed the feat/fetch-multi-order-books branch from 03519a3 to eee1b5d Compare May 22, 2026 15:51
Copy link
Copy Markdown
Contributor

@realfishsam realfishsam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review: FAIL

What This Does

Adds fetchOrderBooks(outcomeIds) — a batch REST endpoint for fetching multiple order books in one call — implemented for Polymarket (via postBooks) and Kalshi (via new GetMarketOrderbooks). Also adds a global paramsSerializer to axios so array query params serialize correctly for Kalshi's GET API.

Blast Radius

  • BaseExchange.ts: new fetchOrderBooks abstract stub, callApi signature widened to accept any[], global paramsSerializer added to every exchange's axios instance
  • Kalshi: new spec entry + api.ts regeneration, fetcher.ts (new fetchRawOrderBooks, return type of fetchRawOrderBook widened to KalshiRawOrderBook), index.ts (new method + indentation reformat)
  • Polymarket: fetcher.ts (new fetchRawOrderBooks), index.ts (new method), PolymarketRawOrderBook type extended with asset_id
  • Server: method-verbs.json (new POST verb entry), openapi.yaml (new endpoint + inline exchange enums replacing $ref)
  • SDKs (both): new fetchOrderBooks wrapper method in hand-written clients
  • Docs: api-doc-config.generated.json, docs/api-reference/openapi.json, API_REFERENCE.md

Consumer Verification

Before (base branch main):
fetchOrderBooks did not exist. Callers had to loop over fetchOrderBook individually.

# fetchOrderBook (singular) still works — non-regression confirmed
curl -X GET "http://localhost:3848/api/polymarket/fetchOrderBook?outcomeId=<oid>" \
  -H "x-pmxt-access-token: $TOKEN"
# {"success":true,"data":{"bids":[],"asks":[91 levels],"timestamp":1779468011943}}

After (PR branch):
Batch call verified live against Polymarket:

curl -X POST http://localhost:3848/api/polymarket/fetchOrderBooks \
  -H "Content-Type: application/json" \
  -H "x-pmxt-access-token: $TOKEN" \
  -d '{"args":[["<oid1>","<oid2>"]]}'
# {"success":true,"data":{
#   "<oid1>":{"bids":[],"asks":[91 levels],"timestamp":...},
#   "<oid2>":{"bids":[7 levels],"asks":[55 levels],"timestamp":...}
# }}

Kalshi returns AuthenticationError without credentials (expected — GetMarketOrderbooks is a private endpoint per the Kalshi spec).


Test Results

  • Build (TypeScript): PASS
  • Python unit tests: FAIL (pre-existing — pmxt_internal module not installed in this env; unrelated to this PR)
  • Server start: PASS (port 3848)
  • E2E smoke — Polymarket fetchOrderBooks: PASS (2 books returned, correct shape)
  • E2E smoke — Polymarket fetchOrderBook (non-regression): PASS
  • Kalshi fetchOrderBooks without creds: PASS (correct AuthenticationError)

Findings

1. CI: client.py generator drift (blocking)
sdks/python/scripts/generate-client-methods.js produces different code from what's committed:

-    def fetch_order_books(self, ids: List[str]) -> Dict[str, OrderBook]:
+    def fetch_order_books(self, outcome_ids: dict) -> dict:
         ...
-            return {key: _convert_order_book(value) for key, value in (data or {}).items()}
+            return data

The CI check "Verify client.py methods are up-to-date" fails because the generator uses the parameter name from BaseExchange.ts (outcomeIdsoutcome_ids) and does not emit the _convert_order_book conversion logic. Fix: run the generator first, then manually override the generated stub with a correctly typed implementation, similar to fetch_order_book. The current PR's implementation is functionally correct but bypasses the CI-enforced generation workflow.

2. CI: docs/llms.txt and docs/llms-full.txt not regenerated (blocking)
After running npm run docs:llms, both files gain a new fetchOrderBooks entry. The "Verify docs are in sync with core" CI check fails because these files were not regenerated and committed. Fix: run npm run docs:llms and commit.

3. CI: Exchange drift — gemini-titan, hyperliquid, mock removed from openapi enum (blocking)
The check-exchange-drift.js script reports:

FAIL openapi enum: missing 3 — gemini-titan, hyperliquid, mock
FAIL TypeScript SDK classes: missing 3 — same
FAIL TypeScript package entry: missing 3 — same

The PR's openapi.yaml replaced the generic $ref: '#/components/parameters/ExchangeParam' with inline per-endpoint enums, omitting these three exchanges from all endpoints. These exchanges ARE present in core/src/exchanges/ and were served by the old ExchangeParam. Note: on main, the Python SDK had similar drift for 4 exchanges (gemini-titan, hyperliquid, mock, polymarket_us) — the PR fixed 3 of those in Python while introducing new openapi/TS drift for the same 3. The net result is worse (5 failing checks vs 2 on main). Fix: add gemini-titan, hyperliquid, and mock to the inline exchange enums where appropriate, or restore the ExchangeParam reference.

4. close method removed from API reference (moderate)
core/api-doc-config.generated.json now registers testDummyMethod where close used to be:

-"close": { "summary": "Close all WebSocket connections...", "returns": {"type": "void"} }
+"testDummyMethod": { "summary": "Close all WebSocket connections...", "returns": {"type": "string"} }

The close() method is still present in code but is now absent from the published API reference. Caused by a trailing whitespace change (line 1360 in BaseExchange.ts had a bare tab removed) that confused the JSDoc extractor. Fix: restore the trailing whitespace, or ensure the extractor correctly delineates the two docblocks.

5. docs/api-reference/openapi.json version regression (moderate)

-"version": "2.42.7"
+"version": "2.17.1"

The hosted API docs now advertise version 2.17.1 instead of 2.42.7. This file is auto-generated by generate-openapi.js from package.json, so the generator produces this value correctly for this repo (package.json is at 2.17.1). The 2.42.7 in the file before was from the hosted pmxt infrastructure's own versioning — not from this repo's package.json. This is a pre-existing discrepancy in how the hosted docs version is tracked, not introduced by this PR. However, committing it here will overwrite the hosted version the next time the hosted sync runs.

6. Code samples for fetchOrderBooks are broken (minor)

# Python sample shows:
result = exchange.fetch_order_books()          # missing required outcomeIds

# TypeScript sample shows:
await exchange.fetchOrderBooks("12345")        # string, not string[]

Auto-generated templates don't know the required arguments. Fix: add a codeExample override in the generator config for this method.

7. paramsSerializer global change (informational, no action needed)
The paramsSerializer in BaseExchange constructor now forces key=v1&key=v2 encoding for all array params on every exchange, replacing axios's non-standard key[0]=v1&key[1]=v2 default. Verified: Polymarket fetchMarkets and fetchOrderBook still work correctly after this change. The change is directionally correct (repeated params is what most APIs expect) and no regression was observed.


PMXT Pipeline Check

  • Field propagation (3-layer): OK — fetchOrderBooks result is Record<string, OrderBook>; OrderBook is already defined in openapi.yaml and SDK converters exist (convertOrderBook)
  • OpenAPI sync: OK (locally regenerated, no diff on openapi.yaml or method-verbs.json)
  • Financial precision: OK — no float arithmetic introduced; price/size remain parseFloat as in existing normalizeOrderBook
  • Type safety: ISSUE — callApi widened to Record<string, any> | any[]; the array branch has no path-parameter support (silent failure if used incorrectly), but no existing callers expose this
  • Auth safety: OK — no credentials logged; Kalshi auth headers applied correctly via sign()

Semver Impact

minor — new fetchOrderBooks API surface, no breaking changes to existing methods.


Risk

Three CI checks are failing and must be fixed before merge:

  1. Run node sdks/python/scripts/generate-client-methods.js, then manually restore the typed implementation for fetch_order_books
  2. Run npm run docs:llms and commit
  3. Add gemini-titan, hyperliquid, mock back to the inline exchange enums in openapi.yaml

The core implementation (Polymarket batch call, Kalshi spec + auth path, paramsSerializer, SDK wrappers) is correct and verified end-to-end. The fixes needed are all in generated/doc artifacts, not in the business logic.


Generated by Claude Code

@psyberck
Copy link
Copy Markdown
Contributor Author

psyberck commented May 22, 2026

Hey @realfishsam, I included some changes of SDK generation scripts for both Python and TypeScript int this PR. Let me know how you think about it, thanks!
aca8251: Fix and extend the functionality of sdks/typescript/scripts/generate-client-methods.js for typescript SDK
67b9dbe: Fix and extend the functionality of sdks/python/scripts/generate-client-methods.js for Python SDK

@psyberck psyberck force-pushed the feat/fetch-multi-order-books branch from 73a1bf6 to 3954b9c Compare May 22, 2026 17:45
@realfishsam realfishsam merged commit 95bf538 into pmxt-dev:main May 22, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants