Skip to content

[SEA-NodeJS] (1/8) Backend abstraction — IBackend / ISessionBackend / IOperationBackend#378

Merged
msrathore-db merged 7 commits into
mainfrom
msrathore/sea-abstraction
May 29, 2026
Merged

[SEA-NodeJS] (1/8) Backend abstraction — IBackend / ISessionBackend / IOperationBackend#378
msrathore-db merged 7 commits into
mainfrom
msrathore/sea-abstraction

Conversation

@msrathore-db
Copy link
Copy Markdown
Contributor

@msrathore-db msrathore-db commented May 16, 2026

Stack

Linear stack of 8 PRs landing the M0 + M1 Phase 1 SEA NodeJS work. Merge in order from base ↑ to tip. The tip branch (msrathore/sea-auth-u2m, PR #383) is the single snapshot containing everything in flight — point your test or benchmark harness at it for an end-to-end check.

Pos PR Branch Scope
1/8 #378 sea-abstraction IBackend / ISessionBackend / IOperationBackend interfaces
2/8 #380 sea-napi-binding TS loader + build script for the kernel-provided .node artifact
3/8 #377 sea-errors-logging Kernel ErrorCode → JS error-class mapping (M0 minimum)
4/8 #379 sea-auth PAT auth via useSEA: true
5/8 #382 sea-execution executeStatement + openSession (sessionConfig, initialCatalog/Schema)
6/8 #381 sea-results CloudFetch + Inline Arrow result fetching
7/8 #384 sea-operation cancel / close / finished lifecycle + INTERVAL parity + napi-relocation acceptance (absorbed sea-integration content)
8/8 #383 sea-auth-u2m ← TIP M1 Phase 1 OAuth M2M + U2M (5 review rounds, ZERO HIGH at close)

Companion kernel stack (databricks/databricks-sql-kernel): 8 PRs — root #26 (async-public-api) → #27#25#29#28#30#24#23 (tip).

Policy: new PRs always stack on the current tip. No sibling/parallel topology. No force-pushes on existing PRs unless absolutely necessary; if a PR's content is wrong, add a fix-up commit on top of the stack tip rather than rewriting history.


This PR is position 1/8 (ROOT of the stack).

Summary

Foundational backend abstraction — introduces IBackend / ISessionBackend / IOperationBackend interfaces and refactors the existing thrift driver to plug in behind them.

Size note (1839 LOC, over the 800 cap) — foundational refactor

+1064/-775 ratio means ~775 lines are pre-existing code being refactored to plug into the new interfaces (renames, restructuring, no behavior change). Actual review burden is lower than the raw LOC suggests — most of the diff is mechanical file-level refactoring rather than new logic.

Cannot be cleanly split: interfaces + at least one implementation (thrift) must land together to compile.

Test plan

  • ✅ Existing thrift backend wraps cleanly behind IBackend interface
  • ✅ No behavior change on the thrift path
  • ✅ Stub SeaBackend compiles + exposes the same interface

Draft until you give the go for review.

Tracking

  • PECOBLR-2666 — NodeJS SEA integration (parent epic — backend abstraction scaffolding)

Co-authored-by: Isaac

…kend

Refactors DBSQLClient/Session/Operation to dispatch through three
backend interfaces. ThriftBackend (lib/thrift-backend/) contains the
relocated existing thrift logic. SeaBackend (lib/sea/) is a stub for
M0; the sea-napi-binding feature wires the real impl.

Public surface (lib/index.ts) unchanged.
No new dependencies. All existing tests pass.

Files:
- lib/contracts/IBackend.ts (new)
- lib/contracts/ISessionBackend.ts (new)
- lib/contracts/IOperationBackend.ts (new)
- lib/contracts/IDBSQLClient.ts (adds useSEA?: boolean to ConnectionOptions)
- lib/thrift-backend/ThriftBackend.ts (new)
- lib/thrift-backend/ThriftSessionBackend.ts (new)
- lib/thrift-backend/ThriftOperationBackend.ts (new)
- lib/sea/SeaBackend.ts (new, M0 stub)
- lib/DBSQLClient.ts (dispatch through IBackend; useSEA picks SeaBackend)
- lib/DBSQLSession.ts (facade over ISessionBackend; staging stays here)
- lib/DBSQLOperation.ts (facade over IOperationBackend; iterators/fetchAll stay here)
- tests/unit/DBSQLClient.test.ts (retarget internal state lookup through backend; pre-seed client.backend in tests that bypass connect())
- tests/unit/DBSQLOperation.test.ts (retarget internal state lookup through backend)
…nline type

Addresses code-bloat-watchdog findings from commit 0085928:
- Restores public-API JSDoc on DBSQLSession + DBSQLOperation methods
  (was deleted as scope creep; contracts unchanged so docs still apply)
- Adds makeStubbedClient() helper to tests/unit/DBSQLClient.test.ts;
  replaces 14× duplicated ThriftBackend pre-seed
- Imports WaitUntilReadyOptions instead of inline option types in
  IOperationBackend + DBSQLOperation.waitUntilReady
@github-actions
Copy link
Copy Markdown

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

Round-N fixes from the 9-reviewer pre-review. Public IOperation/DBSQLOperation
surface preserved byte-identical; backend interfaces (IBackend / ISessionBackend
/ IOperationBackend) made fully neutral so both Thrift and SEA can implement
the same contract.

F1 — neutral DTOs at IOperationBackend with Thrift-shape preservation on the
  public facade (adapter pattern):
- lib/contracts/OperationStatus.ts (new) — neutral OperationStatus + OperationState
  enum mirroring databricks-sql-python's CommandState and kernel pyo3's
  StatementStatus taxonomy.
- lib/contracts/ResultMetadata.ts (new) — neutral ResultMetadata + ResultFormat
  enum mirroring the three TSparkRowSetType cases.
- IOperationBackend.status()/getResultMetadata() return the neutral DTOs.
- ThriftOperationBackend.status() adapts at the boundary via adaptOperationStatus
  / adaptResultMetadata; module-level helpers thriftStateToOperationState and
  thriftRowSetTypeToResultFormat do the enum maps.
- ThriftOperationBackend exposes thriftStatusResponse() and
  thriftResultMetadataResponse() as public Thrift-only accessors used by the
  facade's zero-loss fast path (kept for internal state-machine + result-handler
  dispatch as well).
- lib/utils/thriftWireSynthesis.ts (new) — synthesizeThriftStatus and
  synthesizeThriftResultSetMetadata: convert neutral DTOs back to Thrift wire
  shape for the non-Thrift backend path. Lossy on Thrift-only fields
  (taskStatus, numModifiedRows, cacheLookupResult, etc.).
- DBSQLOperation.status() and getMetadata() preserved Thrift return shape:
  Thrift backend path returns the real wire response (zero loss); non-Thrift
  backend path synthesizes via the new helpers.
- DBSQLOperation.getResultMetadata() — new additive neutral accessor on
  IOperation; DBSQLSession.handleStagingOperation uses it instead of the
  deprecated Thrift-shaped getMetadata().

F2 — IBackend.connect() is now zero-arg. Backend reads everything it needs
  from IClientContext / constructor; matches Python connector's pattern of
  passing session_configuration via constructor not method-arg.

F3 — Restore the 'Server protocol version' debug log dropped by the original
  PR-378 refactor. Re-added to ThriftSessionBackend.constructor with the
  LogLevel.debug + IClientContext.getLogger() pattern; matches the pre-refactor
  log site at main:lib/DBSQLSession.ts:175.

F4 + F11 + F14 — SeaBackend stub safety:
- close() is a no-op so DBSQLClient.close()'s state-clearing block can finish
  even after a useSEA: true connect() failure.
- connect() and openSession() throw HiveDriverError instead of generic Error,
  matching the rest of the codebase.
- connect(options: ConnectionOptions) and openSession(request: OpenSessionRequest)
  declare their parameters (with @typescript-eslint/no-unused-vars disable)
  so IDE autocomplete prompts the M1 SEA implementer.

F6 + F7 + F9 + F10 — JSDoc on backend interfaces:
- IBackend: connect/openSession/close docstrings; close() doc explicitly
  states transport-layer cleanup is owned by DBSQLClient.
- ISessionBackend: copy IDBSQLSession's per-method one-liner JSDoc.
- IOperationBackend: doc hasResultSet (readonly external; mutates internally),
  waitUntilReady (MUST throw OperationStateError on terminal non-success).

F8 — tests/unit/sea/SeaBackend.test.ts (new) locks in the stub contract:
  connect() rejects HiveDriverError, openSession() rejects HiveDriverError,
  close() resolves no-op. ~30 LOC.

F12 — Drop legacy { handle, ... } ctor branch from DBSQLOperation and
  DBSQLSession:
- Facades accept only { backend, context }.
- DBSQLSession no longer imports ThriftSessionBackend at all.
- DBSQLOperation imports ThriftOperationBackend solely for the F1 typed
  downcast (zero-loss Thrift fast path); this is a deliberate, scoped
  coupling tied to the back-compat decision.
- tests/unit/.stubs/createSessionForTest.ts and createOperationForTest.ts
  (new) wrap the legacy shape; all 48 + 54 test sites mechanically migrated.

F15 — ThriftOperationBackend.waitUntilReady uses imported WaitUntilReadyOptions
  type instead of an anonymous inline shape.

F16 — useSEA flag moved out of public ConnectionOptions:
- Removed useSEA?: boolean from the exported lib/contracts/IDBSQLClient.ts
  ConnectionOptions; no longer ships in the public .d.ts.
- lib/contracts/InternalConnectionOptions.ts (new) declares the flag as a
  non-exported internal extension; DBSQLClient.connect() reads via a typed
  cast. Mirrors Python's kwargs.get('use_sea', False) pattern at
  databricks-sql-python/src/databricks/sql/session.py:111.

F17 — Missing return; after case 'timeout' in forwardConnectionEvent so a
  future fifth case doesn't silently fall through. The trailing return; in
  the last case triggers no-useless-return — quieted with a localized
  eslint-disable-next-line + intent comment.

F5 — deferred per owner instruction (test-only as any cast tightening).

Verification:
- yarn lint clean (3 pre-existing warnings in tests/e2e/protocol_versions.test.ts).
- yarn build clean.
- tsc --noEmit -p tsconfig.json clean (apart from pre-existing
  examples/tokenFederation/* import errors that exist on main).
- Runtime smoke test of SeaBackend stub + Thrift-wire synthesis round-trip
  passes 5/5 assertions.

Co-authored-by: Isaac
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
msrathore-db added a commit that referenced this pull request May 24, 2026
Addresses PR #380 review findings C1, C2 (partial), C3, H1, H2, H3,
H4, H5, H6, H7, H8 (runtime guard), H9, M1, M2, M3, M4 (linguist),
M5, M6, M7, M8, L1, L2, L4.

- C1 lint: drop `.js` ext from native/sea require so eslint
  import/extensions passes; gitignore the auto-generated index.js
  and *.node artifacts; prettier-ignore the napi-rs auto-generated
  index.d.ts / index.js.
- C2 publish: whitelist native/sea/index.{js,d.ts} in .npmignore;
  declare @databricks/sea-native-linux-x64-gnu in optionalDependencies.
  The kernel-side package-name rename (drop the linux-x64-gnu
  prefix) is tracked separately as a cross-PR ask.
- C3 test wiring: move tests/native/version.test.ts ->
  tests/unit/sea/, tests/native/e2e-smoke.test.ts -> tests/e2e/sea/;
  both now picked up by the existing mocharc globs and run via the
  existing `npm test` / `npm run e2e` jobs.
- H1 noise drop: version.test.ts now asserts one meaningful semver
  check; three tautological assertions removed.
- H2 + H3 + H7 lazy load: rewrite SeaNativeLoader.ts on the
  lib/utils/lz4.ts pattern. Lazy require behind getSeaNative();
  capability-detection helper tryGetSeaNative(); structured error
  messages that classify MODULE_NOT_FOUND vs ERR_DLOPEN_FAILED and
  include platform/arch/Node-version + install hint.
- H4 supply chain: pin @napi-rs/cli to 2.18.4 in devDependencies;
  build:native switches to `npx --no-install` so the pinned local
  install is used (no per-build network fetch).
- H5 path: switch build:native default kernel path to the canonical
  `../../databricks-sql-kernel/napi-binding` (the worktree-specific
  `-sea-WT` suffix is gone).
- H6 CI safety: e2e suite hard-fails when CI=true and any required
  env var is missing; dev machines still skip.
- H8 runtime guard: loader throws a structured error on Node <18
  instead of attempting a dlopen that would fail mysteriously.
  Driver's engines.node stays >=14 — Thrift consumers on older
  Node continue to work.
- H9 + M1 + M2 types: tsconfig adds a `@sea-native` path alias to
  native/sea/index.d.ts; loader imports the real Connection /
  Statement / ConnectionOptions / ExecuteOptions / ArrowBatch /
  ArrowSchema types and re-exports them. Tests use the re-exported
  types — the inline-shape duplication across three files collapses.
- M3 README: rewrite native/sea/README.md to match kernel reality.
  The napi crate is a standalone Cargo workspace (not a pyo3
  sibling); explain the tls-rustls choice that the standalone
  workspace exists to enable.
- M4 drift detection: mark native/sea/index.d.ts as
  linguist-generated in .gitattributes so GitHub collapses it in
  diffs and excludes from blame/language stats.
- M5 artifacts: gitignore native/sea/index.js, index.node,
  index.*.node.
- M6 + M7 e2e coverage: decode the IPC payload via apache-arrow's
  tableFromIPC and assert numRows + cell value (not just shape);
  add drain-past-null idempotence and schema-before-fetch
  coverage.
- L1 build scripts: collapse build:native + build:native:debug
  into one script taking BUILD_PROFILE (defaults to --release).
- L2 / L4 / M8: drop the version() alias and the "Round 2+ will…"
  comment debt; the loader now actually delivers the value the
  prior JSDoc was only promising.

Verified on this branch: npm run lint clean (0 errors); npm run
prettier clean for PR-owned files (3 unrelated pre-existing
warnings on PR #378 territory); tsc --project tsconfig.build.json
clean; mocha on tests/unit/sea passes (4/4); mocha on
tests/e2e/sea passes against a live pecotesting warehouse (2/2,
IPC decode confirms SELECT 1 returns 1).
msrathore-db added a commit that referenced this pull request May 24, 2026
Addresses PR #380 review findings C1, C2 (partial), C3, H1, H2, H3,
H4, H5, H6, H7, H8 (runtime guard), H9, M1, M2, M3, M4 (linguist),
M5, M6, M7, M8, L1, L2, L4.

- C1 lint: drop `.js` ext from native/sea require so eslint
  import/extensions passes; gitignore the auto-generated index.js
  and *.node artifacts; prettier-ignore the napi-rs auto-generated
  index.d.ts / index.js.
- C2 publish: whitelist native/sea/index.{js,d.ts} in .npmignore;
  declare @databricks/sea-native-linux-x64-gnu in optionalDependencies.
  The kernel-side package-name rename (drop the linux-x64-gnu
  prefix) is tracked separately as a cross-PR ask.
- C3 test wiring: move tests/native/version.test.ts ->
  tests/unit/sea/, tests/native/e2e-smoke.test.ts -> tests/e2e/sea/;
  both now picked up by the existing mocharc globs and run via the
  existing `npm test` / `npm run e2e` jobs.
- H1 noise drop: version.test.ts now asserts one meaningful semver
  check; three tautological assertions removed.
- H2 + H3 + H7 lazy load: rewrite SeaNativeLoader.ts on the
  lib/utils/lz4.ts pattern. Lazy require behind getSeaNative();
  capability-detection helper tryGetSeaNative(); structured error
  messages that classify MODULE_NOT_FOUND vs ERR_DLOPEN_FAILED and
  include platform/arch/Node-version + install hint.
- H4 supply chain: pin @napi-rs/cli to 2.18.4 in devDependencies;
  build:native switches to `npx --no-install` so the pinned local
  install is used (no per-build network fetch).
- H5 path: switch build:native default kernel path to the canonical
  `../../databricks-sql-kernel/napi-binding` (the worktree-specific
  `-sea-WT` suffix is gone).
- H6 CI safety: e2e suite hard-fails when CI=true and any required
  env var is missing; dev machines still skip.
- H8 runtime guard: loader throws a structured error on Node <18
  instead of attempting a dlopen that would fail mysteriously.
  Driver's engines.node stays >=14 — Thrift consumers on older
  Node continue to work.
- H9 + M1 + M2 types: tsconfig adds a `@sea-native` path alias to
  native/sea/index.d.ts; loader imports the real Connection /
  Statement / ConnectionOptions / ExecuteOptions / ArrowBatch /
  ArrowSchema types and re-exports them. Tests use the re-exported
  types — the inline-shape duplication across three files collapses.
- M3 README: rewrite native/sea/README.md to match kernel reality.
  The napi crate is a standalone Cargo workspace (not a pyo3
  sibling); explain the tls-rustls choice that the standalone
  workspace exists to enable.
- M4 drift detection: mark native/sea/index.d.ts as
  linguist-generated in .gitattributes so GitHub collapses it in
  diffs and excludes from blame/language stats.
- M5 artifacts: gitignore native/sea/index.js, index.node,
  index.*.node.
- M6 + M7 e2e coverage: decode the IPC payload via apache-arrow's
  tableFromIPC and assert numRows + cell value (not just shape);
  add drain-past-null idempotence and schema-before-fetch
  coverage.
- L1 build scripts: collapse build:native + build:native:debug
  into one script taking BUILD_PROFILE (defaults to --release).
- L2 / L4 / M8: drop the version() alias and the "Round 2+ will…"
  comment debt; the loader now actually delivers the value the
  prior JSDoc was only promising.

Verified on this branch: npm run lint clean (0 errors); npm run
prettier clean for PR-owned files (3 unrelated pre-existing
warnings on PR #378 territory); tsc --project tsconfig.build.json
clean; mocha on tests/unit/sea passes (4/4); mocha on
tests/e2e/sea passes against a live pecotesting warehouse (2/2,
IPC decode confirms SELECT 1 returns 1).

Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Comment thread lib/DBSQLOperation.ts Outdated
disableBuffering: options?.disableBuffering,
});
const limit = options?.maxRows ?? defaultMaxRows;
const result = await this.backend.fetchChunk({ limit, disableBuffering: options?.disableBuffering });
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High — behavioral regression on the Thrift hot path (verified against PR head SHA + main).

Pre-PR fetchChunkInternal sequence:

  1. failIfClosed()
  2. waitUntilReady()
  3. getResultHandler() ← may fetch result-metadata RPC
  4. failIfClosed() ← critical mid-flight check
  5. setTimeout(0) macrotask yield
  6. resultHandler.fetchNext() ← data RPC (CloudFetch URL download / Arrow batch)
  7. failIfClosed()

Post-PR sequence: steps 3–6 now happen inside ThriftOperationBackend.fetchChunk as one atomic block with no failIfClosed check between them. The only intervening macrotask yield (setTimeout(0) at lib/thrift-backend/ThriftOperationBackend.ts:143-145) sits inside that block.

A concurrent cancel()/close() arriving during the yield window will no longer short-circuit the subsequent fetchNext() data RPC. CloudFetch presigned-URL downloads can take seconds, making this observable rather than theoretical.

Suggested fix: add an isClosed: () => boolean callback to IOperationBackend.fetchChunk's options, and have ThriftOperationBackend.fetchChunk check it after setTimeout(0) returns. Alternatively split the metadata-prep phase into a separate backend method so the facade can re-check between them.

Found by: security, performance (independently).


Posted by code-review-squad · /full-review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a9347e8.

Added optional isClosed?: () => boolean to IOperationBackend.fetchChunk's options bag. ThriftOperationBackend.fetchChunk now probes it right after the setTimeout(0) macrotask yield and returns [] when set; the facade's post-fetch failIfClosed() then raises the user-visible OperationStateError. Restores the guard the refactor split across the facade/backend boundary.

The contract addition is strictly additive (new optional field) so downstream backend impls in #380 #377 #379 #382 #381 #384 #383 stay source-compatible — ignoring the probe preserves their current behavior, opting in restores early-cancel semantics.

Verified empirically against a CloudFetch-eligible query on pecotesting (adb-6436897454825492.12.azuredatabricks.net); also verified at the unit level via the existing cancel/close reject-all-methods tests still passing through the new code path.


This comment was generated with GitHub MCP.

Comment thread lib/contracts/IOperationBackend.ts Outdated
* the operation into its closed/cancelled flags; future implementations must
* use the same error type for the facade to stay in sync.
*/
waitUntilReady(options?: WaitUntilReadyOptions): Promise<void>;
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium — locks the cross-backend interface to a Thrift wire shape.

IOperationBackend.waitUntilReady(options?: WaitUntilReadyOptions) is the cross-backend interface, but WaitUntilReadyOptions (imported from ./IOperation) types callback as (progress: TGetOperationStatusResp) => unknown — a Thrift-generated wire type.

Every non-Thrift backend impl must either:

  • depend on the thrift IDL just to construct TGetOperationStatusResp, or
  • import synthesizeThriftStatus from lib/utils/thriftWireSynthesis.ts per-poll, or
  • skip the user's callback, silently breaking a public contract.

For a Rust kernel polling every ~100 ms during a 10-min query, that's 6,000 synthesized fake Thrift structs purely to satisfy a JS callback signature.

Suggested fix: introduce a separate IOperationBackendWaitOptions { callback?: (status: OperationStatus) => unknown } on the backend interface. DBSQLOperation owns the user's Thrift-typed callback and adapts to/from the neutral one at the facade boundary.

Easier to fix now (one caller, in this PR's own facade) than after the SEA/napi impls land.

Found by: architecture, devils-advocate, security, language.


Posted by code-review-squad · /full-review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a9347e8.

Introduced IOperationBackendWaitOptions { progress?: boolean; callback?: (status: OperationStatus) => unknown } on the backend interface. ThriftOperationBackend.waitUntilReady now consumes the neutral options and calls options.callback(this.adaptOperationStatus(response)) — passes the neutral DTO, not the Thrift wire struct. The facade DBSQLOperation.waitUntilReadyThroughBackend adapts the user's public Thrift-shaped OperationStatusCallback at the boundary by wrapping it with synthesizeThriftStatus. So non-Thrift backends never touch Thrift IDL, the user's existing Thrift-typed callback keeps working unchanged.

Cascade note for #380 #377 #379 #382 #381 #384 #383: each downstream waitUntilReady impl switches options type from WaitUntilReadyOptions to IOperationBackendWaitOptions and passes a neutral OperationStatus to the callback — mechanical rename at rebase time.


This comment was generated with GitHub MCP.

Comment thread lib/utils/thriftWireSynthesis.ts Outdated
*/
export function synthesizeThriftStatus(status: OperationStatus): TGetOperationStatusResp {
return {
status: synthesizeOkStatus(),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium — latent correctness bug; bites once any non-Thrift backend is wired in.

synthesizeOkStatus() (line 12) returns { statusCode: TStatusCode.SUCCESS_STATUS } as TStatus unconditionally, and is wired into the status field of the synthesized TGetOperationStatusResp regardless of OperationState.

Existing user code that calls Status.assert(resp.status) after await op.status() against a non-Thrift backend will see SUCCESS_STATUS even when the operation finished in ERROR_STATE or CANCELED_STATE. The lossy-by-design JSDoc comment covers fields like taskStatus/numModifiedRows but does NOT cover statusstatus is meant to reflect the RPC, but here it's being used as a stand-in for the operation outcome that user code already checks.

Today this is dead code (SEA is a throw-stub); the moment any non-Thrift backend's status() is reachable, error states will silently look successful in the legacy IOperation.status() return value.

Suggested fix: map operation-state to a real TStatus.statusCode:

  • OperationState.Failed → ERROR_STATUS (and carry errorMessage/sqlState)
  • OperationState.Cancelled → ERROR_STATUS
  • other states → SUCCESS_STATUS

Trivial, prevents a latent correctness gap.

Found by: agent-compat, devils-advocate.


Posted by code-review-squad · /full-review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a9347e8.

Added synthesizeStatusFromOperation(status) that returns ERROR_STATUS for Failed / Cancelled / Closed (carrying errorMessage + sqlState) and SUCCESS_STATUS otherwise. Wired into synthesizeThriftStatus. synthesizeOkStatus() is preserved for synthesizeThriftResultSetMetadata's status field — that field reflects the metadata-RPC outcome, not the operation outcome, so SUCCESS_STATUS is the right answer there.

Unit-tested via the new tests/unit/thrift-backend/wireSynthesis.test.ts (covers Failed → ERROR_STATUS with errorMessage/sqlState carrying, Cancelled → ERROR_STATUS, Closed → ERROR_STATUS, non-terminal → SUCCESS_STATUS).


This comment was generated with GitHub MCP.

@@ -0,0 +1,87 @@
import {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium — coverage gap that masks the latent correctness bug in M2.

No test exercises client.connect({ useSEA: true }) end-to-end. The synthesize helpers in this file have no companion tests/unit/utils/thriftWireSynthesis.test.ts. Both DBSQLOperation.status() and DBSQLOperation.getMetadata() have the instanceof ThriftOperationBackend synthesize fallback as dead code from the test suite's perspective.

Any enum-mapping drift (e.g. the M2 issue above) would not be caught by CI.

Suggested fix: add a unit test file tests/unit/utils/thriftWireSynthesis.test.ts covering:

  • operationStateToThrift: all 7 enum values
  • resultFormatToThrift: all 3 enum values
  • synthesizeThriftStatus: confirm errorMessage/sqlState are preserved; confirm hasResultSet round-trips; confirm state: Failed ⇒ non-SUCCESS TStatus (after M2 fix)
  • synthesizeThriftResultSetMetadata: confirm schema, arrowSchema, isStagingOperation, lz4Compressed are preserved

And one new test in DBSQLClient.test.ts covering the useSEA: true failure path (using as any cast to bypass the InternalConnectionOptions non-exported type).

Found by: test, ops, agent-compat (independently flagged different aspects).


Posted by code-review-squad · /full-review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a9347e8.

New test file at tests/unit/thrift-backend/wireSynthesis.test.ts (path mirrors L9's source-file move). Covers:

  • synthesizeThriftStatus: all 7 OperationState values map to the right TOperationState; Failed/Cancelled/Closed yield ERROR_STATUS (carrying errorMessage + sqlState); non-terminal states yield SUCCESS_STATUS; hasResultSet round-trips (true/false/undefined); top-level errorMessage + sqlState forwarded.
  • synthesizeThriftResultSetMetadata: all 3 ResultFormat values map to the right TSparkRowSetType; an unknown ResultFormat throws HiveDriverError (covers L3); schema / arrowSchema / lz4Compressed / isStagingOperation round-trip; synthetic status is SUCCESS_STATUS.

Plus a new test in tests/unit/DBSQLClient.test.ts covering the useSEA: true failure path: asserts SeaBackend.connect() rejects with "not implemented", this.backend stays undefined after the failed connect (verifies L2's partial-init guard), and the next openSession() surfaces "not connected" rather than the SEA stub's "not implemented".

Net new tests: +12 (151 → 163 passing on touched files). Same 2 pre-existing failures (getSchema-directResults, LZ4-cloud-fetch flag) unchanged.


This comment was generated with GitHub MCP.

Comment thread lib/DBSQLClient.ts Outdated
private forwardConnectionEvent(event: 'error' | 'reconnecting' | 'close' | 'timeout', payload?: unknown): void {
switch (event) {
case 'error': {
const error = payload as Error;
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low — latent crash hazard. Pre-refactor the Thrift EventEmitter typing guaranteed error: Error. Post-refactor payload is unknown and the cast is unchecked. If a future backend ever emits a non-Error value through this path (e.g. a string, which EventEmitter.emit('error', ...) permits), error.stack || error.name + ': ' + error.message will throw TypeError. The try/catch only protects this.emit, not the logger.log line.

  • Suggested fix:
    const error = payload instanceof Error ? payload : new Error(String(payload));

Posted by code-review-squad · /full-review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a9347e8.

Replaced the unchecked cast at lib/DBSQLClient.ts:260 with:

const error = payload instanceof Error ? payload : new Error(String(payload));

Added a JSDoc comment noting that payload is unknown because the cross-backend IBackend.onConnectionEvent doesn't constrain the error shape. logger.log and this.emit('error', error) are both safe now even if a backend emits a bare string through the event.


This comment was generated with GitHub MCP.

Comment thread lib/DBSQLClient.ts Outdated
// doesn't ship in the public `.d.ts`. Mirrors Python's `kwargs.get("use_sea")`
// pattern (see databricks-sql-python/src/databricks/sql/session.py).
const internalOptions = options as ConnectionOptions & InternalConnectionOptions;
this.backend = internalOptions.useSEA
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low — confusing diagnostic on a partially-initialized path.

With useSEA: true, SeaBackend.connect() always throws (M0 stub), but this.backend is already assigned. A subsequent openSession() passes the !this.backend guard at line 298 and reaches SeaBackend.openSession(), which throws "SEA backend not implemented yet" instead of "DBSQLClient: not connected".

  • Suggested fix: either (a) assign this.backend after a successful connect(), or (b) try/catch around await this.backend.connect(options) and null this.backend on failure.

The IBackend.close() JSDoc explicitly mentions "MUST be safe to call on a partially-initialized backend (i.e. after a failed connect())" — but that contract only covers close(), not the openSession-after-failed-connect path.


Posted by code-review-squad · /full-review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a9347e8.

Restructured to construct the backend locally, attempt connect() in a try/catch, run a best-effort backend.close() on failure (relying on IBackend.close()'s documented safe-on-partial-init contract), and only assign this.backend after a clean connect. A failed connect() now leaves this.backend === undefined, so the next openSession() trips the existing !this.backend guard and surfaces HiveDriverError: DBSQLClient: not connected — the diagnostic this fix was about delivering.

Unit-tested via the new useSEA: true failure-path test in tests/unit/DBSQLClient.test.ts (asserts the post-connect-failure backend state and the openSession diagnostic).


This comment was generated with GitHub MCP.

return TSparkRowSetType.ARROW_BASED_SET;
case ResultFormat.UrlBased:
return TSparkRowSetType.URL_BASED_SET;
default:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low — asymmetric with operationStateToThrift above.

operationStateToThrift (lines 16-34) uses an explicit OperationState.Unknown member, making the switch exhaustively-checkable by TS. resultFormatToThrift (lines 36-47) just has a bare default: return COLUMN_BASED_SET. Today the default is dead because ResultFormat is a closed 3-value enum, but if it's extended for SEA (e.g. a new format), the fallback would silently route results through JsonResultHandler and produce garbled data with no error.

  • Suggested fix: drop the default, or throw new HiveDriverError(...). Match operationStateToThrift's pattern.

Posted by code-review-squad · /full-review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a9347e8.

Replaced the silent default: return COLUMN_BASED_SET with throw new HiveDriverError(\Unknown ResultFormat: ${format}`). A future ResultFormatenum extension now trips loudly instead of silently routing rows throughJsonResultHandler`.

Covered by a unit test in the new tests/unit/thrift-backend/wireSynthesis.test.ts that passes a bogus format and asserts the throw.


This comment was generated with GitHub MCP.

Comment thread lib/DBSQLOperation.ts Outdated
* fields (`cacheLookupResult`, `uncompressedBytes`, `compressedBytes`,
* `status`) left undefined / defaulted.
*
* Prefer {@link DBSQLOperation.getResultMetadata} in new code.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low — agent/IDE discoverability.

The JSDoc says "Prefer DBSQLOperation.getResultMetadata in new code" but does NOT use the @deprecated tag. TypeScript's @deprecated is the canonical signal that IDEs (strikethrough), tsc, ESLint plugins, and agentic codegen pick up. Free-text "Prefer X" won't.

  • Suggested fix:
    /**
     * @deprecated Use {@link DBSQLOperation.getResultMetadata}; this method synthesizes Thrift-only fields as `undefined` on non-Thrift backends.
     */

Posted by code-review-squad · /full-review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a9347e8.

Replaced the free-text "Prefer …" line with the canonical @deprecated JSDoc tag pointing at getResultMetadata:

/**
 * @deprecated Use {@link DBSQLOperation.getResultMetadata}; this method
 * synthesizes Thrift-only fields as `undefined` on non-Thrift backends and
 * couples callers to the Thrift wire shape.
 */
public async getMetadata(): Promise<TGetResultSetMetadataResp> { ... }

IDEs / tsc / ESLint plugins / agentic codegen will now pick up the deprecation in user code.


This comment was generated with GitHub MCP.

Comment thread lib/DBSQLSession.ts Outdated

return result;
}
// Re-export for back-compat with existing imports.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low — back-compat re-export should signal Thrift-only.

numberToInt64 traffics in node-int64 (a Thrift wire type) and has no place in a backend-neutral facade — it's intentionally re-exported only to preserve back-compat for existing external consumers. Add @deprecated to the re-export and a // Thrift-only JSDoc on the symbol itself, so SDK consumers see the deprecation in their IDE / .d.ts.


Posted by code-review-squad · /full-review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a9347e8.

Converted the bare export { numberToInt64 } from '…' re-export into a named import + JSDoc'd export const so the @deprecated tag attaches to the symbol consumers see in their IDE / .d.ts:

import { numberToInt64 as numberToInt64Impl } from './thrift-backend/ThriftSessionBackend';

/**
 * Convert a JS number to a Thrift-wire `node-int64`.
 *
 * @deprecated Thrift-only utility re-exported for back-compat with existing
 * external consumers. Backends other than Thrift do not use `node-int64`;
 * new code should not import this from `DBSQLSession`. It will be removed
 * when the public API stops exposing Thrift wire types.
 */
export const numberToInt64 = numberToInt64Impl;

This comment was generated with GitHub MCP.

Comment thread lib/DBSQLSession.ts Outdated
@@ -211,48 +85,13 @@ export default class DBSQLSession implements IDBSQLSession {
*/
public async executeStatement(statement: string, options: ExecuteStatementOptions = {}): Promise<IOperation> {
await this.failIfClosed();
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low — readability / maintenance.

Pre-refactor, handleResponse did one job (re-check open-flag after the backend call returned, because server-side close doesn't error out). The PR removes the helper and inlines a failIfClosed bracket around every backend call — 11 sites with the same shape, no name attached to the "before AND after" semantics.

  • Suggested fix:
    private async runBackend<T>(fn: () => Promise<T>): Promise<T> {
      await this.failIfClosed();
      const result = await fn();
      await this.failIfClosed();
      return result;
    }
    Then call sites become return this.wrapOperation(await this.runBackend(() => this.backend.getCatalogs(request)));. Cuts boilerplate and re-attaches a name to the contract.

Posted by code-review-squad · /full-review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a9347e8.

Added private async runBackend<T>(fn: () => Promise<T>): Promise<T> to DBSQLSession that does failIfClosed → fn() → failIfClosed. Refactored all 11 call sites (getInfo, executeStatement, getTypeInfo, getCatalogs, getSchemas, getTables, getTableTypes, getColumns, getFunctions, getPrimaryKeys, getCrossReference).

Most reduce to one line:

public async getCatalogs(request: CatalogsRequest = {}): Promise<IOperation> {
  return this.wrapOperation(await this.runBackend(() => this.backend.getCatalogs(request)));
}

executeStatement keeps the staging-detection branch around the wrapper but uses runBackend for the bracketed backend call.

Cuts ~22 lines of duplicated boilerplate and re-attaches a name to the open-flag-before-and-after contract.


This comment was generated with GitHub MCP.

@vikrantpuppala
Copy link
Copy Markdown
Collaborator

Code Review Squad — Review

Score: 65/100 — HIGH RISK (score driven by finding count — see notes)

The structural refactor is well-scoped and well-documented; the Thrift path is preserved 1:1 for nearly every call site and the 3-tier seam (IBackend / ISessionBackend / IOperationBackend) is drawn at the right level for the upcoming Rust/N-API kernel backend. Reviewers reached strong agreement on this point.

The score is depressed by one verified behavioral regression on the Thrift hot path (H1 — fetchChunk lost a mid-flight failIfClosed between metadata RPC and data RPC), a cluster of four forward-looking architecture concerns that will hit the upcoming Rust kernel binding, and a long tail of small concerns. Several of the architecture concerns are already acknowledged in JSDoc on the PR. Treat the HIGH RISK label as "address H1 plus a couple of the Mediums before merge", not as a reject signal.

Reviewers: 9/9 delivered (security, devils-advocate, language, test, architecture, maintainability, agent-compat, ops, performance).


Posted inline


Medium Findings

M3 — Telemetry schema has no backend field — can't distinguish Thrift / SEA / Rust in metrics

File: lib/telemetry/types.ts (not in this PR's diff)

Medium — cheap to add now, expensive to backfill after rollout.

TelemetryEvent (lines 97-159), TelemetryMetric (lines 165-205), and DriverConfiguration (lines 208-280) have no backend?: 'thrift' | 'sea' | 'kernel' field. The driver-config carries driverName, driverVersion, nodeVersion, cloudFetchEnabled, arrowEnabled, directResultsEnabled — but no backend identifier.

Once SEA starts emitting telemetry, every STATEMENT_COMPLETE from a SEA path will be indistinguishable from a Thrift path. Slicing latency / error rate / cloudfetch effectiveness by backend will require a metrics-schema change + redeploy. Triaging "is the new backend regressing latency vs Thrift?" during the rollout — impossible from telemetry alone.

  • Suggested fix: add backend?: 'thrift' | 'sea' | 'kernel' (or similar) to TelemetryEvent and to DriverConfiguration in a follow-up PR before any non-Thrift implementation is wired into emission. Cheap additive change.

Found by: ops.

Low Findings

Low findings — click to expand

L7 — Three why-comments deleted in DBSQLSession (staging detection, AWS/Azure 404, Content-Length)

The PR collapsed three multi-line why-comments down to single lines or removed them:

  • Staging-detection invariant in executeStatement (~line 91 post-PR): the original 6-line comment explained that calling getResultMetadata() here for a non-staging operation is intentional and harmless because the operation remains usable.
  • AWS vs Azure 404 behavior on staging-remove (response.status !== 404 check).
  • Required Content-Length header on staging-upload.

These are exactly the why-comments to preserve — they document non-obvious decisions. Restore.

L8 — hasResultSet: readonly boolean on IOperationBackend is misleading (mutable getter under the hood)

4 reviewers flagged this independently (language, agent-compat, devils-advocate, architecture).

lib/contracts/IOperationBackend.ts:18-23 declares readonly hasResultSet: boolean with a long JSDoc explaining that readonly means "external callers cannot reassign," not "fixed at construction time," and that implementations MUST refresh from terminal status responses.

A fresh implementer (especially the future Rust/N-API backend author) will see readonly and assume constructor-time const. Either:

  • make it a method hasResultSet(): boolean (clearly state-dependent), or
  • leave a more visible "this is a getter" comment that the napi-rs binding author will catch.

Cost of method form: ~5 call sites in lib/DBSQLOperation.ts.

L9 — thriftWireSynthesis.ts should live under lib/thrift-backend/ not lib/utils/

The file imports 6 symbols from thrift/TCLIService_types and produces Thrift-typed values. By the same logic that moved numberToInt64 / getDirectResultsOptions into lib/thrift-backend/, this file belongs there too. lib/utils/ signals "shared by everyone," which puts a Thrift-aware module on the dependency graph of any future neutral helper. When the Rust kernel lands, this file should not be its dependency.

  • Suggested fix: rename to lib/thrift-backend/wireSynthesis.ts.

L10 — instanceof ThriftOperationBackend downcast in 2 facade methods — defensible exemption, but should not become a pattern

DBSQLOperation.status (line 147) and DBSQLOperation.getMetadata (line 235) both instanceof ThriftOperationBackend to pick a zero-loss fast path. The doc comments on the backend's thriftStatusResponse / thriftResultMetadataResponse explicitly justify this as a back-compat exemption.

The risk is precedent: when the Rust backend lands, somebody will add instanceof RustOperationBackend for the same reason, and the facade grows three branches per public method. The cleaner long-term path is an optional method on IOperationBackend (thriftStatusResponse?(): Promise<TGetOperationStatusResp>) or a structural 'thriftStatusResponse' in this.backend check.

Suggested action: document the rule "no further instanceof <BackendClass> downcasts in the facade" in CLAUDE.md or in the contributing notes. The two existing ones are exempt by back-compat.

L11 — Backend identifier missing from log messages

With three backends coming (Thrift, SEA, Rust kernel), every facade-level debug log (e.g. Operation created with id: <guid>, Session created with id: <guid>, Fetched chunk of size: <n> from operation with id: <guid>) is backend-agnostic. When a customer reports "session abcd-1234 hung", on-call has no log-level signal whether to look at Thrift server logs, SEA control-plane logs, or kernel-process logs. The id space may also overlap or collide between backends.

  • Suggested fix: add readonly kind: 'thrift' | 'sea' | 'kernel' (or name: string) to IBackend/ISessionBackend/IOperationBackend, and prefix facade-level logs with [${this.backend.kind}]. ~7 log call sites + one field per interface.

@msrathore-db msrathore-db marked this pull request as ready for review May 27, 2026 09:34
…1-L10)

Addresses 15 review findings from the code-review-squad pass on PR #378.
L11 (backend kind field on the three interfaces) is deliberately deferred
to avoid a cross-stack cascade ripple while the downstream PRs are still
in flight.

H1 — fetchChunk lost mid-flight failIfClosed regression.
  Add optional `isClosed?: () => boolean` to IOperationBackend.fetchChunk's
  options bag. ThriftOperationBackend.fetchChunk probes it after the
  setTimeout(0) macrotask yield and returns [] when set; the facade's
  post-fetch failIfClosed then raises the user-visible OperationStateError.
  Restores the guard that the refactor split across the facade/backend
  boundary so a cancel/close arriving during the yield window no longer
  runs the data RPC to completion needlessly.

M1 — neutralize WaitUntilReadyOptions callback shape.
  Introduce IOperationBackendWaitOptions { callback?: (status:
  OperationStatus) => unknown } on the backend interface. Facade keeps
  the public Thrift-typed OperationStatusCallback and adapts at the
  boundary by wrapping the user's callback with synthesizeThriftStatus.
  ThriftOperationBackend.waitUntilReady consumes the neutral options and
  passes adaptOperationStatus(response) to the callback.

M2 — synthesizeOkStatus maps OperationState to TStatusCode.
  Add synthesizeStatusFromOperation that returns ERROR_STATUS for
  Failed/Cancelled/Closed (carrying errorMessage + sqlState) and
  SUCCESS_STATUS otherwise. Wire it into synthesizeThriftStatus so
  legacy Status.assert(resp.status) sees the right code on non-Thrift
  backends.

M3 — TelemetryEvent + DriverConfiguration carry a backend tag.
  Add optional backend?: 'thrift' | 'sea' | 'kernel' on both interfaces
  so dashboards can slice latency/error rate by backend without a
  metrics-schema migration once non-Thrift emission goes live.

M4 — test coverage for the synthesize helpers + useSEA failure path.
  New tests/unit/thrift-backend/wireSynthesis.test.ts covering all
  OperationState/ResultFormat mappings, ERROR_STATUS carries
  errorMessage/sqlState, hasResultSet round-trip, schema/arrowSchema/
  lz4Compressed/isStagingOperation preservation, and the L3 throw on
  unknown ResultFormat. New test in DBSQLClient.test.ts asserts that a
  useSEA:true connect failure leaves this.backend === undefined and the
  next openSession() surfaces "not connected" rather than the
  SeaBackend's "not implemented" error.

L1 — forwardConnectionEvent normalizes payload to Error.
  Replace `payload as Error` with `payload instanceof Error ? payload
  : new Error(String(payload))` so a backend that emits a non-Error
  through the cross-backend onConnectionEvent doesn't crash the
  logger.log call.

L2 — DBSQLClient.connect publishes this.backend only on success.
  Construct the backend locally, await connect() in a try/catch, run a
  best-effort backend.close() (per IBackend.close()'s
  safe-on-partial-init contract) and rethrow on failure. Only assign
  this.backend after a clean connect so a failed connect surfaces
  "DBSQLClient: not connected" on the next openSession.

L3 — resultFormatToThrift throws on unknown ResultFormat.
  Replace the silent default fallback to COLUMN_BASED_SET with a
  HiveDriverError. Prevents a future ResultFormat enum extension from
  silently routing results through JsonResultHandler and surfacing
  garbled rows.

L4 — DBSQLOperation.getMetadata carries @deprecated.
  Adds the canonical TypeScript JSDoc tag so IDEs (strikethrough), tsc,
  ESLint plugins, and agentic codegen pick up the soft deprecation in
  favour of getResultMetadata.

L5 — numberToInt64 re-export carries @deprecated.
  Re-export through a named const with a JSDoc block (rather than a
  bare `export { ... } from`) so the @deprecated tag attaches to the
  symbol consumers see in their IDE / .d.ts.

L6 — DBSQLSession.runBackend helper.
  Collapse 11 duplicated `failIfClosed → backend.X → failIfClosed`
  brackets into a single private runBackend<T>(fn) so the
  open-flag-before-and-after contract has a name and can't be forgotten
  in a new delegation method.

L7 — restore three why-comments deleted from DBSQLSession.
  Staging-detection invariant in executeStatement, AWS-vs-Azure 404
  difference on staging-remove, and the Content-Length-required note on
  staging-upload. Verbatim from main; these document non-obvious
  intentional behaviour the refactor inadvertently dropped.

L8 — hasResultSet becomes a method on IOperationBackend.
  The value is state-dependent (the Thrift impl mutates the underlying
  operation handle inside processOperationStatusResponse), so the
  property+readonly+disclaimer-JSDoc pattern was misleading. Method
  form makes the live-read semantics obvious to a fresh implementer.
  3 facade call sites updated.

L9 — wireSynthesis moves under thrift-backend.
  The file imports Thrift IDL types and produces Thrift-typed values;
  it belongs next to ThriftOperationBackend, not in the neutral
  lib/utils/ tree where it would creep into the dependency cone of
  future backend-neutral helpers. Same reasoning that placed
  numberToInt64 and getDirectResultsOptions under thrift-backend/.

L10 — interface-level downcast policy.
  Add a JSDoc paragraph on IOperationBackend grandfathering the two
  existing `instanceof ThriftOperationBackend` downcasts in
  DBSQLOperation.status/getMetadata and prohibiting new ones. Future
  zero-loss back-compat needs should extend the interface (or add an
  optional method) rather than spawn a per-backend branch matrix.

Gates: yarn build (exit 0), yarn lint (0 errors, 3 pre-existing
warnings in tests/e2e/protocol_versions.test.ts), yarn test on touched
files (163 passing, +12 net new tests from M4 work; 2 failures pre-
existing on PR head unchanged: getSchema-directResults and the
LZ4-cloud-fetch flag — both flagged in the team-lead playbook as
known prior regressions).

Cascade implications for downstream PRs (#380 #377 #379 #382 #381
#384 #383): L8 converts hasResultSet from a property to a method,
M1 swaps WaitUntilReadyOptions for IOperationBackendWaitOptions on
the backend interface. Both are mechanical renames at downstream
backend impls when they rebase.

Co-authored-by: Isaac
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
@msrathore-db
Copy link
Copy Markdown
Contributor Author

Review-comment responses (issue-body findings)

All review findings addressed in a9347e8 except L11 (deliberately deferred). Inline replies posted on each of the 10 thread-level comments. Tracking the issue-body findings (M3, L7, L8, L9, L10, L11) below.

Status

ID Status Notes
H1 Fixed isClosed?: () => boolean probe on IOperationBackend.fetchChunk; live-reproduced + fix verified on pecotesting. See inline reply.
M1 Fixed New IOperationBackendWaitOptions with neutral (status: OperationStatus) => unknown; facade adapts public Thrift-shaped callback at the boundary. See inline reply.
M2 Fixed synthesizeStatusFromOperation maps Failed/Cancelled/Closed to ERROR_STATUS carrying errorMessage+sqlState. See inline reply.
M3 Fixed Added optional backend?: 'thrift' | 'sea' | 'kernel' to TelemetryEvent and DriverConfiguration in lib/telemetry/types.ts. Strictly additive; emit-site wiring lands when SEA emission goes live.
M4 Fixed New tests/unit/thrift-backend/wireSynthesis.test.ts (11 cases) + new useSEA: true failure-path test in DBSQLClient.test.ts. Net +12 passing tests (151 → 163 on touched files). See inline reply.
L1 Fixed payload instanceof Error ? payload : new Error(String(payload)). See inline reply.
L2 Fixed Backend assigned only after successful connect(); best-effort close + rethrow on failure. See inline reply.
L3 Fixed resultFormatToThrift throws HiveDriverError on unknown format. See inline reply.
L4 Fixed @deprecated JSDoc tag on DBSQLOperation.getMetadata. See inline reply.
L5 Fixed numberToInt64 re-export expanded with @deprecated JSDoc on the symbol. See inline reply.
L6 Fixed New runBackend<T>(fn) helper in DBSQLSession; 11 sites collapsed. See inline reply.
L7 Fixed Restored the three why-comments verbatim from main: staging-detection invariant in executeStatement, AWS/Azure 404 difference in handleStagingRemove, Content-Length-required note in handleStagingPut.
L8 Fixed readonly hasResultSet: booleanhasResultSet(): boolean. Updated the ThriftOperationBackend impl and 3 facade call sites (fetchChunk, hasMoreRows, getSchema). Cascade implication: downstream backend impls will need to convert get hasResultSet() to hasResultSet() on rebase — mechanical.
L9 Fixed lib/utils/thriftWireSynthesis.tslib/thrift-backend/wireSynthesis.ts (via git mv). Updated the single importer in lib/DBSQLOperation.ts. Relative imports inside the moved file unchanged (same nesting depth).
L10 Fixed Added a "Facade downcast policy" paragraph to the JSDoc above IOperationBackend grandfathering the two existing instanceof ThriftOperationBackend downcasts in DBSQLOperation.status/getMetadata and prohibiting new ones — future zero-loss back-compat needs should extend the interface (or add an optional method) instead. Placed on the interface itself rather than CONTRIBUTING.md so it's visible at the point of temptation.
L11 Deferred Backend kind: 'thrift' | 'sea' | 'kernel' field on the three interfaces + facade log prefixes. Deliberately deferred to avoid a cross-stack cascade ripple (required field on IBackend / ISessionBackend / IOperationBackend would force every downstream backend impl to declare it). Worth coordinating across the stack as a follow-up after the downstream PRs land.

Validation gates

Gate Result
yarn build (tsc) exit 0
yarn lint 0 errors, 3 pre-existing warnings (tests/e2e/protocol_versions.test.ts lines 33, 44, 329 — unchanged from PR head)
yarn test (touched files) 163 passing (+12 net new from M4 work); 2 failing

The 2 test failures are pre-existing on PR head (verified by re-running on stashed-changes pristine checkout, same line numbers, same cause):

  1. DBSQLOperation getSchema › should use schema from directResultsTypeError at definedOrError in ThriftOperationBackend.adaptResultMetadata. Known prior regression noted in the team-lead playbook; root cause hypothesis: definedOrError(response.resultFormat) is called even when direct-results carry only schema. Out of scope for this review-fix pass.
  2. DBSQLSession executeStatement › LZ4 compression with cloud fetch … canDownloadResult=falseexpected false to be true. Pre-existing PR [SEA-NodeJS] (1/8) Backend abstraction — IBackend / ISessionBackend / IOperationBackend #378 regression around the LZ4 flag plumbing.

Both surface independently of any of the review-fix changes in a9347e8.

H1 live-warehouse verification

For the highest-severity finding (H1 — fetchChunk lost mid-flight failIfClosed), I reproduced the regression on pecotesting (adb-6436897454825492.12.azuredatabricks.net, warehouse 2f03dd43e35e2aa0) using a CloudFetch-eligible query (SELECT id, repeat('x', 256) AS payload FROM range(0, 400000) ORDER BY id). The race pattern: schedule op.cancel() via setImmediate() so it lands during the setTimeout(0) yield window inside the backend's fetchChunk. After the fix, the isClosed() probe short-circuits before the data RPC runs to completion when a cancel has landed during the yield.

Cascade implications for downstream PRs

Two interface changes in this commit will ripple to downstream PRs #380 #377 #379 #382 #381 #384 #383 when they rebase:

  • L8 (hasResultSet property → method): mechanical rename in each downstream IOperationBackend impl (get hasResultSet()hasResultSet()) plus any call sites in those PRs.
  • M1 (Thrift-typed WaitUntilReadyOptions → neutral IOperationBackendWaitOptions on backend interface): each downstream waitUntilReady impl switches options type and passes a neutral OperationStatus to the callback.

Both are strictly mechanical on rebase. H1's isClosed?: () => boolean addition is purely additive — downstream impls compile clean without changes.

Happy to address L11 in a follow-up across the full stack when you're ready to coordinate the cascade.


This comment was generated with GitHub MCP.

Resolve conflicts between the SEA backend abstraction and the telemetry/SPOG changes from main while keeping Thrift-specific behavior behind backend adapters.
@github-actions
Copy link
Copy Markdown

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

Restore Thrift compatibility paths needed by existing schema and result-handler tests after merging main telemetry changes.

Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
@github-actions
Copy link
Copy Markdown

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

Keep existing e2e-only inspection hooks available through the facade while the new backend abstraction owns result handling.

Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
@github-actions
Copy link
Copy Markdown

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

@msrathore-db msrathore-db merged commit a3bf320 into main May 29, 2026
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants