Skip to content

Commit 8a22d54

Browse files
committed
sea-abstraction: address full-review findings (F1-F17 except F5)
Round-N fixes from the 9-reviewer pre-review. Public IOperation/DBSQLOperation surface preserved byte-identical; backend interfaces (IBackend / ISessionBackend / IOperationBackend) made fully neutral so both Thrift and SEA can implement the same contract. F1 — neutral DTOs at IOperationBackend with Thrift-shape preservation on the public facade (adapter pattern): - lib/contracts/OperationStatus.ts (new) — neutral OperationStatus + OperationState enum mirroring databricks-sql-python's CommandState and kernel pyo3's StatementStatus taxonomy. - lib/contracts/ResultMetadata.ts (new) — neutral ResultMetadata + ResultFormat enum mirroring the three TSparkRowSetType cases. - IOperationBackend.status()/getResultMetadata() return the neutral DTOs. - ThriftOperationBackend.status() adapts at the boundary via adaptOperationStatus / adaptResultMetadata; module-level helpers thriftStateToOperationState and thriftRowSetTypeToResultFormat do the enum maps. - ThriftOperationBackend exposes thriftStatusResponse() and thriftResultMetadataResponse() as public Thrift-only accessors used by the facade's zero-loss fast path (kept for internal state-machine + result-handler dispatch as well). - lib/utils/thriftWireSynthesis.ts (new) — synthesizeThriftStatus and synthesizeThriftResultSetMetadata: convert neutral DTOs back to Thrift wire shape for the non-Thrift backend path. Lossy on Thrift-only fields (taskStatus, numModifiedRows, cacheLookupResult, etc.). - DBSQLOperation.status() and getMetadata() preserved Thrift return shape: Thrift backend path returns the real wire response (zero loss); non-Thrift backend path synthesizes via the new helpers. - DBSQLOperation.getResultMetadata() — new additive neutral accessor on IOperation; DBSQLSession.handleStagingOperation uses it instead of the deprecated Thrift-shaped getMetadata(). F2 — IBackend.connect() is now zero-arg. Backend reads everything it needs from IClientContext / constructor; matches Python connector's pattern of passing session_configuration via constructor not method-arg. F3 — Restore the 'Server protocol version' debug log dropped by the original PR-378 refactor. Re-added to ThriftSessionBackend.constructor with the LogLevel.debug + IClientContext.getLogger() pattern; matches the pre-refactor log site at main:lib/DBSQLSession.ts:175. F4 + F11 + F14 — SeaBackend stub safety: - close() is a no-op so DBSQLClient.close()'s state-clearing block can finish even after a useSEA: true connect() failure. - connect() and openSession() throw HiveDriverError instead of generic Error, matching the rest of the codebase. - connect(options: ConnectionOptions) and openSession(request: OpenSessionRequest) declare their parameters (with @typescript-eslint/no-unused-vars disable) so IDE autocomplete prompts the M1 SEA implementer. F6 + F7 + F9 + F10 — JSDoc on backend interfaces: - IBackend: connect/openSession/close docstrings; close() doc explicitly states transport-layer cleanup is owned by DBSQLClient. - ISessionBackend: copy IDBSQLSession's per-method one-liner JSDoc. - IOperationBackend: doc hasResultSet (readonly external; mutates internally), waitUntilReady (MUST throw OperationStateError on terminal non-success). F8 — tests/unit/sea/SeaBackend.test.ts (new) locks in the stub contract: connect() rejects HiveDriverError, openSession() rejects HiveDriverError, close() resolves no-op. ~30 LOC. F12 — Drop legacy { handle, ... } ctor branch from DBSQLOperation and DBSQLSession: - Facades accept only { backend, context }. - DBSQLSession no longer imports ThriftSessionBackend at all. - DBSQLOperation imports ThriftOperationBackend solely for the F1 typed downcast (zero-loss Thrift fast path); this is a deliberate, scoped coupling tied to the back-compat decision. - tests/unit/.stubs/createSessionForTest.ts and createOperationForTest.ts (new) wrap the legacy shape; all 48 + 54 test sites mechanically migrated. F15 — ThriftOperationBackend.waitUntilReady uses imported WaitUntilReadyOptions type instead of an anonymous inline shape. F16 — useSEA flag moved out of public ConnectionOptions: - Removed useSEA?: boolean from the exported lib/contracts/IDBSQLClient.ts ConnectionOptions; no longer ships in the public .d.ts. - lib/contracts/InternalConnectionOptions.ts (new) declares the flag as a non-exported internal extension; DBSQLClient.connect() reads via a typed cast. Mirrors Python's kwargs.get('use_sea', False) pattern at databricks-sql-python/src/databricks/sql/session.py:111. F17 — Missing return; after case 'timeout' in forwardConnectionEvent so a future fifth case doesn't silently fall through. The trailing return; in the last case triggers no-useless-return — quieted with a localized eslint-disable-next-line + intent comment. F5 — deferred per owner instruction (test-only as any cast tightening). Verification: - yarn lint clean (3 pre-existing warnings in tests/e2e/protocol_versions.test.ts). - yarn build clean. - tsc --noEmit -p tsconfig.json clean (apart from pre-existing examples/tokenFederation/* import errors that exist on main). - Runtime smoke test of SeaBackend stub + Thrift-wire synthesis round-trip passes 5/5 assertions. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
1 parent 2be1a63 commit 8a22d54

21 files changed

Lines changed: 650 additions & 173 deletions

lib/DBSQLClient.ts

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import IConnectionOptions from './connection/contracts/IConnectionOptions';
1515
import HiveDriverError from './errors/HiveDriverError';
1616
import { buildUserAgentString } from './utils';
1717
import IBackend from './contracts/IBackend';
18+
import { InternalConnectionOptions } from './contracts/InternalConnectionOptions';
1819
import ThriftBackend from './thrift-backend/ThriftBackend';
1920
import SeaBackend from './sea/SeaBackend';
2021
import PlainHttpAuthentication from './connection/auth/PlainHttpAuthentication';
@@ -237,7 +238,11 @@ export default class DBSQLClient extends EventEmitter implements IDBSQLClient, I
237238

238239
this.connectionProvider = this.createConnectionProvider(options);
239240

240-
this.backend = options.useSEA
241+
// M0: `useSEA` is consumed via a non-exported internal-options cast so it
242+
// doesn't ship in the public `.d.ts`. Mirrors Python's `kwargs.get("use_sea")`
243+
// pattern (see databricks-sql-python/src/databricks/sql/session.py).
244+
const internalOptions = options as ConnectionOptions & InternalConnectionOptions;
245+
this.backend = internalOptions.useSEA
241246
? new SeaBackend()
242247
: new ThriftBackend({
243248
context: this,
@@ -272,6 +277,10 @@ export default class DBSQLClient extends EventEmitter implements IDBSQLClient, I
272277
case 'timeout':
273278
this.logger.log(LogLevel.debug, 'Connection timed out.');
274279
this.emit('timeout');
280+
// Explicit return mirrors the other cases and protects against
281+
// fall-through if a new event is added below.
282+
// eslint-disable-next-line no-useless-return
283+
return;
275284
// no default
276285
}
277286
}

lib/DBSQLOperation.ts

Lines changed: 43 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -11,29 +11,23 @@ import IOperation, {
1111
} from './contracts/IOperation';
1212
import {
1313
TGetOperationStatusResp,
14-
TOperationHandle,
15-
TTableSchema,
16-
TSparkDirectResults,
1714
TGetResultSetMetadataResp,
15+
TTableSchema,
1816
} from '../thrift/TCLIService_types';
1917
import Status from './dto/Status';
2018
import { LogLevel } from './contracts/IDBSQLLogger';
2119
import OperationStateError, { OperationStateErrorCode } from './errors/OperationStateError';
2220
import { OperationChunksIterator, OperationRowsIterator } from './utils/OperationIterator';
2321
import IClientContext from './contracts/IClientContext';
2422
import IOperationBackend from './contracts/IOperationBackend';
23+
import { ResultMetadata } from './contracts/ResultMetadata';
2524
import ThriftOperationBackend from './thrift-backend/ThriftOperationBackend';
25+
import { synthesizeThriftStatus, synthesizeThriftResultSetMetadata } from './utils/thriftWireSynthesis';
2626

27-
type DBSQLOperationConstructorOptions =
28-
| {
29-
handle: TOperationHandle;
30-
directResults?: TSparkDirectResults;
31-
context: IClientContext;
32-
}
33-
| {
34-
backend: IOperationBackend;
35-
context: IClientContext;
36-
};
27+
interface DBSQLOperationConstructorOptions {
28+
backend: IOperationBackend;
29+
context: IClientContext;
30+
}
3731

3832
export default class DBSQLOperation implements IOperation {
3933
private readonly context: IClientContext;
@@ -48,14 +42,7 @@ export default class DBSQLOperation implements IOperation {
4842

4943
constructor(options: DBSQLOperationConstructorOptions) {
5044
this.context = options.context;
51-
this.backend =
52-
'backend' in options
53-
? options.backend
54-
: new ThriftOperationBackend({
55-
handle: options.handle,
56-
directResults: options.directResults,
57-
context: options.context,
58-
});
45+
this.backend = options.backend;
5946
this.context.getLogger().log(LogLevel.debug, `Operation created with id: ${this.id}`);
6047
}
6148

@@ -144,14 +131,27 @@ export default class DBSQLOperation implements IOperation {
144131
}
145132

146133
/**
147-
* Requests operation status
134+
* Requests operation status. Returns the Thrift wire response for
135+
* back-compat with existing user code. On the Thrift backend the response
136+
* is returned verbatim; on any other backend (e.g. SEA) the response is
137+
* synthesized from the neutral {@link IOperationBackend.status} result,
138+
* with Thrift-only fields (`taskStatus`, `numModifiedRows`, etc.) left
139+
* undefined.
140+
*
148141
* @param progress
149142
* @throws {StatusError}
150143
*/
151144
public async status(progress: boolean = false): Promise<TGetOperationStatusResp> {
152145
await this.failIfClosed();
153146
this.context.getLogger().log(LogLevel.debug, `Fetching status for operation with id: ${this.id}`);
154-
return this.backend.status(progress);
147+
if (this.backend instanceof ThriftOperationBackend) {
148+
// Zero-loss path: the Thrift backend has the wire response on hand.
149+
return this.backend.thriftStatusResponse(progress);
150+
}
151+
// Non-Thrift backend: synthesize the Thrift-shaped response from the
152+
// neutral OperationStatus DTO.
153+
const status = await this.backend.status(progress);
154+
return synthesizeThriftStatus(status);
155155
}
156156

157157
/**
@@ -213,12 +213,31 @@ export default class DBSQLOperation implements IOperation {
213213
return metadata.schema ?? null;
214214
}
215215

216-
public async getMetadata(): Promise<TGetResultSetMetadataResp> {
216+
public async getResultMetadata(): Promise<ResultMetadata> {
217217
await this.failIfClosed();
218218
await this.waitUntilReadyThroughBackend();
219219
return this.backend.getResultMetadata();
220220
}
221221

222+
/**
223+
* Fetch result-set metadata as the Thrift wire response. Kept for
224+
* back-compat with existing user code. On the Thrift backend the wire
225+
* response is returned verbatim; on any other backend the response is
226+
* synthesized from the neutral {@link ResultMetadata}, with Thrift-only
227+
* fields (`cacheLookupResult`, `uncompressedBytes`, `compressedBytes`,
228+
* `status`) left undefined / defaulted.
229+
*
230+
* Prefer {@link DBSQLOperation.getResultMetadata} in new code.
231+
*/
232+
public async getMetadata(): Promise<TGetResultSetMetadataResp> {
233+
await this.failIfClosed();
234+
await this.waitUntilReadyThroughBackend();
235+
if (this.backend instanceof ThriftOperationBackend) {
236+
return this.backend.thriftResultMetadataResponse();
237+
}
238+
return synthesizeThriftResultSetMetadata(await this.backend.getResultMetadata());
239+
}
240+
222241
private async failIfClosed(): Promise<void> {
223242
if (this.closed) {
224243
throw new OperationStateError(OperationStateErrorCode.Closed);

lib/DBSQLSession.ts

Lines changed: 6 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@ import * as path from 'path';
33
import stream from 'node:stream';
44
import util from 'node:util';
55
import fetch, { HeadersInit } from 'node-fetch';
6-
import { TSessionHandle, TProtocolVersion } from '../thrift/TCLIService_types';
76
import IDBSQLSession, {
87
ExecuteStatementOptions,
98
TypeInfoRequest,
@@ -27,24 +26,17 @@ import StagingError from './errors/StagingError';
2726
import IClientContext from './contracts/IClientContext';
2827
import ISessionBackend from './contracts/ISessionBackend';
2928
import IOperationBackend from './contracts/IOperationBackend';
30-
import ThriftSessionBackend from './thrift-backend/ThriftSessionBackend';
3129

3230
// Explicitly promisify a callback-style `pipeline` because `node:stream/promises` is not available in Node 14
3331
const pipeline = util.promisify(stream.pipeline);
3432

3533
// Re-export for back-compat with existing imports.
3634
export { numberToInt64 } from './thrift-backend/ThriftSessionBackend';
3735

38-
type DBSQLSessionConstructorOptions =
39-
| {
40-
handle: TSessionHandle;
41-
context: IClientContext;
42-
serverProtocolVersion?: TProtocolVersion;
43-
}
44-
| {
45-
backend: ISessionBackend;
46-
context: IClientContext;
47-
};
36+
interface DBSQLSessionConstructorOptions {
37+
backend: ISessionBackend;
38+
context: IClientContext;
39+
}
4840

4941
export default class DBSQLSession implements IDBSQLSession {
5042
private readonly context: IClientContext;
@@ -59,14 +51,7 @@ export default class DBSQLSession implements IDBSQLSession {
5951

6052
constructor(options: DBSQLSessionConstructorOptions) {
6153
this.context = options.context;
62-
this.backend =
63-
'backend' in options
64-
? options.backend
65-
: new ThriftSessionBackend({
66-
handle: options.handle,
67-
context: options.context,
68-
serverProtocolVersion: options.serverProtocolVersion,
69-
});
54+
this.backend = options.backend;
7055
this.context.getLogger().log(LogLevel.debug, `Session created with id: ${this.id}`);
7156
}
7257

@@ -106,7 +91,7 @@ export default class DBSQLSession implements IDBSQLSession {
10691

10792
// Staging detection: only run when stagingAllowedLocalPath is provided.
10893
if (options.stagingAllowedLocalPath !== undefined) {
109-
const metadata = await operation.getMetadata();
94+
const metadata = await operation.getResultMetadata();
11095
if (metadata.isStagingOperation) {
11196
const allowedLocalPath = Array.isArray(options.stagingAllowedLocalPath)
11297
? options.stagingAllowedLocalPath

lib/contracts/IBackend.ts

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,28 @@ import ISessionBackend from './ISessionBackend';
77
* re-selected per-call.
88
*/
99
export default interface IBackend {
10+
/**
11+
* Establish backend-level state before any session is opened. Implementations
12+
* consume `options` to build backend-specific connection parameters (e.g. the
13+
* SEA backend derives napi-binding `SeaNativeConnectionOptions` from the auth
14+
* + host fields here). Transport-layer connection providers are owned by
15+
* `DBSQLClient` (via `IClientContext`) and exposed to backends through
16+
* constructor injection.
17+
*/
1018
connect(options: ConnectionOptions): Promise<void>;
1119

20+
/**
21+
* Open a session. Returned `ISessionBackend` is owned by the caller
22+
* and torn down via its own `close()`.
23+
*/
1224
openSession(request: OpenSessionRequest): Promise<ISessionBackend>;
1325

26+
/**
27+
* Backend-level teardown. Transport-layer cleanup (connection provider,
28+
* thrift client, auth provider) is owned by `DBSQLClient` and runs
29+
* after this returns. Implementations release backend-internal resources
30+
* here, and MUST be safe to call on a partially-initialized backend
31+
* (i.e. after a failed `connect()`).
32+
*/
1433
close(): Promise<void>;
1534
}

lib/contracts/IDBSQLClient.ts

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -54,12 +54,6 @@ export type ConnectionOptions = {
5454
socketTimeout?: number;
5555
proxy?: ProxyOptions;
5656
enableMetricViewMetadata?: boolean;
57-
/**
58-
* Opt-in flag to dispatch through the Statement Execution API (SEA) backend
59-
* instead of the default Thrift backend. Defaults to `false`.
60-
* @internal Not stable; M0 stub only.
61-
*/
62-
useSEA?: boolean;
6357
} & AuthOptions;
6458

6559
export interface OpenSessionRequest {

lib/contracts/IOperation.ts

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import { Readable, ReadableOptions } from 'node:stream';
22
import { TGetOperationStatusResp, TTableSchema } from '../../thrift/TCLIService_types';
33
import Status from '../dto/Status';
4+
import { ResultMetadata } from './ResultMetadata';
45

56
export type OperationStatusCallback = (progress: TGetOperationStatusResp) => unknown;
67

@@ -59,7 +60,10 @@ export default interface IOperation {
5960
fetchAll(options?: FetchOptions): Promise<Array<object>>;
6061

6162
/**
62-
* Request status of operation
63+
* Request status of operation. Returns the Thrift wire response for
64+
* back-compat. New code should prefer {@link IOperation.getResultMetadata}
65+
* for metadata and may consume the neutral `IOperationBackend.status` via
66+
* a typed downcast when implementing alternative backends.
6367
*
6468
* @param progress
6569
*/
@@ -90,6 +94,12 @@ export default interface IOperation {
9094
*/
9195
getSchema(options?: GetSchemaOptions): Promise<TTableSchema | null>;
9296

97+
/**
98+
* Fetch result-set metadata in the backend-neutral `ResultMetadata` shape.
99+
* Prefer this over the Thrift-shaped surface for new code.
100+
*/
101+
getResultMetadata(): Promise<ResultMetadata>;
102+
93103
iterateChunks(options?: IteratorOptions): IOperationChunksIterator;
94104

95105
iterateRows(options?: IteratorOptions): IOperationRowsIterator;

lib/contracts/IOperationBackend.ts

Lines changed: 31 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,55 @@
1-
import { TGetOperationStatusResp, TGetResultSetMetadataResp } from '../../thrift/TCLIService_types';
21
import Status from '../dto/Status';
32
import { WaitUntilReadyOptions } from './IOperation';
3+
import { OperationStatus } from './OperationStatus';
4+
import { ResultMetadata } from './ResultMetadata';
45

56
/**
67
* What a `DBSQLOperation` needs from its backend. Returned by
78
* `ISessionBackend.executeStatement` and the metadata methods.
89
*/
910
export default interface IOperationBackend {
11+
/** Operation identifier. */
1012
readonly id: string;
1113

14+
/**
15+
* Whether this operation has a result set. Initial value may be derived
16+
* from the create-operation response; implementations MUST refresh it
17+
* from terminal status responses (the Thrift impl updates
18+
* `operationHandle.hasResultSet` inside `processOperationStatusResponse`).
19+
* `readonly` here means external callers cannot reassign the property —
20+
* not that the underlying value is fixed at construction time.
21+
*/
1222
readonly hasResultSet: boolean;
1323

24+
/** Fetch the next chunk of result rows. */
1425
fetchChunk(options: { limit: number; disableBuffering?: boolean }): Promise<Array<object>>;
1526

27+
/** Whether more rows are available beyond what has been fetched. */
1628
hasMore(): Promise<boolean>;
1729

30+
/**
31+
* Poll the backend until the operation reaches a terminal state.
32+
*
33+
* MUST throw `OperationStateError` (with one of `OperationStateErrorCode.{Canceled,
34+
* Closed, Error, Timeout, Unknown}`) on terminal non-success states. The
35+
* `DBSQLOperation` facade depends on `Canceled` and `Closed` codes to mirror
36+
* the operation into its closed/cancelled flags; future implementations must
37+
* use the same error type for the facade to stay in sync.
38+
*/
1839
waitUntilReady(options?: WaitUntilReadyOptions): Promise<void>;
1940

20-
status(progress: boolean): Promise<TGetOperationStatusResp>;
41+
/**
42+
* Fetch operation status as a neutral `OperationStatus`. Pass `progress: true`
43+
* to request that the backend include a progress payload.
44+
*/
45+
status(progress: boolean): Promise<OperationStatus>;
2146

22-
getResultMetadata(): Promise<TGetResultSetMetadataResp>;
47+
/** Fetch result-set metadata (schema, format, lz4 flag, arrow schema, staging flag). */
48+
getResultMetadata(): Promise<ResultMetadata>;
2349

50+
/** Cancel the operation. */
2451
cancel(): Promise<Status>;
2552

53+
/** Close the operation. Idempotent. */
2654
close(): Promise<Status>;
2755
}

lib/contracts/ISessionBackend.ts

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,21 +19,42 @@ import InfoValue from '../dto/InfoValue';
1919
* `IBackend.openSession()`. Lifecycle tied to a single `DBSQLSession`.
2020
*/
2121
export default interface ISessionBackend {
22+
/** Session identifier. */
2223
readonly id: string;
2324

25+
/** Returns general information about the data source. */
2426
getInfo(infoType: number): Promise<InfoValue>;
2527

28+
/** Executes DDL/DML statements. */
2629
executeStatement(statement: string, options: ExecuteStatementOptions): Promise<IOperationBackend>;
2730

31+
/** Information about supported data types. */
2832
getTypeInfo(request: TypeInfoRequest): Promise<IOperationBackend>;
33+
34+
/** List of catalogs. */
2935
getCatalogs(request: CatalogsRequest): Promise<IOperationBackend>;
36+
37+
/** List of schemas. */
3038
getSchemas(request: SchemasRequest): Promise<IOperationBackend>;
39+
40+
/** List of tables. */
3141
getTables(request: TablesRequest): Promise<IOperationBackend>;
42+
43+
/** List of supported table types. */
3244
getTableTypes(request: TableTypesRequest): Promise<IOperationBackend>;
45+
46+
/** Full column information for a table. */
3347
getColumns(request: ColumnsRequest): Promise<IOperationBackend>;
48+
49+
/** Information about a function. */
3450
getFunctions(request: FunctionsRequest): Promise<IOperationBackend>;
51+
52+
/** Primary keys of a table. */
3553
getPrimaryKeys(request: PrimaryKeysRequest): Promise<IOperationBackend>;
54+
55+
/** Foreign-key relationships between two tables. */
3656
getCrossReference(request: CrossReferenceRequest): Promise<IOperationBackend>;
3757

58+
/** Close the session. Idempotent. */
3859
close(): Promise<Status>;
3960
}

0 commit comments

Comments
 (0)