spike: port adapter from pyodbc/ODBC to ADBC#633
Conversation
Replace the entire pyodbc/msodbcsql18 stack with ADBC (Arrow Database Connectivity) using the mssql driver from dbc, and remove the dbt-fabric dependency to make dbt-sqlserver a standalone adapter. Key changes: - Remove dbt-fabric dependency; inherit from dbt-adapters base classes - Rewrite connection layer to use adbc-driver-manager dbapi - Build go-mssqldb connection URIs from profile credentials - Inline ~25 fabric macros with sqlserver__ prefix - Rewrite seed loading to inline literal values (ADBC doesn't support parameterized queries with ? placeholders) - Add Arrow type code -> SQL Server type name mapping - Update unit tests for ADBC/URI-based connection model Test results: 123 passed, 8 failed, 45 skipped (10/10 unit tests pass) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
for more information, see https://pre-commit.ci
|
@dataders , You should keep this driver core change isolated from others, if you can please use #628 as base for fabric adapter removal, I will production test the driver change myself if this helps on DBT fusion road but this needs to be behind a flag, it's a core change and should wait for a better schematic version update. |
hey @axellpadilla thanks for checking this out! FWIW, I have no plans to merge this. I was just vibe-coding out a thought experiment. |
Yeah thanks it's interesting, I hate having to install the driver! |
Summary
Spike to replace the entire pyodbc/msodbcsql18 stack with ADBC (Arrow Database Connectivity) using the
mssqldriver from dbc, and remove the dbt-fabric dependency entirely to make dbt-sqlserver a standalone adapter.Why: ODBC driver installation is the #1 pain point for dbt-sqlserver users across platforms. ADBC's mssql driver is a single binary installed via
dbc install mssql— no ODBC driver manager, no msodbcsql18, no platform-specific setup.Scope: SQL auth only, local Docker SQL Server, proving the adapter works end-to-end.
Changes
Python adapter (remove dbt-fabric dependency)
dbt-fabric==1.9.6, addadbc-driver-manager>=1.9.0+pyarrow>=20.0.0__init__.py— Removedependencies=["fabric"]fromAdapterPluginsqlserver_credentials.py— Inherit fromCredentials(wasFabricCredentials); addbuild_adbc_uri()for go-mssqldb connection URIssqlserver_connections.py— Inherit fromSQLConnectionManager(wasFabricConnectionManager); connect viaadbc_dbapi.connect(driver='mssql', ...); adddata_type_code_to_name()mapping Arrow type codes → SQL Server typessqlserver_adapter.py— Inherit fromSQLAdapter(wasFabricAdapter); inline Fabric's type conversion and SQL helper methodssqlserver_column.py— Inherit fromColumn(wasFabricColumn); inline type labels and methodssqlserver_configs.py— Inherit fromAdapterConfig(wasFabricConfigs)Macros (inline fabric macros with sqlserver__ prefix)
Since we removed
dependencies=["fabric"], fabric macros are no longer in the dispatch chain. All ~25 fabric macros that sqlserver depended on were copied in withsqlserver__prefix:list_schemas,check_schema_exists,list_relations_without_caching,get_relation_without_caching,get_relation_last_modifiedmake_temp_relation,get_drop_sql,rename_relationcreate_schema,drop_schemaget_columns_in_relation,alter_relation_add_remove_columnsapply_grants,get_show_grant_sql,get_grant_sql,get_revoke_sqlget_limit_sql(OFFSET/FETCH)get_merge_sql,get_delete_insert_merge_sql,get_incremental_default_sqlpost_snapshot,get_true_sql,snapshot_hash_arguments,build_snapshot_table,snapshot_staging_tablebuild_columns_constraints,build_model_constraintsany_value,array_construct,cast_bool_to_text,concat,date_trunc,dateadd,get_tables_by_pattern,hash,last_day,length,listagg,position,safe_cast,timestampsSeed loading fix
ADBC's mssql driver does not support parameterized queries with
?placeholders. Rewrotesqlserver__load_csv_rowsto inline literal values directly into INSERT statements, with proper handling for NULL, booleans (is sameas true/false), numbers, and strings (with quote escaping).Unit test macro fix
Updated
unit_test_table.sqlto passcolumn_name_to_quotedtoget_expected_sql()(required by newer dbt-core API).Test Results
dbt debug8 remaining failures (all known ADBC behavioral differences)
Known limitations (acceptable for spike)
?fedauth=for future work)handle.timeoutHow to test locally
🤖 Generated with Claude Code