feat(csharp): Add driver manager by davidhcoe · Pull Request #4075 · apache/arrow-adbc

davidhcoe · 2026-03-11T20:37:27Z

Implements C# driver manager matching adbc_driver_manager.h spec:

Core APIs:
• LoadDriver() / FindLoadDriver() - Load native/managed drivers
• OpenDatabaseFromProfile() - TOML profile support
• AdbcLoadFlags - Search control (env/user/system paths)

Features:
• Native + managed (.NET) driver loading
• TOML manifests with env_var() expansion
• Uses a custom TOML parser vs Tomlyn because a strong name is required for dependencies
• Cross-platform driver discovery
• Option merging (profile + explicit)
• Co-located manifest detection

Tests: 40+ unit tests + sample BigQuery/Snowflake implementations (attached, not checked in)

BigQueryDriverManagerTests.txt
SnowflakeDriverManagerTests.txt

…anager

davidhcoe · 2026-03-13T03:06:35Z

I debated whether to put this in its own assembly or as a subfolder under the shared Adbc project. I decided on a subfolder approach to reduce the number of dependencies required (especially since the AdbcDrivers are still using submodules and any tests there would require an additional submodule to be managed).

lidavidm · 2026-03-13T05:46:02Z

Note we noticed various inconsistencies in profiles that we're working on correcting (#4082 / #4085)

CurtHagenlocher · 2026-03-13T15:01:12Z

To what extent have you tried this with .NET 8.0? I'm discovering that dynamic loading of assemblies is fairly different in that environment.

davidhcoe · 2026-03-13T17:05:56Z

To what extent have you tried this with .NET 8.0? I'm discovering that dynamic loading of assemblies is fairly different in that environment.

It has .NET 8 tests that all pass but I added some more items in the recent push.

davidhcoe · 2026-03-13T17:09:54Z

Note we noticed various inconsistencies in profiles that we're working on correcting (#4082 / #4085)

I added some items to address #4082. For #4085 , based on your table, this implementation currently does:

Feature -> How It Works

init_func (entrypoint) priority -> LoadDriver(path, entrypoint) - if provided, uses it directly
Load driver from explicit path -> LoadDriver(driverPath)
Load driver by name (search) -> FindLoadDriver(driverName, ...) with directory search
Load from profile object -> OpenDatabaseFromProfile(profile, ...)
Profile from TOML file -> TomlConnectionProfile.FromFile(path)
Explicit options override profile -> OpenDatabaseFromProfile(profile, explicitOptions, ...)
Managed (.NET) drivers -> LoadManagedDriver(assemblyPath, typeName)
Native drivers -> Via CAdbcDriverImporter.Load(...)
Co-located manifests (.toml next to .dll) -> Auto-detected in LoadDriver
Profile/manifest version fields -> profile_version (new) and version (legacy)
Options section naming -> [Options] (new) and [options] (legacy, case-insensitive)

❌ Not Yet Implemented

URI parsing -> No parsing of snowflake://..., postgresql://... URIs
Profile URI scheme -> No profile:///path/to/file.toml support
Driver URI scheme -> No URI that embeds both driver and connection string
Profile in options -> No automatic detection of profile key in options dict
Multiple profile detection -> No error if profile specified in multiple ways
Driver/profile consistency -> No validation that driver arg matches profile's driver
URI extraction from driver -> No parsing driver argument as potential URI
Non-driver URI detection -> No distinguishing connection URIs from driver URIs

davidhcoe · 2026-05-06T04:27:25Z

Anything else needed for this one @lidavidm @CurtHagenlocher ?

davidhcoe · 2026-05-13T13:10:39Z

We have more users asking for the ability to edit configuration files for their drivers. Is there anything else needed for this @lidavidm or @CurtHagenlocher ?

CurtHagenlocher · 2026-05-13T13:38:58Z

My main concern is being explicit about how dependency resolution is expected to work for .NET 8+, as I think there is trickiness and subtlety there based on my experiences in our own code base. I'll try to find time this week to take a closer look and run a few experiments.

…anager

CurtHagenlocher

I'm still working on reviewing this; there's more to absorb than I'd expected. Here's some initial feedback.

CurtHagenlocher · 2026-05-15T16:40:42Z

+    ///   </item>
+    ///   <item>
+    ///     <description>
+    ///       If a driver does not ship a <c>.deps.json</c>, the loader still works but


In my experience, loading a driver without a .deps.json file did not generally work. Has it worked for you?

CurtHagenlocher · 2026-05-15T16:47:09Z

+            // .deps.json is consulted when the default AssemblyLoadContext fails to
+            // resolve a dependency. AssemblyDependencyResolver throws if no .deps.json
+            // is present next to the assembly; in that case we silently fall back to
+            // the host's normal probing behavior.


I found that it was necessary to always reference the host's version of the following three assemblies regardless of the driver's dependencies: Apache.Arrow, Apache.Arrow.Adbc and System.Diagnostics.DiagnosticSource. That's because these three assemblies contain types that are in the public ADBC contracts. If the driver resolves to a different version of these assemblies than the host, an attempt to use an API with one of these types will produce a MethodMissingException.

A test which covers this scenario would need to have the driver built against a different version of one or more of these than the test code is built against.

CurtHagenlocher · 2026-05-15T19:39:26Z

+                    continue;
+                }
+
+                if (line.StartsWith("[", StringComparison.Ordinal) && line.EndsWith("]", StringComparison.Ordinal))


Should this throw an exception if the section name contains anything other than A-Za-z0-9_-? I know we don't want or need a full-featured parser, but perhaps we should try to be overly strict in what we accept rather than reading something incorrectly.

CurtHagenlocher · 2026-05-18T17:02:05Z

+        // Tracks which driver path "won" the very first contested resolution for a
+        // given simple assembly name (e.g. "Newtonsoft.Json"). Used purely to emit a
+        // diagnostic when a later resolver could also satisfy that name -- the runtime
+        // always returns the assembly already loaded by the first winner, so any later
+        // candidate is silently shadowed.


I'm not sure to what extent this is true under .NET Core. Two assemblies with different load contexts can definitely resolve the same assembly to two different files/versions. Is this code path always used or only if there's no .deps file?

CurtHagenlocher · 2026-05-18T17:16:28Z

On the whole this looks good to me in its current state. The tests need to pass, of course, and I'm not super certain about the managed driver loading code for .NET 8+. But I think the latter can be worked out as people run into problems.

I'll take one last pass over the changes once the test issues are worked out.

davidhcoe · 2026-05-18T21:34:07Z

On the whole this looks good to me in its current state. The tests need to pass, of course, and I'm not super certain about the managed driver loading code for .NET 8+. But I think the latter can be worked out as people run into problems.

I'll take one last pass over the changes once the test issues are worked out.

Everything passes now except a strange Databricks test. Maybe it needs to be re-run?

CurtHagenlocher · 2026-05-18T23:26:51Z

Everything passes now except a strange Databricks test. Maybe it needs to be re-run?

That test has always been flaky and I was never able to figure out why.

CurtHagenlocher · 2026-05-18T23:58:52Z

@davidhcoe. the CI tests seem to be flaky after this checkin :(. See https://github.com/apache/arrow-adbc/actions/runs/26067113103/job/76641108349 for example.

David Coe added 4 commits March 11, 2026 10:43

initial check-in

4e7b4e7

Merge https://github.com/davidhcoe/arrow-adbc into dev/csharp-driverm…

18449bc

…anager

update readme header

3d64362

refactor; add security

6d48037

fix tests for GitHub

1406322

davidhcoe marked this pull request as ready for review March 13, 2026 03:54

davidhcoe requested a review from CurtHagenlocher as a code owner March 13, 2026 03:54

update profile parsing

7e13912

David Coe added 2 commits May 13, 2026 09:50

Merge https://github.com/davidhcoe/arrow-adbc into dev/csharp-driverm…

63c022c

…anager

add net10 support

d20df32

CurtHagenlocher reviewed May 15, 2026

View reviewed changes

David Coe added 4 commits May 16, 2026 00:31

PR feedback

857bbc8

add more tests

8f7b836

Copilot PR feedback

6619f9c

more Copilot feedback

63181fb

CurtHagenlocher reviewed May 18, 2026

View reviewed changes

fix tests

c7dca85

CurtHagenlocher approved these changes May 18, 2026

View reviewed changes

CurtHagenlocher merged commit 298c78f into apache:main May 18, 2026
19 of 20 checks passed

Conversation

davidhcoe commented Mar 11, 2026

Uh oh!

davidhcoe commented Mar 13, 2026

Uh oh!

lidavidm commented Mar 13, 2026

Uh oh!

CurtHagenlocher commented Mar 13, 2026

Uh oh!

davidhcoe commented Mar 13, 2026

Uh oh!

davidhcoe commented Mar 13, 2026

Uh oh!

davidhcoe commented May 6, 2026

Uh oh!

davidhcoe commented May 13, 2026

Uh oh!

CurtHagenlocher commented May 13, 2026

Uh oh!

CurtHagenlocher left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

CurtHagenlocher May 15, 2026

Choose a reason for hiding this comment

Uh oh!

CurtHagenlocher May 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CurtHagenlocher May 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CurtHagenlocher May 18, 2026

Choose a reason for hiding this comment

Uh oh!

CurtHagenlocher commented May 18, 2026

Uh oh!

davidhcoe commented May 18, 2026

Uh oh!

CurtHagenlocher commented May 18, 2026

Uh oh!

Uh oh!

CurtHagenlocher commented May 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants