Skip to content

Add experimental ADBC backend - #783

Open
axellpadilla wants to merge 1 commit into
masterfrom
feat/771-sqlserver-adbc-backend
Open

Add experimental ADBC backend#783
axellpadilla wants to merge 1 commit into
masterfrom
feat/771-sqlserver-adbc-backend

Conversation

@axellpadilla

@axellpadilla axellpadilla commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds an experimental adbc backend (backend: adbc in a profile target) as a third connection path alongside pyodbc and mssql-python. ADBC (Arrow Database Connectivity) talks to SQL Server via go-mssqldb instead of an ODBC/DB-API bridge, avoiding row-based marshalling overhead.

  • New dbt-sqlserver[adbc] extra (adbc-driver-manager, pyarrow); the driver binary itself ships separately via the dbc CLI (not on PyPI). Full setup/config/troubleshooting guide in docs/adbc_backend.md; README.md gets the new backend row and a matching setup section.
  • SQL Server (user/password) authentication only for now — Azure AD and Windows auth return a clear "not yet supported" error.
  • Query placeholders are rewritten from PEP 249 ? to T-SQL @pN (go-mssqldb doesn't support qmark binding), skipping ? characters that appear inside bracket-quoted identifiers or string literals so a literal ? never misaligns a binding.
  • AdapterResponse.rows_affected is recovered via a scoped @@ROWCOUNT query, since ADBC's own cursor.rowcount is always -1: only run when the statement is DML and has no pending result set to fetch, so it never races an unfetched SELECT on the same connection (no MARS support) and isn't paid for on DDL/SELECT/transaction-control statements. Other backends' rowcount values are untouched.
  • Arrow DataType/type-code column metadata is mapped to the same SQL Server type names pyodbc/mssql-python report; an unrecognized type code raises instead of silently mistyping a column.
  • cursor.nextset() (unsupported by the ADBC driver) is drained safely by checking the driver's exception type rather than pyodbc-style return values.

CI / tooling

  • New adbc Docker CI image stage (installs the dbc driver-manager CLI + mssql driver binary) and matrix entry in publish-docker.yml.
  • New backend: adbc row in integration-tests-sqlserver.yml on Python 3.13 / SQL Server 2025 (latest), marked continue-on-error since the backend is still experimental.
  • devcontainer (setup_env.sh) installs the ADBC driver binary and sets ADBC_DRIVER_PATH automatically on container start.

Closes #771

Test plan

  • pytest tests/unit — 400 passed
  • pytest tests/functional against a live SQL Server with backend: adbc — 320 passed, 48 skipped, 2 xfailed, 0 failed
  • pre-commit run --all-files — clean (ruff check/format, mypy, etc.)
  • publish-docker.yml only rebuilds CI images on push to master, so the new CI-3.13-adbc image won't exist until that runs once after merge — the new integration-test row will fail-but-not-block until then (or trigger it manually via workflow_dispatch).

Adds an experimental `adbc` backend (`backend: adbc`) as a third
connection path alongside pyodbc and mssql-python, talking to SQL
Server via go-mssqldb instead of an ODBC/DB-API bridge.

- New dbt-sqlserver[adbc] extra; driver binary installed separately
  via the `dbc` CLI (not on PyPI). Full guide in docs/adbc_backend.md.
- SQL Server (user/password) authentication only for now.
- ? -> @pn placeholder rewriting, quote/bracket-aware so a literal ?
  in an identifier or string literal never misaligns a binding.
- AdapterResponse.rows_affected recovered via a scoped @@rowcount
  query (ADBC's own cursor.rowcount is always -1), gated to DML
  statements with no pending result set so it never races an
  unfetched SELECT (no MARS support) and isn't paid for on
  DDL/SELECT/transaction-control statements.
- Arrow type-code/DataType column metadata mapped to the same SQL
  Server type names as pyodbc/mssql-python; unrecognized codes raise
  instead of silently mistyping a column.
- cursor.nextset() (unsupported by the driver) drained safely by
  checking the driver's exception type.

CI: new adbc Docker image stage + publish-docker.yml matrix entry,
new backend: adbc row in integration-tests-sqlserver.yml (Python
3.13 / SQL Server 2025, continue-on-error while experimental), and
devcontainer setup now installs the driver binary automatically.

Closes #771
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Explore an experimental SQL Server ADBC backend

1 participant