Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .devcontainer/setup_env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,18 @@ source .venv/bin/activate
# plus the dev dependency group (pre-commit, pytest, etc.). Groups are installed
# explicitly rather than relying on uv's default-group behaviour, which varies by version.
uv sync --all-extras --group dev

# `--all-extras` above installs the adbc-driver-manager/pyarrow *client
# library* for the adbc extra, but not the driver binary that speaks the SQL
# Server wire protocol -- that ships separately via the `dbc` CLI and is not
# on PyPI. See docs/adbc_backend.md. `dbc` installs into $VIRTUAL_ENV when a
# venv is active (it is here), so point ADBC_DRIVER_PATH there.
pip install --quiet dbc
dbc install mssql
ADBC_DRIVER_PATH="$(pwd)/.venv/etc/adbc/drivers"
if ! grep -qF 'ADBC_DRIVER_PATH' ~/.bashrc 2>/dev/null; then
echo "export ADBC_DRIVER_PATH=\"$ADBC_DRIVER_PATH\"" >> ~/.bashrc
fi
export ADBC_DRIVER_PATH

uv run pre-commit install
14 changes: 12 additions & 2 deletions .github/workflows/integration-tests-sqlserver.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,9 +97,19 @@ jobs:
sqlserver_version: "2025"
msodbc_version: "18"
collation: SQL_Latin1_General_CP1_CS_AS
# ADBC (experimental, see docs/adbc_backend.md) gets a single
# coverage row on the latest Python and SQL Server baseline.
- backend: adbc
python_version: "3.13"
sqlserver_version: "2025"
msodbc_version: "18"
collation: SQL_Latin1_General_CP1_CI_AS
runs-on: ubuntu-latest
# The ADBC backend is experimental (docs/adbc_backend.md): don't block
# the required check on it while it's still stabilizing.
continue-on-error: ${{ matrix.backend == 'adbc' }}
container:
image: ghcr.io/${{ github.repository }}:CI-${{ matrix.python_version }}-${{ matrix.backend == 'pyodbc' && format('msodbc{0}', matrix.msodbc_version) || 'mssql' }}
image: ghcr.io/${{ github.repository }}:CI-${{ matrix.python_version }}-${{ matrix.backend == 'pyodbc' && format('msodbc{0}', matrix.msodbc_version) || (matrix.backend == 'adbc' && 'adbc' || 'mssql') }}
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -121,7 +131,7 @@ jobs:

- name: Install dependencies
env:
INSTALL_EXTRA: ${{ matrix.backend == 'pyodbc' && 'pyodbc' || 'mssql' }}
INSTALL_EXTRA: ${{ matrix.backend == 'pyodbc' && 'pyodbc' || (matrix.backend == 'adbc' && 'adbc' || 'mssql') }}
run: uv pip install --system -e ".[$INSTALL_EXTRA]" --group dev

- name: Run functional tests
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish-docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
strategy:
matrix:
python_version: ["3.11", "3.12", "3.13", "3.14"]
docker_target: ["msodbc17", "msodbc18", "mssql"]
docker_target: ["msodbc17", "msodbc18", "mssql", "adbc"]
runs-on: ubuntu-latest
permissions:
contents: read
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
- Add Python 3.14 support.
- **Behavior change:** `dbt_sqlserver_use_dbt_transactions` now defaults to `True`: dbt-managed transaction hooks (begin/commit) emit real `BEGIN TRANSACTION` / `COMMIT TRANSACTION` T-SQL instead of no-ops, so a failed model rolls back its own statements instead of leaving a partial result behind on autocommit. The `False` (legacy autocommit) behavior is deprecated and will be removed in a future release.
- **Behavior change:** `dbt_sqlserver_use_native_string_types` now defaults to `True`: `STRING` maps to `VARCHAR(MAX)`, `NCHAR` to `NCHAR(1)`, and `NVARCHAR` to `NVARCHAR(4000)`, instead of the legacy `VARCHAR(8000)`/`CHAR(1)` mappings. The `False` (legacy) behavior is deprecated and will be removed in a future release.
- Add an experimental `adbc` backend (`backend: adbc`), an alternative to `pyodbc`/`mssql-python` built on [ADBC](https://arrow.apache.org/adbc/) that talks to SQL Server via `go-mssqldb` instead of an ODBC/DB-API bridge. Install with `dbt-sqlserver[adbc]` plus the separate `dbc` CLI-installed driver binary; supports SQL Server (user/password) authentication only for now. See [docs/adbc_backend.md](docs/adbc_backend.md). [#771](https://github.com/dbt-msft/dbt-sqlserver/issues/771)

#### Bugfixes

Expand Down
13 changes: 12 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ Latest pre-release: ![GitHub tag (latest SemVer pre-release)](https://img.shield
|---|---|---|
| `pyodbc` | `dbt-sqlserver[pyodbc]` or `pyodbc` | `unixodbc-dev` plus the Microsoft ODBC Driver for SQL Server |
| `mssql-python` | `dbt-sqlserver[mssql]` or `mssql-python` | `libltdl7`, `libkrb5-3`, `libgssapi-krb5-2` |
| `adbc` *(experimental)* | `dbt-sqlserver[adbc]` | none (driver binary installed separately via the `dbc` CLI) |


### `pyodbc` backend
Expand Down Expand Up @@ -118,6 +119,16 @@ your_profile:
backend: mssql-python # <-- enables this backend
```

### `adbc` backend *(experimental)*

An Arrow-native backend built on [ADBC](https://arrow.apache.org/adbc/), avoiding the row-based ODBC/DB-API bridge entirely. SQL Server authentication only (no Azure AD / Windows auth yet).

```shell
pip install -U "dbt-sqlserver[adbc]"
```

The driver binary is not on PyPI and must be installed once via the `dbc` CLI. See [docs/adbc_backend.md](docs/adbc_backend.md) for the full setup, configuration, and known-differences guide.

## Changelog

See [the changelog](CHANGELOG.md)
Expand Down Expand Up @@ -151,7 +162,7 @@ The same setting is also honoured via `vars:` for backwards compatibility; the b

### `backend`

*(default: `pyodbc`)* Set to `mssql-python` in a profile target to use the `mssql-python` backend instead of `pyodbc`. The adapter fails if the required backend package (Python dependency), such as `pyodbc` or `mssql-python`, is not installed.
*(default: `pyodbc`)* Set to `mssql-python` or `adbc` (experimental, see [docs/adbc_backend.md](docs/adbc_backend.md)) in a profile target to use that backend instead of `pyodbc`. The adapter fails if the required backend package (Python dependency), such as `pyodbc`, `mssql-python`, or `adbc-driver-manager`, is not installed.

### `dbt_sqlserver_enable_safe_type_expansion`

Expand Down
45 changes: 42 additions & 3 deletions dbt/adapters/sqlserver/sqlserver_adapter.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from typing import Any, Dict, List, Optional
import datetime as _dt
from typing import Any, Dict, List, Optional, Tuple, Union

import agate
import dbt_common.exceptions
Expand Down Expand Up @@ -37,6 +38,44 @@
logger = AdapterLogger("SQLServer")


def _normalize_result_datetimes(
result: Union[Tuple, List[Tuple], None],
) -> Union[Tuple, List[Tuple], None]:
"""Strip spurious ``tzinfo=UTC`` that ADBC attaches to SQL Server
DATETIME2 / DATETIME values.

SQL Server does **not** store timezone offsets for these types, so the
correct Python representation is a naive ``datetime``. ADBC's Arrow
backend wraps every ``timestamp`` column as ``timestamp[us, tz=UTC]``
regardless of the source semantics, producing ``datetime(…, tzinfo=UTC)``.
We revert that to match SQL Server semantics and to stay compatible with
the existing pyodbc / mssql-python backends.
"""
if result is None:
return None

if isinstance(result, tuple):
return tuple(
(v.replace(tzinfo=None) if isinstance(v, _dt.datetime) and v.tzinfo is not None else v)
for v in result
)

if isinstance(result, list):
return [
tuple(
(
v.replace(tzinfo=None)
if isinstance(v, _dt.datetime) and v.tzinfo is not None
else v
)
for v in row
)
for row in result
]

return result


class SQLServerAdapter(SQLAdapter):
"""
Controls actual implementation of adapter, and ability to override certain methods.
Expand Down Expand Up @@ -267,9 +306,9 @@ def run_sql_for_tests(self, sql, fetch, conn):
if not fetch:
conn.handle.commit()
if fetch == "one":
return cursor.fetchone()
return _normalize_result_datetimes(cursor.fetchone())
elif fetch == "all":
return cursor.fetchall()
return _normalize_result_datetimes(cursor.fetchall())
else:
return
except BaseException:
Expand Down
8 changes: 8 additions & 0 deletions dbt/adapters/sqlserver/sqlserver_auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,14 @@ def is_mssql_python_backend(backend: "SQLServerBackend") -> bool:
return backend.value == SQLSERVER_BACKEND_MSSQL_PYTHON


def is_adbc_backend(backend: "SQLServerBackend") -> bool:
"""Return whether the coerced backend enum targets ``adbc``."""

from dbt.adapters.sqlserver.sqlserver_credentials import SQLServerBackend

return backend == SQLServerBackend.adbc


def normalize_authentication_key(value: Optional[str]) -> str:
"""Normalize a SQL Server auth or lookup key for cross-layer comparisons."""

Expand Down
96 changes: 95 additions & 1 deletion dbt/adapters/sqlserver/sqlserver_backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

from __future__ import annotations

import urllib.parse
from contextlib import suppress
from typing import Any, Callable, Tuple

Expand Down Expand Up @@ -42,6 +43,7 @@
_RUNTIME_STATE,
MssqlPythonModuleProtocol,
PyodbcModuleProtocol,
_get_adbc,
)

logger = AdapterLogger("sqlserver")
Expand Down Expand Up @@ -259,12 +261,19 @@ def log_connection_string(connection_string: str) -> None:


def is_pyodbc_handle(handle: Any) -> bool:
"""Detect a pyodbc handle without importing pyodbc from the caller."""
"""Detect a pyodbc handle without importing pyodbc from the caller.

ADBC handles (from ``adbc_driver_manager``) must always return False,
even when the handle is a mock in test code.
"""

handle_type = type(handle)
module_name = getattr(handle_type, "__module__", "") or ""
class_name = getattr(handle_type, "__name__", "") or ""

if "adbc" in module_name.lower() or "adbc" in class_name.lower():
return False

if "pyodbc" in module_name or "pyodbc" in class_name:
return True

Expand Down Expand Up @@ -340,3 +349,88 @@ def _connect_pyodbc(
timeout=credentials.login_timeout,
)
return _finalize_connection_handle(handle, credentials)


def _sanitize_adbc_uri_for_logging(uri: str) -> str:
"""Redact the password component of an ADBC URI for safe logging."""
parsed = urllib.parse.urlparse(uri)
if parsed.password:
safe_netloc = parsed.netloc.replace(
f":{parsed.password}@",
":***@",
)
sanitized = parsed._replace(netloc=safe_netloc).geturl()
return sanitized
return uri


def build_adbc_connection_uri(credentials: SQLServerCredentials) -> str:
"""Build a go-mssqldb URI for the ADBC backend.

Constructs URIs in the format:
``sqlserver://[user:password@]host[:port]?query_params``

- User and password are URL-encoded so special characters (``@``, ``:``,
``/``, etc.) are escaped per RFC 3986.
- Named instances (host containing ``\\``) omit the port.
- Query parameters: ``database``, ``encrypt``, ``TrustServerCertificate``,
and optional ``connection timeout``.
"""

host = (credentials.host or "").strip()
port = credentials.port or 1433
database = credentials.database or ""
uid = credentials.UID or ""
pwd = credentials.PWD or ""

userinfo = f"{urllib.parse.quote(uid, safe='')}:{urllib.parse.quote(pwd, safe='')}"

if "\\" in host:
authority = f"{userinfo}@{host}"
else:
authority = f"{userinfo}@{host}:{port}"

query_parts = [
f"database={urllib.parse.quote(database, safe='')}",
f"encrypt={str(credentials.encrypt).lower()}",
f"TrustServerCertificate={str(credentials.trust_cert).lower()}",
]

if credentials.login_timeout and credentials.login_timeout > 0:
query_parts.append(f"connection timeout={credentials.login_timeout}")

query_string = "&".join(query_parts)
return f"sqlserver://{authority}?{query_string}"


def _connect_adbc(credentials: SQLServerCredentials, uri: str) -> Any:
"""Open an ADBC connection using the constructed go-mssqldb URI.

The ADBC mssql driver (``"mssql"``) is the only supported driver.
Connections are opened with ``autocommit=True`` so that DDL statements
(``CREATE SCHEMA``, ``CREATE TABLE``, …) auto-commit without extra
round-trips. When ``dbt_sqlserver_use_dbt_transactions: true``, dbt
emits explicit ``BEGIN TRANSACTION`` / ``COMMIT TRANSACTION`` SQL
that correctly nests on top of the autocommit session -- exactly the
same strategy used by the pyodbc backend.
"""

sanitized = _sanitize_adbc_uri_for_logging(uri)
logger.debug(f"Connecting to SQL Server with ADBC: {sanitized}")

adbc_module = _get_adbc()
return adbc_module.connect(
driver="mssql",
db_kwargs={"uri": uri},
autocommit=True,
)


def get_adbc_retryable_exceptions() -> Tuple[type[Exception], ...]:
"""Return the ADBC exception types that the connection manager may retry."""

adbc_module = _get_adbc()
return (
adbc_module.InternalError,
adbc_module.OperationalError,
)
Loading
Loading