Skip to content

feat: dbt-core 1.12 compatibility, Python 3.14 support, and transaction-safety fixes - #772

Open
axellpadilla wants to merge 5 commits into
masterfrom
feat/1.12-issues-upgrade
Open

feat: dbt-core 1.12 compatibility, Python 3.14 support, and transaction-safety fixes#772
axellpadilla wants to merge 5 commits into
masterfrom
feat/1.12-issues-upgrade

Conversation

@axellpadilla

Copy link
Copy Markdown
Collaborator

Summary

Official support for dbt-core 1.12 and Python 3.14, plus two behavior-flag
defaults flipped to their new (non-legacy) values, plus the fixes those
flips exposed.

dbt-core 1.12 / Python 3.14

  • Bump dbt-core minimum to >=1.12.0, dbt-adapters to >=1.24.5.
  • Add Python 3.14 to classifiers and the CI/publish-docker matrices.
  • Drop the pathspec override in pyproject.toml (no longer needed on 1.12).
  • Add a deprecation warning for the vars.dbt_sqlserver_use_default_schema_concat
    fallback in schema.sql (use flags.* instead).

Behavior changes (both now default True; legacy False is deprecated)

  • dbt_sqlserver_use_dbt_transactions: dbt-managed transaction hooks now emit
    real BEGIN TRANSACTION / COMMIT TRANSACTION instead of no-ops, so a
    failed model rolls back its own statements instead of leaving a partial
    result on autocommit.
  • dbt_sqlserver_use_native_string_types: STRINGVARCHAR(MAX), NCHAR
    NCHAR(1), NVARCHARNVARCHAR(4000), replacing the legacy
    VARCHAR(8000)/CHAR(1) mappings as the default.

Bugfixes: making the transactions flip actually safe

Turning transactions on by default means a model's whole build now runs
inside one continuous ambient SQL Server transaction (first statement through
its own trailing adapter.commit()). Two places still assumed the old
autocommit-per-statement behavior and broke:

  • Creating an index with build_options: {online: true} or {resumable: true}
    on a fresh table, or on the rename-swap path — sqlserver__create_indexes
    never split these out of the transaction, so it hit SQL Server error 574
    (RESUMABLE cannot run inside a user transaction). Index reconciliation
    already had a creates_no_txn split, but run_query's auto_begin=false
    only skips starting a new transaction — it doesn't escape one already open.
  • The dbt_full_refresh_incomplete crash marker (full_refresh_build=prebuilt
    and incremental full refreshes) shared the same ambient transaction as the
    rebuild it exists to survive, so a real failure mid-rebuild rolled the
    marker back along with everything else — defeating its purpose.

Fixed with two new adapter primitives, commit_if_open() / begin_if_closed(),
used to flush the ambient transaction around both cases: a shared
sqlserver__create_indexes_no_txn helper for the index case, and a split of
sqlserver__create_table_as_prebuilt into two real statements (setup+marker,
then load) for the marker case.

Test infra

  • Local functional tests now default to the mssql-python backend
    (SQLSERVER_TEST_BACKEND still overrides; CI always sets this explicitly
    per matrix cell, so it's unaffected).
  • Removed the dbt_sqlserver_use_dbt_transactions: False class-level
    opt-outs from test_index_config.py / test_full_refresh_build.py — an
    intermediate stop-gap now made unnecessary by the bugfix above.
  • test_native_string_types.py / test_transactions.py updated for the new
    defaults (default case now asserts native types; legacy case renamed and
    asserts the deprecated mappings under the flag).

Test plan

  • Full tests/functional/adapter/mssql suite (135 passed, 2 skipped) and
    tests/unit (312 passed) under pyodbc.
  • test_index_config.py, test_full_refresh_build.py, test_transactions.py,
    test_xact_abort.py, tests/unit re-verified under mssql-python
    (365 passed, 1 xfailed).
  • Reproduced the two transaction bugs live against SQL Server before
    fixing (SQL error 574 on RESUMABLE; marker lost after a real mid-rebuild
    failure), confirmed both fixed after.

Closes #764, closes #662

- Bump dbt-core minimum to >=1.12.0 and dbt-adapters to >=1.24.5
- Remove pathspec override workaround (no longer needed in 1.12)
- Add Python 3.14 classifier and CI matrix entries
- Flip dbt_sqlserver_use_dbt_transactions default to True
- Flip dbt_sqlserver_use_native_string_types default to True
- Add deprecation warning for legacy var() fallback in schema.sql
- Update test transaction xfail comment for migration

Closes #764, closes #662
With dbt_sqlserver_use_dbt_transactions now defaulting to True, these
tests rely on autocommit behavior:
- full_refresh_build: marker persistence across failed runs
- index_config: RESUMABLE INDEX cannot run inside a user transaction

Explicitly set the flag to False for these test classes since they were
designed for the autocommit path. The transaction-safe equivalents will
be added in a follow-up.
…the full-refresh marker

dbt_sqlserver_use_dbt_transactions now defaults to True (1.12), so a
model's whole build - from its first statement through its own trailing
adapter.commit() - runs inside one continuous ambient transaction. Two
places still assumed the old autocommit-per-statement behavior:

- sqlserver__create_indexes (the create-path index macro) never split
  ONLINE/RESUMABLE builds out of the transaction at all, so a fresh table
  with build_options: {resumable: true} hit SQL Server error 574
  (RESUMABLE cannot run inside a user transaction). reconcile_indexes had
  a creates_no_txn split, but run_query's auto_begin=false only skips
  starting a *new* transaction - it doesn't escape one already open.

- The dbt_full_refresh_incomplete crash marker shared the same ambient
  transaction as the rebuild it's meant to survive, so a real failure
  mid-rebuild rolled the marker back right along with everything else,
  defeating its purpose.

Add adapter.commit_if_open()/begin_if_closed() primitives and use them
to flush the ambient transaction around both: a shared
sqlserver__create_indexes_no_txn helper for the index case, and a split
of sqlserver__create_table_as_prebuilt into two real statements (setup
+ marker, then load) for the marker case.

Verified against live SQL Server (pyodbc and mssql-python backends):
removed the class-level dbt_sqlserver_use_dbt_transactions: False
opt-outs from test_index_config.py/test_full_refresh_build.py, which
were an acknowledged stop-gap, and confirmed all cases pass with
transactions on their real default.
Name the changelog section and bump the version for the 1.12 line
instead of leaving it under Unreleased.
Companion to the conftest.py fallback (already committed): flip which
line is commented out in test.env.sample so a fresh devcontainer setup
matches.
@axellpadilla
axellpadilla force-pushed the feat/1.12-issues-upgrade branch from 759f7a2 to 1b39559 Compare July 27, 2026 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dbt-core 1.12 compatibility: required changes Support for python 3.14

1 participant