Skip to content

Fix SQLite connection reuse after failed transaction commit - #2597

Merged
an-tao merged 2 commits into
drogonframework:masterfrom
mcirsta:bugfix/sqlite-failed-commit-rollback
Sep 27, 2026
Merged

an-tao merged 2 commits into
drogonframework:masterfrom
mcirsta:bugfix/sqlite-failed-commit-rollback

Conversation

@mcirsta

@mcirsta mcirsta commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

A deferred foreign-key violation can make SQLite's COMMIT fail without ending the transaction. Returning that connection to the pool lets later queries see uncommitted rows or fail when starting another transaction.

This patch runs ROLLBACK after a failed SQLite COMMIT and checks sqlite3_get_autocommit() before returning the connection to the pool. If SQLite has already rolled back automatically, the clean connection is reused without another ROLLBACK. The commit callback still receives false, after recovery finishes.

If rollback fails and the transaction remains active, cached statements are finalized and the connection is closed and retired. It is not automatically replaced: a new connection would lose application settings such as PRAGMA foreign_keys, and could lose an in-memory database. Other healthy pool connections continue working. If none remain, queued and future queries fail with BrokenConnection rather than waiting indefinitely; asynchronous transaction requests receive nullptr through their existing callback contract. Recovery then requires explicitly recreating and initializing the client.

The public API and PostgreSQL/MySQL reconnect behavior are unchanged.

The original deferred-constraint regression is retained. A separate test executable uses SQLite's authorizer to deny ROLLBACK and a commit hook to exercise automatic rollback. It covers queued queries and transactions, absent commit callbacks, connection settings, pool exhaustion, lock release and shutdown. The fault-injection hooks exist only in the test.

Validation on the PR baseline with C++17 and ASan/UBSan: the existing SQLite suite passes 135 assertions in four test cases; the new regression passes 87 assertions in three cases, including ten consecutive runs. Leak detection was disabled for this baseline because it lacks the separate cached-statement cleanup fix. The modified sources also compile with SQLite disabled. Formatting checked with clang-format 17.0.6 and cpplint.

@an-tao an-tao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix and for adding the regression test. The test case is quite helpful for reproducing the SQLite transaction issue.

[P1] Handle rollback failure before returning the connection to the pool

One potential issue I noticed is that the connection appears to be released through idleCallback regardless of whether the explicit ROLLBACK succeeds.

If the ROLLBACK itself fails, usedUpCallback() would still be invoked when the connection becomes idle, which could return a connection with an active/inconsistent transaction state back to the pool.

Would it be possible to only release the connection after a successful rollback, and handle the rollback failure by keeping the connection out of the pool (or closing it)?

It would also be helpful to have a regression test covering the rollback-failure case.

Check native transaction state before reusing a connection after COMMIT failure. If rollback leaves the transaction active, close the handle and retire the connection without silently replacing its application settings.

Fail queued and future work when no healthy connections remain. Add authorizer and commit-hook regressions for failed rollback, automatic rollback, pool exhaustion, and shutdown.
@mcirsta

mcirsta commented Sep 27, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, you're right: reaching the idle callback doesn't mean the rollback succeeded. Addressed in 78f5d44.

The connection is now returned to the pool only after sqlite3_get_autocommit() confirms that the transaction has ended. If rollback fails and it is still active, the connection is closed and kept out of the pool. SQLite's automatic-rollback case keeps the existing clean connection.

I also checked the replacement path. It opens a fresh connection without replaying application initialization; in the reproduction, PRAGMA foreign_keys changed from 1 to 0. So this fix deliberately avoids automatic replacement. Other healthy connections can keep working, but losing the last one fails queued and new operations promptly, including when no timeout is configured.

Added a standalone regression that denies ROLLBACK through SQLite's authorizer. It checks that uncommitted rows are discarded, locks are released, the unsafe connection is never reused, and queued/future work fails instead of hanging. It also covers automatic rollback, multiple connections and shutdown. No production fault-injection hooks were added.

The new regression passes ten consecutive runs, and the existing SQLite tests pass under ASan/UBSan. The SQLite-disabled build and clang-format 17/cpplint checks pass too.

@an-tao
an-tao merged commit 7433e9b into drogonframework:master Sep 27, 2026
34 checks passed
@an-tao

an-tao commented Sep 27, 2026

Copy link
Copy Markdown
Member

@mcirsta Thanks so much.

@mcirsta

mcirsta commented Sep 27, 2026

Copy link
Copy Markdown
Contributor Author

@an-tao Thank you very much for reviewing this. The Astra model did most of the job, I mostly check to see if it's not doing some crazy stuff.
I think I have some other SQLite fixes as I have used SQLite for a big project and found some small things here and there.
Great stuff though, Drogon + SQLite works fast and efficient, I've always wanted to use it but never got it till now and it exceeded my expectations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants