Skip to content

Conversation

@onmax
Copy link
Contributor

@onmax onmax commented Jan 3, 2026

Fixes #3669

Problem

During development, processCollectionItems executes SQL INSERT queries one at a time in a loop. SQLite by default wraps each statement in an implicit transaction with fsync to disk. With 300+ queries, this causes ~3.8 seconds of overhead.

Solution

Wrap the SQL execution loop in an explicit transaction (BEGIN TRANSACTION / COMMIT). This batches all queries into a single transaction with one fsync at the end.

Important: D1 doesn't support transactions, so we conditionally skip the transaction wrapper for D1 databases by adding a supportsTransactions flag to the database interface.

Benchmark

Metric Before After
SQL exec (323 queries) 3864ms 51ms
Content processing total 4681ms ~1000ms

StackBlitz

Link Expected
Bug content-3669 ❌ Slow processing (~3-4s)
Fix content-3669-fixed ✅ Fast processing (~200ms)

CLI Reproduction

git clone --depth 1 --filter=blob:none --sparse https://github.com/onmax/repros.git
cd repros && git sparse-checkout set content-3669
cd content-3669 && pnpm i && pnpm dev

Verify fix

git sparse-checkout add content-3669-fixed
cd ../content-3669-fixed && pnpm i && pnpm dev

The -fixed folder uses pnpm patch to apply the fix locally.

Changes

  • Added supportsTransactions property to LocalDevelopmentDatabase interface
  • Set supportsTransactions: database.type === 'sqlite' in getLocalDatabase
  • Wrap SQL execution in transaction when supportsTransactions is true

Why this works

SQLite by default wraps each statement in an implicit transaction. When you run 323 INSERT statements individually, SQLite does 323 separate transactions (with fsync to disk each time). Wrapping them in a single explicit transaction does one fsync at the end.

Discovered in: https://github.com/nimiq/nimiq-website/tree/nuxt-content

@vercel
Copy link

vercel bot commented Jan 3, 2026

@onmax is attempting to deploy a commit to the Nuxt Team on Vercel.

A member of the Team first needs to authorize it.

@pkg-pr-new
Copy link

pkg-pr-new bot commented Jan 3, 2026

npm i https://pkg.pr.new/@nuxt/content@3670

commit: 7d714d6

@onmax onmax changed the title perf: wrap SQL queries in transaction for 75x faster dev startup perf: wrap SQL queries in transaction Jan 3, 2026
@onmax onmax force-pushed the perf/transaction-batch-sql branch 2 times, most recently from 95541ca to a1a5b3f Compare January 3, 2026 09:56
@onmax onmax force-pushed the perf/transaction-batch-sql branch from a1a5b3f to 7d714d6 Compare January 3, 2026 09:57
Copy link
Member

@farnabaz farnabaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 👍

@farnabaz farnabaz merged commit 030dc27 into nuxt:main Jan 7, 2026
4 of 5 checks passed
@onmax onmax deleted the perf/transaction-batch-sql branch January 7, 2026 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: wrap SQL queries in transaction for 75x faster dev startup

2 participants