Skip to content

fix(tag): make tag creation atomic - #8459

Open
dentiny wants to merge 4 commits into
lance-format:mainfrom
dentiny:hjiang/fix-concurrent-tag-creation
Open

fix(tag): make tag creation atomic#8459
dentiny wants to merge 4 commits into
lance-format:mainfrom
dentiny:hjiang/fix-concurrent-tag-creation

Conversation

@dentiny

@dentiny dentiny commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Closes #8460

Hi team, I found for tag creation, in the current implementation, we use two IO operations: check existence first then create, which could lead to data race and tag loss under concurrent accesses.

In this PR, I updated the operation to leverage PutOption to achieve put-if-absent semantics.

@github-actions github-actions Bot added the bug Something isn't working label Aug 11, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: request changes.

The atomic-create direction is correct, but acceptance requires ref creation to fail closed on every supported backend. The current direct conditional PUT regresses unreliable backends from rejecting an ordinary duplicate to silently overwriting it.

Use a backend-aware conditional-publish abstraction and reject unsupported or unreliable stores, or coordinate them with a ref-name-keyed distributed lock.

Comment thread rust/lance/src/dataset/refs.rs Outdated
@lance-gatekeeper lance-gatekeeper Bot added the K-changes Latest Gatekeeper recommendation requests changes. label Aug 11, 2026
@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions github-actions Bot added A-deps Dependency updates A-encoding Encoding, IO, file reader/writer labels Aug 11, 2026
@lance-gatekeeper lance-gatekeeper Bot removed the K-changes Latest Gatekeeper recommendation requests changes. label Aug 11, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: request changes.

1 fixed / 1 new. The backend-aware atomic-create revision resolves the COS overwrite risk, but the new direct dependency leaves the excluded Python and Java lockfiles stale. Regenerate both lockfiles so every supported locked build uses the current lance-io dependency graph.

Comment thread rust/lance-io/Cargo.toml
@lance-gatekeeper lance-gatekeeper Bot added the K-changes Latest Gatekeeper recommendation requests changes. label Aug 11, 2026
@github-actions github-actions Bot added A-python Python bindings A-java Java bindings + JNI labels Aug 11, 2026
@lance-gatekeeper lance-gatekeeper Bot removed the K-changes Latest Gatekeeper recommendation requests changes. label Aug 11, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: approve.

2 fixed / 0 remain. The backend-aware create path now fails closed on COS and preserves atomic ref publication on supported stores; both excluded binding lockfiles are synchronized with the new dependency. This keeps the fix at the storage boundary without adding distributed coordination.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 11, 2026
@dentiny

dentiny commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

I'm not sure on the PR page it shows failure, but all CI pipelines seem to pass.
image

@lance-gatekeeper lance-gatekeeper Bot added K-approved Latest Gatekeeper recommendation permits acceptance. and removed K-approved Latest Gatekeeper recommendation permits acceptance. labels Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-deps Dependency updates A-encoding Encoding, IO, file reader/writer A-java Java bindings + JNI A-python Python bindings bug Something isn't working K-approved Latest Gatekeeper recommendation permits acceptance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tag creation is not atomic

1 participant