erofs: share one inode per hardlink, and fix the stale mode-bit comments - #2422
Open
smoser wants to merge 2 commits into
Open
erofs: share one inode per hardlink, and fix the stale mode-bit comments#2422smoser wants to merge 2 commits into
smoser wants to merge 2 commits into
Conversation
The comment on the Chmod in emitErofsEntry described it as a workaround for a bug in the pinned go-erofs, to revisit once a release containing erofs/go-erofs#41 was out. chainguard-dev#2412 bumped the pin past that merge, so the description is wrong on both halves: probing the pinned version shows Mkdir now honours setuid/setgid/sticky, and FileInfo.Mode() reports them. The Chmod stays, for a reason that has nothing to do with chainguard-dev#41 and is not going away: Writer.Create takes no mode argument at all, so every regular file starts life 0644. Mkdir and Mknod do take one, but this hands them mode.Perm() and lets the single Chmod cover all three rather than splitting the rule across three call sites. pkg/erofsmount/ls.go still reads *erofs.Stat off Sys(), because fs.FileInfo has nowhere to put a uid or a device number, but its comment justified that with the setuid claim, which no longer holds. TestWriteErofs_SpecialModeBits now asserts FileInfo.Mode() equals Stat.Mode for every case it covers -- regular files with setuid and setgid, a sticky directory, a char device and a symlink -- which pins the half that changed. Refs chainguard-dev#2408
go-erofs grew Writer.Link, which gives a second name the same fsInode as the first and keeps nlink for us. apko was still writing every link as an independent copy, because when chainguard-dev#2249 landed there was no API for it -- SetNlink sets the reported count without sharing the inode, so it would only have made the metadata lie. A hardlink reaches the writer as an ordinary second dirent; its linkness is only in the *tar.Header from Sys(), which pkg/tarfs fills in from the apk it unpacked. hardlinkTarget reads it there, the same place the tar layer path finds it via tar.FileInfoHeader. A rootfs read back off disk (apkfs.MemFS, rwosfs) records nothing, so those keep getting a copy per name, as before. Links are held back until the walk finishes. Writer.Link needs the target to exist, and fs.WalkDir is lexicographic, so /usr/bin/[ arrives long before /usr/bin/coreutils. A link whose target is missing from the image falls back to a copy: that is the cross-layer case, where §3.7 requires materialize-or-fail and apko materializes. It cannot happen in a single-layer image, but layer splitting is what puts a link and its target in different writers. Nothing is re-applied to a shared inode -- mode, ownership, timestamps and xattrs came with it. Tests build their fixture with pkg/tarfs, because apkfs.MemFS cannot express a hardlink, and cover the shared inode and nlink, the materialize fallback against a writer that lacks the target, and Linkname cleaning. Against the previous code the three names report nlink 1 with distinct inodes, and three extra names for a 64K file grow the image by exactly 192K. Refs chainguard-dev#2408
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two loose ends from #2408 that both come out of the go-erofs bump in #2412,
neither of which belongs in the layer-splitting PR (#2418). Independent of
that one and of #2415.
1. The mode-bit comments are stale
pkg/build/erofs.godescribed theChmodafterMkdir/Mknod/Createas aworkaround for the pinned go-erofs, to revisit "once a release containing
[erofs/go-erofs#41] is out". #2412 bumped the pin past that merge, so both
halves of the claim are now wrong. Probing the pinned version:
Mkdirwith setuid/setgid/stickyFileInfo.Mode()on readThe
Chmodstill has to stay, for a reason that has nothing to do with #41and is not going away:
Writer.Createtakes no mode argument at all, so everyregular file starts life
0644.MkdirandMknoddo take one, but apkohands them
mode.Perm()and lets the singleChmodcover all three ratherthan splitting the rule across three call sites. So this is a comment change,
not a code change.
pkg/erofsmount/ls.gokeeps reading*erofs.StatoffSys()—fs.FileInfohas nowhere to put a uid or a device number — but its comment justified that
with the setuid claim, which no longer holds.
TestWriteErofs_SpecialModeBitsnow also assertsFileInfo.Mode() == Stat.Modefor every case it covers (setuid and setgid regular files, a stickydirectory, a char device, a symlink), which pins the half that changed.
2. Hardlinks point at one inode
go-erofs grew
Writer.Link(oldname, newname), which gives a second name thesame
fsInodeas the first and maintainsnlink. apko was still materializingevery hardlink as an independent copy, because when #2249 landed there was no
API for it —
SetNlinksets the reported count without sharing the inode, soit would only have made the metadata lie.
A hardlink reaches the writer as an ordinary second dirent; its linkness lives
only in the
*tar.HeaderfromSys(), whichpkg/tarfsfills in from the apkit unpacked.
hardlinkTargetreads it from there — the same place the tarlayer path finds it via
tar.FileInfoHeader. A rootfs read back off disk(
apkfs.MemFS,rwosfs) records nothing, so those keep getting a copy pername, exactly as before.
Two details worth review:
Writer.Linkneeds thetarget to already exist and
fs.WalkDiris lexicographic, so/usr/bin/[arrives long before
/usr/bin/coreutils.is the cross-layer case: §3.7 requires a hardlink spanning layers to be
materialized or the build to fail, and apko materializes. It cannot happen in
a single-layer image, but layer splitting (erofs: restore multi-layer splitting, with the bugs review found fixed #2418) is what puts a link and its
target in different writers, so the fallback is there and tested now rather
than after that lands.
Nothing is re-applied to a shared inode — mode, ownership, timestamps and
xattrs came with it.
Verification
gofmt -lclean,golangci-lint run -nreports 0 issues,go build ./...andGOOS=darwin go build ./...both succeed,SOURCE_DATE_EPOCH=0 go test ./...passes.
The behavioural tests were checked to fail without the change, by making
hardlinkTargetalways return false and re-running:196608 is exactly three more copies of the 64K fixture file. With the change
the three names share one inode, report
nlink3, andfsck.erofsaccepts theimage.
Refs #2408
🤖 Generated with Claude Code