Add macOS CI using FUSE-T and fix max_open_files overflow#528
Closed
petemoore wants to merge 7 commits intogittup:masterfrom
Closed
Add macOS CI using FUSE-T and fix max_open_files overflow#528petemoore wants to merge 7 commits intogittup:masterfrom
petemoore wants to merge 7 commits intogittup:masterfrom
Conversation
On macOS, RLIM_INFINITY is 0x7FFFFFFFFFFFFFFF. After tup_fuse_fs_init() doubles rlim_cur and divides by 2, casting the rlim_t result to int overflows, producing max_open_files = -1. Since open_count >= -1 is always true, every FUSE open immediately closes its fd and sets fh=0. With macFUSE (kernel FUSE) this is harmless — the kernel always delivers FUSE_RELEASE regardless of server-side fd state. With FUSE-T (NFS-backed FUSE), the NFS client may skip sending CLOSE for files the server already closed, causing finfo_wait_open_count() to time out with "FUSE did not appear to release all file descriptors after the sub-process closed." The fix caps the rlim_t value at INT_MAX before casting to int. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
FUSE-T is a kext-free FUSE implementation for macOS that uses an NFS v4 local server instead of a kernel extension. This makes it possible to run tup's FUSE-based test suite on GitHub-hosted macOS runners (which block kernel extensions). CI changes: - Install FUSE-T runtime and create macFUSE-compatible header symlinks - Build patched libfuse (unmount teardown fix, PR pending upstream: macos-fuse-t/libfuse#11) - Bootstrap tup and run full test suite 9 tests are skipped (deterministic FUSE-T NFS backend limitations or macOS platform issues). ~20 additional tests are flaky under CI load due to the NFS client occasionally dropping FUSE callbacks; these are retried up to 3 times to distinguish flakes from regressions. Also update macOS install docs with FUSE-T instructions as an alternative to macFUSE. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The unmount teardown fix (macos-fuse-t/libfuse#11) has landed upstream and shipped in FUSE-T 1.0.54. Remove the custom libfuse build step and rely on whatever brew install fuse-t provides. A verification step logs what files are installed so we can see what's available if the build fails. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
FUSE-T installs headers in a framework and its pkg-config file as fuse-t.pc, but tup's build expects /usr/local/include/fuse/ and fuse.pc. Add symlinks for both. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
FUSE-T's fuse-t.pc doesn't include -D_FILE_OFFSET_BITS=64 in Cflags, but the FUSE headers require it. Create a fuse.pc that adds this flag instead of symlinking directly to fuse-t.pc. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
brew install fuse-t resolved to 1.0.49 via Homebrew's JSON API cache despite the tap containing 1.0.54. Use the fully qualified cask path (macos-fuse-t/cask/fuse-t) to force reading from the local tap. We need >= 1.0.54 which includes the recv-EOF unmount fix (macos-fuse-t/libfuse#11) required for tup's FUSE operations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
FUSE-T's stock libfuse-t.dylib uses @rpath as its install name, but tup doesn't set LC_RPATH, causing dyld to fail at runtime. Use install_name_tool to set the absolute path as the install name so the linker embeds it directly in the tup binary. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2 tasks
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
max_open_fileson macOS that silently broke FUSE fd trackingmax_open_files integer overflow fix
On macOS,
RLIM_INFINITYis0x7FFFFFFFFFFFFFFF. Aftertup_fuse_fs_init()doublesrlim_curand divides by 2, casting therlim_tresult tointoverflows, producingmax_open_files = -1. Sinceopen_count >= -1is always true, every FUSE open immediately closes its fd and setsfh=0.With macFUSE (kernel FUSE) this is harmless — the kernel always delivers
FUSE_RELEASEregardless of server-side fd state. With FUSE-T (NFS-backed FUSE), the NFS client may skip sendingCLOSEfor files the server already closed, causingfinfo_wait_open_count()to time out with"FUSE did not appear to release all file descriptors".The fix caps the
rlim_tvalue atINT_MAXbefore casting toint(3 lines infuse_fs.c).Why FUSE-T?
macFUSE requires a kernel extension that:
FUSE-T uses an NFS v4 local server instead — no kernel extension needed, API-compatible with macFUSE, works on all macOS versions including Apple Silicon.
Patched libfuse dependency
FUSE-T's libfuse has two issues that affect tup:
recv()returns 0 (EOF) after unmount, treated as error instead of clean exit. Fix submitted upstream: Fix clean exit after unmount with NFS backend macos-fuse-t/libfuse#11Both fixes are on a patched fork (
petemoore/libfusebranchfix/recv-eof-and-attrcache) used by CI until merged upstream.CI details
Ubuntu (unchanged): full bootstrap + all tests pass.
macOS (new):
Test plan
max_open_filesoverflow fix verified locally and in CI🤖 Generated with Claude Code