Add macOS CI using FUSE-T and fix max_open_files overflow

[petemoore]

Summary

• Fix integer overflow in max_open_files on macOS that silently broke FUSE fd tracking
• Add macOS CI job using FUSE-T (kext-free FUSE)
• Update macOS install docs with FUSE-T instructions

max_open_files integer overflow fix

On macOS, RLIM_INFINITY is 0x7FFFFFFFFFFFFFFF. After tup_fuse_fs_init() doubles rlim_cur and divides by 2, casting the rlim_t result to int overflows, producing max_open_files = -1. Since open_count >= -1 is always true, every FUSE open immediately closes its fd and sets fh=0.

With macFUSE (kernel FUSE) this is harmless — the kernel always delivers FUSE_RELEASE regardless of server-side fd state. With FUSE-T (NFS-backed FUSE), the NFS client may skip sending CLOSE for files the server already closed, causing finfo_wait_open_count() to time out with "FUSE did not appear to release all file descriptors".

The fix caps the rlim_t value at INT_MAX before casting to int (3 lines in fuse_fs.c).

Why FUSE-T?

macFUSE requires a kernel extension that:

• Can't be loaded on GitHub-hosted CI runners (Apple blocks System Extensions)
• Has been removed from Homebrew due to its closed-source kext
• Requires manual approval in System Settings > Privacy & Security

FUSE-T uses an NFS v4 local server instead — no kernel extension needed, API-compatible with macFUSE, works on all macOS versions including Apple Silicon.

Patched libfuse dependency

FUSE-T's libfuse has two issues that affect tup:

1. Unmount teardown — recv() returns 0 (EOF) after unmount, treated as error instead of clean exit. Fix submitted upstream: Fix clean exit after unmount with NFS backend macos-fuse-t/libfuse#11
2. NFS attribute cache — stale file metadata after rapid rename/stat sequences, causing missed dependencies.

Both fixes are on a patched fork (petemoore/libfuse branch fix/recv-eof-and-attrcache) used by CI until merged upstream.

CI details

Ubuntu (unchanged): full bootstrap + all tests pass.

macOS (new):

• Installs FUSE-T, creates macFUSE-compatible header symlinks
• Builds patched libfuse from source
• Full bootstrap (tup builds itself using FUSE-T)
• 9 tests skipped: 7 deterministic FUSE-T NFS backend limitations + 2 macOS platform issues (also fail with macFUSE)
• ~20 additional tests are flaky under CI load (NFS client occasionally drops FUSE callbacks); these are retried up to 3 times

Test plan

• max_open_files overflow fix verified locally and in CI
• t3083/t6082 confirmed as macOS platform issues (fail with macFUSE too)
• CI passes on both Ubuntu and macOS
• Retry mechanism tested: flaky test failed on first attempt, passed on retry

🤖 Generated with Claude Code