fix: add timeout to snapshot git add to handle large worktrees#18069
Open
ihubanov wants to merge 1 commit intoanomalyco:devfrom
Open
fix: add timeout to snapshot git add to handle large worktrees#18069ihubanov wants to merge 1 commit intoanomalyco:devfrom
ihubanov wants to merge 1 commit intoanomalyco:devfrom
Conversation
When opencode is used in a workspace containing large non-code files (media, datasets, ML models, etc.), the `git add .` in the snapshot system can run indefinitely — consuming excessive CPU and memory. This adds a 15-second timeout to `git add .`. If it times out, the snapshot system generates an exclude file that: - Skips known binary/large file extensions (.mp4, .zip, .onnx, etc.) - Copies the project's .gitignore rules - Scans top-level directories and excludes any over 100MB The retry `git add .` then runs with these exclusions in place. Normal-sized projects see zero overhead since the timeout is never hit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
github-actions
bot
added
needs:compliance
Contributor
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
github-actions
bot
removed
needs:issue
needs:compliance
Contributor
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes #18072
Fixes #8887
Fixes #3182
Type of change
What does this PR do?
The snapshot system runs
git add .on the full worktree at every agentic step. When the worktree contains large non-code files (media, datasets, ML models, Go/npm dependencies), this operation runs indefinitely — observed running for 3+ hours at 90% CPU, with the hourlygit gccleanup consuming 3.7GB RAM. See also #3182 (98GB snapshots from Go deps) and #8887 (snapshot ignoreswatcher.ignore).This fix wraps the
git add .call in a 15-secondEffect.timeout. If it times out:git addprocess (handled by Effect's scoped interruption)info/excludefile for the snapshot repo containing:.gitignorerulesgit add .with the exclusions in placeThe exclude generation is guarded by a flag so it runs at most once per session. Normal-sized code projects see zero overhead since the timeout is never hit.
This is complementary to the
watcher.ignoreapproach proposed in #8887 — this provides a runtime safety net for users who haven't configured ignore rules.How did you verify your code works?
Tested on a workspace containing a ~500MB media directory that previously caused the snapshot system to hang. After the fix, the first
git add .times out at 15s, excludes are generated, and the retry completes quickly. Subsequenttrack()/patch()/diff()calls work normally.Checklist