Skip to content

Fix context window overflow: auto-recovery and proactive compaction#1973

Open
yunus25jmi1 wants to merge 1 commit intodocker:mainfrom
yunus25jmi1:fix/context-window-overflow-recovery
Open

Fix context window overflow: auto-recovery and proactive compaction#1973
yunus25jmi1 wants to merge 1 commit intodocker:mainfrom
yunus25jmi1:fix/context-window-overflow-recovery

Conversation

@yunus25jmi1
Copy link

Problem

When tool results push the conversation past the model's context limit, the agent crashes with raw HTTP errors like \prompt is too long: 226360 tokens > 200000 maximum\ and never recovers.

Fixes #1750

What this does

Prevents overflow before it happens:

  • After tool calls return, estimates how many tokens the new messages add
  • If the total would exceed 90% of the context limit, compacts automatically before the next API call

Recovers if overflow still occurs:

  • Detects overflow errors from all major providers (Anthropic, OpenAI, Google, Bedrock — 14 patterns)
  • Stops wasting retries on errors that won't go away
  • Auto-compacts the session and retries with the shorter context
  • Shows a clear message instead of raw HTTP/JSON errors

Files changed

  • \pkg/runtime/fallback.go\ — overflow detection, skip retries, error wrapping
  • \pkg/runtime/runtime.go\ — auto-recovery loop, proactive compaction, token estimation, friendly errors
  • \pkg/runtime/fallback_test.go\ — 26 new test cases
  • \pkg/runtime/runtime_test.go\ — 7 new test cases

Testing

  • All existing tests pass
  • 33 new test cases added
  • Linter: 0 issues

…compaction

When tool results push the conversation past the model's context limit,
the agent would crash with raw HTTP errors and no recovery path.

This change:
- Detects context overflow errors from all major providers (14 patterns)
- Skips pointless retries when the context is too large
- Auto-compacts and retries when overflow is caught (safety net)
- Estimates token cost of tool results and compacts before hitting the
  limit (root-cause prevention)
- Shows clear error messages instead of raw API responses

Fixes docker#1750
@yunus25jmi1 yunus25jmi1 requested a review from a team as a code owner March 6, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Context window overflow produces raw error messages and no auto-recovery

1 participant