Skip to content

Conversation

@hannesrudolph
Copy link
Collaborator

@hannesrudolph hannesrudolph commented Feb 11, 2026

Summary

Extracts duplicated cache breakpoint logic from 4 providers into a shared utility applyCacheBreakpoints(), fixing a tool message caching gap introduced by PR #11386.

Problem

Cache breakpoint placement was duplicated across 4 providers (Anthropic, Anthropic Vertex, Minimax, Bedrock) with identical targeting logic that only targeted role === "user" messages. After PR #11386, tool results became separate role: "tool" messages, creating a gap where the last message from our side (often a tool response) wouldn't get a cache breakpoint.

Solution

New shared utility in src/api/transform/cache-breakpoints.ts:

  • applyCacheBreakpoints(messages, config) — targets the last message in each non-assistant batch (user + tool), fixing the tool message gap
  • CacheBreakpointConfig interface — supports per-provider differences:
    • Anthropic/Vertex/Minimax: 2 breakpoints (default)
    • Bedrock: 3 breakpoints + anchor at ~1/3 position

Changes

Provider Before After
anthropic.ts 18-line targeting block + applyCacheControlToAiSdkMessages() method Single applyCacheBreakpoints() call
anthropic-vertex.ts 30-line block + method Single call
minimax.ts 18-line block + method Single call
bedrock.ts 42-line block + applyCachePointsToAiSdkMessages() method Single call with useAnchor: true

+244 / -184 lines (net +60, mostly new tests)

Test Coverage

  • 8 new tests in cache-breakpoints.spec.ts: empty array, single user, single tool (gap fix), multi-batch selection, same-batch dedup, anchor placement, existing providerOptions preservation, maxMessageBreakpoints=3
  • 0 regressions across 136 existing provider tests
  • 5,500 total tests pass

Key improvement

// Before (only user messages cached):
messages.reduce((acc, msg, index) => (msg.role === "user" ? [...acc, index] : acc), [])

// After (last message in each non-assistant batch cached):
// user, tool, user → caches the tool msg AND the last user msg (2 separate batches)

Important

Refactored cache breakpoint logic across multiple provider implementations by extracting common functionality into a centralized applyCacheBreakpoints utility function.

Cache Breakpoint Utility Extraction

  • New cache-breakpoints.ts module:
    • Introduced CacheBreakpointConfig interface with options for cache provider settings, maximum message breakpoints, anchor breakpoint support, and anchor threshold configuration.
    • Implemented applyCacheBreakpoints() function that identifies the last message in each non-assistant message batch and applies cache provider options to targeted messages.
    • Supports selecting the last N batches based on maxMessageBreakpoints configuration and optionally adds an anchor breakpoint at one-third position when useAnchor is enabled.
  • Provider implementations simplified:
    • Replaced inline cache control logic in anthropic-vertex.ts, anthropic.ts, bedrock.ts, and minimax.ts with calls to applyCacheBreakpoints().
    • Removed private applyCacheControlToAiSdkMessages() and applyCachePointsToAiSdkMessages() methods from all provider files.
    • Removed detailed comments and manual cache point calculation logic that is now handled by the utility function.
  • Test coverage:
    • Added comprehensive test suite in cache-breakpoints.spec.ts covering empty messages, single/multiple batches, anchor point behavior, existing provider options preservation, and Bedrock-specific configuration.

This description was created by Ellipsis for 205aa9a. You can customize this summary. It will automatically update as commits are pushed.

Create src/api/transform/cache-breakpoints.ts with applyCacheBreakpoints()
that replaces duplicated targeting+apply logic in:
- anthropic.ts (applyCacheControlToAiSdkMessages removed)
- anthropic-vertex.ts (applyCacheControlToAiSdkMessages removed)
- minimax.ts (applyCacheControlToAiSdkMessages removed)
- bedrock.ts (applyCachePointsToAiSdkMessages removed)

Key improvement: targets non-assistant batches (user + tool messages)
instead of only role=user. After PR #11386, tool results are separate
role=tool messages that now correctly receive cache breakpoints.

The shared utility supports per-provider configuration:
- Anthropic/Vertex/Minimax: 2 breakpoints (last 2 non-assistant batches)
- Bedrock: 3 breakpoints + anchor at ~1/3 for 20-block lookback coverage

8 new tests in cache-breakpoints.spec.ts, 0 regressions in provider tests.
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Feb 11, 2026
@roomote
Copy link
Contributor

roomote bot commented Feb 11, 2026

Rooviewer Clock   Follow task

Reviewing your PR now. Feedback coming shortly!

@hannesrudolph
Copy link
Collaborator Author

Closing in favor of a shared-layer approach. Instead of per-provider cache breakpoint calls, cache hints will be applied once at the message preparation layer (Task.ts) before createMessage(), letting each provider's streamText() pick up the relevant providerOptions automatically. See updated Section 13.3 of ai-sdk-migration-guide.md for the revised design.

@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Feb 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant