refactor: extract shared cache breakpoint utility from 4 providers #11393
+244
−184
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Extracts duplicated cache breakpoint logic from 4 providers into a shared utility
applyCacheBreakpoints(), fixing a tool message caching gap introduced by PR #11386.Problem
Cache breakpoint placement was duplicated across 4 providers (Anthropic, Anthropic Vertex, Minimax, Bedrock) with identical targeting logic that only targeted
role === "user"messages. After PR #11386, tool results became separaterole: "tool"messages, creating a gap where the last message from our side (often a tool response) wouldn't get a cache breakpoint.Solution
New shared utility in
src/api/transform/cache-breakpoints.ts:applyCacheBreakpoints(messages, config)— targets the last message in each non-assistant batch (user + tool), fixing the tool message gapCacheBreakpointConfiginterface — supports per-provider differences:Changes
anthropic.tsapplyCacheControlToAiSdkMessages()methodapplyCacheBreakpoints()callanthropic-vertex.tsminimax.tsbedrock.tsapplyCachePointsToAiSdkMessages()methoduseAnchor: true+244 / -184 lines (net +60, mostly new tests)
Test Coverage
cache-breakpoints.spec.ts: empty array, single user, single tool (gap fix), multi-batch selection, same-batch dedup, anchor placement, existing providerOptions preservation, maxMessageBreakpoints=3Key improvement
Important
Refactored cache breakpoint logic across multiple provider implementations by extracting common functionality into a centralized
applyCacheBreakpointsutility function.Cache Breakpoint Utility Extraction
cache-breakpoints.tsmodule:CacheBreakpointConfiginterface with options for cache provider settings, maximum message breakpoints, anchor breakpoint support, and anchor threshold configuration.applyCacheBreakpoints()function that identifies the last message in each non-assistant message batch and applies cache provider options to targeted messages.maxMessageBreakpointsconfiguration and optionally adds an anchor breakpoint at one-third position whenuseAnchoris enabled.anthropic-vertex.ts,anthropic.ts,bedrock.ts, andminimax.tswith calls toapplyCacheBreakpoints().applyCacheControlToAiSdkMessages()andapplyCachePointsToAiSdkMessages()methods from all provider files.cache-breakpoints.spec.tscovering empty messages, single/multiple batches, anchor point behavior, existing provider options preservation, and Bedrock-specific configuration.This description was created by
for 205aa9a. You can customize this summary. It will automatically update as commits are pushed.