refactor: extract shared prompt cache breakpoint layer from 4 providers #11415
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Moves cache breakpoint placement from individual providers to a shared utility called from
Task.tsbeforecreateMessage(). Messages arrive at providers pre-annotated withproviderOptions, and the AI SDK routes the correct options to the active provider automatically.Architecture
The AI SDK's
providerOptionsnamespacing ensures each provider only picks up its own cache hints —anthropic.cacheControlfor Anthropic family,bedrock.cachePointfor Bedrock.New Files
src/api/transform/prompt-cache.tsresolveCacheProviderOptions()— maps provider name → cache config;applyCacheBreakpoints()— annotates last N non-assistant batchessrc/api/transform/__tests__/prompt-cache.spec.tsProvider Changes
anthropic.tsapplyCacheControlToAiSdkMessages()systemProviderOptionsinstreamText()anthropic-vertex.tsapplyCacheControlToAiSdkMessages()systemProviderOptionsinstreamText()minimax.tsapplyCacheControlToAiSdkMessages()systemProviderOptionsinstreamText()bedrock.tsapplyCachePointsToAiSdkMessages()systemProviderOptions,usePromptCache,supportsAwsPromptCache()Key Improvements
Fixes tool message caching gap — Targets non-assistant batches (
role: user+role: tool) instead of onlyrole: user. After PR feat: implement ModelMessage storage layer with AI SDK response messages #11409, tool results are separaterole: toolmessages that now correctly receive cache breakpoints.Single source of truth — Cache strategy defined once in
prompt-cache.tswith provider adapter mapping.Provider-specific config preserved — Bedrock: 3 breakpoints + anchor at ~1/3 for 20-block lookback coverage. Anthropic family: 2 breakpoints. Bedrock opt-in via
awsUsePromptCache.Not Changed
systemProviderOptionsin all 4 providerspromptCacheRetention(provider-level option, not per-message)Stats
+426 / -216 lines • 14 new tests • 5,491 total tests pass • 0 regressions