Problem
smart_routing currently pins provider/model in context once selected. This is useful for tool loops but can be too sticky for reused contexts.
Current behavior
- Pinning reuses prior selection when
_smart_routing_selection exists in context.
- A low-complexity first message can lock model/provider choice for later higher-complexity prompts if context is reused.
Proposal
Add configurable pinning mode:
off: never pin
tool_loop: pin only within a single request/tool-execution loop (recommended default)
request: pin for whole request context (current behavior)
Config shape:
{
"overrides": {
"pinningMode": "tool_loop"
}
}
Acceptance criteria
- Default behavior is
tool_loop.
- Existing behavior preserved when
pinningMode=request.
pinningMode=off always re-routes per call.
- Routing metadata reports active pinning mode and whether a response used pinned selection.
Tests
- No pinning: provider can change between sequential calls.
- Tool-loop pinning: provider remains stable across tool-call iterations.
- Request pinning: current sticky behavior preserved.
- Invalid config value falls back to default safely.
Problem
smart_routingcurrently pins provider/model in context once selected. This is useful for tool loops but can be too sticky for reused contexts.Current behavior
_smart_routing_selectionexists in context.Proposal
Add configurable pinning mode:
off: never pintool_loop: pin only within a single request/tool-execution loop (recommended default)request: pin for whole request context (current behavior)Config shape:
{ "overrides": { "pinningMode": "tool_loop" } }Acceptance criteria
tool_loop.pinningMode=request.pinningMode=offalways re-routes per call.Tests