Skip to content

smart_routing: make provider pinning configurable #2

@bussyjd

Description

@bussyjd

Problem

smart_routing currently pins provider/model in context once selected. This is useful for tool loops but can be too sticky for reused contexts.

Current behavior

  • Pinning reuses prior selection when _smart_routing_selection exists in context.
  • A low-complexity first message can lock model/provider choice for later higher-complexity prompts if context is reused.

Proposal

Add configurable pinning mode:

  • off: never pin
  • tool_loop: pin only within a single request/tool-execution loop (recommended default)
  • request: pin for whole request context (current behavior)

Config shape:

{
  "overrides": {
    "pinningMode": "tool_loop"
  }
}

Acceptance criteria

  • Default behavior is tool_loop.
  • Existing behavior preserved when pinningMode=request.
  • pinningMode=off always re-routes per call.
  • Routing metadata reports active pinning mode and whether a response used pinned selection.

Tests

  • No pinning: provider can change between sequential calls.
  • Tool-loop pinning: provider remains stable across tool-call iterations.
  • Request pinning: current sticky behavior preserved.
  • Invalid config value falls back to default safely.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions