Skip to content

smart_routing: make provider pinning configurable #2

@bussyjd

Description

@bussyjd

Problem

smart_routing currently pins provider/model in context once selected. This is useful for tool loops but can be too sticky for reused contexts.

Current behavior

  • Pinning reuses prior selection when _smart_routing_selection exists in context.
  • A low-complexity first message can lock model/provider choice for later higher-complexity prompts if context is reused.

Proposal

Add configurable pinning mode:

  • off: never pin
  • tool_loop: pin only within a single request/tool-execution loop (recommended default)
  • request: pin for whole request context (current behavior)

Config shape:

{
  "overrides": {
    "pinningMode": "tool_loop"
  }
}

Acceptance criteria

  • Default behavior is tool_loop.
  • Existing behavior preserved when pinningMode=request.
  • pinningMode=off always re-routes per call.
  • Routing metadata reports active pinning mode and whether a response used pinned selection.

Tests

  • No pinning: provider can change between sequential calls.
  • Tool-loop pinning: provider remains stable across tool-call iterations.
  • Request pinning: current sticky behavior preserved.
  • Invalid config value falls back to default safely.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions