Skip to content

feat: add api_format option to OpenAI/Azure targets (chat vs responses) #896

@christso

Description

@christso

Problem

The Vercel AI SDK v3 (@ai-sdk/openai@3.0.41) defaults to the OpenAI Responses API (/responses), but not all OpenAI-compatible endpoints support it (e.g. GitHub Models returns 404). Currently we use a URL heuristic (baseURL.includes('api.openai.com')) to decide between .chat() and default — this works but is fragile.

Proposal

Add an api_format field to OpenAI and Azure target configs in targets.yaml:

targets:
  - name: grader
    provider: openai
    api_format: chat          # "chat" (default) | "responses"
    base_url: https://models.github.ai/inference/v1
    api_key: ${{ GH_MODELS_TOKEN }}
    model: ${{ GH_MODELS_MODEL }}

  - name: openai-direct
    provider: openai
    api_format: responses     # opt in to Responses API for actual OpenAI
    api_key: ${{ OPENAI_API_KEY }}
    model: gpt-5-mini

Behavior

api_format AI SDK method Endpoint
chat (default) openai.chat(model) /chat/completions
responses openai(model) /responses

Precedent

  • promptfoo: openai:chat:model vs openai:responses:model
  • pi-ai: each model has an api field (openai-responses or openai-completions) in its model registry

Why default to chat

  • Universally supported by all OpenAI-compatible endpoints
  • Responses API is only fully supported by api.openai.com and Azure OpenAI
  • For eval grading (our primary use case), Chat Completions is sufficient
  • GitHub Models, local proxies, and other OpenAI-compatible APIs only support /chat/completions

Implementation

  1. Add api_format to BASE_TARGET_SCHEMA in targets.ts
  2. Pass it through OpenAIResolvedConfig and AzureResolvedConfig
  3. Use it in OpenAIProvider and AzureProvider constructors to select .chat() vs default
  4. Remove the URL heuristic (isOpenAI = config.baseURL.includes('api.openai.com'))

Related: #892, PR #893

Metadata

Metadata

Assignees

No one assigned

    Labels

    in-progressClaimed by an agent — do not duplicate work

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions