feat: migrate xAI provider to Responses API with reusable transform utils

[carlesso]

Related GitHub Issue

Closes: #11961

Roo Code Task Context (Optional)

N/A

Description

Migrate the xAI provider from the deprecated Chat Completions API to the Responses API, add grok-4.20 as the new default model, and introduce shared transform utilities for Responses API that other providers can adopt.

New shared utilities (src/api/transform/):

• responses-api-stream.ts: processResponsesApiStream() handles core Responses API stream events (text, reasoning, tool calls, usage) and createUsageNormalizer() provides configurable token/cost extraction. Designed for reuse by openai-native, openai-codex, or any future Responses API provider.
• responses-api-input.ts: convertToResponsesApiInput() converts directly from Anthropic message format to Responses API input format, avoiding the intermediate Chat Completions conversion step. Handles input_text, input_image, function_call, and function_call_output mappings.

xAI provider (src/api/providers/xai.ts):

• Switch from client.chat.completions.create() to client.responses.create()
• Use shared transform utilities for stream handling and input conversion
• Enable reasoning traces via include: ["reasoning.encrypted_content"]
• System prompt via instructions field, store: false for privacy
• completePrompt() also migrated to Responses API

Model updates (packages/types/src/providers/xai.ts):

• Add grok-4.20 as the new default model (2M context, $2/$6 pricing)
• Remove grok-4.20-beta-0309-reasoning and grok-4.20-beta-0309-non-reasoning

Test Procedure

Tested locally

Pre-Submission Checklist

• Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
• Scope: My changes are focused on the linked issue (one major feature/fix per PR).
• Self-Review: I have performed a thorough self-review of my code.
• Testing: New and/or updated tests have been added to cover my changes (if applicable).
• Documentation Impact: I have considered if my changes require documentation updates (see "Documentation Updates" section below).
• Contribution Guidelines: I have read and agree to the Contributor Guidelines.

Screenshots / Videos

N/A

Documentation Updates

Does this PR necessitate updates to user-facing documentation?

• No documentation updates are required.
• Yes, documentation updates are required. (Please describe what needs to be updated or link to a PR in the docs repository).

Additional Notes

N/A

Get in Touch

Discord: ecarlesso

Interactively review PR in Roo Code Cloud

[hannesrudolph]

@roomote do a full review please and leave inline comments.

[roomote-v0]

Fixaroo   See task

Fixed the reported issues. All local checks (tests, lint, type-check) passed.

_{View commit | Revert commit}

[roomote-v0]

Good migration to the Responses API overall. The shared transform utilities are a solid pattern. I found several issues that need addressing -- mostly around tool schema handling, missing metadata pass-through, and a bug in the input converter for assistant string content.

Issues found:

1. mapResponseTools bypasses base provider schema hardening (src/api/providers/xai.ts:52-64) - Doesn't use convertToolSchemaForOpenAI(), doesn't handle MCP tools, sets strict: false for all tools.

2. metadata?.tool_choice is ignored (src/api/providers/xai.ts:89) - Always uses "auto" instead of respecting caller's tool_choice. Also missing parallel_tool_calls.

3. Assistant string content uses input_text instead of output_text (src/api/transform/responses-api-input.ts:20-26) - For assistant role with string content, wraps as input_text but should use output_text / message format.

4. modelInfo parameter unused (src/api/transform/responses-api-stream.ts:113) - createUsageNormalizer accepts modelInfo but never references it.

5. Missing cacheWriteTokens extraction (src/api/transform/responses-api-stream.ts:133) - Normalizer doesn't extract cache_creation_input_tokens.

6. Pervasive any types - Both shared utilities use any extensively. Not blocking but worth noting.

7. Inconsistent test function (src/api/providers/__tests__/xai.spec.ts:85) - Uses test() while all others use it().

Will fix all of these.

[hannesrudolph]

@roomote fix it all

[roomote-v0]

Fixaroo   See task

Fixed the reported issues. Restored getModelParams for proper reasoning effort/temperature/maxTokens handling, simplified completePrompt to use output_text, fixed duplicate cacheWriteTokens property in usage normalizer, and added tests for reasoning effort with mini models. All local checks passed.

_{View commit | Revert commit}

[carlesso]

Hey @hannesrudolph could you kindly give this another go?

[hannesrudolph]

@roomote please re-review this

[roomote-v0]

Fixaroo   See task

Fixed three issues found during re-review: removed double schema processing in mapResponseTools (was calling convertToolSchemaForOpenAI twice), removed redundant stream: true in the request body, and added URL image source support in convertToResponsesApiInput. All tests and type checks pass.

_{View commit | Revert commit}

[hannesrudolph]

@roomote are there any outstanding review issues?

[roomote-v0]

Fixaroo   See task

No outstanding review issues. The original review raised 7 items (schema hardening, tool_choice pass-through, assistant string format, unused param, missing cacheWriteTokens, any types, test consistency) -- all were addressed in the follow-up commits. CI is green, all 55 tests pass, and TypeScript compiles cleanly.

[carlesso]

Manually re-tested with the latest changes, all looks great!

done