feat: add OpenRouter quantization filter to exclude low-bit providers #11366
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related GitHub Issue
Closes: #11325
Description
This PR attempts to address Issue #11325. Feedback and guidance are welcome.
When using OpenRouter with CJK languages (Korean, Chinese, Japanese), low-bit quantization providers (FP4/FP6/Int4) can produce broken Unicode escape sequences instead of readable text. This adds an opt-in checkbox to filter out those providers.
Implementation details:
Schema (
packages/types/src/provider-settings.ts): AddedopenRouterExcludeLowQuantization: z.boolean().optional()to theopenRouterSchema.Handler (
src/api/providers/openrouter.ts): Extracted provider options logic into abuildProviderOptions()method that combines both specific provider routing and quantization filtering. When the setting is enabled,provider.quantizationsis set to["fp16", "bf16", "fp8", "int8"], restricting to higher-precision providers. This is applied to bothcreateMessage()andcompletePrompt().UI (
webview-ui/src/components/settings/providers/OpenRouter.tsx): Added a checkbox labeled "Exclude low-bit quantization (FP4/FP6/Int4)" with a description explaining the CJK encoding fix.i18n (
webview-ui/src/i18n/locales/en/settings.json): Added English translations for the new UI labels.Test Procedure
src/api/providers/__tests__/openrouter.spec.ts:completePrompt()as wellcd src && npx vitest run api/providers/__tests__/openrouter.spec.tsPre-Submission Checklist
Documentation Updates
Additional Notes
The setting defaults to off, preserving existing behavior. When enabled, it uses the OpenRouter documented
provider.quantizationsAPI field to restrict routing to higher-precision providers only.Important
Adds a feature to filter out low-bit quantization providers in OpenRouter settings, with schema, handler, UI, and i18n updates.
openRouterExcludeLowQuantizationtoopenRouterSchemainprovider-settings.tsto filter out low-bit quantization providers.OpenRouterHandlerinopenrouter.tsto include quantization filtering inbuildProviderOptions().OpenRouter.tsxto enable/disable the quantization filter.openrouter.spec.tsto verify quantization filtering behavior when enabled/disabled and its combination with specific provider routing.settings.json.This description was created by
for 0bb4567. You can customize this summary. It will automatically update as commits are pushed.