Problem
Large tool responses and too many requests in a short period of time can lead to 429 (rate limit exceeded) errors from model provider.
Feature
MCP-CLI (and underlying chuk-llm) should gracefully handle rate limiting from providers when possible.
At a low level:
Many providers include header information on responses about rate limits, or provide api's to obtain rate limits that are in effect for a user's account. This information should be taken into account when chuk-llm is making requests. Eg. Tracking token use/payload sizes to stay within limit, tracking #requests per minute, and pausing requests to stay within limits.
The UX of MCP-CLI should display. an indicator to users when rate limiting is in effect in the panel's status line.
Eg. something fun like
🐇 - No rate limiting
🐢 - Rate limiting
The above indicators could be placed next to general network connectivity status indicator, eg.
🟢 Good network connection
🔴 No network connection
🟡 Poor network connection
🟢🐇 - Good network connection, no rate limiting in effect.
🟢🐢 - Good network connection, rate limiting in effect.
Problem
Large tool responses and too many requests in a short period of time can lead to 429 (rate limit exceeded) errors from model provider.
Feature
MCP-CLI (and underlying chuk-llm) should gracefully handle rate limiting from providers when possible.
At a low level:
Many providers include header information on responses about rate limits, or provide api's to obtain rate limits that are in effect for a user's account. This information should be taken into account when chuk-llm is making requests. Eg. Tracking token use/payload sizes to stay within limit, tracking #requests per minute, and pausing requests to stay within limits.
The UX of MCP-CLI should display. an indicator to users when rate limiting is in effect in the panel's status line.
Eg. something fun like
🐇 - No rate limiting
🐢 - Rate limiting
The above indicators could be placed next to general network connectivity status indicator, eg.
🟢 Good network connection
🔴 No network connection
🟡 Poor network connection
🟢🐇 - Good network connection, no rate limiting in effect.
🟢🐢 - Good network connection, rate limiting in effect.