Skip to content

feat:add_request_level_token_callback#1362

Open
fengju0213 wants to merge 11 commits intomainfrom
update_token_logic
Open

feat:add_request_level_token_callback#1362
fengju0213 wants to merge 11 commits intomainfrom
update_token_logic

Conversation

@fengju0213
Copy link
Collaborator

Related Issue

Closes #https://github.com/eigent-ai/eigent_server/issues/46

Description

Summary

This PR adapts Eigent to CAMEL request-level usage reporting for non-stream
mode by introducing a dedicated SSE event request_usage, enabling real-time
token accumulation during a step loop.

Background

Previously, Eigent consumed token usage mainly via deactivate_agent (step-
level). This is too coarse for long multi-request loops inside a single step
and can delay downstream billing/accounting actions.

What Changed

  • Added new backend action/event:
    • Action.request_usage
    • typed payload ActionRequestUsageData
  • Wired CAMEL callback in ListenChatAgent:
    • auto-detects whether CAMEL ChatAgent supports on_request_usage
    • when available, emits request_usage per model request with:
      • tokens (request total tokens)
      • request_index
      • response_id
      • step_total_tokens
      • agent/task identity fields
  • Added SSE forwarding:
    • chat_service now emits request_usage
  • Frontend adaptation:
    • added AgentStep.REQUEST_USAGE
    • chatStore consumes request_usage and calls addTokens(...)
      immediately
    • updated agent message type fields for new payload keys
  • Added dedup logic for non-stream mode:
    • when request-level reporting is enabled, non-stream
      deactivate_agent.tokens is set to 0
    • prevents double counting (request_usage + deactivate_agent)
  • Added unit test:
    • verifies request_usage increments tokens in real time
    • verifies subsequent deactivate_agent(tokens=0) does not over-count

Compatibility

  • If CAMEL version does not expose on_request_usage, Eigent falls back to
    previous step-level behavior automatically.
  • Streaming behavior remains unchanged.

Validation

  • python3 -m py_compile backend/app/agent/listen_chat_agent.py backend/app/ service/task.py backend/app/service/chat_service.py
  • npm run test -- test/unit/store/chatStore.test.ts
  • Result: passed

Scope

  • Non-stream request-level token reporting only.
  • No token-by-token streaming settlement in this PR.

Testing Evidence (REQUIRED)

  • I have included human-verified testing evidence in this PR.
  • This PR includes frontend/UI changes, and I attached screenshot(s) or screen recording(s).
  • No frontend/UI changes in this PR.

What is the purpose of this pull request?

  • Bug fix
  • New Feature
  • Documentation update
  • Other

Contribution Guidelines Acknowledgement

@fengju0213
Copy link
Collaborator Author

fengju0213 commented Feb 24, 2026

Screen.Recording.2026-02-24.at.14.47.55.mp4

The video shows that the token is updated with each tool call (request).

@fengju0213
Copy link
Collaborator Author

pending camel-ai/camel#3872

@Pakchoioioi Pakchoioioi requested review from Wendong-Fan and bytecii and removed request for Wendong-Fan and bytecii February 26, 2026 12:03
Copy link
Contributor

@Wendong-Fan Wendong-Fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems in streaming mode _stream_chunks / _astream_chunks call _send_agent_deactivate(accumulated_content, total_tokens) in their finally block with the full step token count, while request_usage events have already reported those same tokens, the frontend adds both, resulting in inflated token counts?

added pr #1439

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants