Skip to content

Codex totals are inflated when duplicate token_count events repeat unchanged total_token_usage #876

@HeMuling

Description

@HeMuling

Summary

I can reproduce inflated Codex totals with the current released CLI (@ccusage/codex 18.0.8).

On my local logs, the overcount comes from summing info.last_token_usage even when consecutive token_count events carry the same cumulative info.total_token_usage.

I checked this against my own session JSONL files and local Codex SQLite state, and PR #824 appears to be the correct fix for this specific duplicate-total pattern.

Version

  • @ccusage/codex: 18.0.8

Current released output

Running:

npx -y @ccusage/codex@18.0.8 monthly -s 2026-03-01 -u 2026-03-05 -j

reported:

  • totalTokens = 8,321,045,582

Local audit of the same closed window

I audited the matching JSONL files under:

  • ~/.codex/sessions/2026/03/01
  • ~/.codex/sessions/2026/03/02
  • ~/.codex/sessions/2026/03/03
  • ~/.codex/sessions/2026/03/04
  • ~/.codex/sessions/2026/03/05

and compared them with the corresponding thread totals in ~/.codex/state_5.sqlite.

Across that closed window on my machine:

  • Sessions: 88
  • token_count events: 66,386
  • Duplicate events with unchanged total_token_usage: 42,869
  • Current released logic (sum(last_token_usage.total_tokens)): 8,299,097,962
  • PR fix(codex): ignore duplicate token_count totals #824 logic (ignore last_token_usage when total_token_usage is unchanged): 2,899,127,807
  • SQLite threads.tokens_used: 2,899,127,807

So for this dataset, the PR #824 logic matches the local SQLite totals exactly, while the current released logic overcounts by about 2.86x.

Single-session reproduction

For this session file:

  • rollout-2026-03-04T00-00-40-019cb46e-1a1c-79f2-bc67-6551633c9e03.jsonl

I measured:

  • token_count events: 8,545
  • Duplicate events with unchanged totals: 5,703
  • Current released logic: 1,231,236,178
  • PR fix(codex): ignore duplicate token_count totals #824 logic: 411,914,227
  • sum(diff(total_token_usage.total_tokens)): 411,914,227
  • SQLite threads.tokens_used: 411,914,227

So the single-session case also lines up exactly with PR #824.

Why I think PR #824 is the right fix

This is the exact pattern in my logs:

  1. A token_count event is emitted.
  2. The next token_count event repeats the same last_token_usage.
  3. total_token_usage does not advance.
  4. The current loader still adds last_token_usage again.

PR #824 adds the missing guard for that case and makes my local totals line up with both:

  • the cumulative total_token_usage deltas in the JSONL logs
  • the persisted threads.tokens_used values in state_5.sqlite

Related PR

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions