Skip to content

feat(context engine): Index scheduling job changes#111035

Open
Mihir-Mavalankar wants to merge 2 commits intomasterfrom
mihir/fix/context-engine-cache-snapshot-dispatch
Open

feat(context engine): Index scheduling job changes#111035
Mihir-Mavalankar wants to merge 2 commits intomasterfrom
mihir/fix/context-engine-cache-snapshot-dispatch

Conversation

@Mihir-Mavalankar
Copy link
Contributor

@Mihir-Mavalankar Mihir-Mavalankar commented Mar 18, 2026

PR Details

  • Change context engine indexing frequency from weekly (168 slots) to daily (24 slots) so each org is indexed once per day instead of once per week, using the same md5 hash-based slot assignment
  • Known caveats - if Redis loses the cache key, or task fails, newly enabled orgs silently fall back to waiting for their hash slot (up to ~23 hours) rather than being indexed immediately — acceptable risk since tasks are idempotent and Redis is stable across deploys.
  • Worst case a new org might have to wait upto 23 hours if the cache check fails for whatever reason .

Mihir-Mavalankar and others added 2 commits March 18, 2026 13:55
Track context-engine enabled org snapshots separately from scheduled dispatches
so newly-enabled orgs are detected correctly across hourly runs. This prevents
new orgs from being dropped back to weekly scheduling because of cache state.

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Reduce TOTAL_HOURLY_SLOTS from 168 (weekly) to 24 (daily) so each org
is indexed once per day instead of once per week. Fix tests to properly
unpack the tuple return value from get_allowed_org_ids_context_engine_indexing.

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
@Mihir-Mavalankar Mihir-Mavalankar self-assigned this Mar 18, 2026
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Mar 18, 2026
@Mihir-Mavalankar Mihir-Mavalankar changed the title Mihir/fix/context engine cache snapshot dispatch feat(context engine): Index scheduling job changes Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant