Prewarm tiktoken cache to avoid Foundry egress

[anand-testcompare]

Why\nFoundry compute modules run with restricted outbound network. At runtime, tiktoken (via litellm) attempts to download the cl100k_base encoding from openaipublic.blob.core.windows.net, causing DNS/egress failures and 500s on classification routes.\n\n## What changed\n- Pre-warm the cl100k_base encoding during image build and copy it into the runtime image.\n- Set TIKTOKEN_CACHE_DIR in the runtime container to ensure the cached encoding is used.\n\n## Notes\nThis removes the need to allow egress to openaipublic.blob.core.windows.net just for tokenizer initialization.

Summary by CodeRabbit

• Chores
  • Pre-warmed token encoding cache during image build to speed application startup.
  • Exposed cache directory via environment variable so runtime can locate pre-warmed cache.
  • Included pre-warmed cache in the runtime image and ensured proper ownership and permissions.
  • Added an additional runtime cache location to further improve performance and reliability.

[railway-app]

🚅 Deployed to the dspy-reference-examples-pr-18 environment in dspy-reference-example

Service	Status	Web	Updated (UTC)
dspy-reference-examples	✅ Success (View Logs)	Web	Feb 15, 2026 at 2:56 am

[coderabbitai]

Walkthrough

The Dockerfile now pre-warms the tiktoken encoding cache (initializes cl100k_base) during the builder stage and creates a dspy cache; those cache directories are copied into the runtime image with TIKTOKEN_CACHE_DIR/PYTHONPATH/PATH adjustments and ownership set for the application user.

Changes

Cohort / File(s)	Summary
Docker Build Cache Optimization Dockerfile	Added pre-warm step to initialize cl100k_base and create /app/.tiktoken_cache in the builder. Exposed TIKTOKEN_CACHE_DIR and adjusted PYTHONPATH/PATH. Copied /app/.tiktoken_cache (and created /app/data/.dspy_cache) into the runtime image and ensured correct ownership for the app user.

Poem

🐰 Whiskers twitch at build-time light,

Caches nestle warm and tight,
cl100k hums its tune,
dspy naps beneath the moon,
Ready hops for speedy flight!

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Prewarm tiktoken cache to avoid Foundry egress' directly and clearly summarizes the main change—pre-warming the tiktoken cache to prevent network egress issues in Foundry environments.
Merge Conflict Detection	✅ Passed	✅ No merge conflicts detected when merging into main

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix/foundry-runtime-network

---

No actionable comments were generated in the recent review. 🎉

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@Dockerfile`:
- Around line 19-22: The Dockerfile pre-warm step uses the system python which
lacks tiktoken; change the command to run the virtualenv interpreter under
/app/.venv so TIKTOKEN_CACHE_DIR=/app/.tiktoken_cache python -c 'import
tiktoken; tiktoken.get_encoding("cl100k_base")' uses /app/.venv/bin/python
instead, ensuring the tiktoken import and tiktoken.get_encoding call succeed
during build.

🧹 Nitpick comments (1)

Dockerfile (1)

42-45: Minor: Redundant directory creation for .tiktoken_cache.

The /app/.tiktoken_cache directory is already copied from the builder on line 42 with correct ownership (--chown=5000:5000), so creating it again on line 45 is unnecessary.

♻️ Proposed cleanup

 COPY --from=builder --chown=5000:5000 /app/.tiktoken_cache /app/.tiktoken_cache
 COPY --chown=5000:5000 pyproject.toml uv.lock README.md ./

-RUN mkdir -p /app/data/.dspy_cache /app/.tiktoken_cache && chown -R 5000:5000 /app
+RUN mkdir -p /app/data/.dspy_cache && chown -R 5000:5000 /app/data