Pre-render blog posts for LLM and search engine accessibility

[MaxGhenis]

Summary

• Adds build-time pre-rendering of all blog posts as standalone HTML files
• Middleware now detects LLM user agents (GPTBot, ChatGPT-User, Claude-Web, PerplexityBot, etc.) and serves full article content instead of the empty SPA shell
• Search engine bots (Googlebot, Bingbot, etc.) also get full pre-rendered content, improving indexing beyond just OG tags
• Social media crawlers continue to get the existing OG-tag-only response

How it works

1. Build step: generate-prerender.ts reads all articles, transforms charts/iframes to text placeholders, converts markdown to HTML, and writes /prerender/{slug}.html files
2. Middleware: Detects user agent type and routes accordingly:
   • LLM bots + search engines → full pre-rendered HTML with article content
   • Social media crawlers → OG tags only (existing behavior)
   • Browsers → SPA (existing behavior)

Closes #750

Test plan

• Verify 30 middleware tests pass (14 new tests for LLM bot detection)
• Verify pre-rendered files are generated during build
• After deploy, test with curl -H "User-Agent: GPTBot" https://policyengine.org/us/research/tanf-50-state-coverage to confirm full content is returned

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
policyengine-app-v2	Building	Preview, Comment	Feb 26, 2026 4:01am
policyengine-calculator	Building	Preview, Comment	Feb 26, 2026 4:01am