feat(ci): add GitHub Actions workflow to run evals#893

Merged

christso merged 21 commits intomainfrom

feat/892-ci-evals

Apr 1, 2026

Commits on Apr 1, 2026

feat(ci): add GitHub Actions workflow to run evals

christso
and
claude
committed
fix(ci): use GH_MODELS_TOKEN with default OpenAI target

christso
and
claude
committed
feat(ci): add GH Models default target with copilot-cli/copilot-sdk

christso
and
claude
committed
refactor: consolidate targets.yaml to root .agentv/

christso
and
claude
committed
fix(ci): use bunx agentv instead of dist path

christso
and
claude
committed
feat(ci): use COPILOT_MODEL env var for copilot targets

christso
and
claude
committed
refactor: use OPENROUTER_MODEL env var for pi targets

christso
and
claude
committed
refactor: remove duplicate azure target, keep azure-llm

christso
and
claude
committed
fix(ci): use gpt-5-mini without openai/ prefix for GitHub Models

christso
and
claude
committed
fix(ci): use GITHUB_TOKEN directly for GitHub Models

christso
and
claude
committed
fix(ci): restore GH_MODELS_TOKEN with GITHUB_TOKEN fallback

christso
and
claude
committed
fix(core): use chat completions API for non-OpenAI endpoints

christso
and
claude
committed
fix(ci): add models:read permission for GitHub Models API

christso
and
claude
committed
fix(ci): prefer COPILOT_PAT for GitHub Models token

christso
and
claude
committed
fix(ci): use bun apps/cli/dist/cli.js instead of bunx

christso
and
claude
committed
feat(ci): expand eval patterns to include examples/features evals

christso
and
claude
committed
refactor(targets): add dedicated grader target, run CI with copilot-cli

christso
and
claude
committed
fix(ci): force root targets.yaml to override per-eval local targets

christso
and
claude
committed
fix(ci): narrow eval pattern to exclude csv-analyzer (no workspace template)

christso
and
claude
committed
refactor(ci): use comma-separated eval patterns instead of space-separated

christso
and
claude
committed
refactor(openai): default to Chat Completions, opt in to Responses API

christso
and
claude
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(ci): add GitHub Actions workflow to run evals#893

feat(ci): add GitHub Actions workflow to run evals#893
christso merged 21 commits intomainfrom
feat/892-ci-evals

Commits on Apr 1, 2026

feat(ci): add GitHub Actions workflow to run evals

fix(ci): use GH_MODELS_TOKEN with default OpenAI target

feat(ci): add GH Models default target with copilot-cli/copilot-sdk

refactor: consolidate targets.yaml to root .agentv/

fix(ci): use bunx agentv instead of dist path

feat(ci): use COPILOT_MODEL env var for copilot targets

refactor: use OPENROUTER_MODEL env var for pi targets

refactor: remove duplicate azure target, keep azure-llm

fix(ci): use gpt-5-mini without openai/ prefix for GitHub Models

fix(ci): use GITHUB_TOKEN directly for GitHub Models

fix(ci): restore GH_MODELS_TOKEN with GITHUB_TOKEN fallback

fix(core): use chat completions API for non-OpenAI endpoints

fix(ci): add models:read permission for GitHub Models API

fix(ci): prefer COPILOT_PAT for GitHub Models token

fix(ci): use bun apps/cli/dist/cli.js instead of bunx

feat(ci): expand eval patterns to include examples/features evals

refactor(targets): add dedicated grader target, run CI with copilot-cli

fix(ci): force root targets.yaml to override per-eval local targets

fix(ci): narrow eval pattern to exclude csv-analyzer (no workspace template)

refactor(ci): use comma-separated eval patterns instead of space-separated

refactor(openai): default to Chat Completions, opt in to Responses API