feat(evals): set default targets so all evals work out of the box#898

Merged

christso merged 40 commits intomainfrom

feat/default-targets

Apr 2, 2026

Commits on Apr 1, 2026

feat(evals): set default targets so all evals work out of the box

christso
and
claude
committed
feat(evals): set default targets so all evals work out of the box

christso
and
claude
committed
feat(evals): make default target env-var-driven for out-of-box evals

christso
and
claude
committed
fix(ci): use explicit include patterns instead of negated globs

christso
and
claude
committed
feat(cli): support negation patterns (!glob) in eval path resolution

christso
and
claude
committed
fix(ci): remove --targets override so per-example targets auto-discover

christso
and
claude
committed
fix: remove deprecated workspace_template from mock target configs

christso
and
claude
committed
fix(ci): add Gemini credentials to workflow .env

christso
and
claude
committed
feat(evals): add llm target and classify all evals as llm or agent

christso
and
claude
committed
fix(evals): use default (copilot) instead of pi-cli for agent evals

christso
and
claude
committed
chore(ci): increase eval workers from 1 to 3

christso
and
claude
committed
fix(ci): exclude evals with local script providers from CI

christso
and
claude
committed
fix(ci): add missing echo provider and install uv for local script evals

christso
and
claude
committed
fix(evals): make LLM eval assertions pass with generic models

christso
and
claude
committed
fix(evals): switch llm and grader targets to OpenRouter

christso
and
claude
committed
fix(evals): switch per-example grader targets from azure to root grader

christso
and
claude
committed
feat(core): add target alias support for single-env-var provider switching

christso
and
claude
committed
feat(core): add use_target for target delegation

christso
and
claude
committed
refactor(targets): use use_target for llm and grader targets

christso
and
claude
committed
refactor(core): make provider optional when use_target is set

christso
and
claude
committed
fix(core): allow provider to be omitted when use_target is set

christso
and
claude
committed
fix(core): allow use_target in targets-file.ts parser

christso
and
claude
committed
fix(ci): exclude copilot-log-eval from CI

christso
and
claude
committed
fix(cli): catch before_all failures per eval file instead of aborting

christso
and
claude
committed
fix(core): resolve use_target chains in orchestrator for grader targets

christso
and
claude
committed
fix(evals): restore workspace.template for mock agent evals

christso
and
claude
committed
fix(ci): exclude evals with pre-existing workspace/batch bugs

christso
and
claude
committed
fix(evals): fix remaining CI failures

christso
and
claude
committed
fix(ci): remove --verbose to reduce log size, make JUnit step non-fatal

christso
and
claude
committed
fix(ci): use --output instead of -o for JUnit path

christso
and
claude
committed
feat(ci): add eval results summary to GitHub Actions step summary

christso
and
claude
committed

Commits on Apr 2, 2026