feat: introduce evaluation feature#518

Merged

tejaskash merged 22 commits intoaws:mainfrom

notgitika:feat/eval-support

Mar 19, 2026

Commits on Mar 19, 2026

feat: add evals control plane operations
notgitika
committed
feat: add functionality to run evaluation and online evals
notgitika
committed
fix: eval data plane code clean up
notgitika
committed
tests: add test coverage for evals
notgitika
committed
add support for running evals on agents created outside the cli
notgitika
committed
feat: add eval discovery commands, status enrichment, and schema updates
notgitika
committed
fix: add API limit of 10 on spanIds and TUI changes
notgitika
committed
feat: add evals in resourcegraph
notgitika
committed
feat: add eval TUI screens, online eval dashboard, and run eval wizard
notgitika
committed
feat: remove stop online-eval command, use remove + deploy instead
notgitika
committed
chore: remove unused get-eval-run module, add ARN mode tests for pause-resume
notgitika
committed
docs: update AGENTS.md with eval primitives, clarify sampling rate and default model
notgitika
committed
fix: bump aws-cdk-lib to 2.243.0 and remove description from UpdateOnlineEvalOptions
notgitika
committed
fix: skip requireProject for run eval in ARN mode
notgitika
committed
feat: add session discovery and selection step to run eval TUI
notgitika
committed
feat: improve command docs, rating scale clarity, ARN support messaging
notgitika
committed
feat: add eval level guidance, sampling rate context, score interpretation, placeholder docs
notgitika
committed
chore: rename eval command to evals throughout codebase
notgitika
committed
fix: require --instructions and validate placeholders for add evaluator CLI mode
notgitika
committed
fix: restore MCP/A2A code lost during rebase conflict resolution, fix dev tests
notgitika
committed
Merge branch 'main' into feat/eval-support
notgitika
authored
fix: agents.md
notgitika
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: introduce evaluation feature#518

feat: introduce evaluation feature#518
tejaskash merged 22 commits intoaws:mainfrom
notgitika:feat/eval-support

Commits on Mar 19, 2026

feat: add evals control plane operations

feat: add functionality to run evaluation and online evals

fix: eval data plane code clean up

tests: add test coverage for evals

add support for running evals on agents created outside the cli

feat: add eval discovery commands, status enrichment, and schema updates

fix: add API limit of 10 on spanIds and TUI changes

feat: add evals in resourcegraph

feat: add eval TUI screens, online eval dashboard, and run eval wizard

feat: remove stop online-eval command, use remove + deploy instead

chore: remove unused get-eval-run module, add ARN mode tests for pause-resume

docs: update AGENTS.md with eval primitives, clarify sampling rate and default model

fix: bump aws-cdk-lib to 2.243.0 and remove description from UpdateOnlineEvalOptions

fix: skip requireProject for run eval in ARN mode

feat: add session discovery and selection step to run eval TUI

feat: improve command docs, rating scale clarity, ARN support messaging

feat: add eval level guidance, sampling rate context, score interpretation, placeholder docs

chore: rename eval command to evals throughout codebase

fix: require --instructions and validate placeholders for add evaluator CLI mode

fix: restore MCP/A2A code lost during rebase conflict resolution, fix dev tests

Merge branch 'main' into feat/eval-support

fix: agents.md