Add eval harness for testing AGENTS.md changes#69308

Open

RoyLee1224 wants to merge 12 commits into

apache:mainfrom

RoyLee1224:feat/skill-eval-harness

Commits on Jul 3, 2026

Add skill-eval harness scaffold with promptfoo
RoyLee1224
committed
ci: Add prek hook to remind eval on AGENTS.md and SKILL.md changes
RoyLee1224
committed
ci: add OAuth auth and runtime config generation to skill-eval harness
RoyLee1224
committed
feat: add skill-eval harness for AGENTS.md regression testing
RoyLee1224
committed
docs: update skill-eval README and use Helm routing as starter case
RoyLee1224
committed
fix: point skill-eval reminder hook at eval.py, not nonexistent eval.sh
RoyLee1224
committed
refactor: skip working arm when unchanged, show model in output
RoyLee1224
committed
refactor: skip working arm when unchanged, show model, use promptfoo@latest
RoyLee1224
committed
refactor: replace command-routing cases with newsfragment cases from real PRs
RoyLee1224
committed
fix: harden skill-eval harness and add prek hook tests
RoyLee1224
committed

Commits on Jul 4, 2026