feat(eval): add read-only Eval TUI - #1920
Merged
Merged
Conversation
Wire the interactive TUI over the existing eval command tree for the read-only surface: evaluator and online-eval menus, paginated lists, and resource detail hubs (with a raw JSON view). Mutating commands (create/update/delete/pause/resume) stay CLI-only for now. - New pickers: EvaluatorPicker, OnlineEvalPicker (over the existing listEvaluators / listOnlineEvaluationConfigs Core calls). - New screens: eval / evaluator / online-eval menus, list + get(+json). get hubs reuse the shared ResourceDetailScreen; evaluator kind is derived from the evaluatorConfig union (no type field on the get response), online-eval sampling reads rule.samplingConfig. - RouterScreen gains an `omit` prop so the eval menus hide the mutating subcommands. Without it those items would fall through to the HelpScreen catch-all, which exits the app. - eval routers swap createHelpDefault for withTuiOnEmptyFlagsAndArgs + renderTui, matching harness/runtime/memory. Bare read leaves now open the TUI instead of erroring on a missing required flag; existing leaf tests assert the headless path under --json, matching the memory read-only TUI convention.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## refactor #1920 +/- ##
============================================
+ Coverage 96.17% 96.28% +0.10%
============================================
Files 236 245 +9
Lines 11778 12105 +327
============================================
+ Hits 11328 11655 +327
Misses 450 450 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
jariy17
marked this pull request as ready for review
August 5, 2026 17:38
AlexanderRichey
previously approved these changes
Aug 5, 2026
aidandaly24
previously approved these changes
Aug 5, 2026
…y-tui # Conflicts: # src/components/Root.tsx
jariy17
dismissed stale reviews from aidandaly24 and AlexanderRichey
via
August 5, 2026 19:38
4c0712c
aidandaly24
approved these changes
Aug 5, 2026
AlexanderRichey
approved these changes
Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds the interactive TUI over the existing
evalcommand tree, scoped to read-only screens. It opens a working menu → list → detail → JSON flow, matchingruntime/memory.Screens
eval,eval evaluator,eval online-evalRouterScreenmenuseval evaluator list/online-eval listPaginatedTablePickereval evaluator get [/json]ResourceDetailScreenhub →JsonDetaileval online-eval get [/json]ResourceDetailScreenhub →JsonDetailNotes for reviewers
ResourceDetailScreenfrom refactor(tui): share resource detail screen #1890,PaginatedTablePicker,JsonDetail,RouterScreen). Only genuinely new code is two pickers + four screen files + three thin menu screens.RouterScreengains anomit?: string[]prop. The eval command tree still contains the mutation commands, andRouterScreenbuilds its menu from that tree. Without filtering, a mutation item in the menu would route to nothing and fall through to the*→HelpScreencatch-all, which callsexit()and tears down the app.omithides them from the menu; they remain usable from the CLI. Backwards-compatible (defaults undefined; existing menus unchanged). Read-only,llm-as-a-judge/code-basedhave onlycreate/updatechildren, so those submenus are omitted too —evaluatorshows justget/list.kind(LLM-as-a-Judge vs code-based) is derived from which arm of theevaluatorConfigunion is populated — theGetEvaluatorresponse has no type field of its own (unlike the list summary'sevaluatorType).rule.samplingConfig.samplingPercentage; sampling/evaluator-count are only on thegetresponse, not the list summary, so they live on the detail screen.Behaviour change
Mounting
withTuiOnEmptyFlagsAndArgsmeans a bare read command with no flags (e.g.eval evaluator get) now opens the TUI instead of erroringrequired option '--id' not specified. This matchesruntime/memory. Existing leaf tests were updated to assert the headless path under--json(same convention as the memory read-only TUI), plus new tests assert bare read leaves open the TUI.Testing
evaluator.screen.test.tsx,online-eval.screen.test.tsx— menus omit mutations, lists render/paginate/select, detail hubs render + route to JSON, retry, empty states.--jsonheadless contract.Plan docs: read-only spec and Phase 2 (mutations) spec are tracked separately.