Add ui watch command for UI event streaming#602
Open
yeelam-gordon wants to merge 1 commit into
Open
Conversation
Adds a new `winapp ui watch` command that listens for UIA / WinEvent notifications from a running app and streams them as they occur, with JSON/NDJSON output for tooling and bounded capture via event filters, duration, and max-event limits. Also updates the generated CLI schema and UI automation skill/docs to surface the new command. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a new winapp ui watch subcommand that streams live UI events (focus, window open/close, invoke, selection, property/text/structure changes, notifications, live regions) from a running app, instead of forcing callers to poll for state transitions. A dedicated STA thread creates its own CUIAutomation8, registers UIA automation-event/property/structure/focus handlers plus a process-scoped SetWinEventHook for window lifecycle, and pumps messages until a duration/--max-events limit or Ctrl+C. Output is human-readable by default or NDJSON (one compact object per line) plus a trailing summary line with --json, with an optional -o tee to file. The change is wired into DI/command registration and the auto-generated + hand-written docs surfaces are (partly) updated.
Changes:
- New
UiWatchCommand+IUiEventWatcher/UiEventWatcher(COM CCW sinks + WinEvent hook) and aWindowLifecycleCoalescerto collapse paired CREATE+SHOW / DESTROY+HIDE events. - Event/property validation, element-scope-requires-window fail-fast, selector re-resolution, NDJSON JSON context, and unit tests for the command and coalescer.
- Regenerated
cli-schema.json/SKILL.md and updateddocs/usage.md+ skill fragments (but notdocs/ui-automation.md/winapp.agent.md).
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
src/winapp-CLI/WinApp.Cli/Commands/UiWatchCommand.cs |
New command: options, validation, streaming output, summary. |
src/winapp-CLI/WinApp.Cli/Services/UiEventWatcher.cs |
Real watcher: STA pump, UIA handlers, WinEvent hook, teardown. |
src/winapp-CLI/WinApp.Cli/Services/IUiEventWatcher.cs |
Contract, event-name constants, request/outcome models. |
src/winapp-CLI/WinApp.Cli/Services/WindowLifecycleCoalescer.cs |
De-dupes paired window lifecycle WinEvents. |
src/winapp-CLI/WinApp.Cli/Helpers/UiWatchJsonContext.cs |
Compact source-gen JSON context + event/summary DTOs. |
src/winapp-CLI/WinApp.Cli/NativeMethods.txt |
CsWin32 declarations for event handlers/WinEvent/message pump. |
src/winapp-CLI/WinApp.Cli/Helpers/HostBuilderExtensions.cs |
Registers watcher service + command handler. |
src/winapp-CLI/WinApp.Cli/Commands/UiCommand.cs |
Adds watch as a ui subcommand. |
src/winapp-CLI/WinApp.Cli.Tests/* |
New command + coalescer tests and FakeUiEventWatcher. |
scripts/generate-llm-docs.ps1 |
Adds ui watch to the skill command map. |
docs/usage.md, docs/fragments/.../ui-automation.md, .github/plugin/.../SKILL.md, .claude/.../SKILL.md, docs/cli-schema.json |
Documentation and generated-schema updates for the new command. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1100
to
+1102
| #### `winapp ui watch` | ||
|
|
||
| Stream live UIA / WinEvent notifications from a running app as they occur. With `--json`, emits NDJSON (one compact JSON object per event line) followed by a summary line. |
Comment on lines
+112
to
+115
| PostQuitMessage | ||
| GetCurrentThreadId | ||
| MSG | ||
| GetAncestor |
|
|
||
| if (!string.IsNullOrWhiteSpace(outputPath)) | ||
| { | ||
| logFile = new StreamWriter(outputPath, append: false, Encoding.UTF8); |
Contributor
Build Metrics ReportBinary Sizes
Test Results✅ 1562 passed, 1 skipped out of 1563 tests in 427.6s (+17 tests, -50.4s vs. baseline) Test Coverage❌ 18.2% line coverage, 37.1% branch coverage · ✅ no change vs. baseline CLI Startup Time40ms median (x64, Updated 2026-07-06 03:16:58 UTC · commit |
nmetulev
reviewed
Jul 6, 2026
nmetulev
left a comment
nmetulev
left a comment
Member
There was a problem hiding this comment.
🤖 AI-generated review — produced by GitHub Copilot CLI at a maintainer's request. Every finding comes from an automated, multi-dimensional review plus a hands-on build-and-test of this branch in an isolated worktree. Treat
file:linereferences as pointers to verify, not gospel. Nothing was pushed. Posting as a comment (no approve / request-changes).
Verdict: 🟡 SHIP-WITH-CHANGES — Good architecture (dedicated STA pump, correct COM handler teardown), but a buffering bug currently defeats the streaming use case this command exists for.
What I actually ran
Built (~33s, 0 warnings); 17 unit tests pass. Watched Notepad while driving it from a second shell:
--duration-secand--max-eventsbounding stop cleanly; NDJSON is one event per line + a summary object. ✅- Reading the
-ofile mid-run: empty until the process exits (C01) — real-time streaming doesn't work. - Opening one WinUI3 Notepad emitted 25+
window-openevents for internal panes (DesktopWindowXamlSource,Default IME,CicMarshalWnd, …); only 1 was the real window (H01). - Focus-event latency <100ms; timestamps ISO-8601 UTC.
Must-fix (Critical / High)
- C01 — no flush after each event (
UiWatchCommand.cs:173).StreamWriterdefaults toAutoFlush = false, so neither stdout nor the-ofile streams until the process exits. This is the headline "agents read events live" scenario, and it's currently broken. One-line fix (Flush()after each write). - H01 — window-open/close filter too broad (
UiEventWatcher.cs:225-230): 20–30 internal-pane events per window open. Add a top-level-window filter (parent == desktop, orWS_CAPTION/WS_OVERLAPPEDWINDOW). - H02 — event
selectorisn't a winapp slug. It's the raw AutomationId (UiEventWatcher.cs:314-319) and is frequently null, soevent.element.selectorcan't be piped intoui invoke/set-value. The unit-test fakes use slug-format strings, which hides this. Route throughSelectorService.
Medium / Low
- M01
RequestStop()is a non-atomic check-then-set. M02 thes_activestatic is unguarded against a concurrentWatchAsync. M03--outputdescription is copy-pasted fromscreenshot. M04 npmwinapp-commands.tsis missinguiWatch(). - L: no integration test for the real watcher (all 17 use
FakeUiEventWatcher); the summary line lacks atypediscriminant;ui-json-envelope.mdisn't updated; thedetailfield format varies across event types.
Fit & recommendation
Push notifications are genuinely more efficient than polling. But this is a long-running streaming command inside an otherwise request/response CLI, and it overlaps the existing wait-for verb. The agent workflow (fork a subprocess, drive the app from another, parse after exit) is awkward, and the flush bug amplifies it. I'd gate this on fixing streaming (C01), then decide whether the streaming model earns its complexity versus investing in wait-for.
nmetulev
added a commit
that referenced
this pull request
Jul 14, 2026
…ndependent multi-model (#613) ## What & why Evidence-driven improvements to the hand-written `pr-review` skill (`.github/skills/pr-review/`). Grounded in **5 real past pr-review runs** mined from the local session store — in every one the user had to *separately* ask for real testing, simplicity/necessity, or a genuine second-model opinion, and the skill's own rules ("No build/test execution", "no fixes", "stdout only") fought that intent. Concrete signals these changes address: - **pester5 (`fix/pester5-count`)** — confidence in the missing `exit 1` only came from *running* Pester 5.7.1, not from reading the diff; it also surfaced a GH Actions vs Azure DevOps `$ErrorActionPreference` divergence and a "will this regress?" question a static pass couldn't answer. - **UI-input verbs (#599–603)** — hands-on found #602's streaming watch writing a **completely empty output file** mid-run, and the user explicitly asked whether a new verb "earns its complexity over just improving `wait-for`" ("can merge" ≠ "should merge"). Turn 1 also hit a **"Policy hook failed"** sub-agent abort. - **crash-stacks (`zt/549`)** — the **JsProvider cold-cache version drift** was invisible to unit tests and only showed up for a freshly-installed user; found via E2E + red-team, not diff review. This is a **DRAFT** for review — no behavior in the repo build/tests changes (the skill is hand-written and not in any auto-generation pipeline). ## The 9 changes 1. **New "Validate findings for real" phase** (SKILL.md step 6, before Report): build the branch, install/run the CLI **as a user would** (npm pack+install, built MSIX, or global tool — explicitly *not* `dotnet run`/dev mode), scaffold a throwaway app (may delegate to `winui-dev`), exercise each changed command, then **confirm or drop** each critical/high finding with runtime evidence and record what couldn't be validated. Replaces the old "No build/test execution" rule. Every finding is tagged **validated** vs **static-only**. 2. **New `necessity-and-simplicity` dimension** (#5): does it earn its complexity? in scope for winapp's mission? smaller / fewer commands / staged? pros & cons of shipping vs *not*. Uses the real "can merge ≠ should merge", "earns its complexity", and "fills a need" framings. Registered everywhere. 3. **Strengthened `alternative-solution`**: require ≥1 **concrete named alternative** (in-repo helper *or* ecosystem tool/pattern/API) with a tradeoff, backed by a quick search — and softened the guidance that suppressed out-of-scope/necessity critiques. 4. **Reworked `multi-model`**: independent pass from the **raw diff + actual code files** (not the consolidated findings, which anchored it into a rubber-stamp), **records which model family ran**, and supports 2–3 families (opus/gemini/gpt) for high-risk PRs. 5. **Regression + threat-model rigor**: explicit **regression** analysis in `correctness` (defect vs test-gap vs intended change); a **threat-model checklist** + red-team in `security` (input-injection / process-invocation / credentials-secrets / signing / supply-chain); static-only findings tagged "needs runtime confirmation". 6. **Multi-PR / multi-target mode** (trackable sub-agent per PR) + **sub-agent fault-tolerance** — a blocked/policy tool call must not abort the sub-agent, and the orchestrator retries a dead one (from the real "Policy hook failed" incident). 7. **Opt-in follow-through**: AI-**labelled** PR review comment, a test-setup / prerequisites handoff checklist, and (only on explicit request) a follow-up fix PR. **Default stays stdout-only / no-fix.** 8. **Contract + report format**: findings carry a **Validation** status; the Coverage block gains the new dimension, a **regression** line, and a **validation** line. 9. **Internal consistency**: dimension counts, tables, fan-out list, and both examples all reflect **9 dimensions + the Validate phase**. ## Files - `SKILL.md` — new Validate phase, multi-PR mode, fault-tolerance, opt-in follow-through, updated rules/report/coverage/examples, renumbered to 9 dims. - `dimensions/necessity-and-simplicity.md` — new. - `dimensions/alternative-solution.md`, `multi-model.md`, `correctness.md`, `security.md`, `_shared-contract.md` — updated per above. All changes are confined to `.github/skills/pr-review/`. --------- Co-authored-by: Nikola Metulev <711864+nmetulev@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
winapp ui watchto listen for UIA/WinEvent notifications from a running app--duration-sec,--max-events), and JSON/NDJSON outputWhy
This adds a first-class way to observe focus, selection, window, and related UI events instead of polling for every state transition.