Skip to content

fix(model): resolve CLI exec paths for Windows .cmd shims - #250

Open
WODE25500 wants to merge 6 commits into
microsoft:mainfrom
WODE25500:fix/model-windows-exec
Open

fix(model): resolve CLI exec paths for Windows .cmd shims#250
WODE25500 wants to merge 6 commits into
microsoft:mainfrom
WODE25500:fix/model-windows-exec

Conversation

@WODE25500

Copy link
Copy Markdown
Contributor

Cross-platform exec fixes for the model backends.

  • Resolve codex/claude/cursor/copilot exec paths via shutil.which at config load, so bare npm .cmd shims spawn on Windows (bare 'codex' -> codex.CMD; CreateProcess does not search PATHEXT, so bare names raise WinError 2). Mirrors the proven pattern in the sleep layer.
  • os.symlink uses target_is_directory + copytree/copy2 fallback (fixes OfficeQA on Windows).
  • Scrub the copilot exec stdout/stderr trace (was unredacted, unlike cursor/codex/claude).
  • Add tests for _resolve_cli_path.

Verified: _resolve_cli_path('codex') -> codex.CMD on Windows; targeted tests pass.

- Resolve codex/claude/cursor/copilot exec paths via shutil.which at config
  load so bare npm .cmd shims spawn on Windows (bare 'codex' -> codex.CMD;
  CreateProcess does not search PATHEXT, so bare names raise WinError 2).
- os.symlink uses target_is_directory + copytree/copy2 fallback (OfficeQA
  on Windows).
- Add tests for _resolve_cli_path.
@Yif-Yang

Copy link
Copy Markdown
Contributor

The CLI path-resolution change is useful, but the symlink fallback needs to fail closed before merge.

In prepare_workspace(), every OSError from os.symlink() is treated as “symlinks unavailable”, and copytree(..., dirs_exist_ok=True) then merges into whatever already occupies dst. I reproduced two failures using valid helper arguments:

  • If extra_files creates the destination first, the call succeeds and silently mixes stale and source files.
  • If two link_dirs entries use the same destination, the first creates a symlink to source A; the second gets EEXIST, then copytree follows the existing symlink and copies source B into source A, modifying data outside the workspace.

The current OfficeQA caller generates unique docs/root_N paths, so its normal Windows privilege fallback works, but the helper must not turn any collision into source-tree mutation. At minimum, re-raise when os.path.lexists(dst) and use shutil.copytree(src_abs, dst) without dirs_exist_ok=True; preferably fall back only for the expected Windows “symlink privilege not held” error. Add tests for forced symlink-permission fallback, an existing destination, and duplicate destinations, asserting that neither source is modified.

There is also an incomplete redaction path introduced here: Copilot stdout/stderr is passed through _redact_cursor_error(), but JSON such as {"token":"plain-secret"} remains unchanged because quoted mapping keys do not match its assignment regex. Please sanitize Copilot JSONL structurally using the existing mapping-key-aware sanitizer (or a shared equivalent), and add a run_copilot_exec regression proving secret fields are absent from returned traces and raised error details.

Address maintainer review on microsoft#250:
- prepare_workspace symlink fallback fails closed: re-raise when the
  destination already exists (lexists), never copytree into an existing dst,
  and only fall back for the Windows symlink-privilege-not-held error.
  Prevents a colliding extra_files or duplicate link_dirs entry from mutating
  source data outside the work dir.
- Sanitize Copilot stdout/stderr structurally (mapping-key aware) so JSON
  objects carrying a token/secret field are redacted even when the value is
  a quoted mapping (which the string-level redactor missed).
- Add tests: symlink privilege fallback, existing/duplicate dst, non-privilege
  re-raise, and Copilot JSONL secret-field redaction.
@WODE25500

WODE25500 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor Author

Yifan Yang (@Yif-Yang) — thank you for the thorough review and guidance! I've addressed the feedback:

  • The symlink fallback now fails closed: an existing destination raises FileExistsError; copyree/copytree no longer uses dirs_exist_ok; and we only fall back for the Windows "symlink privilege not held" error (WinError 1314 / EPERM / ENOTSUP), re-raising other errors — so a colliding or duplicate destination can no longer mutate the source outside the work dir.
  • Copilot JSON output is now sanitized structurally (mapping-key aware, _redact_copilot_trace/_redact_copilot_json), so {"token": "..."} is redacted too, while content/prompt are kept for debugging.
  • Added tests/test_workspace_symlink.py (privilege fallback, existing/duplicate destination, non-privilege re-raise, JSONL secret-field redaction) — all pass.

Thanks again for the detailed review!

@Yif-Yang

Copy link
Copy Markdown
Contributor

Thanks — the symlink collision/source-mutation blocker is fixed. One Copilot redaction gap remains.

_redact_copilot_trace() sanitizes structurally only when the entire line is valid JSON. Prefixed or pretty-printed stderr such as warning: {"token":"plain-secret"} falls back to _redact_cursor_error(), whose assignment regex does not match quoted JSON keys. I reproduced plain-secret verbatim both in the combined trace returned by a successful run_copilot_exec() call and in the RuntimeError detail on a non-zero exit.

Please sanitize quoted/embedded JSON in the fallback path and add run_copilot_exec-level regressions for both returned traces and raised errors, including prefixed and multiline JSON. Once this boundary is covered, the original workspace issue looks resolved.

- Fold quoted-JSON whole-value capture into the shared _redact_cursor_error
  (run first) so a quoted value with spaces is scrubbed whole instead of
  truncating at the first space and leaking the remainder. One change covers
  the copilot fallback, the cursor stderr paths, and all string-leaf callers.
- Regressions: prefixed/multiline quoted JSON, space-in-value, cursor parity.
- The substring regex over-scrubbed diagnostics like token_count / token_budget /
  secret_version. Decide with the project's endswith-based rule (matching
  _is_secret_mapping_key) so only real credential keys are redacted, while a
  quoted value that contains spaces is still captured whole.
- Regressions: token-count diagnostic retention + nested secret key.
- Add "bearer" to the exact secret-key set so a standalone "bearer": <value>
  JSON key is scrubbed (the common "Authorization": "Bearer ..." form was
  already covered via the authorization key).
- Regression: bare bearer key redaction.
@WODE25500

WODE25500 commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

Already addressed the review feedback and updated this branch (#250):

  • Commits 84b72a8 / b33b12e: folded quoted-JSON whole-value capture into the shared _redact_cursor_error (one fix covers the copilot fallback, the cursor stderr paths, and all string-leaf callers); secret-key detection now uses endswith on the normalized key rather than substring (keeping token_count/token_budget/secret_version diagnostics while still redacting token/api_key/refreshToken), and bare "bearer" keys are covered.
    Please re-review, thanks.

@Yif-Yang

Copy link
Copy Markdown
Contributor

Thanks — the scalar prefixed cases are fixed, but two redaction gaps remain. _redact_copilot_json() still uses the old exact-field set (plus endswith("apikey")), so valid JSON such as {"githubToken":"plain-secret"} is emitted unchanged. The fallback regex also supports only single-level object values and processes input line by line, so deeply nested or pretty-printed embedded JSON can still leak secrets into both returned traces and RuntimeError details. Please use one mapping-aware key policy consistently and parse/redact embedded JSON without a fixed nesting limit, with run-level regressions for both success and error paths.

…SON parsing

- _redact_copilot_json now uses the same endswith-based _is_copilot_secret_key as
  the embedded-JSON fallback, so camelCase keys like githubToken are redacted.
- Replace the single-level regex and line-by-line parsing with a bracket-matched,
  unbounded-nesting embedded-JSON scan, so deeply nested / pretty-printed JSON in
  non-JSON text is structurally redacted.
- Add run-level regressions (success trace + error detail) for camelCase keys and
  deep-nested embedded JSON.
@WODE25500

Copy link
Copy Markdown
Contributor Author

Thanks for the re-review. Addressed both redaction gaps: _redact_copilot_json now uses the same endswith-based _is_copilot_secret_key as the embedded-JSON fallback (so githubToken is redacted), and embedded JSON is parsed with an unbounded bracket-matching scan (deeply nested / pretty-printed JSON in non-JSON text is structurally redacted). Added run-level regressions for both the returned trace and the raised error. Commits 5a5a1f8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants