Skip to content

🩺 Runner Doctor UpdateRunner Doctor catalog update β€” 2026-07-17 to 2026-07-20 scan (B10 repair entrypoint, Copilot config.json mask, sbx PATH, difcPro [Content truncated due to length]Β #6444

Description

@github-actions

Summary


Proposed knowledge-base changes

File: .github/workflows/shared/self-hosted-failure-modes.md

1. Update B10 β€” add PR #6342/6356 as an additional root cause and fix

The current B10 entry attributes the ~30 s per-directory timeout in fixArtifactPermissionsForRootless() solely to compound tag@digest image refs causing registry manifest verification (fixed by PR #6025). PRs #6342 and #6356 reveal a second, independent root cause: even after the image ref was corrected, docker run was invoked without --entrypoint sh, so AWF's own entrypoint.sh ran instead of the chown/chmod repair command. entrypoint.sh waits up to 30 s for the iptables-init container (which never starts in the repair context), then exits non-zero β€” producing the same ~30 s Γ— 3 observable.

Proposed addition to the B10 "Fix" cell (append after the existing PR #6025 paragraph):

Additional fix (PRs #6342 / #6356, merged 2026-07-18): Even with a tag-only image ref, docker run was missing --entrypoint sh, causing AWF's entrypoint.sh to execute in place of the repair command. entrypoint.sh waits ~30 s for an iptables-init container that never starts in this context, then fails. The fix adds --entrypoint sh and passes the chown/chmod command via -c; stdout is now captured alongside stderr so timeout messages surfaced on stdout are not silently swallowed.

Proposed addition to the B10 "Citations" cell: github/gh-aw-firewall#6342, github/gh-aw-firewall#6356

2. New row B13 β€” topology MCP peers and difcProxyHost denied through Squid in network-isolation mode

Field Value
ID B13
Signal In --network-isolation mode, MCP tool calls to topology-attached peers (e.g. awmg-mcpg) or to a difcProxyHost configured independently of topologyAttach return connection errors or are logged as TCP_DENIED in Squid's access log; the agent can reach the DIFC proxy host during startup (B5/B12) but individual in-session MCP/HTTP calls to those same hosts fail or produce Squid 403 noise
Root cause buildProxyEnvironment() assembled NO_PROXY from a static list of AWF-internal hosts but did not include (a) config.topologyAttach peer hostnames or (b) config.difcProxyHost. Proxy-aware clients (Node undici, curl) honoured HTTPS_PROXY for those hosts; Squid denied them because they are not in the domain allowlist
Fix / flag Fixed in AWF (PR #6189, merged 2026-07-20): config.topologyAttach peer hostnames are now added to NO_PROXY/no_proxy. Fixed in AWF (PR #6438, merged 2026-07-20): config.difcProxyHost (stripped of any :port suffix) is also added to NO_PROXY. Upgrade AWF to version including both PRs.
Probe Inspect Squid access.log for TCP_DENIED entries whose target host is a topology peer or the DIFC proxy host; inside the agent container check echo $NO_PROXY β€” the topology peer and DIFC proxy hostname should appear
Citations #6189, #6438

3. New row B14 β€” Copilot CLI silent crash (exit 1, ~0.5 s, no output) after ~/.copilot/config.json is masked as a credential

Field Value
ID B14
Signal After upgrading to AWF including the centralized mount-policy refactor (PR #6339), the Copilot CLI agent exits immediately (exit code 1, ~0.5 s, zero stdout/stderr); no network calls are made; the issue appears on Docker and gVisor runtimes but not sbx (which moves files aside rather than overlay-masking them)
Root cause PR #6339 added ~/.copilot/config.json to the credential deny list with reason "Copilot CLI persisted auth token". This is incorrect: config.json contains only experiment flags, first-launch timestamps, and plugin metadata β€” no token. AWF masks denied files with a read-only /dev/null bind-mount overlay. The Copilot CLI reads and atomically rewrites config.json via temp-file + rename at startup; a rename cannot target a bind-mounted path and the overlay is read-only, so the CLI crashes silently.
Fix / flag Fixed in AWF (PR #6374, merged 2026-07-18): ~/.copilot/config.json is removed from the credential deny list; the file is bind-mounted read-write as part of the .copilot tool subdir (pre-#6339 behaviour). Upgrade to the AWF version including #6374. Workaround (older AWF): pass --keep-containers and inspect the agent container for EROFS or rename errors; downgrade AWF past #6339 if unpatched.
Probe awf ... copilot agent --version β€” if it exits 1 with no output in ~0.5 s, confirm AWF version includes #6374; inspect agent container logs for EROFS/rename errors on ~/.copilot/config.json
Citations #6339, #6374

4. New row D10 β€” Rootless Copilot CLI (~/.local/bin/copilot) not found in sbx microVM PATH

Field Value
ID D10
Signal On --container-runtime sbx, Copilot CLI installed via install_copilot_cli.sh --rootless (redirects install to ~/.local/bin) is not found even though ~/.local is mounted into the microVM; copilot: command not found or ENOENT at agent startup
Root cause sbx microVMs do not run AWF's entrypoint.sh (which prepends $HOME/.local/bin to PATH). buildCoreEnvironment() injected only /usr/local/sbin:...:/bin as PATH. The binary is present in the VM but unreachable by name. Docker/gVisor compose runtimes were unaffected because entrypoint.sh fixed PATH for them.
Fix / flag Fixed in AWF (PR #6407, merged 2026-07-19): buildCoreEnvironment() now prepends ${HOME}/.local/bin to the injected PATH, fixing sbx without affecting compose runtimes. Upgrade to the AWF version including #6407. Workaround (older AWF): use the non-rootless install (requires sudo on the runner) or pass --env PATH=/home/runner/.local/bin:$PATH explicitly.
Probe Inside the sbx agent: echo $PATH β€” ~/.local/bin should appear first; which copilot or ls ~/.local/bin/copilot confirms binary presence vs PATH visibility
Citations #6407

Error-string quick-lookup additions (same file)

Append to the lookup table:

Observable Likely mode
Copilot CLI exits immediately (exit 1, ~0.5 s, zero stdout/stderr) after AWF upgrade; works in sbx but not Docker/gVisor B14 (.copilot/config.json masked as credential; fixed in #6374)
copilot: command not found inside sbx microVM when binary is at ~/.local/bin/copilot D10 (~/.local/bin absent from sbx injected PATH; fixed in #6407)
TCP_DENIED in Squid access log for topology peer or DIFC proxy host during agent run; MCP tool calls silently fail or return connection errors in network-isolation mode B13 (topology peers and difcProxyHost missing from NO_PROXY; fixed in #6189 and #6438)

Proposed doctor changes

File: .github/workflows/self-hosted-runner-doctor.md

Apply the same B10 update, B13/B14/D10 new rows, and three new error-string lookup entries as described above. The playbook ordering (Category A β†’ B β†’ C β†’ D) remains unchanged; B13/B14 are appended after B12, D10 after D9.


Proposed portable agent changes

File: .github/agents/self-hosted-runner-doctor.md

Apply identical changes to the embedded failure-mode catalog and error-string lookup table in this file so the portable agent stays in sync:

  1. B10 β€” add the entrypoint-bypass paragraph and cite PRs Bypass agent ENTRYPOINT in rootless artifact permission repairΒ #6342/fix: bypass AWF entrypoint in rootless artifact permission repairΒ #6356.
  2. B13 β€” add new row (topology/difcProxyHost NO_PROXY; PRs Exclude topology-attached MCP peers from agent proxy routingΒ #6189, fix: exempt DIFC/cli-proxy host from proxy routing in isolation modeΒ #6438).
  3. B14 β€” add new row (Copilot CLI config.json mask; PR fix: don't mask Copilot CLI's ~/.copilot/config.jsonΒ #6374).
  4. D10 β€” add new row (sbx rootless PATH; PR fix: add ~/.local/bin to sandbox PATH for rootless copilot installsΒ #6407).
  5. Error-string lookup β€” add the three new entries listed above.

Source issues and PRs

Citation Title Relevance
#6342 Bypass agent ENTRYPOINT in rootless artifact permission repair B10 update β€” entrypoint not overridden in repair container
#6356 fix: bypass AWF entrypoint in rootless artifact permission repair B10 update β€” refined fix (stdout capture)
#6374 fix: don't mask Copilot CLI's ~/.copilot/config.json New B14 β€” silent CLI crash from credential deny-list false positive
#6407 fix: add ~/.local/bin to sandbox PATH for rootless copilot installs New D10 β€” sbx PATH gap for rootless installs
#6189 Exclude topology-attached MCP peers from agent proxy routing New B13 β€” topology peers not in NO_PROXY
#6438 fix: exempt DIFC/cli-proxy host from proxy routing in isolation mode New B13 β€” difcProxyHost not in NO_PROXY
#6326 ARC/DinD implementation gaps identified from community feedback Context for B5/B12/B13 relationship; gaps 3 and 4 confirmed addressed

Generated by Runner Doctor Updater Β· 50.2 AIC Β· ⊞ 19K Β· β—·

  • expires on Aug 19, 2026, 7:26 PM UTC

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions