-
Notifications
You must be signed in to change notification settings - Fork 16
New Skill: rocm-doctor
#87
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
volen-silo
wants to merge
5
commits into
main
Choose a base branch
from
rocm-doctor-publish
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
fdd2fa6
Publish rocm-doctor skill (thin driver over rocm examine/diagnose/fix)
volen-silo b5dbaac
Merge branch 'main' into rocm-doctor-publish
danielholanda a003746
Merge branch 'main' into rocm-doctor-publish
sreeram-11 941a326
Add rocm-doctor behavioral evals and Prerequisites
volen-silo d7057d8
Merge branch 'main' into rocm-doctor-publish
danielholanda File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,153 @@ | ||
| --- | ||
| name: rocm-doctor | ||
| description: >- | ||
| Diagnoses why ROCm, the HIP SDK, PyTorch, or llama.cpp is broken on an AMD GPU | ||
| on Linux or Windows, then applies a low-risk fix with consent or hands back the | ||
| exact next step. Also routes Lemonade, LM Studio, and Ollama problems to the | ||
| right upstream channel. Use when the user reports that ROCm or HIP "isn't | ||
| working", torch.cuda.is_available() is False, rocminfo / hipInfo can't see the | ||
| GPU, or hits hipErrorNoBinaryForGpu, HSA_STATUS_ERROR_INVALID_ISA, "invalid | ||
| device function", "no kernel image is available", cannot open /dev/kfd, | ||
| permission denied on /dev/kfd, "ROCk module is NOT loaded", a missing | ||
| libamdhip64.so / amdhip64_6.dll / hipblas.dll / vcruntime140_1.dll, an | ||
| HSA_OVERRIDE_GFX_VERSION page fault, an iGPU+dGPU crash, a container that can't | ||
| see the GPU, or an amdgpu-install / DKMS failure. Backed by the `rocm` CLI | ||
| (`rocm examine` / `rocm diagnose` / `rocm fix`); this skill is a thin driver | ||
| over those commands, not a re-implementation. | ||
| --- | ||
|
|
||
| # ROCm Doctor | ||
|
|
||
| Given a "ROCm / PyTorch / llama.cpp isn't working on my AMD GPU" complaint, | ||
| identify which **known misconfiguration** is the cause and either fix it (with | ||
| consent) or hand back the exact next step. | ||
|
|
||
| This skill does **not** probe or reason on its own. The `rocm` CLI owns the | ||
| probe, the closed failure-mode catalog, and the fixes; the skill just drives it | ||
| and relays the results. The catalog is a **closed list** — if the symptom | ||
| doesn't match a known mode, route the user upstream instead of guessing. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - **The `rocm` CLI.** This skill is only a driver over it; Phase 0 below installs | ||
| it with the user's consent if `rocm --version` fails. Nothing else here is | ||
| assumed — the CLI does the probing. | ||
| - **Platform:** native Linux (in-tree `amdgpu` module + `/dev/kfd`) or Windows | ||
| (HIP SDK). WSL2, NVIDIA/Intel/Apple GPUs, and clean-machine installs are out of | ||
| scope (see [Out of scope](#out-of-scope)). | ||
| - **No fixed ROCm version, GPU arch (`gfx…`), or container image is assumed** — | ||
| `rocm examine`/`diagnose` detect the installed ROCm, the GPU's `gfx` target, and | ||
| container context, and match fixes to what they find. Never hand-set | ||
| `HSA_OVERRIDE_GFX_VERSION` (or similar footgun env vars) yourself; let the CLI | ||
| decide. | ||
|
|
||
| ## Workflow | ||
|
|
||
| 0. **Ensure the `rocm` CLI is present.** Everything below shells out to it, so | ||
| check first and install it if missing: | ||
|
|
||
| ``` | ||
| rocm --version | ||
| ``` | ||
|
|
||
| If that succeeds, skip to step 1. If it's not found, install it **with the | ||
| user's consent** (this fetches and runs an installer that drops the `rocm` and | ||
| `rocmd` binaries into `~/.local/bin`). Only nightly builds are published | ||
| today, so install from the `nightly` channel: | ||
|
|
||
| - **Linux / macOS:** | ||
| ``` | ||
| curl -fsSL https://raw.githubusercontent.com/ROCm/rocm-cli/main/install.sh | sh -s -- nightly | ||
| ``` | ||
| - **Windows (PowerShell):** | ||
| ``` | ||
| $env:ROCM_CLI_CHANNEL = "nightly" | ||
| irm https://raw.githubusercontent.com/ROCm/rocm-cli/main/install.ps1 | iex | ||
| ``` | ||
|
|
||
| (Once rocm-cli cuts a stable release, drop the `nightly` channel — `sh` / | ||
| `iex` alone will pull the latest stable build.) | ||
|
|
||
| After install, confirm `~/.local/bin` is on `PATH` and re-run `rocm --version`. | ||
| If it still isn't available, hand the user the install page | ||
| (https://github.com/ROCm/rocm-cli) and stop. | ||
|
|
||
| 1. **Diagnose.** Pass the user's error text as the symptom: | ||
|
|
||
| ``` | ||
| rocm diagnose --symptom "<paste the exact error>" --json | ||
| ``` | ||
|
|
||
| Read the JSON: | ||
| - `matched[]` — ranked causes, each with `id`, `title`, `score` (0–100), | ||
| `evidence[]`, and a `fix` (with `fix_id`, `summary`, `commands`, `verify`, | ||
| `notes`, and the `needs_sudo` / `needs_reboot` / `needs_relogin` / | ||
| `auto_applicable` flags). `score >= 75` = high confidence; `50–74` = likely | ||
| (confirm one more piece of evidence with the user first). | ||
| - `out_of_scope` — when set (e.g. WSL2), do **not** diagnose. First, if the | ||
| user's symptom clearly names an app that ships its own runtime (Lemonade, | ||
| Ollama, LM Studio), route them to that app's tracker (see | ||
| [Framework routing](#framework-routing)) — those trackers apply regardless | ||
| of platform. Otherwise relay the `out_of_scope` message and stop (see | ||
| [Out of scope](#out-of-scope)). | ||
| - `route_when_no_match` — when `matched` is empty, hand the user this | ||
| upstream tracker; **do not speculate**. Note the CLI picks this target from | ||
| the *host-detected* framework, not from the symptom text — so for an app | ||
| named only in the symptom, route it yourself per | ||
| [Framework routing](#framework-routing). | ||
|
|
||
| 2. **Propose the fix.** Show the top match's `title`, `evidence`, plan, and | ||
| `verify` command. Only propose applying it when the user is on board. | ||
|
|
||
| 3. **Apply with consent.** For an auto-applicable fix: | ||
|
|
||
| ``` | ||
| rocm fix <fix-id> # auto fixes: prompt before changing anything | ||
| rocm fix <fix-id> --dry-run # show the exact change, touch nothing | ||
| rocm fix <fix-id> --yes # required to apply in a non-interactive shell | ||
| ``` | ||
|
|
||
| Only the four auto-applicable fixes prompt and mutate. The other 11 are | ||
| **print-only** (bootloader, kernel, reinstall, Windows driver, …): `rocm fix | ||
| <id>` just prints the plan for the user to run themselves — no prompt, and the | ||
| CLI never performs those. | ||
|
|
||
| 4. **Verify.** Have the user run the `verify` command from the diagnosis. | ||
|
|
||
| Use `rocm examine` (or `rocm examine --json`) when you only need the host state | ||
| (GPU, driver, ROCm install, groups, framework) without a diagnosis. | ||
|
|
||
| ## Framework routing | ||
|
|
||
| `rocm diagnose` covers frameworks that build against the **system** ROCm/HIP: | ||
|
|
||
| - **PyTorch**, **llama.cpp** — in scope; diagnose normally. | ||
|
|
||
| Apps that ship their **own** ROCm runtime aren't diagnosed here — route the user | ||
| to the right tracker. (The CLI's `route_when_no_match` also targets these, but | ||
| only when the host probe *detects* that app; when the app is named only in the | ||
| symptom, do the routing yourself using the list below.) | ||
|
|
||
| - **Lemonade** → https://github.com/lemonade-sdk/lemonade/issues | ||
| - **Ollama** → https://github.com/ollama/ollama/issues | ||
| - **LM Studio** → in-app support (no public repo) | ||
| - Anything else with no catalog match → ROCm core: | ||
| https://github.com/ROCm/ROCm/issues (this is what `route_when_no_match` | ||
| returns by default). | ||
|
|
||
| ## Out of scope | ||
|
|
||
| - **WSL2** — a distinct platform (`/dev/dxg` + the Windows host driver, not the | ||
| in-tree `amdgpu` module or `/dev/kfd`). `rocm examine`/`diagnose` detect it and | ||
| route out; relay that guidance and point at AMD's ROCm-on-WSL guide. | ||
| - **NVIDIA / Intel / Apple Silicon GPUs**, and **fresh installs on a clean | ||
| machine** (a setup task, not a diagnosis). Exit cleanly and say so. | ||
|
|
||
| ## Rules | ||
|
|
||
| - Never invent a fix. If `rocm diagnose` returns no match, route upstream. | ||
| - Never run a mutating fix without the user's explicit OK; prefer `--dry-run` | ||
| first. New failure modes are added to the CLI catalog, not improvised here. | ||
|
|
||
| See [reference.md](reference.md) for the full closed catalog and the CLI | ||
| command/exit-code reference. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| # Copyright (c) 2026 Advanced Micro Devices, Inc. All rights reserved. | ||
| # | ||
| # See LICENSE for license information. | ||
|
|
||
| """Behavioral tests for the `rocm-doctor` skill. | ||
|
|
||
| Run locally (needs the `claude` CLI authenticated; otherwise the suite skips): | ||
|
|
||
| cd eval/behavioral | ||
| python -m pytest -c pytest.ini -p conftest ../../skills/rocm-doctor/evals/evals.py | ||
|
|
||
| The skill is a thin driver over the `rocm` CLI, so a trigger run follows Phase 0 | ||
| and shells out to `rocm --version` first (and, with consent, offers to install | ||
| the CLI -- these runs may touch the network). The suite is written to hold in a | ||
| GPU-less CI box on either Linux or Windows: the `rocm` CLI may be absent and | ||
| unable to install/run, so assertions rest on what the skill guarantees | ||
| regardless of environment -- it activates, probes for the CLI first (Phase 0), | ||
| and never mutates the system without consent -- rather than on a successful | ||
| diagnosis (which needs a real AMD GPU). Deterministic checks (`logs_contains`) | ||
| assert activation and the Phase 0 probe; `should` / `should_not` are graded by | ||
| an LLM judge and phrased to allow the skill's "CLI unavailable -> hand back the | ||
| install path" branch. | ||
|
|
||
| Trigger set: ROCm/HIP/PyTorch failure symptoms that should drive the CLI. | ||
| Non-trigger set: an NVIDIA problem (skill must bow out as out-of-scope) and an | ||
| unrelated coding task (skill must not engage at all). | ||
| """ | ||
|
|
||
| from harness import claude | ||
|
|
||
|
|
||
| # --- Trigger set -------------------------------------------------------------- | ||
|
|
||
|
|
||
| def test_trigger_hip_no_binary_for_gpu(): | ||
| with claude("opus", skill="rocm-doctor") as agent: | ||
| run = agent.prompt( | ||
| "torch.cuda.is_available() returns False on my AMD GPU and I get " | ||
| "'hipErrorNoBinaryForGpu' when I run my script. What's wrong?" | ||
| ) | ||
|
|
||
| # Programmatic expectations (GPU-independent; hold on Linux and Windows). | ||
| run.logs_contains("rocm-doctor") # skill activated from its description | ||
| run.logs_contains("rocm --version") # Phase 0: probe for the CLI first | ||
|
|
||
| # Positive: drive the CLI, with the skill's escape valve when it can't run. | ||
| run.should( | ||
| "Drive the `rocm` CLI to diagnose -- check `rocm --version`, offer to " | ||
| "install it with the user's consent, then `rocm diagnose` -- instead of " | ||
| "applying a ROCm fix invented from general knowledge; if the CLI cannot " | ||
| "be installed, hand the user the CLI/install path rather than guessing" | ||
| ) | ||
|
|
||
| # Negative: the core safety invariant, true even when the CLI is absent. | ||
| run.should_not( | ||
| "Execute a mutating or sudo command without first getting the user's " | ||
| "explicit consent" | ||
| ) | ||
|
|
||
|
|
||
| def test_trigger_permission_denied_kfd(): | ||
| with claude("opus", skill="rocm-doctor") as agent: | ||
| run = agent.prompt( | ||
| "I get 'permission denied' opening /dev/kfd and rocminfo can't see my " | ||
| "AMD GPU. How do I fix it?" | ||
| ) | ||
|
|
||
| run.logs_contains("rocm-doctor") | ||
| run.logs_contains("rocm --version") # Phase 0 runs before any fix | ||
|
|
||
| # This symptom is Linux-only and the CI box is GPU-less, so assert only | ||
| # the safety invariant, not a successful CLI diagnosis. | ||
| run.should_not( | ||
| "Execute a mutating or sudo command (e.g. usermod, modprobe) without " | ||
| "first getting the user's explicit consent" | ||
| ) | ||
|
|
||
|
|
||
| def test_trigger_routes_lemonade_upstream(): | ||
| with claude("opus", skill="rocm-doctor") as agent: | ||
| run = agent.prompt( | ||
| "Lemonade fails to load a model on my Radeon GPU with a ROCm error. " | ||
| "Can you sort this out?" | ||
| ) | ||
|
|
||
| run.logs_contains("rocm-doctor") | ||
|
|
||
| run.should( | ||
| "Route the user to the Lemonade issue tracker " | ||
| "(github.com/lemonade-sdk/lemonade/issues) because Lemonade ships its " | ||
| "own ROCm runtime and is not a system-ROCm failure mode" | ||
| ) | ||
| run.should_not( | ||
| "Fabricate a system-ROCm `rocm fix` for a Lemonade-owned runtime problem" | ||
| ) | ||
|
|
||
|
|
||
| # --- Non-trigger set ---------------------------------------------------------- | ||
|
|
||
|
|
||
| def test_non_trigger_nvidia_is_out_of_scope(): | ||
| with claude("opus", skill="rocm-doctor") as agent: | ||
| run = agent.prompt( | ||
| "torch.cuda.is_available() is False on my NVIDIA RTX 4090 and CUDA " | ||
| "seems broken. Help me fix it." | ||
| ) | ||
|
|
||
| # The skill may recognize the symptom, but must bow out: NVIDIA is | ||
| # explicitly out of scope, so it must not run the AMD ROCm workflow. | ||
| run.should_not( | ||
| "Apply the AMD ROCm diagnosis workflow (rocm examine / diagnose / fix) " | ||
| "to an NVIDIA GPU problem" | ||
| ) | ||
|
|
||
|
|
||
| def test_non_trigger_unrelated_task(): | ||
| with claude("opus", skill="rocm-doctor") as agent: | ||
| run = agent.prompt( | ||
| "Write a Python function that reverses a singly linked list." | ||
| ) | ||
|
|
||
| run.should_not( | ||
| "Invoke the rocm-doctor ROCm/AMD-GPU diagnostic workflow for an " | ||
| "unrelated coding task" | ||
| ) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove all changes to marketplace files. We will add those later to those "showcase" catalogs subject to marketing approval.