[ExecuTorch][WebGPU] Reuse pool for single-op scratch (-86.7 MiB)

[JCNTH]

Stack from ghstack (oldest at bottom):

• [ExecuTorch][WebGPU] Test coverage for the f16-accumulate steel GEMM #20801
• [ExecuTorch][WebGPU] f16-accumulate (pwdqf16acc) steel q4gsw prefill GEMM #20800
• [ExecuTorch][WebGPU] Test coverage for the packed-word-dequant f16 steel GEMM #20799
• [ExecuTorch][WebGPU] Packed-word-dequant f16 steel q4gsw prefill GEMM #20798
• -> [ExecuTorch][WebGPU] Reuse pool for single-op scratch (-86.7 MiB) #20797
• [ExecuTorch][WebGPU] Test coverage for the f16 KV cache #20773
• [ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20772
• [ExecuTorch][WebGPU] Test coverage for the f16-multiply steel q4gsw GEMM #20753
• [ExecuTorch][WebGPU] Add f16-multiply variant of the steel q4gsw prefill GEMM #20752
• [ExecuTorch][WebGPU] Steel q4gsw prefill GEMM — test coverage #20731
• [ExecuTorch][WebGPU] Add 256-thread "steel" q4gsw prefill GEMM #20730
• [ExecuTorch][WebGPU] Request shader-f16 device feature (enablement, fail-open) #20729
• [ExecuTorch][WebGPU] Unit tests for the WGSL shader-variant codegen #20728
• [ExecuTorch][WebGPU] WGSL shader-variant codegen (vec1/vec4, fp16-ready) + rms_norm dedup #20727

Problem: WebGPUGraph::create_scratch_buffer keeps every fused-op scratch WGPUBuffer alive in scratch_buffers_ until graph teardown. The SDPA prefill workspace (attn_weights + attn_weights_softmax, each Hq*S*Cmax*4 bytes) and the FlashDecoding partials (part_o/part_ml) are allocated fresh in every decoder layer's lowering, so a 16-layer Llama holds ~16x copies of scratch that is only live during a single op — a large slice of the device compute/scratch footprint.

Solution: add an acquire/release reuse pool for SINGLE-OP-LIFETIME scratch. acquire_scratch() returns a free slot (best-fit, size in [n, 2n]) or creates one; the caller releases it at op-lowering scope exit via the RAII ScopedScratch guard. Across the layers the SDPA/FD scratch now recycles a small constant of buffers instead of Nx. Whole-graph-lifetime scratch stays on create_scratch_buffer unchanged.

Correctness: reuse is bit-identical because WebGPU/Dawn auto-inserts RAW hazard barriers between dispatches that share a storage buffer regardless of pass structure — the same guarantee the shipped mem_obj_id tensor aliasing already relies on. A still-in_use slot is never handed to a co-live requester (co-live safety). WEBGPU_NO_SCRATCH_POOL falls back to the per-call path for A/B.

Differential Revision: D110948389

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20797

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures

As of commit cf7c21f with merge base 4af91c3 ():

NEW FAILURES - The following jobs have failed:

• pull / android / build-android (gh)
  ImportError: /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/executorch/extension/training/pybindings/_training_lib.cpython-311-x86_64-linux-gnu.so: undefined symbol: _ZN3c104impl3cow23materialize_cow_storageERNS_11StorageImplE
• pull / test-binary-size-linux-gcc / linux-job (gh)
  RuntimeError: Command docker exec -t 530f593a8398bfd591db5e67f99b7708eb044c816d870b1970c1403635324910 /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 68e71c8dfb4b3ddf7059b1995e702123515dc34fb3cbc1d2a2e0ed2ac5ad25df /exec failed with exit code 1
• pull / unittest / macos / macos-job (gh)
  export/tests/test_target_recipes.py::TestTargetRecipes::test_mv2_model
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 3d32ea44d973e6a5b5549cb00b91ea2f5deb2453acfb0beb3b4dc6b8a6e40dff /exec failed with exit code 1
• pull / unittest-editable / macos / macos-job (gh)
  export/tests/test_target_recipes.py::TestTargetRecipes::test_ic4_model

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.