Metal backend: unsupported op 'PAD' crash on Apple Silicon

[jasagiri]

Environment

• Device: Apple M4 Pro (macOS 15.5, Darwin 24.6.0, arm64)
• Model: CosyVoice3-2512_Q4_K_S.gguf from Lourdle/Fun-CosyVoice3-0.5B-2512-GGUF
• Frontend: speech_tokenizer_v3.onnx + campplus.onnx
• GGML version: 0.9.11 (bundled)
• Build: cmake -DGGML_METAL=ON -DGGML_METAL_EMBED_LIBRARY=ON with SIMDe patch for ARM64 AVX2→NEON translation

Build

Build succeeds with the SIMDe patch (PR #2). libcosyvoice.dylib and cosyvoice-cli are produced without errors.

Issue

Metal backend — crashes with unsupported op

$ cosyvoice-cli --model CosyVoice3-2512_Q4_K_S.gguf \
    --speech-tokenizer speech_tokenizer_v3.onnx \
    --campplus campplus.onnx \
    --prompt-audio ref.wav --prompt-text "テスト" \
    --text "こんにちは、テストです。" \
    --output out.wav --mode zero-shot

ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_op_encode_impl: error: unsupported op 'PAD'

The crash occurs during token2wav (Flow + HiFT stage), specifically at:

ggml_metal_op_encode + 2500
ggml_metal_graph_compute + 588
ggml_backend_sched_graph_compute + 2244
cosyvoice_model_3::token2wav + 948
cosyvoice_tts + 168

CPU backend — "Too many stop tokens"

With GGML_METAL=OFF (CPU-only build), generation starts but produces only 0.48–0.96 seconds of audio before aborting:

Too many stop tokens sampled, something might be wrong with the model or the sampling parameters.
Error: TTS generation failed.

This is consistent with the README Known Issues ("CPU / Vulkan backends: Produced noisy output in tests").

Analysis

The PAD operation is used in the CosyVoice Flow/HiFT architecture but is not implemented in the GGML Metal backend. This is likely an upstream GGML issue — the PAD op exists in the CPU backend but hasn't been ported to Metal shaders.

Possible solutions:

1. Upstream GGML: Implement PAD in ggml-metal.metal (preferred long-term fix)
2. Workaround: Force PAD ops to CPU via ggml_backend_sched offloading while keeping other ops on Metal
3. Graph rewrite: Replace PAD with equivalent ops that are already Metal-supported (e.g., concatenation with zero tensors)

Related

• Build PR: feat: Apple Silicon support — SIMDe, Metal PAD beg-padding, op fallback, stop token fix #2 (SIMDe patch for ARM64 compilation)
• GGML Metal op support: the bundled ggml 0.9.11 does not list PAD in its Metal shader implementations

[jasagiri]

Progress Update

Fixed: Metal PAD and IM2COL crashes

The PAD crash was caused by two issues:

1. Metal shader only supported right-padding (end-only) — CosyVoice's Flow/HiFT uses left-padding (beg), which was rejected by ggml_metal_device_supports_op. Fixed by:

   • Adding p0_beg/p1_beg/p2_beg/p3_beg fields to ggml_metal_kargs_pad
   • Updating kernel_pad_f32 shader to offset source reads by beg values
   • Relaxing supports_op to accept any non-circular PAD

2. set_graph_backend() force-assigned all nodes to Metal — Ops like IM2COL that Metal doesn't support were sent to Metal anyway. Fixed by checking ggml_backend_supports_op() before assignment and falling back to CPU for unsupported ops.

Remaining: LLM sampling failure

After fixing Metal dispatch, the pipeline reaches the LLM stage but fails with:

Too many stop tokens sampled, something might be wrong with the model or the sampling parameters.

This occurs on both Metal+CPU (with the fixes above) and pure CPU builds. The LLM generates only stop tokens immediately, producing no speech tokens for token2wav. This appears to be a model/sampling issue separate from the backend, matching the existing Known Issues.

Will prepare a PR with the Metal PAD + op fallback fixes.