Skip to content

tests : load backends before init when built with GGML_BACKEND_DL - #4031

Open
apollo-2006 wants to merge 1 commit into
ggml-org:masterfrom
apollo-2006:tests/backend-dl-load-all
Open

tests : load backends before init when built with GGML_BACKEND_DL#4031
apollo-2006 wants to merge 1 commit into
ggml-org:masterfrom
apollo-2006:tests/backend-dl-load-all

Conversation

@apollo-2006

@apollo-2006 apollo-2006 commented Sep 3, 2026

Copy link
Copy Markdown

Fixes #4030.

ggml_backend_load_all() at the top of main() in the four tests that
initialise a context. Every example already does this:
examples/cli/cli.cpp, bench, quantize, lsp,
vad-speech-segments. The tests were the only callers that did not.

With GGML_BACKEND_DL=ON no backend is registered until something calls it,
so whisper_init_* and parakeet_init_* ran with devices = 0 and aborted
on GGML_ASSERT(device) in ggml_backend_dev_backend_reg.

No new include: ggml_backend_load_all() is already reachable through
whisper.h and parakeet.h, both of which pull in ggml-cpu.h, which
includes ggml-backend.h.

Before / after

Full suite on a -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON build,
macOS 26.6.2 / arm64, Apple clang 21.0.0:

test before after
test-whisper-zero-samples aborted passes
test-vad aborted passes
test-parakeet aborted passes
test-vad-full aborted reaches init, see below

ctest -L gh goes from 2 of 4 to 4 of 4. The full suite goes from three
aborts to none.

test-vad-full is labelled base;en rather than gh because it needs a real
models/ggml-base.en.bin, which I do not have locally. It had the same fault
and the fix is verified the same way: before, it printed devices = 0 and hit
GGML_ASSERT(device); after, it prints devices = 3 and gets through
initialisation. Substituting the stub model it then fails its own
assert(n_segments == 1), identically on a GGML_BACKEND_DL and a normal
build, which is the stub having no segments rather than anything to do with
this change.

Regression check

The same suite on a build without GGML_BACKEND_DL is unchanged; 14 of 15
before and after, the one failure being test-vad-full for the missing model
described above. Both configurations now produce identical results.

Why CI did not catch it

None of the workflows pairs the two. build-clang.yml, build-gcc.yml and
build-sanitize.yml run ctest -L gh but never set GGML_BACKEND_DL;
release.yml sets it but runs no tests. Adding a ctest step to a
GGML_BACKEND_DL job would cover the combination, but that felt like a
separate change from fixing the tests, so I have left it out here.

With GGML_BACKEND_DL=ON no backend is registered until something calls
ggml_backend_load_all(). Every example does; the tests never did, so
whisper_init_* and parakeet_init_* ran with devices = 0 and aborted on
GGML_ASSERT(device) in ggml_backend_dev_backend_reg.

Three tests aborted on such a build - test-whisper-zero-samples,
test-vad and test-parakeet - taking ctest -L gh to 2 of 4.
test-vad-full has the same fault and reaches it only once a real
base.en model is present.

No workflow caught this because none pairs the two: build-clang,
build-gcc and build-sanitize run ctest -L gh but never set
GGML_BACKEND_DL, while release.yml sets it and runs no tests.

ggml_backend_load_all() is already reachable through whisper.h and
parakeet.h via ggml-cpu.h, so no new include is needed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are minimal and directly address the documented GGML_BACKEND_DL test abort by ensuring backends are registered before context initialization.

Pull request overview

Ensures tests that initialize a whisper_context / whisper_vad_context / parakeet_context explicitly load GGML backends first, matching the behavior of existing examples and preventing devices = 0 aborts when built with GGML_BACKEND_DL=ON.

Changes:

  • Add ggml_backend_load_all() at the start of main() in four affected test binaries.
  • Align test startup behavior with example binaries that already perform backend loading.
  • Fixes aborts caused by no backends/devices being registered under dynamic-backend loading builds.
File summaries
File Description
tests/test-whisper-zero-samples.cpp Load GGML backends before initializing a Whisper context.
tests/test-vad.cpp Load GGML backends before initializing VAD context/model usage.
tests/test-vad-full.cpp Load GGML backends before Whisper/VAD initialization path.
tests/test-parakeet.cpp Load GGML backends before initializing Parakeet contexts.
Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@apollo-2006

Copy link
Copy Markdown
Author

@danbev one thing I could not settle from the source. test-vad is labelled unit and test-vad-full is base;en, so neither runs under ctest -L gh, while test-whisper-buffer-loader beside it is unit;gh. That puts two of the four tests this PR fixes outside the label CI actually runs, so a regression here would go unnoticed the same way this did.

Is that split deliberate, keeping anything that needs a model out of the gh job, or should test-vad pick up gh the way the buffer-loader test does?

@danbev

danbev commented Sep 7, 2026

Copy link
Copy Markdown
Member

Is that split deliberate, keeping anything that needs a model out of the gh job, or should test-vad pick up gh the way the buffer-loader test does?

There is a separate workflow, build-vad.yml which runs test-vad so it does not require the gh label.
And test-vad-full is left as a manual test as it requires a larger whisper model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tests : ctest -L gh fails on a GGML_BACKEND_DL=ON build

3 participants