Skip to content

Fix TextToAudioPipeline crash for tokenizer-only models - #48505

Open
jiqing-feng wants to merge 1 commit into
huggingface:mainfrom
jiqing-feng:fix-tts-pipeline-tokenizer-dtype
Open

Fix TextToAudioPipeline crash for tokenizer-only models#48505
jiqing-feng wants to merge 1 commit into
huggingface:mainfrom
jiqing-feng:fix-tts-pipeline-tokenizer-dtype

Conversation

@jiqing-feng

@jiqing-feng jiqing-feng commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

CPU CI GPU run-slow

Overview

Any text-to-audio model without a processor is broken on main:

from transformers import pipeline
pipeline(task="text-to-audio", model="facebook/mms-tts-eng")("This is a test")
# TypeError: BatchEncoding.to() got an unexpected keyword argument 'dtype'

tests/pipelines/test_pipelines_text_to_audio.py::TextToAudioPipelineTests::test_vits_model_pt fails on main.

Cause

#40546 (VibeVoice) added an unconditional cast at the end of TextToAudioPipeline.preprocess:

output = output.to(dtype=self.model.dtype)

Models with a processor get a BatchFeature, whose to() accepts dtype. Models without one (VITS, etc.) go through a plain tokenizer and get a BatchEncoding, whose to() only accepts a device.

Fix

Only cast when the preprocessor returned a BatchFeature. A BatchEncoding holds integer tensors only, and BatchFeature.to casts floating point tensors only, so this is a no-op for the tokenizer path either way.

if isinstance(output, BatchFeature):
    output = output.to(dtype=self.model.dtype)

Verification

  • pytest tests/pipelines/test_pipelines_text_to_audio.py -k "vits or speecht5" — passes (fails on main).
  • facebook/mms-tts-eng, float16, CPU: pipeline runs, returns 45312 samples @ 16 kHz.

@jiqing-feng
jiqing-feng force-pushed the fix-tts-pipeline-tokenizer-dtype branch from dd29709 to 8e9a1f7 Compare September 3, 2026 08:50
@jiqing-feng
jiqing-feng marked this pull request as ready for review September 3, 2026 08:51
Since huggingface#40546, `TextToAudioPipeline.preprocess` unconditionally calls
`output.to(dtype=self.model.dtype)`. Models without a processor (e.g. VITS /
`facebook/mms-tts-eng`) go through a plain tokenizer, which returns a
`BatchEncoding` whose `to()` only accepts a device, so this raised
`TypeError: BatchEncoding.to() got an unexpected keyword argument 'dtype'`.

Only cast when the preprocessor returned a `BatchFeature`; a `BatchEncoding`
holds integer tensors only, so the cast was a no-op there anyway.
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

CI recap

Dashboard: View test results in Grafana
Latest run: 33735433068:2
Result: success | Jobs: 1 | Tests: 1 | Failures: 0 | Duration: 2s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant