Skip to content

feat(sdks/python): allow language selection for local Whisper transcription - #13076

Open
Aj2280 wants to merge 1 commit into
BasedHardware:mainfrom
Aj2280:feat/whisper-language-selection
Open

feat(sdks/python): allow language selection for local Whisper transcription#13076
Aj2280 wants to merge 1 commit into
BasedHardware:mainfrom
Aj2280:feat/whisper-language-selection

Conversation

@Aj2280

@Aj2280 Aj2280 commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR enables language selection when using local Whisper transcription via the Python SDK, resolving #13071.

Previously, sdks/python/omi/stt/whisper.py hardcoded language="en" in calls to model.transcribe(), preventing callers from selecting other languages (e.g., German, Spanish) or requesting automatic language detection (language=None) even when using multilingual models.

Changes

  • Python SDK (sdks/python/omi/stt/whisper.py):
    • Added keyword-only language: Optional[str] = "en" to WhisperTranscriber.__init__.
    • Saved self.language and forwarded it as language=self.language to self._model.transcribe(audio, fp16=False, language=self.language).
    • Preserved the injected custom runner contract (receives raw PCM bytes and manages its own language).
  • Unit Tests (sdks/python/tests/test_whisper_language.py):
    • Added hermetic unit tests using a mocked whisper module covering default ("en"), explicit multilingual code ("de"), and automatic detection (None).
    • Added verification that custom runner continues to receive raw PCM bytes only.
  • Documentation (docs/doc/developer/sdk/python.mdx, sdks/python/README.md):
    • Added Local Whisper language selection documentation and usage examples.
  • CI Workflow (.github/workflows/python-cli-ci.yml):
    • Added sdks/python/** path triggers to pull requests and push events.
    • Added sdk-tests job to run hermetic Python SDK unit tests using pytest.

Verification

  • Ran PYTHONPATH=sdks/python pytest sdks/python/tests/ locally: all 11 tests passed.
  • Ran make preflight: all 18 local manifest checks passed.

Fixes #13071

Review in cubic

…iption

Allow passing language (e.g. "de", "es", or None for auto-detection) to
WhisperTranscriber instead of hardcoding language="en".

- Update WhisperTranscriber.__init__ with language: Optional[str] = "en"
- Pass language=self.language to model.transcribe()
- Preserve injected runner bytes-only contract
- Add unit tests verifying language forwarding and runner contract
- Document local Whisper language selection in README and docs
- Run hermetic SDK unit tests in Python CLI CI workflow

Fixes BasedHardware#13071

Signed-off-by: Abhi <108084481+Aj2280@users.noreply.github.com>
Co-authored-by: Theodor Frank <33786175+TheoFrk@users.noreply.github.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 5 files

Confidence score: 4/5

  • .github/workflows/python-cli-ci.yml installs pytest and numpy without pinned versions, so SDK test outcomes can change as PyPI releases move; use a checked-in constraints or lock file.
  • .github/workflows/python-cli-ci.yml now triggers the full tests matrix for SDK-only changes, adding unnecessary Linux and Windows CI work for a suite that does not depend on omi-sdk; narrow the path/job conditions or separate the SDK trigger.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/python-cli-ci.yml">

<violation number="1" location=".github/workflows/python-cli-ci.yml:8">
P3: Adding `sdks/python/**` to the workflow's `paths` filter makes the unrelated `tests` matrix job (the omi-cli suite across Linux 3.10/3.12 and Windows) run on every SDK-only change. omi-cli does not depend on omi-sdk, so those jobs add no signal to SDK PRs and can block them on unrelated CLI failures. Give the SDK test job its own workflow (or separate path trigger) scoped to `sdks/python/**` instead of widening the shared workflow-wide filter.</violation>

<violation number="2" location=".github/workflows/python-cli-ci.yml:37">
P2: The SDK test job is not reproducible because `pip install pytest numpy` resolves unpinned PyPI versions on every run. Install test dependencies from a checked-in, pinned constraints or lock file.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

with:
python-version: '3.10'
- name: Install hermetic SDK test dependencies
run: python -m pip install pytest numpy

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The SDK test job is not reproducible because pip install pytest numpy resolves unpinned PyPI versions on every run. Install test dependencies from a checked-in, pinned constraints or lock file.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/python-cli-ci.yml, line 37:

<comment>The SDK test job is not reproducible because `pip install pytest numpy` resolves unpinned PyPI versions on every run. Install test dependencies from a checked-in, pinned constraints or lock file.</comment>

<file context>
@@ -20,6 +22,24 @@ defaults:
+        with:
+          python-version: '3.10'
+      - name: Install hermetic SDK test dependencies
+        run: python -m pip install pytest numpy
+      - name: Run SDK tests without devices or model downloads
+        run: python -m pytest -q
</file context>

branches: [main]
paths:
- 'sdks/python-cli/**'
- 'sdks/python/**'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: Adding sdks/python/** to the workflow's paths filter makes the unrelated tests matrix job (the omi-cli suite across Linux 3.10/3.12 and Windows) run on every SDK-only change. omi-cli does not depend on omi-sdk, so those jobs add no signal to SDK PRs and can block them on unrelated CLI failures. Give the SDK test job its own workflow (or separate path trigger) scoped to sdks/python/** instead of widening the shared workflow-wide filter.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/python-cli-ci.yml, line 8:

<comment>Adding `sdks/python/**` to the workflow's `paths` filter makes the unrelated `tests` matrix job (the omi-cli suite across Linux 3.10/3.12 and Windows) run on every SDK-only change. omi-cli does not depend on omi-sdk, so those jobs add no signal to SDK PRs and can block them on unrelated CLI failures. Give the SDK test job its own workflow (or separate path trigger) scoped to `sdks/python/**` instead of widening the shared workflow-wide filter.</comment>

<file context>
@@ -5,11 +5,13 @@ on:
     branches: [main]
     paths:
       - 'sdks/python-cli/**'
+      - 'sdks/python/**'
       - '.github/workflows/python-cli-ci.yml'
   pull_request:
</file context>

@Git-on-my-level

Copy link
Copy Markdown
Collaborator

Thanks for this — the core change is verified and well built. Review notes below.

Verified implementation

  • sdks/python/omi/stt/whisper.py: the new keyword-only language: Optional[str] = "en" is stored and forwarded as language=self.language to model.transcribe(); default behavior is unchanged, and the injected-runner path still receives only PCM bytes.
  • Wiring confirmed through omi/transcribe.py (**engine_kwargs) and omi/stt/__init__.py (create_transcriber(**kwargs)), so transcribe(..., engine="whisper", language="de") reaches the model call unchanged.
  • sdks/python/tests/test_whisper_language.py: hermetic (mocked whisper module) yet exercises the real ~5-second batching loop, int16-to-float32 conversion, and callback delivery; it asserts the exact kwargs (fp16=False plus language) for default "en", explicit "de", and None auto-detection, and keeps the runner bytes-only contract covered.

CI status

  • The new sdk-tests job in .github/workflows/python-cli-ci.yml has not actually executed on this PR yet: workflow runs for this head are still awaiting first-time-contributor approval. I replicated the job locally (fresh venv, pip install pytest numpy, PYTHONPATH=. from sdks/python) and all 11 SDK tests pass. The job itself is safe (read-only contents: read permissions, standard actions, no secrets); the two minor follow-ups already flagged by the bot review (pinning pytest/numpy, and the widened sdks/python/** trigger now also running the CLI matrix on SDK-only changes) are reasonable but non-blocking.

Attribution — needs maintainer ruling before merge

Docs (docs/doc/developer/sdk/python.mdx and sdks/python/README.md) are accurate against the code, including the "" api_key quirk of the shared wrapper and the multilingual-model requirement.


by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.

@Git-on-my-level Git-on-my-level added needs-maintainer-review Needs a human maintainer to sign off before merge python labels Sep 8, 2026
@Aj2280

Aj2280 commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Hi @Git-on-my-level, thanks for the review and local verification!

Regarding attribution:

  1. I saw @TheoFrk's proposed patch in Python SDK: allow language selection for local Whisper transcription #13071, verified the implementation, and packaged it into this PR with the necessary preflight checks, CI wiring, and hermetic tests so it would meet the repository's Definition of Done and merge smoothly into main.
  2. Full credit to @TheoFrk for the original patch design — I included Co-authored-by: Theodor Frank <33786175+TheoFrk@users.noreply.github.com> in the commit metadata from the start (GitHub displays @TheoFrk as co-author on this commit/PR).
  3. Regarding the proposed $50 bounty/sponsorship, I defer completely to the maintainers (@josancamon19 / @kodjima33) on how to allocate or share the reward with @TheoFrk. My primary goal was to ensure the contribution was cleanly integrated and verified for the Omi community.

I've also posted this clarification in #13071 for visibility. Thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-maintainer-review Needs a human maintainer to sign off before merge python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python SDK: allow language selection for local Whisper transcription

2 participants