Skip to content

HuBERT support rollout#1111

Open
david-wei-01001 wants to merge 89 commits intoTransformerLensOrg:devfrom
david-wei-01001:hubert
Open

HuBERT support rollout#1111
david-wei-01001 wants to merge 89 commits intoTransformerLensOrg:devfrom
david-wei-01001:hubert

Conversation

@david-wei-01001
Copy link

@david-wei-01001 david-wei-01001 commented Nov 7, 2025 •
edited
Loading

Description

This add support for the HuBERT model, extending Transformer Lens to speech/audio models. Codes has been thoroughly tested, test codes can be found in demos/HuBERT_test folder. No additional dependencies needed

Fixes # (issue)

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

@david-wei-01001
Create hubert_block.py
f7e76b6
@david-wei-01001
Delete transformer_lens/components/hubert_block.py
926c1c4
@david-wei-01001
Create HookedAudioEncoder.py
95d50bc
@david-wei-01001
Update HookedAudioEncoder.py
9a29295
@david-wei-01001
Update HookedAudioEncoder.py
48c6efe
@david-wei-01001
Update HookedAudioEncoder.py
1b7559c
@david-wei-01001
Update HookedAudioEncoder.py
94fa33e
@david-wei-01001
Update loading_from_pretrained.py
6e93a5b
@david-wei-01001
Update HookedAudioEncoder.py
4edde8d
@david-wei-01001
Update loading_from_pretrained.py
a5ef321
@david-wei-01001
Update loading_from_pretrained.py
cd930f3
@david-wei-01001
Update loading_from_pretrained.py
4621730
@david-wei-01001
Create hubert.py
548e693
@david-wei-01001
Update HookedAudioEncoder.py
5dc88a1
@david-wei-01001
Update HookedAudioEncoder.py
8282805
@david-wei-01001
Create hubert_test.py
7f0c373
@david-wei-01001
Update hubert_test.py
e8bbf84
@david-wei-01001
Update HookedAudioEncoder.py
86ac1d9
@david-wei-01001
Create hubert_ctc_test.py
8f1b889
@david-wei-01001
Update HookedAudioEncoder.py
afc2a35
@david-wei-01001
Create hubert_hook_test.py
f94fa40
@david-wei-01001
Update hubert_hook_test.py
cff50b3
@david-wei-01001
done
764810a
@david-wei-01001
done
7e844a3
@david-wei-01001
done
9a6bc7a
@david-wei-01001
done
1ddbf7f
@david-wei-01001
done
c646ee5
@david-wei-01001
Rename hubert_ctc_test.py to demos/HuBERT_test/hubert_ctc_test.py
7d5fe2a
@david-wei-01001
Rename hubert_hook_test.py to demos/HuBERT_test /hubert_hook_test.py
21a0256
@david-wei-01001
Rename hubert_hook_test.py to hubert_hook_test.py
c9f7c68
david-wei-01001 and others added 24 commits November 9, 2025 14:27
@david-wei-01001
Update HookedAudioEncoder.py
fefcea2
@david-wei-01001
Update HookedAudioEncoder.py
b1414e0
@david-wei-01001
Update HookedAudioEncoder.py
94bd3d7
@david-wei-01001
Update HookedAudioEncoder.py
fbae9c1
@david-wei-01001
Update HookedAudioEncoder.py
f23d0d9
@david-wei-01001
Update HookedAudioEncoder.py
d20ee07
@david-wei-01001
Update HookedAudioEncoder.py
00c12cb
@david-wei-01001
Update HookedAudioEncoder.py
14ab5bb
@david-wei-01001
Update loading_from_pretrained.py
41402ba
@david-wei-01001
Update loading_from_pretrained.py
b5cb2e1
@david-wei-01001
Update loading_from_pretrained.py
f8200bc
@david-wei-01001
Update loading_from_pretrained.py
6926e2b
@david-wei-01001
Update loading_from_pretrained.py
e8e958c
@david-wei-01001
Update requirements.txt
fa89321
@david-wei-01001
Update requirements.txt
5a7c5c7
@david-wei-01001
Update loading_from_pretrained.py
cd8e922
@david-wei-01001
Update loading_from_pretrained.py
77285ba
@david-wei-01001
Update loading_from_pretrained.py
9fa6464
@david-wei-01001
Update HookedAudioEncoder.py
fc9327e
@david-wei-01001
Update bert_pooler.py
c6a43a7
@david-wei-01001
Update bert_pooler.py
9427068
@jlarson4
Merge remote-tracking branch 'origin/dev' into hubert
e72aee1
@jlarson4
Apply isort and black formatting to HuBERT files
e0a649d
@jlarson4
Merge branch 'dev' into hubert
fc6ec7e
jlarson4
jlarson4 reviewed Mar 16, 2026
View reviewed changes
demos/HuBERT_test/hubert_ctc_test.py
Copy link
Collaborator

@jlarson4 jlarson4 Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, we build all our demos as Jupiter Notebooks, which can be run in an IDE or in Google Colab. This is for ease of use. Please reformat these demos/HuBERT_test/ files into a notebook, and create a set of tests for your new functionality in tests/

@jlarson4
Copy link
Collaborator

jlarson4 commented Mar 16, 2026

Hello @david-wei-01001! I took some time to look over this contribution today, thank you for contributing! There are a couple of changes I've requested above that will need to be done before I can consider this for inclusion in a release. Please feel free to reach out to me if you have any questions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jlarson4 jlarson4 jlarson4 left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@david-wei-01001 @jlarson4