Skip to content

kernels: support conditions on use_kernel_forward_from_hub - #796

Merged
danieldk merged 3 commits into
mainfrom
conditional-layer
Sep 3, 2026
Merged

kernels: support conditions on use_kernel_forward_from_hub#796
danieldk merged 3 commits into
mainfrom
conditional-layer

Conversation

@danieldk

Copy link
Copy Markdown
Member

Add and optional condition argument to use_kernel_forward_from_hub.
This condition accepts an instance of the layer and returns whether
kernelization of the layer should proceed. For instance,

@use_kernel_forward_from_hub(
    "SwiGLUMLP",
    condition=lambda module: module.config.hidden_act == "silu",
)
class MyMLP(nn.Module):
    ...

would only kernelize the layer using a mapping registered for SwiGLUMLP
its hidden_act configuration is silu.

Add and optional `condition` argument to `use_kernel_forward_from_hub`.
This condition accepts an instance of the layer and returns whether
kernelization of the layer should proceed. For instance,

```
@use_kernel_forward_from_hub(
    "SwiGLUMLP",
    condition=lambda module: module.config.hidden_act == "silu",
)
class MyMLP(nn.Module):
    ...
```

would only kernelize the layer using a mapping registered for `SwiGLUMLP`
its `hidden_act` configuration is `silu`.
@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown

Coverage report — kernels/

Measured on: Python 3.10 / Torch 2.13.0.
Other CI configurations are not included in this number.
Hardware-gated code paths (ROCm/XPU/NPU/Darwin/Windows) are excluded or unreachable on the Linux+CUDA runner.

Total coverage: 86.9% — threshold: 80% — ✅

Per-file breakdown
Name Stmts Miss Cover Missing
src/kernels/__init__.py 14 0 100%
src/kernels/_system.py 6 1 83% 10
src/kernels/_versions.py 78 9 88% 47, 53-54, 57-58, 97, 119, 130, 136
src/kernels/archs.py 56 1 98% 95
src/kernels/backends.py 212 62 71% 40, 44, 48-51, 68, 90, 108, 117, 121, 125-127, 148, 157, 161, 165-167, 188, 199, 201, 208-211, 224, 228, 232-252, 260, 283-303
src/kernels/compat.py 8 1 88% 5
src/kernels/deps.py 103 2 98% 59, 132
src/kernels/hf_hub.py 62 2 97% 18, 20
src/kernels/importer.py 57 5 91% 112, 116, 119, 133-134
src/kernels/install.py 21 7 67% 77-101
src/kernels/layer/__init__.py 6 0 100%
src/kernels/layer/_interval_tree.py 103 4 96% 23, 52, 147, 150
src/kernels/layer/device.py 48 14 71% 42, 47-49, 91, 96-98, 101, 149, 152, 155-157
src/kernels/layer/func.py 87 6 93% 91, 124, 198, 320, 346, 376
src/kernels/layer/globals.py 5 0 100%
src/kernels/layer/kernelize.py 80 8 90% 258, 293, 301-302, 308, 312, 328-330
src/kernels/layer/layer.py 217 14 94% 183, 230, 256, 390, 470-471, 492, 500, 511, 540, 544, 557, 610, 640
src/kernels/layer/mode.py 14 0 100%
src/kernels/layer/repos.py 144 42 71% 27, 33, 36-43, 63-64, 70, 73-76, 90, 94, 103-104, 110, 113-116, 123-124, 130, 133-136, 143-144, 150, 153-156, 163-164, 170, 173-176, 257
src/kernels/load.py 69 2 97% 329, 368
src/kernels/locking.py 89 64 28% 35-83, 91-98, 102-125, 137, 152-159, 165-175, 179-186
src/kernels/python_deps.py 58 6 90% 59-60, 64-65, 101, 104
src/kernels/resolver.py 141 2 99% 182, 188
src/kernels/status.py 49 2 96% 23, 81
src/kernels/variants.py 278 19 93% 64, 95, 116, 146, 255-256, 298-301, 303, 387-394, 400-406, 437-443, 455-461
src/kernels/verify.py 88 1 99% 32
TOTAL 2093 274 87%

Updated by the Test kernels workflow on commit cdee0896f17c0032c541a9711011396b3d6f2a8f.

@vasqu vasqu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Careful approval but I sanity checked on huggingface/transformers#48335 (llama) and it worked as expected, e.g. when passing a new config with activation == gelu, we no longer kernelize that module

Ofc, docs are missing but since it's a draft I expect it tbh 🤗

@danieldk

danieldk commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Thanks for testing! I'll add the docs.

@danieldk
danieldk marked this pull request as ready for review September 1, 2026 09:31
Comment thread kernels/tests/test_layer.py

@sayakpaul sayakpaul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just single comment regarding testing. Otherwise, looks good.

drbh
drbh previously approved these changes Sep 1, 2026

@drbh drbh left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm to me too - should be good to merge after the comments above

@sayakpaul sayakpaul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really cool. Particularly the tests are 🤌

@danieldk
danieldk merged commit d7c10c0 into main Sep 3, 2026
52 of 54 checks passed
@danieldk
danieldk deleted the conditional-layer branch September 3, 2026 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants