Skip to content

[WIP][DO NOT MERGE] XNNPACK BYOC backend for Relax CPU inference #19580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
mshr-h wants to merge 18 commits into
base: main
Choose a base branch
from
Draft

[WIP][DO NOT MERGE] XNNPACK BYOC backend for Relax CPU inference #19580

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
f1c166e
Add XNNPACK BYOC skeleton
mshr-h May 17, 2026
c3ad896
Add minimal XNNPACK ReLU BYOC pipeline
mshr-h May 17, 2026
e0cd433
Add minimal CNN BYOC operator support
mshr-h May 17, 2026
fa646af
Add end-to-end validation and benchmark script
mshr-h May 17, 2026
917bda9
Harden runtime options and capability handling
mshr-h May 17, 2026
3b819ef
Add runtime FP16 precision policy options
mshr-h May 17, 2026
bab5c59
Add partition cost policy and decision report
mshr-h May 17, 2026
30436ac
Add XNNPACK quantization metadata plumbing
mshr-h May 17, 2026
866b1ed
Add QS8 weighted op plumbing
mshr-h May 17, 2026
cb45e36
Add signed-int8 TFLite QDQ import plumbing
mshr-h May 17, 2026
812286c
Add QS8 QDQ island ops for pool/reshape/add
mshr-h May 17, 2026
36214a4
Add deployment hardening and benchmark matrix support
mshr-h May 17, 2026
ea8bac9
Add opt-in dynamic-range fully-connected path
mshr-h May 17, 2026
1e25a89
Add opt-in dynamic batch support for Relax BYOC
mshr-h May 17, 2026
ca88094
Harden JSON runtime validation and size checks
mshr-h May 17, 2026
d0a18df
Add fp32 MLP GELU and softmax BYOC coverage
mshr-h May 17, 2026
a4ce7c1
Add larger benchmark fixtures and reporting
mshr-h May 17, 2026
e6446d4
Prune unstable BYOC paths and add typed configs
mshr-h May 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ tvm_option(USE_NNAPI_CODEGEN "Build with NNAPI Codegen support" OFF)
tvm_option(USE_NNAPI_RUNTIME "Build with NNAPI runtime" OFF)
tvm_option(USE_EXAMPLE_NPU_CODEGEN "Build with Example NPU Codegen support" OFF)
tvm_option(USE_EXAMPLE_NPU_RUNTIME "Build with Example NPU runtime" OFF)
tvm_option(USE_XNNPACK "Build with XNNPACK Relax BYOC support" OFF)
tvm_option(SUMMARIZE "Print CMake option summary after configuring" OFF)
tvm_option(USE_CLML "Build with CLML Codegen support" OFF)
tvm_option(USE_CLML_GRAPH_EXECUTOR "Build with CLML graph runtime" OFF)
Expand Down Expand Up @@ -468,6 +469,7 @@ include(cmake/modules/contrib/CoreML.cmake)
include(cmake/modules/contrib/TensorRT.cmake)
include(cmake/modules/contrib/NNAPI.cmake)
include(cmake/modules/contrib/ExampleNPU.cmake)
include(cmake/modules/contrib/XNNPACK.cmake)
include(cmake/modules/contrib/vllm.cmake)
include(cmake/modules/Git.cmake)

Expand Down
7 changes: 7 additions & 0 deletions cmake/config.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,13 @@ set(USE_CLML OFF)
# USE_CLML_GRAPH_EXECUTOR - CLML SDK PATH or ON or OFF
set(USE_CLML_GRAPH_EXECUTOR OFF)

# Whether to build with XNNPACK Relax BYOC support.
# Possible values:
# - ON: enable with CMake's default library/header search paths
# - /path/to/xnnpack/prefix: use a specific XNNPACK install prefix
# - OFF: disable XNNPACK support
set(USE_XNNPACK OFF)

# Whether use Thrust
# Possible values:
# - ON: enable Thrust with CMake's auto search
Expand Down
Loading
Loading