[WIP][DO NOT MERGE] XNNPACK BYOC backend for Relax CPU inference#19580
[WIP][DO NOT MERGE] XNNPACK BYOC backend for Relax CPU inference#19580mshr-h wants to merge 18 commits into
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request integrates XNNPACK as a Relax BYOC backend, enabling support for static-shape float32 and quantized CNN subgraphs. The changes encompass CMake build support, Relax pattern registration with a cost-based partitioning policy, TFLite frontend updates for QDQ models, and a JSON-based external codegen. Reviewers identified multiple typos in the Python implementation where tvm.tirx was incorrectly used instead of tvm.tir, and suggested replacing a hardcoded float literal in the C++ codegen with std::numeric_limits<float>::max() for improved robustness.
|
|
||
| shape = [] | ||
| for dim in sinfo.shape.values: | ||
| if not isinstance(dim, (tvm.tirx.expr.IntImm, int)): |
There was a problem hiding this comment.
The submodule tvm.tirx does not exist in the standard TVM library. This appears to be a typo for tvm.tir.
| if not isinstance(dim, (tvm.tirx.expr.IntImm, int)): | |
| if not isinstance(dim, (tvm.tir.IntImm, int)): |
|
|
||
|
|
||
| def _symbol_name(dim) -> str | None: | ||
| if isinstance(dim, (tvm.tirx.expr.IntImm, int)): |
There was a problem hiding this comment.
The submodule tvm.tirx does not exist in the standard TVM library. This appears to be a typo for tvm.tir.
| if isinstance(dim, (tvm.tirx.expr.IntImm, int)): | |
| if isinstance(dim, (tvm.tir.IntImm, int)): |
| result: list[int | None] = [] | ||
| symbol: str | None = None | ||
| for index, dim in enumerate(dims): | ||
| if isinstance(dim, (tvm.tirx.expr.IntImm, int)): |
There was a problem hiding this comment.
The submodule tvm.tirx does not exist in the standard TVM library. This appears to be a typo for tvm.tir.
| if isinstance(dim, (tvm.tirx.expr.IntImm, int)): | |
| if isinstance(dim, (tvm.tir.IntImm, int)): |
| if not isinstance(expr, relax.PrimValue): | ||
| return None | ||
| value = expr.value | ||
| if isinstance(value, tvm.tirx.expr.FloatImm): |
There was a problem hiding this comment.
The submodule tvm.tirx does not exist in the standard TVM library. This appears to be a typo for tvm.tir.
| if isinstance(value, tvm.tirx.expr.FloatImm): | |
| if isinstance(value, tvm.tir.FloatImm): |
| value = expr.value | ||
| if isinstance(value, tvm.tirx.expr.FloatImm): | ||
| return float(value.value) | ||
| if isinstance(value, tvm.tirx.expr.IntImm): |
There was a problem hiding this comment.
The submodule tvm.tirx does not exist in the standard TVM library. This appears to be a typo for tvm.tir.
| if isinstance(value, tvm.tirx.expr.IntImm): | |
| if isinstance(value, tvm.tir.IntImm): |
| } | ||
|
|
||
| private: | ||
| static constexpr double kXNNPACKInfinity = 3.4028234663852886e38; |
There was a problem hiding this comment.
Using a hardcoded literal for the maximum float value is less robust than using standard library constants. Consider using std::numeric_limits<float>::max().
| static constexpr double kXNNPACKInfinity = 3.4028234663852886e38; | |
| static constexpr double kXNNPACKInfinity = std::numeric_limits<float>::max(); |
1 participant
Just experimenting.
Benchmark on NVIDIA DGX Spark:
xnnpack_tiny_cnnxnnpack_static_qs8_tiny_cnnxnnpack_large_cnn_fp32xnnpack_large_mlp_fp32xnnpack_large_qs8_cnntorchvision:mobilenet_v2torchvision:mobilenet_v3_smalltorchvision:resnet18