support 5,6,7 bits on cuda and add humming backend

[wenhuach21]

Description

Please briefly describe your main changes, the motivation.

Type of Change

Bug fix

Related Issues

Fixes or relates to #

Checklist Before Submitting

• My code has been tested locally.
• Documentation has been updated as needed.
• New or updated tests are included where applicable.
• The CUDA CI has passed. You can trigger it by commenting /azp run Unit-Test-CUDA-AutoRound.

[wenhuach21]

vllm-project/vllm#52890
huggingface/transformers#48481

[wenhuach21]

@copilot resolve the merge conflicts in this pull request

[Copilot]

@copilot resolve the merge conflicts in this pull request

Resolved the merge conflicts by merging main in commit 53d4571.

[Copilot]

🟡 Changes recommended

There are correctness/docs issues (corrupted docs snippet, backend constraint mismatch), misleading format support reporting, missing targeted CUDA test coverage for 5/6/7-bit paths, and unrelated dataset-scope changes bundled into the same PR.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR extends AutoRound’s CUDA path to support 5/6/7-bit weight-only quantization by introducing a generic bit-stream packing layout and adding a new optional CUDA inference backend backed by the external humming kernels, alongside documentation and format-matrix updates.

Changes:

• Add generic little-endian bit-stream pack/unpack utilities and wire them into GPTQ/AWQ-style torch fallback packing for 5/6/7 bits.
• Register new CUDA backend variants (auto_round:humming*) and add post-init handling to repack/JIT kernels when humming is selected.
• Update docs/tests and export-format scheme lists to reflect 5/6/7-bit support in the auto_round family.

File summaries

File	Description
test/unit/test_cpu/export/test_export.py	Updates expected error message for AWQ scheme support text.
docs/step_by_step.md	Documents 5/6/7-bit packing and humming usage; updates scheme/format tables.
docs/step_by_step_CN.md	Chinese translation updates for the new 5/6/7-bit and humming documentation.
auto_round/utils/bit_packing.py	New generic bit-stream packing implementation (1–8 bits) plus AWQ reorder helpers.
auto_round/inference/convert_model.py	Adds humming QuantLinear construction and humming post-init repack pass.
auto_round/inference/backend.py	Expands torch backend bit support; registers humming backends; adds callable requirement handling.
auto_round/export/formats/backends/autoround.py	Adds W5A16/W6A16/W7A16 to AutoRound format support list.
auto_round/export/formats/backends/auto_gptq.py	Allows W5/6/7 schemes but restricts them to auto_round* formats.
auto_round/export/formats/backends/auto_awq.py	Allows W5/6/7 schemes but restricts them to auto_round* formats; adjusts alignment checks.
auto_round/export/export_to_awq/utils.py	Implements 5/6/7-bit AWQ pack/unpack via generic bit-stream packing and updates alignment rules.
auto_round/compressors/mllm/dataset.py	Refactors LLaVA dataset download logic (hf_hub_download preferred; retries; better errors).
auto_round_extension/torch/qlinear_torch.py	Adds generic 5/6/7-bit pack/unpack path for torch fallback QuantLinear.
auto_round_extension/torch/qlinear_torch_zp.py	Mirrors generic 5/6/7-bit support for the GPTQ zp-1 variant.
auto_round_extension/humming/qlinear_humming.py	New inference-only humming QuantLinear implementations (GPTQ/AWQ variants).
auto_round_extension/humming/init.py	Exposes humming QuantLinear classes and availability helper.

Review details

Suppressed comments (1)

auto_round/export/formats/backends/auto_awq.py:50

• support_schemes/check_scheme_args now include W5A16/W6A16/W7A16 for the auto_awq format, but check_and_reset_format rejects 5/6/7 bits unless the format starts with auto_round. This can mislead users (and auto_round list format) into thinking the plain auto_awq export supports 5/6/7-bit when it doesn't.

    support_schemes = ["W4A16", "W5A16", "W6A16", "W7A16"]
    format_name = "auto_awq"

    # See AutoGPTQFormat.EXTENDED_BITS -- 5/6/7-bit uses the generic bit-stream
    # layout, which upstream AutoAWQ kernels cannot read.
    EXTENDED_BITS = (5, 6, 7)

    @classmethod
    def check_scheme_args(cls, scheme: QuantizationScheme) -> bool:
        error_logs = []
        if scheme.bits not in (4,) + cls.EXTENDED_BITS:
            error_logs.append(f"bits={scheme.bits}")
        if not re.search("int", scheme.data_type):
            error_logs.append(f"data_type={scheme.data_type}")
        if scheme.super_bits:
            error_logs.append(f"super_bits={scheme.super_bits}")
        if scheme.super_group_size:
            error_logs.append(f"super_group_size={scheme.super_group_size}")
        if error_logs:
            raise ValueError(
                f"{cls.format_name} format support quantization scheme with {','.join(cls.support_schemes)} "
                f"but got {', '.join(error_logs)}, please have a check."
            )

• Files reviewed: 15/15 changed files
• Comments generated: 8
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[AutoRoundBot]

/azp run Unit-Test-CUDA-AutoRound

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[AutoRoundBot]

/azp run Unit-Test-CUDA-AutoRound

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[AutoRoundBot]

/azp run Unit-Test-CUDA-AutoRound

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[wenhuach21]

need to add uts later