fix(minimax): repair HF parity reference - map legacy rotary_dim to partial_rotary_factor - #48486
fix(minimax): repair HF parity reference - map legacy rotary_dim to partial_rotary_factor#48486dajiaohuang wants to merge 3 commits into
Conversation
|
Fixed the missed modular conversion output in commit 26c0e74. The PR's new convert_rope_params_to_dict method is now present in generated configuration_minimax_m2.py, resolving the repository consistency diff and the KeyError in test_legacy_rotary_dim_maps_to_partial_rotary_factor. Validation: Python compileall passed and git diff --check passed. |
…elope Released MiniMaxAI/MiniMax-M2* checkpoints are trained with partial RoPE (rotary_dim 64 of head_dim 128) and express it through a legacy rotary_dim config field; their original remote-code config derived partial_rotary_factor = rotary_dim / head_dim in __init__. The in-tree MiniMaxM2Config keeps rotary_dim as an opaque extra attribute, so rope_parameters never receives a partial factor and MiniMaxM2RotaryEmbedding silently rotates the full head dimension with a full-width frequency ladder. Map the legacy field in convert_rope_params_to_dict, following the GPTNeoXConfig rotary_pct precedent; an explicit partial_rotary_factor still takes priority and configs without rotary_dim are unaffected. Fixes huggingface#48241
26c0e74 to
e434db8
Compare
|
Follow-up after the latest consistency run: the failure also compared against upstream main at c119ec3, which had advanced beyond the fork's stale main and introduced inherited conversion output in minimax_m3_vl/step3p7. I rebased the existing branch onto current upstream main and regenerated the MiniMax M2 config output, including exact converter formatting, in commit e434db8. The branch now contains only the modular source, matching generated configuration, and focused test changes. The earlier CI failure was reproduced from job 100931027864. |
|
The latest consistency failure (run 33845763052 / job 100937312164) was from generated descendants of MiniMaxM2Config: minimax_m3_vl and step3p7 were missing the inherited converter override, while the MiniMaxM2 generated file itself was correct. Regenerated both configuration files on top of upstream main and pushed as a20b527. The focused modular conversion check now passes with UTF-8 enabled. |
|
[For maintainers] Suggested jobs to run (before merge) run-slow: minimax_m2, minimax_m3_vl, step3p7 |
CI recapDashboard: View test results in Grafana |
What does this PR do?
Fixes #48241.
Released MiniMaxAI/MiniMax-M2* checkpoints are trained with partial RoPE (
rotary_dim: 64ofhead_dim: 128) and express it through a legacyrotary_dimconfig field; their original remote-code config derivedpartial_rotary_factor = rotary_dim / head_dimin__init__. The in-treeMiniMaxM2Configkeepsrotary_dimas an opaque extra attribute, sorope_parametersnever receives a partial factor andMiniMaxM2RotaryEmbeddingsilently rotates the full head dimension with a full-width frequency ladder — wrong frequency spacing and wrong rotate-half pairing in every attention layer (measured: mean KL 13.1 / cosine 0.02 vs a faithful reference on MiniMax-M2.7 over 2,048 tokens; details in the issue).This maps the legacy field in a
convert_rope_params_to_dictoverride onMiniMaxM2Config(inmodular_minimax_m2.py), following theGPTNeoXConfigrotary_pctprecedent:MiniMaxM2Config(rotary_dim=64)→rope_parameters["partial_rotary_factor"] == 0.5,inv_freqlength 32;partial_rotary_factorinrope_parametersstill wins;rotary_dimare unaffected.Before submitting
rotary_dimfield #48241test_legacy_rotary_dim_maps_to_partial_rotary_factor(covers legacy mapping, explicit-factor priority, and the no-legacy-field case)Who can review?
@ArthurZucker @Cyrilvallez