Skip to content

Xcode 26.5-built MLX-C runtime returns incorrect BF16 gather/RMSNorm/matmul result #115

Description

@pd95

While porting Apertus to the MLX-based Ollama runner, we observed corrupted output after rebuilding the runtime with Xcode 26.5.
I reduced this to a standalone MLX-C repro with constant tensors and no model/application code.

The same repro passes when MLX-C/MLX is built with Xcode 26.4, but fails when built with Xcode 26.5.

Result Matrix

MLX-C pinned ref fba4470 + Xcode 26.5: FAIL, first=512
MLX-C main + Xcode 26.5:               FAIL, first=512
MLX-C pinned ref fba4470 + Xcode 26.4: PASS, first=1024

Full pinned MLX-C ref:

fba4470b89073180056c9ea46c443051375f7399

The included run used CMAKE_OSX_DEPLOYMENT_TARGET=26.0.

Repro Operation

The repro computes:

BF16 take_axis/gather -> fast_rms_norm -> BF16 matmul

with constant tensors:

embedding:      [131072, 4096] BF16, all 0.5
RMSNorm weight: [4096] BF16, all 1.0
q_proj weight:  [4096, 4096] BF16, all 0.25

Expected:

4096 * (0.5 / sqrt(0.5^2 + 1e-5)) * 0.25 ~= 1023.9795

Observed with Xcode 26.5-built runtime:

first=512.00000000
status=FAIL

Observed with Xcode 26.4-built runtime:

first=1024.00000000
status=PASS

The 512 result is equivalent to the matmul consuming the unnormalized 0.5 input rather than the RMSNorm output.

Toolchain Details

From summary.log:

Xcode 26.4 SDK: 26.4
Metal Toolchain: com.apple.MobileAsset.MetalToolchain-v17.5.188.0.mGsQWM
Result: PASS

Xcode 26.5 SDK: 26.5
Metal Toolchain: com.apple.MobileAsset.MetalToolchain-v17.6.42.0.1BIDXR
Result: FAIL

Both toolchains report:

Apple metal version 32023.883 (metalfe-32023.883)
Target: air64-apple-darwin25.5.0

Reproduction

The attached archive contains:

mlx-c-bug/run.sh
mlx-c-bug/repro.c
mlx-c-bug/README.md
mlx-c-bug/results/20260514-193925/summary.log
per-case repro logs, commit refs, runtime file lists, and checksums

Run:

./mlx-c-bug/run.sh --clean-work

The script clones ml-explore/mlx-c, builds MLX-C/MLX directly with the selected Xcode versions, collects the produced runtime files, compiles repro.c against libmlxc.dylib, and runs the constant tensor check.

Question

Is this expected to be an Xcode/Metal Toolchain regression, or is there a build flag/configuration needed for MLX-C with Xcode 26.5 and deployment target 26.0?

mlx-c-bug-repro-20260514.zip

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions