Skip to content

cc's scalar half uniform1 consumes 16 bits of the key where CUDA and Metal consume 32, so one key draws different half values per backend (bf16 and the lane forms agree) #951

Description

@lukstafi

Seen while building gh-ocannl-754's width-parity corpus (staging PR #663): its discrimination control reads back the draw of one fixed key, uniform1(Constant_bits 0x9E3779B9), at each narrow precision, and printed on stderr:

precision cc (M4 Max and x86 Linux) Metal
bf16 0x1.3cp-1 0x1.3cp-1
f16 0x1.e7p-2 0x1.3c8p-1

The bf16 draws agree; the f16 draws are different numbers, not different roundings of one number. The builtins say why: builtins_cc.ml's scalar uint4x32_to_half_uniform computes (x.v[0] & 0xFFFF) * (1/65536) — the low 16 bits — while CUDA's (builtins_cuda.ml) and Metal's (builtins_metal.ml) compute uint32_to_single_uniform(x.v[0]) then narrow to half, i.e. all 32 bits, exactly as every backend's scalar bf16 conversion does (cc's bf16 also goes through uint32_to_single_uniform). The _vec/_lane half forms consume 16 bits per lane on all three backends, so the tensor uniform () path is consistent; only the scalar uniform1 at half is the odd one out, and only on cc (HIP mirrors CUDA).

gh-ocannl-517's contract is that a conversion picks which bits it consumes from the precision it renders at — per precision, not per backend — and narrow_rng_nesting pins virtual-vs-materialized parity within one backend, so nothing checks a draw ACROSS backends today. A half model whose scalar parameters are initialized through uniform1 therefore starts from different values on cc than on the GPUs, which also silently breaks any cc-vs-GPU parity test that seeds through it.

Fix: cc's scalar half conversion takes the 32-bit route like its siblings (one line in builtins_cc.ml, plus the host copy in builtins.c — see gh-ocannl-656 for why the two exist). The 16-bit spelling was presumably meant to match the lane form, but the scalar bf16 conversion already chose the other convention on every backend. A cross-backend draw golden — the two rows above, printed by every backend that runs reduction_forms — would pin it; today the value is deliberately stderr-only because it was known to differ.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions