Skip to content

Split long non-causal SDPA queries to avoid Ultra corruption - #64

Open
quantz8a wants to merge 1 commit into
antirez:mainfrom
quantz8a:fix/16-sdpa-query-block
Open

Split long non-causal SDPA queries to avoid Ultra corruption#64
quantz8a wants to merge 1 commit into
antirez:mainfrom
quantz8a:fix/16-sdpa-query-block

Conversation

@quantz8a

@quantz8a quantz8a commented Sep 12, 2026

Copy link
Copy Markdown

Summary

  • Work around MPSGraph scaledDotProductAttention silent corruption on M2/M3 Ultra for long non-causal sequences (Corrupted video (and audio) outputs #16 / Apple FB24605554).
  • Chunk query rows (default 2048) when sequence > 12288; leave smaller jobs on the fast single-op path.
  • Tunables: --sdpa-query-block N, H3_SDPA_MAX_QUERY_ROWS (0 disables), H3_SDPA_SPLIT_MIN_ROWS.

Fixes #16
Related: #15 (same size class; assert vs silent garbage)

Notes

  • Non-causal only (DiT path). Causal GQA unchanged.
  • Split can increase MPSGraph intermediate residency; combine with --ssd-streaming for long HQ jobs (as discussed in Corrupted video (and audio) outputs #16).

Test plan

  • Compiles h3_gpu.m / main.c with stock flags
  • make -j && ./h3_tests
  • On M2/M3 Ultra: 896×512 or 1344×768, ≥100 frames, steps≥4 previously produced static → now coherent scene/audio
  • Short job (512×512, 22–107f) unchanged wall-clock path (no split below gate)
  • Local M3 Ultra validation in progress (896×512, 107f, steps=4, --ssd-streaming)

MPSGraph scaledDotProductAttention can silently return
garbage on M2/M3 Ultra above ~15k query rows (antirez#16).
Chunk non-causal queries (default block 2048, only when
sequence > 12288). Tune with --sdpa-query-block / env
H3_SDPA_MAX_QUERY_ROWS and H3_SDPA_SPLIT_MIN_ROWS;
set MAX to 0 to disable.
@quantz8a
quantz8a force-pushed the fix/16-sdpa-query-block branch from d5ad814 to 39e9569 Compare September 12, 2026 15:55
@quantz8a

Copy link
Copy Markdown
Author

Local M3 Ultra (96GB) validation for this PR:

  • 896×512, frames=90, steps=4, layers=45, --ssd-streaming, H3_DIT_COMMAND_BLOCKS=5
  • Completed exit=0; wrote a playable MP4 (duration=3.75s)
  • Mid-frame luma contrast check: mean≈17.5, std≈17.5 (not collapsed static)

This geometry is in the Ultra corruption zone discussed in #16; output is coherent rather than noise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Corrupted video (and audio) outputs

1 participant