Add ATom Kimi-K3 for MI355X - #184
Open
itej89 wants to merge 6 commits into
Open
Conversation
Introduces ATom (ROCm/ATOM) integration to MAD, modeled after MAD-Private client-perf. Includes Dockerfile, run scripts, and Kimi-K3 config based on ROCm/ATom PR #1718 recipe (gfx950 TP8, MXFP4 w/ FlyDSL SiTUv2, FP8 KV cache). Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
itej89
requested review from
Rohan138,
amathews-amd,
coketaste,
gargrahul and
ppalaniappan-amd
as code owners
July 28, 2026 23:17
…overs Must-fix: - Remove internal references (MAD-private, client-perf-hub, PR #380) - Remove dead --profile flag and rocm-trace-lite clone from run.sh - Remove duplicate --kv_cache_dtype from perf.yaml, fix batched tokens to 10240 - Add missing env vars (ATOM_USE_UNIFIED_ATTN, ATOM_FORCE_ATTN_TRITON) - Simplify skip_gpu_arch to gfx942 (matches convention) - Remove atom_default tag (prevents accidental sweep of 1TB+ checkpoint) - Remove dev leftover scripts/atom/models.json Should-fix: - Guard HF_HUB_CACHE in run.sh (match vLLM pattern) Co-Authored-By: Claude <noreply@anthropic.com>
- Dockerfile: use rocm/atom-dev:rocm7.2.4_..._kimi_k3 image - Config: add all ATOM/AITER env vars from the MI355X recipe - Fix --max-num-batched-tokens to 10240 (recipe value) Co-Authored-By: Claude <noreply@anthropic.com>
Use MAD_DATAHOME as the model path directly instead of joining with model name. Matches vLLM (run_vllm.py:436) and SGLang (run_sglang.py:317) behavior. Removes CHECK_LOCAL_DATA gate. Co-Authored-By: Claude <noreply@anthropic.com>
vLLM and SGLang use model-specific Dockerfile names (pyt_vllm_kimi_k3, pyt_sglang_kimi_k3). ATom should too. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
client-perfbranchpyt_atom_kimi-k3tag for Kimi-K3 on MI355X (gfx950, TP8)recipes/Kimi-K3.md)Files
docker/pyt_atom.ubuntu.amd.DockerfileFROM rocm/atom-dev:latest)scripts/atom/run.shrun_atom.pyscripts/atom/run_atom.pyscripts/atom/models.jsonscripts/atom/configs/default.yamlscripts/atom/configs/perf.yamlscripts/atom/configs/accuracy.yamlmodels.jsonpyt_atom_kimi-k3entryATom recipe env vars
Notes
rocm/atom-dev:latestdoes not yet include K3 — overrideBASE_DOCKERvia--additional-contextwith a K3-enabled image until [ATOM][Kimi-K3] ATOM main support Kimi-K3 ATOM#1718 mergesskip_gpu_arch: "gfx942, gfx906, gfx908, gfx90a, A100, H100, V100"matches upstream MXFP4 patternTest plan
python3 -c "import json; json.load(open('models.json'))"madengine run --tags pyt_atom_kimi-k3 --dry-run🤖 Generated with Claude Code