[WIP] [AMD/ROCM] atom minimaxm2.5 fp4 on mi355x #1042
+106
−0
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| source "$(dirname "$0")/../benchmark_lib.sh" | ||
|
|
||
| check_env_vars \ | ||
| MODEL \ | ||
| TP \ | ||
| CONC \ | ||
| ISL \ | ||
| OSL \ | ||
| RANDOM_RANGE_RATIO \ | ||
| RESULT_FILENAME \ | ||
| EP_SIZE \ | ||
| DP_ATTENTION | ||
|
|
||
| if [[ -n "$SLURM_JOB_ID" ]]; then | ||
| echo "JOB $SLURM_JOB_ID running on $SLURMD_NODENAME" | ||
| fi | ||
|
|
||
| echo "TP: $TP, CONC: $CONC, ISL: $ISL, OSL: $OSL, EP_SIZE: $EP_SIZE, DP_ATTENTION: $DP_ATTENTION" | ||
|
|
||
| SERVER_LOG=/workspace/server.log | ||
| PORT=${PORT:-8888} | ||
|
|
||
| export OMP_NUM_THREADS=1 | ||
|
|
||
| # Calculate max-model-len based on ISL and OSL | ||
| if [ "$ISL" = "1024" ] && [ "$OSL" = "1024" ]; then | ||
| CALCULATED_MAX_MODEL_LEN="" | ||
| else | ||
| CALCULATED_MAX_MODEL_LEN=" --max-model-len 10240 " | ||
| fi | ||
|
|
||
| if [ "$EP_SIZE" -gt 1 ]; then | ||
| EP=" --enable-expert-parallel" | ||
| else | ||
| EP=" " | ||
| fi | ||
|
|
||
| # Start GPU monitoring (power, temperature, clocks every second) | ||
| start_gpu_monitor | ||
|
|
||
| set -x | ||
|
|
||
| python3 -m atom.entrypoints.openai_server \ | ||
| --model $MODEL \ | ||
| --server-port $PORT \ | ||
| -tp $TP \ | ||
| --kv_cache_dtype fp8 $CALCULATED_MAX_MODEL_LEN $EP \ | ||
| --trust-remote-code \ | ||
| > $SERVER_LOG 2>&1 & | ||
|
|
||
| SERVER_PID=$! | ||
|
|
||
| # Wait for server to be ready | ||
| wait_for_server_ready --port "$PORT" --server-log "$SERVER_LOG" --server-pid "$SERVER_PID" | ||
|
|
||
| export PYTHONDONTWRITEBYTECODE=1 | ||
| run_benchmark_serving \ | ||
| --model "$MODEL" \ | ||
| --port "$PORT" \ | ||
| --backend vllm \ | ||
| --input-len "$ISL" \ | ||
| --output-len "$OSL" \ | ||
| --random-range-ratio "$RANDOM_RANGE_RATIO" \ | ||
| --num-prompts "$((CONC * 10))" \ | ||
| --max-concurrency "$CONC" \ | ||
| --result-filename "$RESULT_FILENAME" \ | ||
| --result-dir /workspace/ \ | ||
| --trust-remote-code | ||
|
|
||
| # After throughput, run evaluation only if RUN_EVAL is true | ||
| if [ "${RUN_EVAL}" = "true" ]; then | ||
| run_eval --framework lm-eval --port "$PORT" | ||
| append_lm_eval_summary | ||
| fi | ||
|
|
||
| # Stop GPU monitoring | ||
| stop_gpu_monitor | ||
| set +x |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 The
minimaxm2.5-fp4-mi355x-atomconfig entry hasimage: TBDinstead of a valid Docker image tag, which will cause any CI pipeline targeting this config to fail immediately when attempting to pull the image. Before merging, replaceTBDwith a real atom image (the fp8 equivalent usesrocm/atom:rocm7.2.1-ubuntu24.04-pytorch2.9.1-atom0.1.2as a reference point).Extended reasoning...
What the bug is and how it manifests
The
minimaxm2.5-fp4-mi355x-atomconfig entry in.github/configs/amd-master.yaml(line 420) setsimage: TBD. This is not a valid Docker image reference. When any CI job targets this config entry, the container runtime will attemptdocker pull TBD(or equivalent), which will fail with an image-not-found error and abort the pipeline.The specific code path that triggers it
The CI benchmark workflow reads the
imagefield from the YAML config and passes it directly as the Docker image to pull and run (e.g.,IMAGE: ${{ inputs.image }}). There is no guard or filtering logic that would skip entries with placeholder values — the config is consumed as-is. Any automated benchmark job that selects theminimaxm2.5-fp4-mi355x-atomkey will immediately fail at the Docker pull step.Why existing code doesn't prevent it
The YAML config file has no schema validation or image-existence checks at parse time. The
# TODO:comments in the diff confirm the author is aware the image value is incomplete, but these are source comments only — they have no runtime effect and do not prevent the broken entry from being picked up by automation.What the impact would be
Any CI benchmark run that triggers the
minimaxm2.5-fp4-mi355x-atomconfig entry will fail with a Docker image pull error, producing a broken pipeline and potentially blocking other benchmark results from the same run.How to fix it
Replace
image: TBDwith the correct atom Docker image tag for the MiniMax M2.5 FP4 workload on MI355X. Based on the fp8 equivalent (minimaxm2.5-fp8-mi355x-atom), a likely candidate isrocm/atom:rocm7.2.1-ubuntu24.04-pytorch2.9.1-atom0.1.2or newer, but the exact image should be verified with the atom team before merging.Step-by-step proof
.github/configs/amd-master.yamland finds theminimaxm2.5-fp4-mi355x-atomkey.image: TBDand setsIMAGE=TBDin the job environment.docker pull TBD(or equivalent container invocation).Error: No such image: TBD/ registry lookup fails.