chore: refactor Docker runner launch to be like SLURM #227

cquil11 · 2025-11-12T22:36:27Z

H100: https://github.com/InferenceMAX/InferenceMAX/actions/runs/19379670915
H200: https://github.com/InferenceMAX/InferenceMAX/actions/runs/19435068076
B200: https://github.com/InferenceMAX/InferenceMAX/actions/runs/19379803427
B200-TRT: https://github.com/InferenceMAX/InferenceMAX/actions/runs/19433901216
MI300X: https://github.com/InferenceMAX/InferenceMAX/actions/runs/19380029693
MI325X: https://github.com/InferenceMAX/InferenceMAX/actions/runs/19433922587
MI355X: https://github.com/InferenceMAX/InferenceMAX/actions/runs/19380049049

"Server died before becoming healthy" test: https://github.com/InferenceMAX/InferenceMAX/actions/runs/19378513021/job/55451808912
Simulated by running on a Docker node, exec'ing into the container and killing a vLLM worker process.

Introduction

Currently, the logic for launching and running benchmarks on Docker nodes is different from that of SLURM nodes. This causes redundancy, confusion for first time contributors, and maintenance overhead (Excalidraw link). The below figure describes the current implementation

Implementation

First and foremost, we propose standardizing the way SLURM and Docker jobs are launched. Henceforth, all benchmark/eval/etc. logic should take place inside a single container. As with SLURM currently, this container should be responsible for both running the server and running the client (benchmarks). Furthermore, we introduce a utility script benchmarks/benchmark_lib.sh that will hold common functions among all benchmark scripts. In this PR, we add two functions:

wait_for_server_ready: a function that continuously polls the server on 0.0.0.0/health and waits for it to become ready
- Will exit 1 upon the server process failing
run_benchmark_serving: a function that standardizes the running of the client benchmark script

As we add more runners and different types of hardware, the number of benchmark scripts will grow exponentially. These standardized functions avoid redundant code throughout all the benchmark scripts and make the code more extensible. I.e., when a change is made to the bench_serving script, we now only have to change in one place instead of 27.

There is no “heavy lifting” involved in switching the Docker logic to the existing SLURM logic. All we must do is copy and paste the existing client logic from the runners/launch_XXXX.sh into the corresponding benchmarks/XXXX.sh scripts. This process is quite straightforward. Below is a diagram showing what all workflows will look like after the refactor.

Copilot

Pull Request Overview

This PR refactors Docker runner launch scripts to follow the same pattern as SLURM runners, where both the server and client logic execute within a single container. This eliminates the need for separate server and client containers, reduces code duplication, and introduces shared utility functions in benchmark_lib.sh.

Key Changes:

Introduced benchmarks/benchmark_lib.sh with reusable wait_for_server_ready() and run_benchmark_serving() functions
Migrated client benchmark logic from runner launch scripts into individual benchmark scripts
Changed Docker containers from detached multi-container setup to single foreground container running both server and client
Added new MI300X SLURM runners to the configuration

Reviewed Changes

Copilot reviewed 37 out of 39 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
benchmarks/benchmark_lib.sh	New utility library providing standardized server health checking and benchmark execution functions
benchmarks/*_docker.sh	Updated to run server in background, wait for health, then execute benchmarks using benchmark_lib.sh functions
benchmarks/*_slurm.sh	Updated to use benchmark_lib.sh utility functions, replacing inline wait loops and benchmark calls
runners/launch_mi355x-amd.sh	Removed separate client container logic, now launches single container running complete benchmark script
runners/launch_mi300x-*.sh	Removed network creation and client container, simplified to single container execution
runners/launch_h100-cr.sh	Removed client container logic, passes additional environment variables for in-container benchmarking
runners/launch_b200-*.sh	Removed client container separation, consolidated to single container pattern
runners/launch_mi300x-amds.sh	New SLURM-based runner for MI300X on AMD infrastructure
.github/workflows/benchmark-tmpl.yml	Added retry logic for result file detection to handle timing between benchmark completion and file availability
.github/configs/runners.yaml	Added four new MI300X AMDS runner configurations

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

runners/launch_b200-tg.sh

benchmarks/gptoss_fp4_b200_docker.sh

runners/launch_b200-nvd.sh

benchmarks/benchmark_lib.sh

.github/workflows/benchmark-tmpl.yml

benchmarks/gptoss_fp4_h100_docker.sh

benchmarks/dsr1_fp4_mi355x_docker.sh

benchmarks/dsr1_fp8_b200_docker.sh

runners/launch_mi355x-amd.sh

Copilot

Pull Request Overview

Copilot reviewed 37 out of 39 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

benchmarks/gptoss_fp4_h100_docker.sh

benchmarks/benchmark_lib.sh

Copilot · 2025-11-17T20:16:43Z

@cquil11 I've opened a new pull request, #244, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

benchmarks/benchmark_lib.sh

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 34 out of 36 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (2)

benchmarks/dsr1_fp8_mi325x_docker.sh:50

[nitpick] This line contains trailing whitespace. Consider removing it for cleaner code.
benchmarks/dsr1_fp8_mi355x_docker.sh:62
[nitpick] This line contains trailing whitespace. Consider removing it for cleaner code.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-21T21:06:01Z

benchmarks/dsr1_fp8_h200_trt_slurm.sh

    --tp_size=$TP --ep_size=$EP_SIZE \
    --extra_llm_api_options=$EXTRA_CONFIG_FILE \
    > $SERVER_LOG 2>&1 &
+


[nitpick] This line contains trailing whitespace. Consider removing it for cleaner code.

Suggested change

Copilot · 2025-11-21T21:06:01Z

benchmarks/gptoss_fp4_h100_docker.sh

+    --output-len "$OSL" \
+    --random-range-ratio "$RANDOM_RANGE_RATIO" \
+    --num-prompts $(( $CONC * 10 )) \
+    --max-concurrency 512 \


The max-concurrency parameter is hardcoded to 512 instead of using the $CONC variable like other benchmark scripts. This should be $CONC to match the intended concurrency level.

Suggested change

--max-concurrency 512 \

--max-concurrency $CONC \

Copilot · 2025-11-21T21:06:02Z

benchmarks/benchmark_lib.sh

+    tail -f "$server_log" &
+    local TAIL_PID=$!
+    until curl --output /dev/null --silent --fail http://0.0.0.0:$port/health; do
+        if ! kill -0 "$server_pid" 2>/dev/null; then
+            echo "Server died before becoming healthy. Exiting."
+            kill $TAIL_PID
+            exit 1
+        fi
+        sleep "$sleep_interval"
+    done
+    kill $TAIL_PID


The tail -f background process could become orphaned if the kill $TAIL_PID command fails. Consider using kill $TAIL_PID 2>/dev/null || true to ensure the function doesn't exit with an error status, or add a trap to ensure cleanup happens.

cquil11 · 2025-11-21T21:10:00Z

@copilot please go through all of the benchmarks/ files and make sure the "required env vars" at the top of each file are correct and commented correctly. recall these should be all of the env vars that are required to be set BEFORE calling this script
additionally, make sure that the corresponding runners/ scripts set the variables accordingly

Copilot · 2025-11-21T21:10:11Z

@cquil11 I've opened a new pull request, #252, to work on those changes. Once the pull request is ready, I'll request review from you.

… benchmark scripts (#252) * Initial plan * Update required env vars documentation in all benchmark scripts Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com> * Fix required env vars - remove NF, PREFILL_SIZE, and correct PORT/PORT_OFFSET Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com> * Remove internally-calculated vars from required env vars (EXTRA_CONFIG_FILE, MAX_NUM_TOKENS, MOE_BACKEND) Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 33 out of 35 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-21T21:47:20Z

benchmarks/benchmark_lib.sh

+# All parameters are required
+# Parameters:
+#   --port: Server port
+#   --server-log: Path to server log file
+#   --server-pid: Server process ID (required)
+#   --sleep-interval: Sleep interval between health checks (optional, default: 5)


The comment states "All parameters are required" but --sleep-interval is listed as "(optional, default: 5)" in the parameter list. This is inconsistent. The comment should either say "Parameters (all required unless marked optional):" or the first line should be updated to reflect that not all parameters are required.

Copilot · 2025-11-21T21:47:21Z

benchmarks/benchmark_lib.sh

+            kill $TAIL_PID
+            exit 1
+        fi
+        sleep "$sleep_interval"
+    done
+    kill $TAIL_PID


The kill $TAIL_PID command at line 65 and 70 should include error suppression (e.g., kill $TAIL_PID 2>/dev/null || true) to prevent the function from failing if the tail process has already terminated or doesn't exist. This is particularly important at line 65 where the script continues to exit 1 afterward.

Suggested change

kill $TAIL_PID

exit 1

fi

sleep "$sleep_interval"

done

kill $TAIL_PID

kill $TAIL_PID 2>/dev/null || true

exit 1

fi

sleep "$sleep_interval"

done

kill $TAIL_PID 2>/dev/null || true

kimbochen · 2025-11-22T20:05:14Z

benchmarks/dsr1_fp4_b200_docker.sh

+# Wait for server to be ready
+wait_for_server_ready --port "$PORT" --server-log "$SERVER_LOG" --server-pid "$SERVER_PID"
+
+pip install -q datasets pandas


nitpick: maybe put the install command in benchmark_lib.sh?

* initial poc * remove -d flag when launching docker container * syntax error * compatibility fixes * add correct endpoint prefix * remove reference env var * run vllm serve in background * unescape sequences * stop vllm to stdout after it stops * stop vllm to stdout after it stops pt 2 * get rid of docker stop as no longer in detatched * clone bench serving to tmp dir * clone bench serving to tmp dir pt 2 * add explanatory comment * cleaning up * cleaning up * adding mi355x refactor * adding h200 initial refactor * different way to see server logs * cleanup * now fail if server fails * starting on b200 * doign b200 * reverting erroneous change * fixing b200 * fixing b200 pt 2 * updating mi300 * updating mi300 pt 2 * updating mi300 pt 3 -- remove detached mode * cleaning up mi355x * fixing mi300x and updating 325x * reverting max conc to 512 on gptoss fp4 b200 docker * fixing mi300x and updating 325x * cleanng up * add wait for h200 slurm dsr1 * max num seqs back to 512 for gptoss fpr b200 docker * fix port issue for dsr1 mi300x docker * fix mi355x docker NUM_PROMPTS * adding prop of failure for server logs * add utils function for benchmark * add utils function for benchmark * function-ize the waiting for server to start * dont show arg parsing set -x * dont show arg parsing set +x oops * dont show arg parsing set +x oops * capture server pid * nebdius dont scancel * changes to comments in benchmark lib . sh * Update benchmarks/dsr1_fp4_mi355x_docker.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update .github/workflows/benchmark-tmpl.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * adding back whitespace * adding back whitespace * adding back whitespace * remove tg launch script * Update benchmarks/gptoss_fp4_h100_docker.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update benchmarks/dsr1_fp8_mi325x_docker.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update benchmarks/dsr1_fp8_mi355x_docker.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update benchmarks/gptoss_fp4_b200_trt_slurm.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Audit and correct required environment variables documentation in all benchmark scripts (InferenceMAX#252) * Initial plan * Update required env vars documentation in all benchmark scripts Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com> * Fix required env vars - remove NF, PREFILL_SIZE, and correct PORT/PORT_OFFSET Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com> * Remove internally-calculated vars from required env vars (EXTRA_CONFIG_FILE, MAX_NUM_TOKENS, MOE_BACKEND) Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com> * removing oci node rebase with main --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>

… 0.5.5.post2 for AMD MI355 (#247) * Change AMD MI355 docker image to lmsysorg/sglang:v0.5.5.post2-rocm700-mi35x for dsr1-fp8 * Adjust preview for dark mode and light mode (#250) * Adjust preview for dark mode and light mode picture element for better display based on color scheme. * Rounded * Update image alt text in README.md * adding ISL/OSL to collect results table summary (#249) Co-authored-by: Jatin Gangani <jgangani@dc2-container-xterm-014.prd.it.nvidia.com> * chore: refactor Docker runner launch to be like SLURM (#227) * initial poc * remove -d flag when launching docker container * syntax error * compatibility fixes * add correct endpoint prefix * remove reference env var * run vllm serve in background * unescape sequences * stop vllm to stdout after it stops * stop vllm to stdout after it stops pt 2 * get rid of docker stop as no longer in detatched * clone bench serving to tmp dir * clone bench serving to tmp dir pt 2 * add explanatory comment * cleaning up * cleaning up * adding mi355x refactor * adding h200 initial refactor * different way to see server logs * cleanup * now fail if server fails * starting on b200 * doign b200 * reverting erroneous change * fixing b200 * fixing b200 pt 2 * updating mi300 * updating mi300 pt 2 * updating mi300 pt 3 -- remove detached mode * cleaning up mi355x * fixing mi300x and updating 325x * reverting max conc to 512 on gptoss fp4 b200 docker * fixing mi300x and updating 325x * cleanng up * add wait for h200 slurm dsr1 * max num seqs back to 512 for gptoss fpr b200 docker * fix port issue for dsr1 mi300x docker * fix mi355x docker NUM_PROMPTS * adding prop of failure for server logs * add utils function for benchmark * add utils function for benchmark * function-ize the waiting for server to start * dont show arg parsing set -x * dont show arg parsing set +x oops * dont show arg parsing set +x oops * capture server pid * nebdius dont scancel * changes to comments in benchmark lib . sh * Update benchmarks/dsr1_fp4_mi355x_docker.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update .github/workflows/benchmark-tmpl.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * adding back whitespace * adding back whitespace * adding back whitespace * remove tg launch script * Update benchmarks/gptoss_fp4_h100_docker.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update benchmarks/dsr1_fp8_mi325x_docker.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update benchmarks/dsr1_fp8_mi355x_docker.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update benchmarks/gptoss_fp4_b200_trt_slurm.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Audit and correct required environment variables documentation in all benchmark scripts (#252) * Initial plan * Update required env vars documentation in all benchmark scripts Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com> * Fix required env vars - remove NF, PREFILL_SIZE, and correct PORT/PORT_OFFSET Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com> * Remove internally-calculated vars from required env vars (EXTRA_CONFIG_FILE, MAX_NUM_TOKENS, MOE_BACKEND) Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com> * removing oci node rebase with main --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> * Bump actions/checkout from 5.0.0 to 6.0.0 in the github-actions group (#253) Bumps the github-actions group with 1 update: [actions/checkout](https://github.com/actions/checkout). Updates `actions/checkout` from 5.0.0 to 6.0.0 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@08c6903...1af3b93) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 6.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add b200 DGXC node to b200 runners list (#245) * Bump actions/setup-python in the github-actions group (#259) Bumps the github-actions group with 1 update: [actions/setup-python](https://github.com/actions/setup-python). Updates `actions/setup-python` from 6.0.0 to 6.1.0 - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@e797f83...83679a8) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: 6.1.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * feat: refresh GB200 SGLang DSR1 submission (#257) * Bumps DSR1 SGLang code * update how we get the resulting log files --------- Co-authored-by: Elnifio <elnifio0519@gmail.com> Co-authored-by: Cameron Quilici <cjquilici@gmail.com> * Update GPTOSS B200 TRTLLM (#266) * Update GPTOSS B200 AGG * set dp attention env vars * Add DP attn comment --------- Co-authored-by: Jatin Gangani <jgangani@dc2-container-xterm-014.prd.it.nvidia.com> * Fixed community container for MI35x dsr1 for fp8 for real * Update dsr1_fp4_mi355x_docker.sh with env flags * Update dsr1_fp4_mi355x_slurm.sh * from post2 to post3 for fp8 * tidy formatting --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Austen Stone <austenstone@github.com> Co-authored-by: Jatin Gangani <5560074+jgangani@users.noreply.github.com> Co-authored-by: Jatin Gangani <jgangani@dc2-container-xterm-014.prd.it.nvidia.com> Co-authored-by: Cameron Quilici <cjquilici@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ankur Singh <ankusingh@nvidia.com> Co-authored-by: yunzhoul-nv <yunzhoul@nvidia.com> Co-authored-by: Elnifio <elnifio0519@gmail.com> Co-authored-by: ppalanga <ppalanga@amd.com>

cquil11 force-pushed the refactor-docker-runner-launch branch 5 times, most recently from 06612ca to 49e0e0c Compare November 17, 2025 14:53

cquil11 changed the title ~~WIP: Refactor docker runner launch~~ chore: refactor Docker runner launch to be like SLURM Nov 17, 2025

cquil11 marked this pull request as ready for review November 17, 2025 16:37

cquil11 requested a review from a team as a code owner November 17, 2025 16:37

Copilot AI review requested due to automatic review settings November 17, 2025 16:37

Copilot started reviewing on behalf of cquil11 November 17, 2025 16:37 View session

cquil11 force-pushed the refactor-docker-runner-launch branch from 47604f7 to 31ca015 Compare November 17, 2025 16:38

Copilot finished reviewing on behalf of cquil11 November 17, 2025 16:39

Copilot AI reviewed Nov 17, 2025

View reviewed changes

Copilot AI review requested due to automatic review settings November 17, 2025 19:17

Copilot started reviewing on behalf of cquil11 November 17, 2025 19:18 View session

Copilot finished reviewing on behalf of cquil11 November 17, 2025 19:19

Copilot AI reviewed Nov 17, 2025

View reviewed changes

benchmarks/gptoss_fp4_h100_docker.sh Show resolved Hide resolved

benchmarks/benchmark_lib.sh Show resolved Hide resolved

benchmarks/benchmark_lib.sh Show resolved Hide resolved

cquil11 added 13 commits November 17, 2025 13:59

initial poc

f4c7925

remove -d flag when launching docker container

d068894

syntax error

0a90d87

compatibility fixes

79c49cf

add correct endpoint prefix

de78e18

remove reference env var

6a38bfb

run vllm serve in background

f2e40ee

unescape sequences

272c81d

stop vllm to stdout after it stops

4d27a9d

stop vllm to stdout after it stops pt 2

5f549dc

get rid of docker stop as no longer in detatched

a3e7064

clone bench serving to tmp dir

cabe362

clone bench serving to tmp dir pt 2

3020926

Copilot AI mentioned this pull request Nov 17, 2025

[WIP] WIP Address feedback on Docker runner launch refactor #244

Closed

Copilot AI reviewed Nov 17, 2025

View reviewed changes

functionstackx reviewed Nov 19, 2025

View reviewed changes

benchmarks/benchmark_lib.sh Show resolved Hide resolved

Update benchmarks/gptoss_fp4_h100_docker.sh

97edfee

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings November 21, 2025 21:02

Copilot started reviewing on behalf of cquil11 November 21, 2025 21:02 View session

cquil11 and others added 3 commits November 21, 2025 15:03

Update benchmarks/dsr1_fp8_mi325x_docker.sh

b6a80fb

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update benchmarks/dsr1_fp8_mi355x_docker.sh

fa1e2c0

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update benchmarks/gptoss_fp4_b200_trt_slurm.sh

a5ebc4a

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot finished reviewing on behalf of cquil11 November 21, 2025 21:05

Copilot AI reviewed Nov 21, 2025

View reviewed changes

Copilot AI mentioned this pull request Nov 21, 2025

Audit and correct required environment variables documentation in all benchmark scripts #252

Merged

8 tasks

Copilot AI and others added 2 commits November 21, 2025 15:41

removing oci node rebase with main

6eb8285

Copilot AI review requested due to automatic review settings November 21, 2025 21:44

Merge branch 'main' into refactor-docker-runner-launch

da48d92

Copilot started reviewing on behalf of cquil11 November 21, 2025 21:44 View session

cquil11 merged commit d22bf9c into main Nov 21, 2025
1 check passed

cquil11 deleted the refactor-docker-runner-launch branch November 21, 2025 21:44

Copilot finished reviewing on behalf of cquil11 November 21, 2025 21:46

Copilot AI reviewed Nov 21, 2025

View reviewed changes

functionstackx mentioned this pull request Nov 21, 2025

Add b200 DGXC node to b200 runners list #245

Merged

kimbochen reviewed Nov 22, 2025

View reviewed changes

cquil11 mentioned this pull request Nov 27, 2025

feat: multinode first class integration #251

Merged

cquil11 mentioned this pull request Dec 8, 2025

fix: gptoss h100 docker bug #310

Merged

chore: refactor Docker runner launch to be like SLURM #227

chore: refactor Docker runner launch to be like SLURM #227

Uh oh!

Conversation

cquil11 commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Introduction

Implementation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI commented Nov 17, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

cquil11 commented Nov 21, 2025

Uh oh!

Copilot AI commented Nov 21, 2025

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

kimbochen Nov 22, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cquil11 commented Nov 12, 2025 •

edited

Loading