We've added a compute_scale option that handles this — see PR #9.

Short answer: the evaluations are not sensitive to GPU size. Every adapted task keeps its macro-batch constant, so only the GPU count / per-GPU batch / wall-clock change — not the metric.

Set compute_scale: 0.5 in your config for H200 (≈ 2× H100). Both LLM families are adapted through the same per-entry h200 block (no script switching):

LLM pretraining: 4 → 2 GPUs, with per-GPU BATCH_SIZE ×2 and GRAD_ACCUM ÷2. The macro-batch BATCH_SIZE × GRAD_ACCUM is unchanged (grad-accum is divided by world size internally), so the gradient — and the result — is identical; the larger micro-batch just uses the H200's extra memory.
LLM RL: 2 → 1 GPU, with vLLM tensor_model_parallel_size 2 → 1, per-GPU token budget 17408 → 20480 and gpu_memory_utilization 0.4 → 0.5 — all injected via env into the same train.sh. The global data.train_batch_size is unchanged, so the result is unchanged.
Other tasks: optional — fractional / single-GPU jobs just pack denser; a multi-GPU task without an H200 profile is left as-is (with a warning), since blindly cutting its GPU count would change its global batch.

The default compute_scale: 1.0 (H100) is a strict no-op, so existing setups are unaffected. Full design and rationale are in PR #9.

Adaptation to different GPU types #4

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions