Context

The float16/bf16 branch (line 372) already uses detach().

shard_model_param = model_param.detach().view(-1)[
    param_range.start : param_range.end
]

This change makes the fp32 branch consistent. The fp32 branch becomes relevant when parameters are intentionally kept in fp32 (e.g., Qwen3.5's A_log via enforce_marked_param_dtypes).

Why it won't break anything

detach() only disconnects the shard from the autograd graph (making it a leaf tensor). It does not copy data — the shard still shares the same underlying storage as model_param. Every downstream consumer of shard_fp32_groups uses manual data operations, never autograd:

Gradient flow (_copy_model_grads_to_main_grads): Reads from model_param.main_grad, slices it, and assigns to shard.grad via direct attribute assignment — autograd is not involved.

Parameter writeback (copy_main_params_to_model_params): Re-slices from the param buffer and calls shard_model_param.data.copy() — autograd is not involved.

Checkpoint load (copy_model_params_to_main_params): Re-slices from model_param.view(-1)[...] and calls shard_main_param.data.copy() — autograd is not involved.

zero_grad: Clears .grad attribute or sets it to None — autograd is not involved.

Optimizer step: PyTorch optimizer operates on shard.data and shard.grad directly — in fact, it prefers leaf tensors and may warn or error on non-leaf params.

In short: Megatron's DistributedOptimizer completely bypasses PyTorch autograd for all gradient and parameter movement. detach() is a no-op in terms of data and behavior — it only satisfies PyTorch's expectation that optimizer params are leaf tensors.