Adds a standalone VLM training playground under
``examples/multimodal_dev/`` with Qwen3.5-VL end-to-end.

Highlights
- Model-agnostic entry point (``pretrain_multimodal.py``) with a
  ``MODEL_REGISTRY`` so adding a new architecture is just a registry
  entry plus a backing module.
- Qwen3.5-VL model: vision encoder, MRoPE, decoder, factory, specs,
  configurations covering proxy / 9B / 397B-A17B variants.
- Datasets: mock data and CORD-V2 VLM dataset, with THD pack/pad in the
  collate function.
- THD + CP support consolidated in ``forward_step.py`` and the model
  layer (uses MRoPE THD pre-computation and ``cu_seqlens_q_padded`` CP
  partitioning).
- Run script + README, plus tests for MRoPE parity, CP correctness, CP
  support, and THD correctness / e2e.

Also gates the torch DataLoader vanilla-collate path on the new
``use_vanilla_collate_fn`` arg (one-line change to
``megatron/training/datasets/data_samplers.py``) so CORD-V2 works under
BSHD.

Functional dependency: the new model arch sets ``mrope_interleaved=True``
in its config and relies on the core MRoPE interleaved layout introduced
in a separate PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Co-Authored-By: BestJuly <19769279+BestJuly@users.noreply.github.com>

… preprocessing

Fixes 8 issues in vlm_dataset.py found by review against Megatron-Bridge's
qwen2_5_collate_fn reference implementation.

- loss_mask off-by-one (Bug 1): the previous mask was built on input_ids
  while labels were shifted, dropping the image->text supervision signal
  at the boundary. Now masks structural tokens on the shifted labels and
  also shifts loss_mask itself left by 1.
- missing SFT prompt masking (Bug 2): user-turn and chat-template tokens
  were trained on. Now uses backward substring token search (mirroring
  create_multiturn_loss_mask_by_search) to unmask only the assistant
  answer span.
- seq_length not enforced (Bug 3): long CORD-V2 samples could overflow.
  Now end-truncates input_ids in __getitem__ with a warning.
- unsafe pad_token_id fallback (Bug 4): falling back to 0 silently masked
  a real vocab token. Now falls back to EOS and raises if neither is set.
- silent image_token_id miss (Bug 6): fallback could return None, causing
  dataset / model disagreement. Now raises ValueError.
- stale docstrings (Bug 8): updated Qwen2.5-VL / --image-size references
  to Qwen3.5-VL / --total-seq-length.
- narrow skipped_tokens set (Bug 14): vision_start/end, im_start/end,
  video_pad, endoftext were not masked on labels. Now uses
  tok.all_special_ids union {pad_id, image_token_id}.
- lost Qwen-VL dynamic resolution (Bugs 15/17/19): fixed-square resize
  removed; conversation content carries the image object;
  qwen_vl_utils.process_vision_info extracts images; processor is called
  with min_pixels / max_pixels.
- pixel_values bf16 conversion (Bug 18): moved from forward_step into the
  dataset so per-step dtype checks become no-ops.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Update the 'Copyright (c) 2025, NVIDIA CORPORATION' line to 2026 across
all newly-added Python files under examples/multimodal_dev/ for the
Qwen3.5-VL training example.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Document the HF -> Megatron-FSDP DTensor conversion path needed before
pretraining from pretrained weights: setup (clone Bridge, pin its
3rdparty/Megatron-LM submodule to this branch), the `torchrun
convert_checkpoints_fsdp.py import` command with EP=8 default topology,
expected output layout, and the open Bridge dependency
(NVIDIA-NeMo/Megatron-Bridge#3987) to skip the post-save tokenizer
build that otherwise crashes on this branch.

Resolves the inline comments from @Victarry's PR review on NVIDIA#4751.

* vision_encoder.py — patch merger GELU was `approximate='tanh'` while
  the in-code NOTE acknowledged HF uses `approximate='none'`. Switched
  to `approximate='none'` to match the official Qwen3VLVisionPatchMerger
  numerics for HF -> Megatron checkpoint parity.

* pretrain_multimodal.py — added an explicit guard against
  `--pipeline-model-parallel-size > 1`. The model_provider builds the
  full model on every rank and ignores pre_process / post_process
  stage flags, so PP>1 would silently break Megatron's pipeline-parallel
  contract. Fail fast instead.

* scripts/run_qwen35_vl.sh — three fixes:
    1. `EP` now defaults to 1 (was 2). MoE variants must opt in via
       the environment override.
    2. After the variant case block, fail fast if
       `NUM_EXPERTS=0 && EP>1` so a dense run such as
       `MODEL_VARIANT=9b ./run_qwen35_vl.sh` no longer trips Megatron's
       arg validation downstream.
    3. `--moe-router-force-load-balancing` was unconditionally added to
       GPT_MODEL_ARGS (and therefore enabled even when no MoE args
       were emitted). It is now gated behind `FORCE_LOAD_BALANCING=1`,
       defaults off, and is appended to MOE_ARGS only when MoE is
       active. Real finetuning runs no longer freeze router routing
       decisions by default.

* data/{vlm_dataset.py -> cord_v2.py} + models/__init__.py — renamed
  the CORD-V2-specific module from the generic-sounding
  `vlm_dataset.py` to `cord_v2.py`, updated the model registry path
  string accordingly, and added an "Adding another VLM dataset" section
  to the module docstring documenting the per-dataset module +
  `MODEL_REGISTRY["..."]["dataset_providers"]` registration pattern.

* models/qwen35_vl/mrope.py — added a performance note on the
  `_build_sample_mrope_positions` helper documenting the
  `.tolist()` / `.item()` GPU<->CPU sync points and CUDA-graph
  incompatibility, and the precompute-in-collate / cache-by-shape
  follow-up plan. Behavior preserved here pending a follow-up data
  pipeline change.

The other tests-import comment (test_thd_*.py importing `_pack_batch`)
is already addressed on this branch: the helper is now named
`pack_or_pad_batch` and the tests import that symbol.