Related issues

1. Use dataclass instead of TypedDict for config #1675
2. Use dataclasses instead of TypedDict to handle defaults (but not #2102

Summary

This PR refactors MasterConfig (in all algorithm entry points: grpo.py, sft.py, dpo.py, rm.py, distillation.py, eval.py) and ClippedPGLossConfig from TypedDict to pydantic.BaseModel.

We can now get four abilities when overriding config fields via Hydra/OmegaConf:

1. Default values — Fields omitted from the YAML use their Python-defined defaults.

   # disable_ppo_ratio is defined with `= False` in ClippedPGLossConfig
   # but absent from examples/configs/grpo_math_1B.yaml — resolves to False
   loss_config.disable_ppo_ratio  # False

2. Custom/extra fields — Ad-hoc fields can be injected from the command line without modifying the config class.

   uv run python examples/run_grpo.py ++loss_fn.custom=1
   # cfg.custom is now accessible inside ClippedPGLossFn.__init__

3. Type validation — Pydantic validates field types at runtime.

   uv run python examples/run_grpo.py ++loss_fn.disable_ppo_ratio=1.2
   # Will raise error: "Input should be a valid boolean [type=bool_type, input_value=1.2, input_type=float]"

4. Go-to-definition — IDE navigation works on attribute access. Clicking on config.disable_ppo_ratio jumps directly to the field definition in ClippedPGLossConfig, whereas the previous dict-style access (config["disable_ppo_ratio"]) was opaque to static analysis tools.

Changes

• MasterConfig in grpo.py, sft.py, dpo.py, rm.py, distillation.py, eval.py: TypedDict → BaseModel(extra="allow")
• ClippedPGLossConfig in loss_functions.py: TypedDict → BaseModel(extra="allow")
• All dict-style access (config["key"]) on these two config types updated to attribute access (config.key)
• skills/config-conventions/SKILL.md updated to reflect the new BaseModel convention

Results before / after the changes