transformers>=4.54.0 (4.54.0 and 4.54.1 for now) all failed on rm models.
It can be reproduced by running the functional test uv run --no-sync bash ./tests/functional/rm.sh.

vllm==0.10.0 needs torch==2.7.1, but it seems torch==2.7.1's FSDP haven't good support with transformers>=4.54.0. We may still need to hold this limit longer.

Error log:

  File "/home/scratch.yukih_gpu/depot/reinforcer/venvs/nemo_rl.models.policy.dtensor_policy_worker.DTensorPolicyWorker/lib/python3.12/site-packages/torch/distributed/fsdp/_fully_shard/_fully_shard.py", line 245, in fully_shard
    module.__class__ = new_cls
    ^^^^^^^^^^^^^^^^
  File "/home/scratch.yukih_gpu/depot/reinforcer/venvs/nemo_rl.models.policy.dtensor_policy_worker.DTensorPolicyWorker/lib/python3.12/site-packages/torch/nn/modules/module.py", line 2041, in __setattr__
    super().__setattr__(name, value)
TypeError: __class__ assignment: 'FSDPLlamaForSequenceClassification' object layout differs from 'LlamaForSequenceClassification'

Remove transformers limit #811

Description

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions