Skip to content

[Train] Replace Checkpoint Manager pydantic v2 APIs with v1 APIs#57147

Merged
justinvyu merged 10 commits intoray-project:masterfrom
JasonLi1909:fix-pydantic-checkpoint-manager-dep
Oct 7, 2025
Merged

[Train] Replace Checkpoint Manager pydantic v2 APIs with v1 APIs#57147
justinvyu merged 10 commits intoray-project:masterfrom
JasonLi1909:fix-pydantic-checkpoint-manager-dep

Conversation

@JasonLi1909
Copy link
Copy Markdown
Contributor

@JasonLi1909 JasonLi1909 commented Oct 2, 2025 •
edited
Loading

Problem: The Ray Train CheckpointManager currently depends on pydantic v2, using v2 API's like model_dump_json, from_json, and model_validate. However, users might not have v2 installed.

This PR:

  • Replaces CheckpointManager's usage of v2 APIs with v1 APIs instead

@JasonLi1909
updated checkpoint manager to use pydantic v1 and v2 compatible utili…
748ff13 
…ty funcs

Signed-off-by: JasonLi1909 <jasli1909@gmail.com>
@JasonLi1909 JasonLi1909 requested review from a team, edoakes and jjyao as code owners October 2, 2025 23:55
gemini-code-assist[bot]
gemini-code-assist bot reviewed Oct 2, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces compatibility utility functions to support both Pydantic v1 and v2 within Ray Train's CheckpointManager. The changes are well-isolated in ray._common.pydantic_compat and correctly applied in CheckpointManager. My review includes a minor suggestion to improve the type hinting for one of the new utility functions to enhance code clarity and correctness.

@JasonLi1909 JasonLi1909 requested a review from justinvyu October 3, 2025 00:03
justinvyu
justinvyu reviewed Oct 3, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@justinvyu justinvyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@ray-gardener ray-gardener bot added the train Ray Train Related Issue label Oct 3, 2025
JasonLi1909 and others added 3 commits October 3, 2025 13:34
@JasonLi1909 @justinvyu
Update python/ray/_common/pydantic_compat.py
6868aec 
Co-authored-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com>
@JasonLi1909 @justinvyu
Update python/ray/_common/pydantic_compat.py
7bf9393 
Co-authored-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com>
@JasonLi1909
updated util names to match pydantic v2 and moved pydantic try catch
08411cf 
Signed-off-by: JasonLi1909 <jasli1909@gmail.com>
cursor[bot]

This comment was marked as outdated.

Sign in to view

justinvyu
justinvyu reviewed Oct 3, 2025
View reviewed changes
JasonLi1909 and others added 2 commits October 3, 2025 16:18
@JasonLi1909 @justinvyu
Update python/ray/_common/pydantic_compat.py
2604b4f 
Co-authored-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com>
@JasonLi1909 @justinvyu
Update python/ray/_common/pydantic_compat.py
17a0caa 
Co-authored-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com>
cursor[bot]

This comment was marked as outdated.

Sign in to view

JasonLi1909 and others added 2 commits October 3, 2025 16:21
@JasonLi1909 @justinvyu
Update python/ray/_common/pydantic_compat.py
ba69c0a 
Co-authored-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com>
@JasonLi1909
update
8cf0b72 
Signed-off-by: JasonLi1909 <jasli1909@gmail.com>
cursor[bot]

This comment was marked as outdated.

Sign in to view

@JasonLi1909
pivot to using v1 apis
c13f684 
Signed-off-by: JasonLi1909 <jasli1909@gmail.com>
@JasonLi1909 JasonLi1909 changed the title [Train] Replace Checkpoint Manager pydantic v2 APIs with v1/v2 compatible utils [Train] Replace Checkpoint Manager pydantic v2 APIs with v1 APIs Oct 6, 2025
cursor[bot]

This comment was marked as outdated.

Sign in to view

@JasonLi1909 JasonLi1909 added the go add ONLY when ready to merge, run all tests label Oct 7, 2025
justinvyu
justinvyu approved these changes Oct 7, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@justinvyu justinvyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@justinvyu justinvyu merged commit 6351471 into ray-project:master Oct 7, 2025
6 of 7 checks passed
liulehui pushed a commit to liulehui/ray that referenced this pull request Oct 9, 2025
@JasonLi1909 @justinvyu @liulehui
[Train] Replace Checkpoint Manager pydantic v2 APIs with v1 APIs (ray…
c23c0e5 
…-project#57147)

Replace `CheckpointManager`'s usage of pydantic v2 APIs with v1 APIs instead.

---------

Signed-off-by: JasonLi1909 <jasli1909@gmail.com>
Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com>
Co-authored-by: Justin Yu <justinvyu@anyscale.com>
joshkodi pushed a commit to joshkodi/ray that referenced this pull request Oct 13, 2025
@JasonLi1909 @justinvyu @joshkodi
[Train] Replace Checkpoint Manager pydantic v2 APIs with v1 APIs (ray…
80dfced 
…-project#57147)

Replace `CheckpointManager`'s usage of pydantic v2 APIs with v1 APIs instead.

---------

Signed-off-by: JasonLi1909 <jasli1909@gmail.com>
Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com>
Co-authored-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: Josh Kodi <joshkodi@gmail.com>
ArturNiederfahrenhorst pushed a commit to ArturNiederfahrenhorst/ray that referenced this pull request Oct 13, 2025
@JasonLi1909 @justinvyu @ArturNiederfahrenhorst
[Train] Replace Checkpoint Manager pydantic v2 APIs with v1 APIs (ray…
4b7f107 
…-project#57147)

Replace `CheckpointManager`'s usage of pydantic v2 APIs with v1 APIs instead.

---------

Signed-off-by: JasonLi1909 <jasli1909@gmail.com>
Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com>
Co-authored-by: Justin Yu <justinvyu@anyscale.com>
justinyeh1995 pushed a commit to justinyeh1995/ray that referenced this pull request Oct 20, 2025
@JasonLi1909 @justinvyu @justinyeh1995
[Train] Replace Checkpoint Manager pydantic v2 APIs with v1 APIs (ray…
1640fbd 
…-project#57147)

Replace `CheckpointManager`'s usage of pydantic v2 APIs with v1 APIs instead.

---------

Signed-off-by: JasonLi1909 <jasli1909@gmail.com>
Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com>
Co-authored-by: Justin Yu <justinvyu@anyscale.com>
xinyuangui2 pushed a commit to xinyuangui2/ray that referenced this pull request Oct 22, 2025
@JasonLi1909 @justinvyu @xinyuangui2
[Train] Replace Checkpoint Manager pydantic v2 APIs with v1 APIs (ray…
7e489a0 
…-project#57147)

Replace `CheckpointManager`'s usage of pydantic v2 APIs with v1 APIs instead.

---------

Signed-off-by: JasonLi1909 <jasli1909@gmail.com>
Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com>
Co-authored-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: xgui <xgui@anyscale.com>
landscapepainter pushed a commit to landscapepainter/ray that referenced this pull request Nov 17, 2025
@JasonLi1909 @justinvyu @landscapepainter
[Train] Replace Checkpoint Manager pydantic v2 APIs with v1 APIs (ray…
f85a59e 
…-project#57147)

Replace `CheckpointManager`'s usage of pydantic v2 APIs with v1 APIs instead.

---------

Signed-off-by: JasonLi1909 <jasli1909@gmail.com>
Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com>
Co-authored-by: Justin Yu <justinvyu@anyscale.com>
Aydin-ab pushed a commit to Aydin-ab/ray-aydin that referenced this pull request Nov 19, 2025
@JasonLi1909 @justinvyu @Aydin-ab
[Train] Replace Checkpoint Manager pydantic v2 APIs with v1 APIs (ray…
7c43187 
…-project#57147)

Replace `CheckpointManager`'s usage of pydantic v2 APIs with v1 APIs instead.

---------

Signed-off-by: JasonLi1909 <jasli1909@gmail.com>
Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com>
Co-authored-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Future-Outlier pushed a commit to Future-Outlier/ray that referenced this pull request Dec 7, 2025
@JasonLi1909 @justinvyu @Future-Outlier
[Train] Replace Checkpoint Manager pydantic v2 APIs with v1 APIs (ray…
41fdd12 
…-project#57147)

Replace `CheckpointManager`'s usage of pydantic v2 APIs with v1 APIs instead.

---------

Signed-off-by: JasonLi1909 <jasli1909@gmail.com>
Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com>
Co-authored-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: Future-Outlier <eric901201@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cursor cursor[bot] cursor[bot] left review comments

@justinvyu justinvyu justinvyu approved these changes

@edoakes edoakes Awaiting requested review from edoakes

@jjyao jjyao Awaiting requested review from jjyao

+1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

go add ONLY when ready to merge, run all tests train Ray Train Related Issue

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@JasonLi1909 @justinvyu