[Data] Return 'inf' for max object store memory usage#59751
Merged
bveeramani merged 1 commit intomasterfrom Dec 30, 2025
Merged
[Data] Return 'inf' for max object store memory usage#59751bveeramani merged 1 commit intomasterfrom
bveeramani merged 1 commit intomasterfrom
Conversation
Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Contributor
There was a problem hiding this comment.
Code Review
This pull request correctly addresses the issue of capping object store memory usage by setting the maximum to infinity for ActorPoolMapOperator and TaskPoolMapOperator. The logic is sound, as the output size of tasks can be unpredictable. The changes are clear and the update to the test for ActorPoolMapOperator is appropriate.
Comment on lines
+191
to
194
| # Set the max `object_store_memory` requirement to 'inf', because we | ||
| # don't know how much data each task can output. | ||
| object_store_memory=float("inf"), | ||
| ) |
Contributor
There was a problem hiding this comment.
This change to set object_store_memory to float("inf") is correct. However, the existing tests for min_max_resource_requirements in test_task_pool_map_operator.py do not cover the case where _max_concurrency is set. It would be beneficial to add a test case to ensure this code path is validated, similar to how it's tested for ActorPoolMapOperator.
raulchen
approved these changes
Dec 30, 2025
AYou0207
pushed a commit
to AYou0207/ray
that referenced
this pull request
Jan 13, 2026
) ray-project#59412 updated the reservation resource allocator to cap resource allocations based on the resources returned by `min_max_resource_usage`. The problem is that running tasks can produce any amounts of data, so it doesn't make sense to cap by `obj_store_mem_max_pending_output_per_task * concurrency`. This PR fixes that issue by setting the max object store memory usage to 'inf'. Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu> Signed-off-by: jasonwrwang <jasonwrwang@tencent.com>
lee1258561
pushed a commit
to pinterest/ray
that referenced
this pull request
Feb 3, 2026
) ray-project#59412 updated the reservation resource allocator to cap resource allocations based on the resources returned by `min_max_resource_usage`. The problem is that running tasks can produce any amounts of data, so it doesn't make sense to cap by `obj_store_mem_max_pending_output_per_task * concurrency`. This PR fixes that issue by setting the max object store memory usage to 'inf'. Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
peterxcli
pushed a commit
to peterxcli/ray
that referenced
this pull request
Feb 25, 2026
) ray-project#59412 updated the reservation resource allocator to cap resource allocations based on the resources returned by `min_max_resource_usage`. The problem is that running tasks can produce any amounts of data, so it doesn't make sense to cap by `obj_store_mem_max_pending_output_per_task * concurrency`. This PR fixes that issue by setting the max object store memory usage to 'inf'. Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu> Signed-off-by: peterxcli <peterxcli@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#59412 updated the reservation resource allocator to cap resource allocations based on the resources returned by
min_max_resource_usage. The problem is that running tasks can produce any amounts of data, so it doesn't make sense to cap byobj_store_mem_max_pending_output_per_task * concurrency.This PR fixes that issue by setting the max object store memory usage to 'inf'.