[data] remove concurrency lock by iamjustinhsu · Pull Request #56798 · ray-project/ray

iamjustinhsu · 2025-09-22T19:51:45Z

Why are these changes needed?

Currently, users who specify max_concurrency>1 don't actually experience multi-threaded concurrency in their actors. This PR addresses that by allowing users to override actor pool max_concurrency behavior. Changes I made

BY DEFAULT, the behavior before and after this PR is preserved
IF users want to respect max_concurrency, they can set enable_true_multi_threading=True in their ActorComputeStrategy

Related issue number

#55354

Checks

Tests:

NONE YET

I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
I've run scripts/format.sh to lint the changes in this PR.
I've included any doc changes needed for https://docs.ray.io/en/master/.
- I've added any new APIs to the API Reference. For example, if I added a
  method in Tune, I've added it in doc/source/tune/api/ under the
  corresponding .rst file.
I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
Testing Strategy
- Unit tests
- Release tests
- This PR is not tested :(

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

github-actions · 2025-10-08T00:35:37Z

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

github-actions · 2025-10-22T12:25:54Z

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

github-actions · 2025-11-06T00:38:08Z

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

github-actions · 2025-11-28T00:37:42Z

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

…/remove-udf-lock

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

python/ray/data/_internal/planner/plan_udf_map_op.py

python/ray/data/_internal/compute.py

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

alexeykudinkin · 2025-12-02T21:45:19Z

python/ray/data/_internal/compute.py

        max_size: Optional[int] = None,
        initial_size: Optional[int] = None,
        max_tasks_in_flight_per_actor: Optional[int] = None,
+        single_threaded: bool = True,


Suggested change

single_threaded: bool = True,

enable_true_multi_threading: bool = False,

Also add ample commentary to explain the difference

python/ray/data/_internal/execution/util.py

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

python/ray/data/_internal/compute.py

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

python/ray/data/_internal/planner/plan_udf_map_op.py

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

cursor · 2025-12-03T18:48:46Z

python/ray/data/_internal/compute.py

            f"initial_size={self.initial_size}, "
            f"max_tasks_in_flight_per_actor={self.max_tasks_in_flight_per_actor})"
            f"num_workers={self.num_workers}, "
+            f"enable_true_multi_threading={self.enable_true_multi_threading}, "


Bug: Malformed repr string with misplaced closing parenthesis

The __repr__ method has a closing parenthesis ) at the end of the max_tasks_in_flight_per_actor line, but continues with more fields (num_workers, enable_true_multi_threading, ready_to_total_workers_ratio) followed by another ). This produces malformed output like ActorPoolStrategy(...)num_workers=..., enable_true_multi_threading=..., ...) where fields after the first ) appear outside the constructor notation.

python/ray/data/_internal/compute.py

python/ray/data/_internal/planner/plan_udf_map_op.py

alexeykudinkin · 2025-12-03T19:41:21Z

python/ray/data/_internal/planner/plan_udf_map_op.py

+    enable_true_multi_threading: bool = (
+        compute.enable_true_multi_threading
+        if isinstance(compute, ActorPoolStrategy)
+        else True
+    )


Just do it inside _get_udf to avoid duplication

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

…/remove-udf-lock

alexeykudinkin · 2025-12-03T23:59:31Z

python/ray/data/_internal/planner/plan_udf_map_op.py

+        if (
+            not is_async_udf
+            and isinstance(compute, ActorPoolStrategy)
+            and not compute.enable_true_multi_threading
+        ):
+            udf = make_callable_class_single_threaded(udf)


Add a comment to explain the behavior here

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

python/ray/data/_internal/planner/plan_udf_map_op.py

Signed-off-by: Alexey Kudinkin <alexey.kudinkin@gmail.com>

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

## Why are these changes needed? Currently, users who specify `max_concurrency>1` don't actually experience multi-threaded concurrency in their actors. This PR addresses that by allowing users to override actor pool `max_concurrency` behavior. Changes I made - BY DEFAULT, the behavior before and after this PR is preserved - IF users want to respect `max_concurrency`, they can set `enable_true_multi_threading=True` in their `ActorComputeStrategy` ## Related issue number ray-project#55354 ## Checks ## Tests: NONE YET - [ ] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [ ] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: iamjustinhsu <jhsu@anyscale.com> Signed-off-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> Co-authored-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> Signed-off-by: peterxcli <peterxcli@gmail.com>

iamjustinhsu added 2 commits September 22, 2025 12:51

[wip][data] remove concurrency lock

7ce2b3d

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

watch more tests fail

6d4644c

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

iamjustinhsu added the go add ONLY when ready to merge, run all tests label Sep 23, 2025

github-actions bot added the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Oct 8, 2025

iamjustinhsu removed the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Oct 8, 2025

github-actions bot added the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Oct 22, 2025

iamjustinhsu removed the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Oct 22, 2025

github-actions bot added the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Nov 6, 2025

iamjustinhsu removed the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Nov 6, 2025

github-actions bot added the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Nov 28, 2025

iamjustinhsu removed the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Dec 1, 2025

iamjustinhsu added 2 commits December 1, 2025 12:34

Merge branch 'master' of https://github.com/ray-project/ray into jhsu…

a641ea7

…/remove-udf-lock

add flag to gatekeep existing behavior

3d87637

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

iamjustinhsu changed the title ~~[wip][data] remove concurrency lock~~ [data] remove concurrency lock Dec 2, 2025

iamjustinhsu marked this pull request as ready for review December 2, 2025 18:44

iamjustinhsu requested a review from a team as a code owner December 2, 2025 18:44

cursor bot reviewed Dec 2, 2025

View reviewed changes

python/ray/data/_internal/planner/plan_udf_map_op.py Outdated Show resolved Hide resolved

python/ray/data/_internal/compute.py Outdated Show resolved Hide resolved

ray-gardener bot added the data Ray Data-related issues label Dec 2, 2025

cursor

a161a70

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

alexeykudinkin reviewed Dec 2, 2025

View reviewed changes

iamjustinhsu force-pushed the jhsu/remove-udf-lock branch from e2ed5f3 to 37b9e50 Compare December 2, 2025 22:04

rename flag

ca15c6d

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

iamjustinhsu force-pushed the jhsu/remove-udf-lock branch from 37b9e50 to ca15c6d Compare December 2, 2025 22:06

cursor bot reviewed Dec 2, 2025

View reviewed changes

python/ray/data/_internal/compute.py Show resolved Hide resolved

beefy docstring

5ca0d31

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

cursor bot reviewed Dec 2, 2025

View reviewed changes

python/ray/data/_internal/planner/plan_udf_map_op.py Outdated Show resolved Hide resolved

iamjustinhsu added 2 commits December 2, 2025 14:30

update repr

da434ae

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

lint

8c97fb7

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

iamjustinhsu force-pushed the jhsu/remove-udf-lock branch from e3dd37c to 8c97fb7 Compare December 2, 2025 22:32

iamjustinhsu added 5 commits December 2, 2025 14:32

clean

d03842d

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

rename

d12aeda

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

clear

63f65b7

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

api policy check

5da7ad8

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

clearer docstring

81c559a

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

cursor bot reviewed Dec 3, 2025

View reviewed changes

alexeykudinkin reviewed Dec 3, 2025

View reviewed changes

iamjustinhsu added 4 commits December 3, 2025 12:04

address comments

aed080d

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

forgot compute flag

708704e

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

lint

65f9728

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

Merge branch 'master' of https://github.com/ray-project/ray into jhsu…

a10e010

…/remove-udf-lock

alexeykudinkin approved these changes Dec 3, 2025

View reviewed changes

comment

83a0bb5

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

alexeykudinkin reviewed Dec 4, 2025

View reviewed changes

python/ray/data/_internal/planner/plan_udf_map_op.py Outdated Show resolved Hide resolved

Tidying up

a2b06bc

Signed-off-by: Alexey Kudinkin <alexey.kudinkin@gmail.com>

alexeykudinkin enabled auto-merge (squash) December 4, 2025 18:17

github-actions bot disabled auto-merge December 4, 2025 18:17

lint

81cae9a

Signed-off-by: iamjustinhsu <jhsu@anyscale.com>

alexeykudinkin merged commit edff0f6 into ray-project:master Dec 5, 2025
6 checks passed

	single_threaded: bool = True,
	enable_true_multi_threading: bool = False,

Conversation

iamjustinhsu commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why are these changes needed?

Related issue number

Checks

Tests:

Uh oh!

github-actions bot commented Oct 8, 2025

Uh oh!

github-actions bot commented Oct 22, 2025

Uh oh!

github-actions bot commented Nov 6, 2025

Uh oh!

github-actions bot commented Nov 28, 2025

Uh oh!

Uh oh!

Uh oh!

alexeykudinkin Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

alexeykudinkin Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot Dec 3, 2025

Choose a reason for hiding this comment

Bug: Malformed repr string with misplaced closing parenthesis

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexeykudinkin Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

alexeykudinkin Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

iamjustinhsu commented Sep 22, 2025 •

edited

Loading