Skip to content

[runtime env] Short-circuit protobuf serialization/deserialization for empty runtime envs#21788

Merged
rkooo567 merged 2 commits intoray-project:masterfrom
architkulkarni:runtime-env-protobuf-performance-fix
Jan 22, 2022
Merged

[runtime env] Short-circuit protobuf serialization/deserialization for empty runtime envs#21788
rkooo567 merged 2 commits intoray-project:masterfrom
architkulkarni:runtime-env-protobuf-performance-fix

Conversation

@architkulkarni
Copy link
Copy Markdown
Contributor

@architkulkarni architkulkarni commented Jan 21, 2022

Why are these changes needed?

stress_test_many_tasks had a performance regression.

A test PR attributed a significant part of the performance loss to OverrideTaskOrActorRuntimeEnv: #21780

Inside this function, we short circuit some serialization/deserialization logic if the runtime environment is empty. However, an empty serialized runtime environment is "{}" (a serialized empty JSON dict.), but the existing code only checked for equality to "". This PR fixes the check.

We should probably wait to merge this until we verify that the stress_test_many_tasks performance was improved.

Related issue number

Partially addresses #21760

Checks

  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

std::shared_ptr<rpc::RuntimeEnv> parent = nullptr;
if (options_.worker_type == WorkerType::DRIVER) {
if (serialized_runtime_env == "") {
if (serialized_runtime_env == "{}") {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to be "" as well in some cases? Should we check both?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I only observed "{}" but it's probably better to be safe and also check "". I've updated the PR.

std::shared_ptr<rpc::RuntimeEnv> parent = nullptr;
if (options_.worker_type == WorkerType::DRIVER) {
if (serialized_runtime_env == "") {
if (serialized_runtime_env == "" || serialized_runtime_env == "{}") {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking now, can we just create a util function? Like IsRuntimeEnvEmpty(serialized_runtime_env) and use it all across the codebase

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't need to block this PR

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, we should clean this up.

@architkulkarni
Copy link
Copy Markdown
Contributor Author

Results:

INFO:__main__:Finished stage 3 actor creation in 0.06654715538024902 seconds.
INFO:__main__:Finished stage 3 in 2339.6809968948364 seconds.

@rkooo567
Copy link
Copy Markdown
Contributor

rkooo567 commented Jan 22, 2022

Seems to be consistent with jiajun. but there’s still 300-400 seconds regression..

@rkooo567
Copy link
Copy Markdown
Contributor

I believe it is good enough to unblock the release though.. (about 20%). Thoughts @iycheng ?

Copy link
Copy Markdown
Contributor

@SongGuyang SongGuyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you for this fix!

@rkooo567 rkooo567 merged commit c4bf68a into ray-project:master Jan 22, 2022
@rkooo567
Copy link
Copy Markdown
Contributor

Hey @architkulkarni can you make sure to follow up on #21788 (comment)?

architkulkarni added a commit that referenced this pull request Jan 24, 2022
…r empty runtime envs (#21788)

* Fix check for empty runtime_env ("{}", not "")

* also check for ""
edoakes pushed a commit that referenced this pull request Feb 7, 2022
#22129)

Followup from #21788.  Previously we had a lot of `serialized_runtime_env == "{}" || serialized_runtime_env == ""` scattered around the C++ code; this PR puts this in a helper function.
simonsays1980 pushed a commit to simonsays1980/ray that referenced this pull request Feb 27, 2022
ray-project#22129)

Followup from ray-project#21788.  Previously we had a lot of `serialized_runtime_env == "{}" || serialized_runtime_env == ""` scattered around the C++ code; this PR puts this in a helper function.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants