[core] tighten ray export symbol allowlists to prevent non-ray symbol leakage#61298
[core] tighten ray export symbol allowlists to prevent non-ray symbol leakage#61298edoakes merged 9 commits intoray-project:masterfrom
Conversation
… leakage Signed-off-by: Rueian Huang <rueiancsie@gmail.com>
There was a problem hiding this comment.
Code Review
This pull request introduces a new sh_test to validate the exported symbols of the _raylet.so library on Unix-like systems. This is a valuable addition for maintaining ABI stability and preventing unintended symbol leakage. The symbol allowlists in src/ray/ray_exported_symbols.lds and src/ray/ray_version_script.lds have been tightened by using more precise C++ mangled names, which is a significant improvement for controlling the public API surface.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Rueian <rueiancsie@gmail.com>
6aa7015 to
8f164e4
Compare
c1df1e8 to
0d5018a
Compare
Signed-off-by: Rueian Huang <rueiancsie@gmail.com>
Signed-off-by: Rueian Huang <rueiancsie@gmail.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e039c5fe55
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… leakage (#61298) ## Description This is a follow-up to #61147. This will prevent us from accidentally leaking non-ray symbols by: 1. Changing all `*ray` symbol rules to starts with `*3ray`. That makes sure symbols are actually related to `ray` but not something like `Array`, according to [Itanium ABI](https://medium.com/@bengisu.batmaz/name-mangling-with-itanium-abi-00a5c4dbc3c4). 2. Golden tests (raylet_exported_symbols_linux.txt/raylet_exported_symbols_macos.txt) to see if there are any exported non-ray symbols not in the golden white lists. With the above, we can make sure the symbols we exported are either in the golden white lists or ray-related. Without the above, we currently have these leaked symbols: ``` _Z58descriptor_table_src_2fray_2fprotobuf_2fgcs_2eproto_getterv _Z68descriptor_table_src_2fray_2fprotobuf_2fgcs_5fservice_2eproto_getterv _ZN46TableStruct_src_2fray_2fprotobuf_2fgcs_2eproto7offsetsE _ZN56TableStruct_src_2fray_2fprotobuf_2fgcs_5fservice_2eproto7offsetsE _ZN9grpc_core17ServerCallContext26MakeTopOfServerCallPromiseENS_8CallArgsEP21grpc_completion_queueP19grpc_metadata_arrayN4absl12lts_2023080211FunctionRefIFvP9grpc_callEEE _ZN9grpc_core20GetAuthPropertyArrayEP17grpc_auth_contextPKc _ZN9grpc_core6Server11RequestCallEPP9grpc_callP17grpc_call_detailsP19grpc_metadata_arrayP21grpc_completion_queueS9_Pv _ZN9grpc_core6Server21RequestRegisteredCallEPNS0_16RegisteredMethodEPP9grpc_callP12gpr_timespecP19grpc_metadata_arrayPP16grpc_byte_bufferP21grpc_completion_queueSE_Pv _ZNSt5arrayISt3mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN9grpc_core19XdsListenerResource14FilterChainMap8SourceIpESt4lessIS6_ESaISt4pairIKS6_SA_EEELm3EED1Ev _ZNSt5arrayISt3mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN9grpc_core19XdsListenerResource14FilterChainMap8SourceIpESt4lessIS6_ESaISt4pairIKS6_SA_EEELm3EED2Ev _ZNSt5arrayISt6vectorIN9grpc_core19XdsListenerResource14FilterChainMap8SourceIpESaIS4_EELm3EED1Ev _ZNSt5arrayISt6vectorIN9grpc_core19XdsListenerResource14FilterChainMap8SourceIpESaIS4_EELm3EED2Ev descriptor_table_src_2fray_2fprotobuf_2fgcs_2eproto descriptor_table_src_2fray_2fprotobuf_2fgcs_5fservice_2eproto ``` Most of them are accidentally matched because of `Array`. ## Related issues #61147. --------- Signed-off-by: Rueian Huang <rueiancsie@gmail.com> Signed-off-by: Rueian <rueiancsie@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Kamil Kaczmarek <kamil@anyscale.com>
… leakage (ray-project#61298) ## Description This is a follow-up to ray-project#61147. This will prevent us from accidentally leaking non-ray symbols by: 1. Changing all `*ray` symbol rules to starts with `*3ray`. That makes sure symbols are actually related to `ray` but not something like `Array`, according to [Itanium ABI](https://medium.com/@bengisu.batmaz/name-mangling-with-itanium-abi-00a5c4dbc3c4). 2. Golden tests (raylet_exported_symbols_linux.txt/raylet_exported_symbols_macos.txt) to see if there are any exported non-ray symbols not in the golden white lists. With the above, we can make sure the symbols we exported are either in the golden white lists or ray-related. Without the above, we currently have these leaked symbols: ``` _Z58descriptor_table_src_2fray_2fprotobuf_2fgcs_2eproto_getterv _Z68descriptor_table_src_2fray_2fprotobuf_2fgcs_5fservice_2eproto_getterv _ZN46TableStruct_src_2fray_2fprotobuf_2fgcs_2eproto7offsetsE _ZN56TableStruct_src_2fray_2fprotobuf_2fgcs_5fservice_2eproto7offsetsE _ZN9grpc_core17ServerCallContext26MakeTopOfServerCallPromiseENS_8CallArgsEP21grpc_completion_queueP19grpc_metadata_arrayN4absl12lts_2023080211FunctionRefIFvP9grpc_callEEE _ZN9grpc_core20GetAuthPropertyArrayEP17grpc_auth_contextPKc _ZN9grpc_core6Server11RequestCallEPP9grpc_callP17grpc_call_detailsP19grpc_metadata_arrayP21grpc_completion_queueS9_Pv _ZN9grpc_core6Server21RequestRegisteredCallEPNS0_16RegisteredMethodEPP9grpc_callP12gpr_timespecP19grpc_metadata_arrayPP16grpc_byte_bufferP21grpc_completion_queueSE_Pv _ZNSt5arrayISt3mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN9grpc_core19XdsListenerResource14FilterChainMap8SourceIpESt4lessIS6_ESaISt4pairIKS6_SA_EEELm3EED1Ev _ZNSt5arrayISt3mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN9grpc_core19XdsListenerResource14FilterChainMap8SourceIpESt4lessIS6_ESaISt4pairIKS6_SA_EEELm3EED2Ev _ZNSt5arrayISt6vectorIN9grpc_core19XdsListenerResource14FilterChainMap8SourceIpESaIS4_EELm3EED1Ev _ZNSt5arrayISt6vectorIN9grpc_core19XdsListenerResource14FilterChainMap8SourceIpESaIS4_EELm3EED2Ev descriptor_table_src_2fray_2fprotobuf_2fgcs_2eproto descriptor_table_src_2fray_2fprotobuf_2fgcs_5fservice_2eproto ``` Most of them are accidentally matched because of `Array`. ## Related issues ray-project#61147. --------- Signed-off-by: Rueian Huang <rueiancsie@gmail.com> Signed-off-by: Rueian <rueiancsie@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Ayush Kumar <ayushk7102@gmail.com>
Description
This is a follow-up to #61147. This will prevent us from accidentally leaking non-ray symbols by:
*raysymbol rules to starts with*3ray. That makes sure symbols are actually related toraybut not something likeArray, according to Itanium ABI.With the above, we can make sure the symbols we exported are either in the golden white lists or ray-related.
Without the above, we currently have these leaked symbols:
Most of them are accidentally matched because of
Array.Related issues
#61147.