Fix suppressions for librdkafka data-race for statistics code#63039
Merged
alexey-milovidov merged 4 commits intoClickHouse:masterfrom Apr 27, 2024
Merged
Fix suppressions for librdkafka data-race for statistics code#63039alexey-milovidov merged 4 commits intoClickHouse:masterfrom
alexey-milovidov merged 4 commits intoClickHouse:masterfrom
Conversation
See: ClickHouse#49829 (comment) Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
The problem is that ignorelist `fun` does not work recursively.
<details>
<summary>example</summary>
```c
// test.cpp
bool flag = false;
// avoid mangling
extern "C" {
void set_flag_impl()
{
flag = true;
}
void set_flag()
{
set_flag_impl();
}
void set_flag_if()
{
if (flag)
flag = false;
}
}
int main()
{
std::thread t1([]{ set_flag(); });
std::thread t2([]{ set_flag_if(); });
t1.join();
t2.join();
return 0;
}
```
```
// ignorelist
[thread]
fun:set_flag
```
```
$ clang++ -g -fno-omit-frame-pointer -fsanitize=thread -fsanitize-ignorelist=ignorelist -o test test.cpp && ./test
SUMMARY: ThreadSanitizer: data race /tmp/tsan-test/test.cpp:18:9 in set_flag_if
$ sed -i 's/set_flag/set_flag_impl/' ignorelist
$ clang++ -g -fno-omit-frame-pointer -fsanitize=thread -fsanitize-ignorelist=ignorelist -o test test.cpp && ./test
OK
```
</details>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Contributor
|
This is an automated comment for commit eb4c095 with description of existing statuses. It's updated for the latest CI running ❌ Click here to open a full report in a separate page
Successful checks
|
Algunenano
approved these changes
Apr 26, 2024
…ualifier)
The problem is that TSan still fails [1] is that ignorelist does not
work for static functions without asterisk:
// test.cpp
#include <thread>
bool flag = false;
// avoid mangling
extern "C" {
static void set_flag_impl()
{
flag = true;
}
void set_flag()
{
set_flag_impl();
}
void set_flag_if()
{
if (flag)
flag = false;
}
}
int main()
{
std::thread t1([]{ set_flag(); });
std::thread t2([]{ set_flag_if(); });
t1.join();
t2.join();
return 0;
}
// ignorelist
[thread]
fun:set_flag_impl
$ clang++ -g -fno-omit-frame-pointer -fsanitize=thread -fsanitize-ignorelist=ignorelist -o test test.cpp && ./test
SUMMARY: ThreadSanitizer: data race /tmp/test-tsan-ignorelist/test.cpp:19:9 in set_flag_if
$ sed -i 's/set_flag_impl/*set_flag_impl*/' ignorelist
$ clang++ -g -fno-omit-frame-pointer -fsanitize=thread -fsanitize-ignorelist=ignorelist -o test test.cpp && ./test
OK
But, note that ignorelist is tricky, and will not work for
functions with __always_inline__ attribute for example.
P.S. set_flag_impl also has brackets in the output (i.e.
set_flag_impl()), while ther eis brackets for rd_avg_calc on CI [1].
[1]: https://s3.amazonaws.com/clickhouse-test-reports/63039/84bebc534ba7cf6e9dbfc1d91e8350939a84f87c/integration_tests__tsan__[6_6]//home/ubuntu/actions-runner/_work/_temp/test/output_dir/integration_run_parallel4_0.log
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Member
Author
|
One more issue due to rd_avg_calc marked static - https://s3.amazonaws.com/clickhouse-test-reports/63039/84bebc534ba7cf6e9dbfc1d91e8350939a84f87c/integration_tests__tsan__[6_6]//home/ubuntu/actions-runner/_work/_temp/test/output_dir/integration_run_parallel4_0.log Should be fixed by P.S. I tested when the function in the header, and it should also work |
alexey-milovidov
approved these changes
Apr 27, 2024
Member
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Fix suppressions for librdkafka data-race for statistics code
The problem is that ignorelist
fundoes not work recursively.example
Fixes: #60939 (cc @Algunenano )
Fixes: #60443
Fixes: #60604
Refs: #61828