Skip to content

Click house/v1.58.2#37

Closed
rschu1ze wants to merge 6789 commits intomasterfrom
ClickHouse/v1.58.2
Closed

Click house/v1.58.2#37
rschu1ze wants to merge 6789 commits intomasterfrom
ClickHouse/v1.58.2

Conversation

@rschu1ze
Copy link
Copy Markdown
Member

No description provided.

github-actions bot and others added 30 commits July 17, 2023 16:34
PanCakes to the rescue!

We noticed that our 'sanity' test was going to fail, but we think we can
fix that automatically, so we put together this PR to do just that!

If you'd like to opt-out of these PR's, add yourself to NO_AUTOFIX_USERS
in .github/workflows/pr-auto-fix.yaml

Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
Custom strip invalidates the signature, caused the arm64-darwin gem
failed to run on M1 macs.
Error in mac console: `Exception Type: EXC_BAD_ACCESS (SIGKILL (Code
Signature Invalid))`

Warning in gem build log:
```
# linking shared-object grpc/grpc_c.bundle
# Stripping grpc_c.bundle
# /opt/osxcross/target/bin/aarch64-apple-darwin20.2-strip: warning: changes being made to the file will invalidate the code signature in: /Users/jeff/work/ruby/grpc/tmp/arm64-darwin/grpc_c/3.2.0/grpc_c.bundle
```

Custom stripping is not really needed (any more), this diff removes it
from `src/ruby/ext/grpc/extconf.rb`, the [gem
artifact](https://console.cloud.google.com/storage/browser/grpc-testing-kokoro-prod/test_result_public/prod/grpc/core/pull_request/linux/grpc_distribtests_ruby/19833/20230708-181642/github/grpc/artifacts?pageState=(%22StorageObjectListTable%22:(%22f%22:%22%255B%255D%22))&prefix=&forceOnObjectsSortingFiltering=false)
size didn't change and arm64-darwin gem runs fine on M1 mac.

fixes: #28631
fixes: #29100
This should resolve breakage on master caused by the jump to Cython
3.0.0 this morning.
`black` doesn't generate an error for the `--diff` command line
argument, but does for `--check` - so that's the one we should use in
sanity checks.
Going forward `[[nodiscard]]` is the portable way to spell this;
requires yanking a bunch of usage from after the param list to before.

We should further refine the GRPC_MUST_USE_RESULT macro to make it work
uniformly for any compilers that it doesn't today (most likely by making
it expand to nothing).

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
…xceeds the BIO buffer size. (#33638)

There is a bug in the SSL stack that was only partially fixed in #29176:
if more than 17kb is written to the BIO buffer, then everything over
17kb will be discarded, and the SSL handshake will fail with a bad
record mac error or hang if not enough bytes have arrived yet.

It's relatively uncommon to hit this bug, because the TLS handshake
messages need to be much larger than normal for you to have a chance of
hitting this bug. However, there was a separate bug in the SSL stack
(recently fixed in #33558) that causes the ServerHello produced by a
gRPC-C++ TLS server to grow linearly in size with the size of the trust
bundle; these 2 bugs combined to cause a large number of TLS handshake
failures for gRPC-C++ clients talking to gRPC-C++ servers when the
server had a large trust bundle.

This PR fixes the bug by ensuring that all bytes are successfully
written to the BIO buffer. An initial quick fix for this bug was planned
in #33611, but abandoned because we were worried about temporarily
doubling the memory footprint of all SSL channels.

The complexity in this PR is mostly in the test: it is fairly tricky to
force gRPC-C++'s SSL stack to generate a sufficiently large ServerHello
to trigger this bug.
…unique_ptr<DNSResolver>>` (#33744)

Based on the discussion at:
https://github.com/grpc/grpc/pull/32701/files/595a75cc5d56f8d7341a9533a24428ad37810269..e3b402a8fa27fc8787a73775068721bf9f322640#r1244325752


<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
This test has been disabled for a long time now due to flakiness, but
it's now causing problems with the import. And stress tests don't
provide positive ROI anyway, so let's just get rid of it.
Will be needed in order to fix some internal tests for the promise
conversion
… to the app (#33782)

Promises code can prevent these bad requests from even reaching the
application, which is beneficial but this test needs a minor update to
handle it.

---------

Co-authored-by: ctiller <ctiller@users.noreply.github.com>
As title - use a deadline that rules out starvation as a source of
flakiness
Fixes b/282107030

I tested this locally by building the images and running some PSM
interop tests.
The case of `!grpc_ruby_initial_pid()` can be a *cause* of the second
case, `!grpc_ruby_initial_thread`, so check the pid first.
We are seeing `g++: fatal error: Killed signal terminated program
cc1plus` on PHP distribtest builds. In case it's an OOM, let's try
reducing the build parallelism to see if it helps.
barracuda156 and others added 28 commits August 22, 2023 12:52
…o pull in __STDC_FORMAT_MACROS define) (#32159)

Some versions of MacOS (as well as some `glibc`-based platforms) require
`__STDC_FORMAT_MACROS` to be defined prior to including `<cinttypes>` or
`<inttypes.h>`, otherwise build fails with undeclared `PRIdMAX`,
`PRIdPTR` etc.

See note here: https://en.cppreference.com/w/cpp/types/integer

label: (release notes: no)
Add example for TLS.

<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
…… (#34129)

…th Bazel on Windows (#34107)"

This reverts commit d540b4c.
…34127)

The tests are skipped incorrectly because `config.server_lang` is
incorrectly compared with the string value "java", instead of
`skips.Lang.JAVA`.

This has been broken since #26998. 

```
xds_url_map_testcase.py:372] ----- Testing TestTimeoutInRouteRule -----
xds_url_map_testcase.py:373] Logs timezone: UTC
skips.py:121] Skipping TestConfig(client_lang='java', server_lang='java', version='v1.57.x')
[  SKIPPED ] setUpClass (timeout_test.TestTimeoutInRouteRule)
xds_url_map_testcase.py:372] ----- Testing TestTimeoutInApplication -----
xds_url_map_testcase.py:373] Logs timezone: UTC
skips.py:121] Skipping TestConfig(client_lang='java', server_lang='java', version='v1.57.x')
[  SKIPPED ] setUpClass (timeout_test.TestTimeoutInApplication)
```
- Add Github Action to conditionally run PSM Interop unit tests:
- Only run when changes are detected in
`tools/run_tests/xds_k8s_test_driver` or any of the proto files used by
the driver
  - Only run against PRs and pushes to `master`, `v1.*.*` branches
  - Runs using `python3.9` and `python3.10`
  - Ready to be added to the list of required GitHub checks
- Add `tools/run_tests/xds_k8s_test_driver/tests/unit/__main__.py` test
loader that recursively discovers all unit tests in
`tools/run_tests/xds_k8s_test_driver/tests/unit`
- Add basic coverage for `XdsTestClient` and `XdsTestServer` to verify
the test loader picks up all folders

Related:
- First unit tests without automated CI added in #34097
We enabled OpenSSL3 testing with #31256 and missed a failing test

It wasn't running before, so this isn't a regression - disabling it so
master doesn't fail while we figure out how to fix it.
This WorkStealingThreadPool change improves the (lagging)
`cpp_protobuf_async_streaming_qps_unconstrained_secure` 32-core
benchmark.

Baseline OriginalThreadPool QPS: 830k
Previous average WorkStealingThreadPool QPS: 755k
New WorkStealingThreadPool average (2 runs) QPS: 850k
…(#34130)

Fixes an issue when an active context selected automatically picked up
as context for `secondary_k8s_api_manager`.

This was introducing an error in GAMMA Baseline PoC

```
sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('100.71.2.143', 56723), raddr=('35.199.174.232', 443)>
```

Here's how the secondary context is incorrectly falls back to the
default context when `--secondary_kube_context` is not set:
```
k8s.py:142] Using kubernetes context "gke_grpc-testing_us-central1-a_psm-interop-security", active host: https://35.202.85.90
k8s.py:142] Using kubernetes context "None", active host: https://35.202.85.90
```
Change was created by the release automation script. See go/grpc-release
Change was created by the release automation script. See go/grpc-release
…ble with older c-ares versions" to v1.58.x (#34324)

Backport grpc/grpc#34314 to v1.58.x.

<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
Change was created by the release automation script. See go/grpc-release
Backport: [PSM Interop] Enable
GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST (#34205)
Change was created by the release automation script. See go/grpc-release
@rschu1ze rschu1ze closed this Nov 15, 2023
@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 16 committers have signed the CLA.

✅ rschu1ze
❌ stanley-cheung
❌ jtattermusch
❌ ctiller
❌ drfloob
❌ jrandolf
❌ sampajano
❌ XuanWang-Amos
❌ markdroth
❌ veblush
❌ apolcyn
❌ barracuda156
❌ yijiem
❌ eugeneo
❌ gtcooke94
❌ sergiitk
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.