Upgrade netty to 4.1.75 #15142
Description
Description of the problem / feature request:
Please upgrade Netty 4.1.69 -> 4.1.75
On linux, we frequently run into netty/netty#11815
This is likely solved in Netty 4.1.75 via netty/netty@1df38d2
Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
There is no easy repro, it seems to be a race condition. Only seems to appear on Linux, not Mac. See below for stack trace.
What operating system are you running Bazel on?
CircleCI image:
Build-agent version 1.0.117438-72c2fbdd (2022-03-29T17:30:21+0000)
System information:
Server Version: 20.10.12
Storage Driver: overlay2
Backing Filesystem: xfs
Cgroup Driver: cgroupfs
Cgroup Version: 1
Kernel Version: 5.13.0-1017-aws
Operating System: Ubuntu 20.04.4 LTS
OSType: linux
Architecture: x86_64
What's the output of bazel info release?
root@62d8983e8522:/src/mn/projects/fullstory# bazel version
Bazelisk version: v1.4.0
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Build label: 5.0.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed Jan 19 14:08:54 2022 (1642601334)
Build timestamp: 1642601334
Build timestamp as int: 1642601334
root@62d8983e8522:/src/mn/projects/fullstory# bazel info release
INFO: Invocation ID: 92faf5b1-4236-49ef-a704-9c2ee74d5bcb
INFO: Reading 'startup' options from /src/mn/projects/fullstory/.bazelrc.user: --shutdown_on_low_sys_mem, --max_idle_secs=5
INFO: Options provided by the client:
Inherited 'common' options: --isatty=0 --terminal_columns=179
INFO: Reading rc options for 'info' from /src/mn/projects/fullstory/.bazelrc:
Inherited 'common' options: --enable_platform_specific_config --repository_cache=./bazel-cache/repos --experimental_repository_cache_hardlinks
INFO: Reading rc options for 'info' from /src/mn/projects/fullstory/.bazelrc.user:
Inherited 'common' options: --repository_cache=/tmp/bazel/repo_cache
INFO: Reading rc options for 'info' from /src/mn/projects/fullstory/.bazelrc:
Inherited 'build' options: --strategy=TypeScriptCompile=worker --remote_cache=https://bazel-cache.ops.fs.team --google_default_credentials --remote_upload_local_results=false --disk_cache=./bazel-cache --incompatible_remote_results_ignore_disk --incompatible_exclusive_test_sandboxed --define=GCR_REPOSITORY=fs-ops/feature-builds --define=GCR_REGISTRY_PREFIX=gcr.io --incompatible_strict_action_env --workspace_status_command=./tools/bazelbuild/load_build_info.sh
INFO: Reading rc options for 'info' from /src/mn/projects/fullstory/.bazelrc.user:
Inherited 'build' options: --show_progress --build_tag_filters=-no-ci --local_cpu_resources=2 --local_ram_resources=2048 --action_env=NODE_OPTIONS --worker_max_instances=TypeScriptCompile=1 --discard_analysis_cache --nokeep_state_after_build --notrack_incremental_state --announce_rc --google_credentials=/root/gcloud-service-key.json --disk_cache=/tmp/bazel/action_cache --remote_upload_local_results=true --jobs 10
release 5.0.0
What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?
fatal: not a git repository (or any of the parent directories): .git
Have you found anything relevant by searching the web?
Yes.. see above Netty issue. netty/netty#11815
Any other information, logs, or outputs that you want to share?
FATAL: bazel crashed due to an internal error. Printing stack trace:
java.lang.RuntimeException: Unrecoverable error while evaluating node 'ActionLookupData{actionLookupKey=ConfiguredTargetKey{label=//go/src/fs/services/alexa/main/alexa:alexa, config=BuildConfigurationValue.Key[48a7a349feeea004bb328fa5c50a7b593c6e17033e218e2151db12a21b868035]}, actionIndex=1}' (requested by nodes 'TargetCompletionKey{topLevelArtifactContext=com.google.devtools.build.lib.analysis.TopLevelArtifactContext@90904c3b, actionLookupKey=ConfiguredTargetKey{label=//go/src/fs/services/alexa/main/alexa:alexa, config=BuildConfigurationValue.Key[48a7a349feeea004bb328fa5c50a7b593c6e17033e218e2151db12a21b868035]}, willTest=false}', 'ActionLookupData{actionLookupKey=ConfiguredTargetKey{label=//go/src/fs/services/alexa/main/alexa:alexa, config=BuildConfigurationValue.Key[48a7a349feeea004bb328fa5c50a7b593c6e17033e218e2151db12a21b868035]}, actionIndex=4}')
at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:674)
at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:382)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: error:1e000065:Cipher functions:OPENSSL_internal:BAD_DECRYPT
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:477)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 more
Caused by: javax.net.ssl.SSLException: error:1e000065:Cipher functions:OPENSSL_internal:BAD_DECRYPT
at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.shutdownWithError(ReferenceCountedOpenSslEngine.java:1071)
at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.sslReadErrorResult(ReferenceCountedOpenSslEngine.java:1365)
at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1305)
at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1392)
at io.netty.handler.ssl.SslHandler$SslEngineType$1.unwrap(SslHandler.java:217)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1342)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1235)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446)
... 21 more