Skip to content

//test/integration:access_log_integration_test is flaky #2327

@lizan

Description

@lizan

Description:

//test/integration:access_log_integration_test is TIMEOUT in a few runs per large runs.
Discovered when I was trying to reproduce https://circleci.com/gh/envoyproxy/envoy/16714

Repro steps:

At master (commit 48ebd9f):

$ bazel test -c opt --runs_per_test=1000 //test/integration:access_log_integration_test
DEBUG: /home/zlizan/github/lizan/envoy-master/bazel/repositories.bzl:76:5: Fetching external dependencies...
DEBUG: /home/zlizan/github/lizan/envoy-master/bazel/repositories.bzl:82:5:
DEBUG: /home/zlizan/github/lizan/envoy-master/bazel/repositories.bzl:83:5: External dependency cache directory /home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/external/envoy_deps_cache_2c744dffd279d7e9e0910ce594eb4f4f
make: Entering directory '/home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/external/envoy_deps'
make: '/home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/external/envoy_deps_cache_2c744dffd279d7e9e0910ce594eb4f4f/cares.dep' is up to date.
make: '/home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/external/envoy_deps_cache_2c744dffd279d7e9e0910ce594eb4f4f/libevent.dep' is up to date.
make: '/home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/external/envoy_deps_cache_2c744dffd279d7e9e0910ce594eb4f4f/gperftools.dep' is up to date.
make: '/home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/external/envoy_deps_cache_2c744dffd279d7e9e0910ce594eb4f4f/luajit.dep' is up to date.
make: '/home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/external/envoy_deps_cache_2c744dffd279d7e9e0910ce594eb4f4f/boringssl.dep' is up to date.
make: '/home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/external/envoy_deps_cache_2c744dffd279d7e9e0910ce594eb4f4f/yaml-cpp.dep' is up to date.
make: '/home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/external/envoy_deps_cache_2c744dffd279d7e9e0910ce594eb4f4f/zlib.dep' is up to date.
Building in /home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/external/envoy_deps_cache_2c744dffd279d7e9e0910ce594eb4f4f/nghttp2.dep.build, logs at /home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/external/envoy_deps_cache_2c744dffd279d7e9e0910ce594eb4f4f/nghttp2.dep.log
Successful build of /home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/external/envoy_deps_cache_2c744dffd279d7e9e0910ce594eb4f4f/nghttp2.dep
make: Leaving directory '/home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/external/envoy_deps'

real    0m20.636s
user    0m13.504s
sys     0m3.028s
DEBUG: /home/zlizan/github/lizan/envoy-master/bazel/repositories.bzl:84:5: External dep build exited with return code: 0
INFO: Analysed target //test/integration:access_log_integration_test (0 packages loaded).
INFO: Found 1 test target...

TIMEOUT: //test/integration:access_log_integration_test (Summary)
      /home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/execroot/envoy/bazel-out/k8-opt/testlogs/test/integration/access_log_integration_test/run_202_of_1000/test.log
Target //test/integration:access_log_integration_test up-to-date:
  bazel-bin/test/integration/access_log_integration_test
INFO: Elapsed time: 322.997s, Critical Path: 300.09s
INFO: Build completed, 1 test FAILED, 1001 total actions
//test/integration:access_log_integration_test                          TIMEOUT in 1 out of 1000 in 300.1s
  Stats over 1000 runs: max = 300.1s, min = 0.7s, avg = 1.0s, dev = 9.5s
  /home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/execroot/envoy/bazel-out/k8-opt/testlogs/test/integration/access_log_integration_test/run_202_of_1000/test.log

Executed 1 out of 1 test: 1 fails locally.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
$ cat /home/zlizan/.cache/bazel/_bazel_zlizan/38b042ddc5bbd24c5b8f7b3c8c150f08/execroot/envoy/bazel-out/k8-opt/testlogs/test/integration/access_log_integration_test/run_202_of_1000/test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
-----------------------------------------------------------------------------
WARNING: Perftools heap leak checker is active -- Performance may suffer
[==========] Running 2 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 2 tests from IpVersions/AccessLogIntegrationTest
[ RUN      ] IpVersions/AccessLogIntegrationTest.BasicAccessLogFlow/0
Segmentation fault

Config:
N/A

Logs:
N/A

Call Stack:
N/A

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions