Skip to content

[7.x](backport #26260) Handle data returned with io.EOF in LineReader#26299

Merged
andrewkroh merged 1 commit into7.xfrom
mergify/bp/7.x/pr-26260
Jun 14, 2021
Merged

[7.x](backport #26260) Handle data returned with io.EOF in LineReader#26299
andrewkroh merged 1 commit into7.xfrom
mergify/bp/7.x/pr-26260

Conversation

@mergify
Copy link
Copy Markdown
Contributor

@mergify mergify bot commented Jun 14, 2021

This is an automatic backport of pull request #26260 done by Mergify.


Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.io/

The libbeat LineReader implementation did not handle the case where the underlying io.Reader
it was reading from returns bytes and io.EOF. It was discarding the data in this case.

As per the io.Reader contract:

    a Reader returning a non-zero number of bytes at the end of the input stream may return either err == EOF or err == nil.

This occurs often with the gzip.Reader. It returns a large chunk of data at the end of the file and io.EOF at the same time.

(cherry picked from commit 6896e77)
@mergify mergify bot added the backport label Jun 14, 2021
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 14, 2021
@botelastic
Copy link
Copy Markdown

botelastic bot commented Jun 14, 2021

This pull request doesn't have a Team:<team> label.

@elasticmachine
Copy link
Copy Markdown
Contributor

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #26299 opened

  • Start Time: 2021-06-14T17:18:24.238+0000

  • Duration: 146 min 1 sec

  • Commit: 7bba1a2

Test stats 🧪

Test Results
Failed 1
Passed 47448
Skipped 5198
Total 52647

Trends 🧪

Image of Build Times

Image of Tests

Test errors 1

Expand to view the tests failures

Extended / heartbeat-windows-10-windows-10 / TestQueueRunsInOrder – github.com/elastic/beats/v7/heartbeat/scheduler/timerqueue
    Expand to view the error details

     Failed 
    

    Expand to view the stacktrace

     === RUN   TestQueueRunsInOrder
    coverage: 81.4% of statements
    panic: test timed out after 10m0s
    
    goroutine 55 [running]:
    testing.(*M).startAlarm.func1()
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/.gvm/versions/go1.16.4.windows.amd64/src/testing/testing.go:1700 +0xe6
    created by time.goFunc
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/.gvm/versions/go1.16.4.windows.amd64/src/time/sleep.go:180 +0x4b
    
    goroutine 1 [chan receive]:
    testing.(*T).Run(0xc00003d080, 0x77e96d, 0x14, 0x78dba0, 0x61592d)
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/.gvm/versions/go1.16.4.windows.amd64/src/testing/testing.go:1239 +0x2da
    testing.runTests.func1(0xc00003cf00)
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/.gvm/versions/go1.16.4.windows.amd64/src/testing/testing.go:1511 +0x7f
    testing.tRunner(0xc00003cf00, 0xc00010fda8)
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/.gvm/versions/go1.16.4.windows.amd64/src/testing/testing.go:1193 +0xef
    testing.runTests(0xc000004150, 0x8d0ec0, 0x2, 0x2, 0xc02a09884aab0e00, 0x8bb3321129, 0x8daa20, 0x77c381)
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/.gvm/versions/go1.16.4.windows.amd64/src/testing/testing.go:1509 +0x310
    testing.(*M).Run(0xc00007c100, 0x0)
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/.gvm/versions/go1.16.4.windows.amd64/src/testing/testing.go:1417 +0x1f5
    main.main()
    	_testmain.go:97 +0x1c8
    
    goroutine 6 [chan receive]:
    github.com/elastic/beats/v7/heartbeat/scheduler/timerqueue.testQueueRunsInOrderOnce(0xc00003d080)
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/src/github.com/elastic/beats/heartbeat/scheduler/timerqueue/queue_test.go:79 +0x238
    github.com/elastic/beats/v7/heartbeat/scheduler/timerqueue.TestQueueRunsInOrder(0xc00003d080)
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/src/github.com/elastic/beats/heartbeat/scheduler/timerqueue/queue_test.go:33 +0x3b
    testing.tRunner(0xc00003d080, 0x78dba0)
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/.gvm/versions/go1.16.4.windows.amd64/src/testing/testing.go:1193 +0xef
    created by testing.(*T).Run
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/.gvm/versions/go1.16.4.windows.amd64/src/testing/testing.go:1238 +0x2b3
    
    goroutine 179 [runnable]:
    github.com/elastic/beats/v7/heartbeat/scheduler/timerqueue.(*TimerQueue).Start.func1.1(0xc000414d80, 0xa, 0x10, 0xc02a09884b925ecc, 0x8bb41961f5, 0x8daa20)
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/src/github.com/elastic/beats/heartbeat/scheduler/timerqueue/queue.go:82
    created by github.com/elastic/beats/v7/heartbeat/scheduler/timerqueue.(*TimerQueue).Start.func1
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/src/github.com/elastic/beats/heartbeat/scheduler/timerqueue/queue.go:82 +0x1dd
    
    goroutine 91 [runnable]:
    github.com/elastic/beats/v7/heartbeat/scheduler/timerqueue.(*TimerQueue).Start.func1(0xc000187000)
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/src/github.com/elastic/beats/heartbeat/scheduler/timerqueue/queue.go:74 +0x11d
    created by github.com/elastic/beats/v7/heartbeat/scheduler/timerqueue.(*TimerQueue).Start
    	C:/Users/jenkins/workspace/PR-26299-1-d614fbcd-66a9-45f2-a665-edc589d9026f/src/github.com/elastic/beats/heartbeat/scheduler/timerqueue/queue.go:72 +0x56
     
    

Steps errors 2

Expand to view the steps failures

heartbeat-windows-10-windows-10 - mage build unitTest
  • Took 12 min 13 sec . View more details on here
  • Description: mage build unitTest
Error signal
  • Took 0 min 0 sec . View more details on here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output

[2021-06-14T19:42:25.004Z] 2.46s call     packetbeat/tests/system/test_0060_flows.py::Test::test_community_id_icmp6
[2021-06-14T19:42:25.004Z] 2.46s call     packetbeat/tests/system/test_0060_flows.py::Test::test_mysql_flow
[2021-06-14T19:42:25.004Z] 2.44s call     packetbeat/tests/system/test_0060_flows.py::Test::test_community_id_icmp
[2021-06-14T19:42:25.004Z] 2.44s call     packetbeat/tests/system/test_0060_flows.py::Test::test_icmp6_ping
[2021-06-14T19:42:25.004Z] 2.43s call     packetbeat/tests/system/test_0060_flows.py::Test::test_q_in_q_flow
[2021-06-14T19:42:25.004Z] 2.43s call     packetbeat/tests/system/test_0060_flows.py::Test::test_community_id_ipv4_tcp
[2021-06-14T19:42:25.004Z] 2.43s call     packetbeat/tests/system/test_0060_flows.py::Test::test_community_id_ipv4_udp
[2021-06-14T19:42:25.004Z] 2.43s call     packetbeat/tests/system/test_0060_flows.py::Test::test_memcache_udp_flow
[2021-06-14T19:42:25.004Z] 2.42s call     packetbeat/tests/system/test_0060_flows.py::Test::test_icmp4_ping
[2021-06-14T19:42:25.004Z] 1.74s call     packetbeat/tests/system/test_0063_http_body.py::Test::test_include_body_for_both_request_response
[2021-06-14T19:42:25.004Z] 1.35s call     packetbeat/tests/system/test_0032_dns.py::Test::test_MX
[2021-06-14T19:42:25.004Z] 1.34s call     packetbeat/tests/system/test_0005_mysql_integration.py::Test::test_string_operations
[2021-06-14T19:42:25.004Z] 1.34s call     packetbeat/tests/system/test_0025_mongodb_basic.py::Test::test_opmsg
[2021-06-14T19:42:25.004Z] 1.30s call     packetbeat/tests/system/test_0040_memcache_tcp_bin_basic.py::Test::test_delete
[2021-06-14T19:42:25.004Z] 1.29s call     packetbeat/tests/system/test_0032_dns.py::Test::test_include_additionals
[2021-06-14T19:42:25.004Z] 1.29s call     packetbeat/tests/system/test_0002_thrift_basics.py::Test::test_thrift_integration
[2021-06-14T19:42:25.004Z] ============ 187 passed, 5 skipped, 1 warning in 259.87s (0:04:19) ============
[2021-06-14T19:42:25.004Z] >> python test: Unit Testing Complete
[2021-06-14T19:42:25.747Z] warn: failed to upgrade pip (ignoring): running "null\build\ve\windows\Scripts\pip install -U pip" failed with exit code 1============================= test session starts =============================
[2021-06-14T19:42:25.747Z] platform win32 -- Python 3.8.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
[2021-06-14T19:42:25.747Z] rootdir: C:\Users\jenkins\workspace\PR-26299-1-ca1b61a4-f88e-482e-830b-c039263aeba9\src\github.com\elastic\beats, configfile: pytest.ini
[2021-06-14T19:42:25.747Z] plugins: rerunfailures-9.1.1, timeout-1.4.2
[2021-06-14T19:42:25.747Z] timeout: 90.0s
[2021-06-14T19:42:25.747Z] timeout method: thread
[2021-06-14T19:42:25.747Z] timeout func_only: True
[2021-06-14T19:42:25.747Z] collected 23 items
[2021-06-14T19:42:25.747Z] 
[2021-06-14T19:42:25.747Z] tests\system\test_exports.py .....                                       [ 21%]
[2021-06-14T19:42:25.747Z] tests\system\test_metricsets.py .ss.s                                    [ 43%]
[2021-06-14T19:42:25.747Z] tests\system\test_system_socket.py sssssssssssss                         [100%]
[2021-06-14T19:42:25.747Z] 
[2021-06-14T19:42:25.747Z] - generated xml file: C:\Users\jenkins\workspace\PR-26299-1-ca1b61a4-f88e-482e-830b-c039263aeba9\src\github.com\elastic\beats\x-pack\auditbeat\build\TEST-python-unit.xml -
[2021-06-14T19:42:25.747Z] ============================ slowest 20 durations =============================
[2021-06-14T19:42:25.747Z] 1.99s call     x-pack/auditbeat/tests/system/test_metricsets.py::Test::test_metricset_host
[2021-06-14T19:42:25.747Z] 0.41s call     x-pack/auditbeat/tests/system/test_exports.py::Test::test_export_index_pattern_migration
[2021-06-14T19:42:25.747Z] 0.40s call     x-pack/auditbeat/tests/system/test_exports.py::Test::test_export_index_pattern
[2021-06-14T19:42:25.747Z] 0.39s call     x-pack/auditbeat/tests/system/test_metricsets.py::Test::test_metricset_process
[2021-06-14T19:42:25.747Z] 0.37s call     x-pack/auditbeat/tests/system/test_exports.py::Test::test_export_template
[2021-06-14T19:42:25.747Z] 0.32s call     x-pack/auditbeat/tests/system/test_exports.py::Test::test_export_config
[2021-06-14T19:42:25.747Z] 0.25s call     x-pack/auditbeat/tests/system/test_exports.py::Test::test_export_ilm_policy
[2021-06-14T19:42:25.747Z] 
[2021-06-14T19:42:25.747Z] (13 durations < 0.005s hidden.  Use -vv to show these durations.)
[2021-06-14T19:42:25.747Z] ======================== 7 passed, 16 skipped in 4.40s ========================
[2021-06-14T19:42:25.747Z] >> python test: Unit Testing Complete
[2021-06-14T19:42:26.134Z] 
[2021-06-14T19:42:26.135Z] C:\Users\jenkins\workspace\PR-26299-1-d23d7578-5605-4367-afd6-e5313676cb88\src\github.com\elastic\beats>FOR / %d IN ("ve") DO @IF EXIST "%d" rmdir /s /q "%d" 
[2021-06-14T19:42:26.965Z] 
[2021-06-14T19:42:26.966Z] C:\Users\jenkins\workspace\PR-26299-1-ca1b61a4-f88e-482e-830b-c039263aeba9\src\github.com\elastic\beats>FOR / %d IN ("ve") DO @IF EXIST "%d" rmdir /s /q "%d" 
[2021-06-14T19:42:27.123Z] 
[2021-06-14T19:42:27.124Z] C:\Users\jenkins\workspace\PR-26299-1-9494e214-4063-4aa6-8522-755cd141ce36\src\github.com\elastic\beats>python .ci/scripts/pre_archive_test.py 
[2021-06-14T19:42:30.502Z] Copy .\filebeat\build into build\filebeat\build
[2021-06-14T19:42:30.502Z] Copy .\filebeat\null\build into build\filebeat\null\build
[2021-06-14T19:42:30.651Z] Running in C:\Users\jenkins\workspace\PR-26299-1-9494e214-4063-4aa6-8522-755cd141ce36\src\github.com\elastic\beats\build
[2021-06-14T19:42:30.766Z] Recording test results
[2021-06-14T19:42:31.635Z] 
[2021-06-14T19:42:31.635Z] C:\Users\jenkins\workspace\PR-26299-1-ca1b61a4-f88e-482e-830b-c039263aeba9\src\github.com\elastic\beats>python .ci/scripts/pre_archive_test.py 
[2021-06-14T19:42:32.001Z] 
[2021-06-14T19:42:32.002Z] C:\Users\jenkins\workspace\PR-26299-1-d23d7578-5605-4367-afd6-e5313676cb88\src\github.com\elastic\beats>python .ci/scripts/pre_archive_test.py 
[2021-06-14T19:42:32.064Z] Copy .\x-pack\auditbeat\build into build\x-pack\auditbeat\build
[2021-06-14T19:42:32.065Z] Copy .\x-pack\auditbeat\null\build into build\x-pack\auditbeat\null\build
[2021-06-14T19:42:32.220Z] Running in C:\Users\jenkins\workspace\PR-26299-1-ca1b61a4-f88e-482e-830b-c039263aeba9\src\github.com\elastic\beats\build
[2021-06-14T19:42:32.431Z] Recording test results
[2021-06-14T19:42:35.522Z] Copy .\packetbeat\build into build\packetbeat\build
[2021-06-14T19:42:35.527Z] Copy .\packetbeat\null\build into build\packetbeat\null\build
[2021-06-14T19:42:35.702Z] Running in C:\Users\jenkins\workspace\PR-26299-1-d23d7578-5605-4367-afd6-e5313676cb88\src\github.com\elastic\beats\build
[2021-06-14T19:42:37.997Z] [Checks API] No suitable checks publisher found.
[2021-06-14T19:42:38.072Z] Recording test results
[2021-06-14T19:42:40.943Z] [Checks API] No suitable checks publisher found.
[2021-06-14T19:42:42.890Z] [Checks API] No suitable checks publisher found.
[2021-06-14T19:42:44.111Z] 
[2021-06-14T19:42:44.111Z] C:\Users\jenkins\workspace\PR-26299-1-9494e214-4063-4aa6-8522-755cd141ce36\src\github.com\elastic\beats>go clean -modcache 
[2021-06-14T19:42:44.306Z] 
[2021-06-14T19:42:44.306Z] C:\Users\jenkins\workspace\PR-26299-1-d23d7578-5605-4367-afd6-e5313676cb88\src\github.com\elastic\beats>go clean -modcache 
[2021-06-14T19:42:44.379Z] 
[2021-06-14T19:42:44.379Z] C:\Users\jenkins\workspace\PR-26299-1-ca1b61a4-f88e-482e-830b-c039263aeba9\src\github.com\elastic\beats>go clean -modcache 
[2021-06-14T19:43:56.950Z] + gsutil --version
[2021-06-14T19:43:58.479Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-06-14T19:43:58.878Z] + gcloud auth activate-service-account --key-file ****
[2021-06-14T19:43:59.491Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-06-14T19:43:59.961Z] + gsutil -m -q cp -a public-read ZmlsZWJlYXQtd2luZG93cy03LTMyLXdpbmRvd3MtNy0zMi1iaXQ3YmJhMWEyMDc4ZTg5ZGU2ZTEyNzU4MzAzNjBiMTY1MmNkYWIzYmQ5 gs://beats-ci-temp/ci/cache/
[2021-06-14T19:44:10.961Z] + gsutil --version
[2021-06-14T19:44:11.629Z] + gsutil --version
[2021-06-14T19:44:12.488Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-06-14T19:44:12.904Z] + gcloud auth activate-service-account --key-file ****
[2021-06-14T19:44:13.158Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-06-14T19:44:13.518Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-06-14T19:44:13.573Z] + gcloud auth activate-service-account --key-file ****
[2021-06-14T19:44:14.033Z] + gsutil -m -q cp -a public-read cGFja2V0YmVhdC13aW5kb3dzLTctMzItd2luZG93cy03LTMyLWJpdDdiYmExYTIwNzhlODlkZTZlMTI3NTgzMDM2MGIxNjUyY2RhYjNiZDk gs://beats-ci-temp/ci/cache/
[2021-06-14T19:44:14.283Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-06-14T19:44:15.080Z] + gsutil -m -q cp -a public-read eC1wYWNrL2F1ZGl0YmVhdC13aW5kb3dzLTctMzItd2luZG93cy03LTMyLWJpdDdiYmExYTIwNzhlODlkZTZlMTI3NTgzMDM2MGIxNjUyY2RhYjNiZDk gs://beats-ci-temp/ci/cache/
[2021-06-14T19:44:21.605Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-06-14T19:44:21.779Z] Stage "Packaging-Pipeline" skipped due to earlier failure(s)
[2021-06-14T19:44:22.041Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-26299/src/github.com/elastic/beats
[2021-06-14T19:44:23.190Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-26299
[2021-06-14T19:44:23.438Z] [INFO] getVaultSecret: Getting secrets
[2021-06-14T19:44:23.561Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-06-14T19:44:25.497Z] + chmod 755 generate-build-data.sh
[2021-06-14T19:44:25.497Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-26299/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-26299/runs/1 FAILURE 8760944
[2021-06-14T19:44:25.497Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-26299/runs/1/steps/?limit=10000 -o steps-info.json
[2021-06-14T19:44:40.270Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-26299/runs/1/tests/?status=FAILED -o tests-errors.json

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Test stats 🧪

Test Results
Failed 1
Passed 47448
Skipped 5198
Total 52647

Genuine test errors 1

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Extended / heartbeat-windows-10-windows-10 / TestQueueRunsInOrder – github.com/elastic/beats/v7/heartbeat/scheduler/timerqueue

@andrewkroh andrewkroh merged commit 67b37d0 into 7.x Jun 14, 2021
@mergify mergify bot deleted the mergify/bp/7.x/pr-26260 branch June 14, 2021 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport needs_team Indicates that the issue/PR needs a Team:* label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants