Cherry-pick #18824 to 7.8: Improve error messages in s3 input by kaiyan-sheng · Pull Request #18837 · elastic/beats

kaiyan-sheng · 2020-05-28T22:18:12Z

Cherry-pick of PR #18824 to 7.8 branch. Original message:

This PR is to improve error messages in S3 input Filebeat to capture more information such as the S3 bucket name and log file name.

(cherry picked from commit f6bd8c6)

elasticmachine · 2020-05-28T22:18:33Z

Pinging @elastic/integrations-platforms (Team:Platforms)

elasticmachine · 2020-05-28T22:56:58Z

💔 Build Failed

Expand to view the summary

Build stats

Build Cause: [Pull request #18837 opened]
Start Time: 2020-05-28T22:19:01.367+0000
Duration: 38 min 54 sec

Test stats 🧪

Test	Results
Failed	0
Passed	1007
Skipped	254
Total	1261

Steps errors

Expand to view the steps failures

Name: Mage update build test
- Description: mage update build test
- Duration: 7 min 9 sec
- Start Time: 2020-05-28T22:42:36.345+0000
- log

Log output

Expand to view the last 100 lines of log output

[2020-05-28T22:48:15.827Z] SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
[2020-05-28T22:48:15.827Z] ----------------------------------------------------------------------
[2020-05-28T22:48:15.827Z] Ran 120 tests in 0.012s
[2020-05-28T22:48:15.828Z] 
[2020-05-28T22:48:15.828Z] OK (SKIP=120)
[2020-05-28T22:48:15.828Z] >> python test: Unit Testing Complete
[2020-05-28T22:48:23.949Z] >> Building docker images
[2020-05-28T22:48:23.949Z] Building googlepubsub
[2020-05-28T22:49:32.837Z] >> go test: Unit Testing
[2020-05-28T22:49:45.404Z] Service 'googlepubsub' failed to build: The command '/bin/sh -c apt-get update     && apt-get install -y         google-cloud-sdk         google-cloud-sdk-pubsub-emulator 	&& rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100
[2020-05-28T22:49:45.404Z] Error: running "docker-compose -p filebeat_7_8_0_005cb61857-snapshot build --force-rm --pull" failed with exit code 1
[2020-05-28T22:49:45.551Z] Recording test results
[2020-05-28T22:49:47.034Z] Stashed 2 file(s)
[2020-05-28T22:49:47.052Z] Archiving artifacts
[2020-05-28T22:49:47.811Z] + curl -sSLo codecov https://codecov.io/bash
[2020-05-28T22:49:48.069Z] + FILE=auditbeat/build/coverage/full.cov
[2020-05-28T22:49:48.069Z] + [ -f auditbeat/build/coverage/full.cov ]
[2020-05-28T22:49:48.069Z] + FILE=filebeat/build/coverage/full.cov
[2020-05-28T22:49:48.069Z] + [ -f filebeat/build/coverage/full.cov ]
[2020-05-28T22:49:48.069Z] + FILE=heartbeat/build/coverage/full.cov
[2020-05-28T22:49:48.069Z] + [ -f heartbeat/build/coverage/full.cov ]
[2020-05-28T22:49:48.069Z] + FILE=libbeat/build/coverage/full.cov
[2020-05-28T22:49:48.069Z] + [ -f libbeat/build/coverage/full.cov ]
[2020-05-28T22:49:48.069Z] + FILE=metricbeat/build/coverage/full.cov
[2020-05-28T22:49:48.069Z] + [ -f metricbeat/build/coverage/full.cov ]
[2020-05-28T22:49:48.069Z] + FILE=packetbeat/build/coverage/full.cov
[2020-05-28T22:49:48.069Z] + [ -f packetbeat/build/coverage/full.cov ]
[2020-05-28T22:49:48.069Z] + FILE=winlogbeat/build/coverage/full.cov
[2020-05-28T22:49:48.069Z] + [ -f winlogbeat/build/coverage/full.cov ]
[2020-05-28T22:49:48.069Z] + FILE=journalbeat/build/coverage/full.cov
[2020-05-28T22:49:48.069Z] + [ -f journalbeat/build/coverage/full.cov ]
[2020-05-28T22:49:49.483Z] Failed in branch Filebeat x-pack
[2020-05-28T22:52:54.384Z] SUMMARY:
[2020-05-28T22:52:54.384Z]   Fail:     0
[2020-05-28T22:52:54.384Z]   Skip:     7
[2020-05-28T22:52:54.385Z]   Pass:     496
[2020-05-28T22:52:54.385Z]   Packages: 19
[2020-05-28T22:52:54.385Z]   Duration: 3m17.5036664s
[2020-05-28T22:52:54.385Z]   Coverage Report: C:\Users\jenkins\workspace\Beats_beats-beats-mbp_PR-18837\src\github.com\elastic\beats\x-pack\filebeat\build\TEST-go-unit.html
[2020-05-28T22:52:54.385Z]   JUnit Report:    C:\Users\jenkins\workspace\Beats_beats-beats-mbp_PR-18837\src\github.com\elastic\beats\x-pack\filebeat\build\TEST-go-unit.xml
[2020-05-28T22:52:54.385Z]   Output File:     C:\Users\jenkins\workspace\Beats_beats-beats-mbp_PR-18837\src\github.com\elastic\beats\x-pack\filebeat\build\TEST-go-unit.out
[2020-05-28T22:52:54.385Z] >> go test: Unit Test Passed
[2020-05-28T22:55:15.903Z] >> python test: Unit Testing
[2020-05-28T22:56:23.608Z] WARNING: You are using pip version 19.2.3, however version 20.1.1 is available.
[2020-05-28T22:56:23.608Z] You should consider upgrading via the 'python -m pip install --upgrade pip' command.
[2020-05-28T22:56:23.608Z] SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
[2020-05-28T22:56:23.608Z] ----------------------------------------------------------------------
[2020-05-28T22:56:23.608Z] Ran 120 tests in 0.019s
[2020-05-28T22:56:23.608Z] 
[2020-05-28T22:56:23.608Z] OK (SKIP=120)
[2020-05-28T22:56:23.608Z] >> python test: Unit Testing Complete
[2020-05-28T22:56:23.752Z] Recording test results
[2020-05-28T22:56:28.577Z] Stashed 2 file(s)
[2020-05-28T22:56:28.596Z] Archiving artifacts
[2020-05-28T22:56:32.532Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18837/src/github.com/elastic/beats
[2020-05-28T22:56:32.868Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-05-28T22:56:32.889Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18837/src/github.com/elastic/beats/Lint
[2020-05-28T22:56:33.039Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18837/src/github.com/elastic/beats/Filebeat-x-pack
[2020-05-28T22:56:33.185Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18837/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-05-28T22:56:33.622Z] + cat
[2020-05-28T22:56:33.622Z] + /usr/local/bin/runbld ./runbld-script
[2020-05-28T22:56:33.622Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-05-28T22:56:40.220Z] runbld>>> runbld started
[2020-05-28T22:56:40.220Z] runbld>>> 1.6.11/a66728ff8f4356963772e6e6d2069392fa06acbe
[2020-05-28T22:56:41.606Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-18837' in order of occurrence in the config (last value wins).
[2020-05-28T22:56:42.993Z] runbld>>> Debug logging enabled.
[2020-05-28T22:56:42.993Z] runbld>>> Storing result
[2020-05-28T22:56:42.993Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-05-28T22:56:42.993Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200528225642-382CDF74
[2020-05-28T22:56:42.993Z] runbld>>> Adding system facts.
[2020-05-28T22:56:43.934Z] runbld>>> Adding vcs info for the latest commit:  005cb6185796638740ce83eeb44f42c6408d764f
[2020-05-28T22:56:44.195Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-05-28T22:56:44.195Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-05-28T22:56:44.195Z] + echo 'Processing JUnit reports with runbld...'
[2020-05-28T22:56:44.195Z] Processing JUnit reports with runbld...
[2020-05-28T22:56:44.765Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-05-28T22:56:44.765Z] runbld>>> DURATION: 28ms
[2020-05-28T22:56:44.765Z] runbld>>> STDOUT: 40 bytes
[2020-05-28T22:56:44.765Z] runbld>>> STDERR: 49 bytes
[2020-05-28T22:56:44.765Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-05-28T22:56:44.765Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18837/src/github.com/elastic/beats
[2020-05-28T22:56:46.147Z] runbld>>> Storing build metadata: 
[2020-05-28T22:56:46.147Z] runbld>>> Adding test report.
[2020-05-28T22:56:46.147Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18837/src/github.com/elastic/beats
[2020-05-28T22:56:47.102Z] runbld>>> Found 4 test output files
[2020-05-28T22:56:47.362Z] runbld>>> Test output logs contained: Errors: 0 Failures: 0 Tests: 1261 Skipped: 240
[2020-05-28T22:56:47.623Z] runbld>>> Storing result
[2020-05-28T22:56:47.623Z] runbld>>> FAILURES: 0
[2020-05-28T22:56:47.884Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-05-28T22:56:47.884Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200528225642-382CDF74
[2020-05-28T22:56:47.884Z] runbld>>> Email notification disabled by environment variable.
[2020-05-28T22:56:47.884Z] runbld>>> Slack notification disabled by environment variable.
[2020-05-28T22:56:53.938Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18837
[2020-05-28T22:56:54.224Z] [INFO] getVaultSecret: Getting secrets
[2020-05-28T22:56:54.277Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-05-28T22:56:55.272Z] + chmod 755 generate-build-data.sh
[2020-05-28T22:56:55.272Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18837/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18837/runs/1 FAILURE 2273634
[2020-05-28T22:56:55.272Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18837/runs/1/steps/?limit=10000 -o steps-info.json
[2020-05-28T22:56:55.522Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18837/runs/1/tests/?status=FAILED -o tests-errors.json
[2020-05-28T22:56:55.773Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18837/runs/1/log/ -o pipeline-log.txt

Improve error messages in s3 input (#18824)

005cb61

(cherry picked from commit f6bd8c6)

kaiyan-sheng added backport review labels May 28, 2020

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 28, 2020

kaiyan-sheng self-assigned this May 28, 2020

kaiyan-sheng added the Team:Platforms Label for the Integrations - Platforms team label May 28, 2020

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 28, 2020

ChrsMark approved these changes May 29, 2020

View reviewed changes

kaiyan-sheng merged commit 767914c into elastic:7.8 May 29, 2020

kaiyan-sheng deleted the backport_18824_7.8 branch May 29, 2020 13:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cherry-pick #18824 to 7.8: Improve error messages in s3 input#18837

Cherry-pick #18824 to 7.8: Improve error messages in s3 input#18837
kaiyan-sheng merged 1 commit intoelastic:7.8from
kaiyan-sheng:backport_18824_7.8

kaiyan-sheng commented May 28, 2020

Uh oh!

elasticmachine commented May 28, 2020

Uh oh!

elasticmachine commented May 28, 2020

Build stats

Test stats 🧪

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

kaiyan-sheng commented May 28, 2020

Uh oh!

elasticmachine commented May 28, 2020

Uh oh!

elasticmachine commented May 28, 2020

💔 Build Failed

Build stats

Test stats 🧪

Steps errors

Log output

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants