Skip to content

Cherry-pick #18956 to 7.x: Add automatic retries and exponential backoff to Filebeat httpjson input#19538

Merged
marc-gr merged 1 commit intoelastic:7.xfrom
marc-gr:backport_18956_7.x
Jul 13, 2020
Merged

Cherry-pick #18956 to 7.x: Add automatic retries and exponential backoff to Filebeat httpjson input#19538
marc-gr merged 1 commit intoelastic:7.xfrom
marc-gr:backport_18956_7.x

Conversation

@marc-gr
Copy link
Copy Markdown
Contributor

@marc-gr marc-gr commented Jul 1, 2020 •
edited by zube bot
Loading

Cherry-pick of PR #18956 to 7.x branch. Original message:

Add automatic retries and exponential backoff to Filebeat httpjson input. If an HTTP error is returned by the client, e.g., connection errors, or if a 500-range response code is received (except 501), then a retry is automatically invoked after a defined wait period.

go test result:

=== RUN   TestProviderCanonical
--- PASS: TestProviderCanonical (0.00s)
=== RUN   TestGetProviderIsCanonical
--- PASS: TestGetProviderIsCanonical (0.00s)
=== RUN   TestIsEnabled
--- PASS: TestIsEnabled (0.00s)
=== RUN   TestGetTokenURL
--- PASS: TestGetTokenURL (0.00s)
=== RUN   TestGetTokenURLWithAzure
--- PASS: TestGetTokenURLWithAzure (0.00s)
=== RUN   TestGetEndpointParams
--- PASS: TestGetEndpointParams (0.00s)
=== RUN   TestGetEndpointParamsWithAzure
--- PASS: TestGetEndpointParamsWithAzure (0.00s)
=== RUN   TestConfigValidationCase1
--- PASS: TestConfigValidationCase1 (0.00s)
=== RUN   TestConfigValidationCase2
--- PASS: TestConfigValidationCase2 (0.00s)
=== RUN   TestConfigValidationCase3
--- PASS: TestConfigValidationCase3 (0.00s)
=== RUN   TestConfigValidationCase4
--- PASS: TestConfigValidationCase4 (0.00s)
=== RUN   TestConfigValidationCase5
--- PASS: TestConfigValidationCase5 (0.00s)
=== RUN   TestConfigValidationCase6
--- PASS: TestConfigValidationCase6 (0.00s)
=== RUN   TestConfigValidationCase7
--- PASS: TestConfigValidationCase7 (0.00s)
=== RUN   TestConfigOauth2Validation
=== RUN   TestConfigOauth2Validation/can't_set_oauth2_and_api_key_together
=== RUN   TestConfigOauth2Validation/can_set_oauth2_and_api_key_together_if_oauth2_is_disabled
=== RUN   TestConfigOauth2Validation/can't_set_oauth2_and_authentication_scheme
=== RUN   TestConfigOauth2Validation/token_url_and_client_credentials_must_be_set
=== RUN   TestConfigOauth2Validation/must_fail_with_an_unknown_provider
=== RUN   TestConfigOauth2Validation/azure_must_have_either_tenant_id_or_token_url
=== RUN   TestConfigOauth2Validation/azure_must_have_only_one_of_token_url_and_tenant_id
=== RUN   TestConfigOauth2Validation/azure_must_have_client_credentials_set
=== RUN   TestConfigOauth2Validation/azure_config_is_valid
=== RUN   TestConfigOauth2Validation/google_can't_have_token_url_or_client_credentials_set
=== RUN   TestConfigOauth2Validation/google_must_fail_if_no_ADC_available
=== RUN   TestConfigOauth2Validation/google_must_fail_if_credentials_file_not_found
=== RUN   TestConfigOauth2Validation/google_must_fail_if_ADC_is_wrongly_set
=== RUN   TestConfigOauth2Validation/google_must_work_if_ADC_is_set_up
=== RUN   TestConfigOauth2Validation/google_must_work_if_credentials_file_is_correct
=== RUN   TestConfigOauth2Validation/google_must_work_if_jwt_file_is_correct
=== RUN   TestConfigOauth2Validation/google_must_work_if_credentials_json_is_correct
=== RUN   TestConfigOauth2Validation/google_must_fail_if_credentials_json_is_not_a_valid_JSON
=== RUN   TestConfigOauth2Validation/google_must_fail_if_the_provided_credentials_file_is_not_a_valid_JSON
--- PASS: TestConfigOauth2Validation (0.00s)
    --- PASS: TestConfigOauth2Validation/can't_set_oauth2_and_api_key_together (0.00s)
    --- PASS: TestConfigOauth2Validation/can_set_oauth2_and_api_key_together_if_oauth2_is_disabled (0.00s)
    --- PASS: TestConfigOauth2Validation/can't_set_oauth2_and_authentication_scheme (0.00s)
    --- PASS: TestConfigOauth2Validation/token_url_and_client_credentials_must_be_set (0.00s)
    --- PASS: TestConfigOauth2Validation/must_fail_with_an_unknown_provider (0.00s)
    --- PASS: TestConfigOauth2Validation/azure_must_have_either_tenant_id_or_token_url (0.00s)
    --- PASS: TestConfigOauth2Validation/azure_must_have_only_one_of_token_url_and_tenant_id (0.00s)
    --- PASS: TestConfigOauth2Validation/azure_must_have_client_credentials_set (0.00s)
    --- PASS: TestConfigOauth2Validation/azure_config_is_valid (0.00s)
    --- PASS: TestConfigOauth2Validation/google_can't_have_token_url_or_client_credentials_set (0.00s)
    --- PASS: TestConfigOauth2Validation/google_must_fail_if_no_ADC_available (0.00s)
    --- PASS: TestConfigOauth2Validation/google_must_fail_if_credentials_file_not_found (0.00s)
    --- PASS: TestConfigOauth2Validation/google_must_fail_if_ADC_is_wrongly_set (0.00s)
    --- PASS: TestConfigOauth2Validation/google_must_work_if_ADC_is_set_up (0.00s)
    --- PASS: TestConfigOauth2Validation/google_must_work_if_credentials_file_is_correct (0.00s)
    --- PASS: TestConfigOauth2Validation/google_must_work_if_jwt_file_is_correct (0.00s)
    --- PASS: TestConfigOauth2Validation/google_must_work_if_credentials_json_is_correct (0.00s)
    --- PASS: TestConfigOauth2Validation/google_must_fail_if_credentials_json_is_not_a_valid_JSON (0.00s)
    --- PASS: TestConfigOauth2Validation/google_must_fail_if_the_provided_credentials_file_is_not_a_valid_JSON (0.00s)
=== RUN   TestGetNextLinkFromHeader
--- PASS: TestGetNextLinkFromHeader (0.00s)
=== RUN   TestCreateRequestInfoFromBody
--- PASS: TestCreateRequestInfoFromBody (0.00s)
=== RUN   TestGetRateLimitCase1
--- PASS: TestGetRateLimitCase1 (0.00s)
=== RUN   TestGetRateLimitCase2
--- PASS: TestGetRateLimitCase2 (0.00s)
=== RUN   TestGetRateLimitCase3
--- PASS: TestGetRateLimitCase3 (0.00s)
=== RUN   TestGET
2020-06-16T20:53:14.947-0700	INFO	[httpjson]	httpjson/input.go:123	Initialized httpjson input.	{"url": "http://127.0.0.1:53858"}
2020-06-16T20:53:14.947-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:502	[DEBUG] GET http://127.0.0.1:53858
--- PASS: TestGET (0.00s)
=== RUN   TestGetHTTPS
2020-06-16T20:53:14.948-0700	INFO	[httpjson]	httpjson/input.go:123	Initialized httpjson input.	{"url": "http://127.0.0.1:53860"}
2020-06-16T20:53:14.948-0700	WARN	[tls]	tlscommon/tls_config.go:83	SSL/TLS verifications disabled.
2020-06-16T20:53:14.948-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:502	[DEBUG] GET http://127.0.0.1:53860
--- PASS: TestGetHTTPS (0.00s)
=== RUN   TestRateLimitRetry
2020-06-16T20:53:14.949-0700	INFO	[httpjson]	httpjson/input.go:123	Initialized httpjson input.	{"url": "http://127.0.0.1:53862"}
2020-06-16T20:53:14.949-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:502	[DEBUG] GET http://127.0.0.1:53862
2020-06-16T20:53:14.949-0700	DEBUG	[httpjson]	httpjson/input.go:325	HTTP request failed	{"url": "http://127.0.0.1:53862", "http.response.status_code": 429, "http.response.body": ""}
2020-06-16T20:53:14.949-0700	DEBUG	[httpjson]	httpjson/input.go:262	Rate Limit: No need to apply rate limit.	{"url": "http://127.0.0.1:53862"}
2020-06-16T20:53:14.949-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:502	[DEBUG] GET http://127.0.0.1:53862
--- PASS: TestRateLimitRetry (0.00s)
=== RUN   TestErrorRetry
2020-06-16T20:53:14.950-0700	INFO	[httpjson]	httpjson/input.go:123	Initialized httpjson input.	{"url": "http://127.0.0.1:53865"}
2020-06-16T20:53:14.950-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:502	[DEBUG] GET http://127.0.0.1:53865
2020-06-16T20:53:14.950-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:600	[DEBUG] GET http://127.0.0.1:53865 (status: 506): retrying in 1s (5 left)
2020-06-16T20:53:15.951-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:600	[DEBUG] GET http://127.0.0.1:53865 (status: 508): retrying in 2s (4 left)
--- PASS: TestErrorRetry (3.01s)
=== RUN   TestArrayResponse
2020-06-16T20:53:17.955-0700	INFO	[httpjson]	httpjson/input.go:123	Initialized httpjson input.	{"url": "http://127.0.0.1:53900"}
2020-06-16T20:53:17.956-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:502	[DEBUG] GET http://127.0.0.1:53900
--- PASS: TestArrayResponse (0.00s)
=== RUN   TestPOST
2020-06-16T20:53:17.957-0700	INFO	[httpjson]	httpjson/input.go:123	Initialized httpjson input.	{"url": "http://127.0.0.1:53902"}
2020-06-16T20:53:17.957-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:502	[DEBUG] POST http://127.0.0.1:53902
--- PASS: TestPOST (0.00s)
=== RUN   TestRepeatedPOST
2020-06-16T20:53:17.958-0700	INFO	[httpjson]	httpjson/input.go:123	Initialized httpjson input.	{"url": "http://127.0.0.1:53904"}
2020-06-16T20:53:17.958-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:502	[DEBUG] POST http://127.0.0.1:53904
2020-06-16T20:53:20.963-0700	INFO	[httpjson]	httpjson/input.go:438	Process another repeated request.	{"url": "http://127.0.0.1:53904"}
2020-06-16T20:53:20.963-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:502	[DEBUG] POST http://127.0.0.1:53904
2020-06-16T20:53:23.962-0700	INFO	[httpjson]	httpjson/input.go:438	Process another repeated request.	{"url": "http://127.0.0.1:53904"}
2020-06-16T20:53:23.962-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:502	[DEBUG] POST http://127.0.0.1:53904
2020-06-16T20:53:23.963-0700	INFO	[httpjson]	httpjson/input.go:435	Context done.	{"url": "http://127.0.0.1:53904"}
--- PASS: TestRepeatedPOST (6.01s)
=== RUN   TestRunStop
2020-06-16T20:53:23.963-0700	INFO	[httpjson]	httpjson/input.go:123	Initialized httpjson input.	{"url": "http://127.0.0.1:53970"}
2020-06-16T20:53:23.963-0700	INFO	[httpjson]	httpjson/input.go:133	httpjson input worker has started.	{"url": "http://127.0.0.1:53970"}
2020-06-16T20:53:23.963-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:502	[DEBUG] GET http://127.0.0.1:53970
2020-06-16T20:53:23.963-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:551	[ERR] GET http://127.0.0.1:53970 request failed: Get http://127.0.0.1:53970: context canceled
2020-06-16T20:53:23.963-0700	ERROR	[httpjson]	httpjson/input.go:138	failed to execute http client.Do: Get http://127.0.0.1:53970: context canceled	{"url": "http://127.0.0.1:53970"}
2020-06-16T20:53:23.963-0700	INFO	[httpjson]	httpjson/input.go:139	httpjson input worker has stopped.	{"url": "http://127.0.0.1:53970"}
--- PASS: TestRunStop (0.00s)
=== RUN   TestOAuth2
2020-06-16T20:53:23.963-0700	INFO	[httpjson]	httpjson/input.go:123	Initialized httpjson input.	{"url": "http://127.0.0.1:53972"}
2020-06-16T20:53:23.964-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:502	[DEBUG] POST http://127.0.0.1:53971
2020-06-16T20:53:23.964-0700	DEBUG	[httpjson.retryablehttp]	go-retryablehttp@v0.6.6/client.go:502	[DEBUG] GET http://127.0.0.1:53972
--- PASS: TestOAuth2 (0.00s)
PASS
ok  	github.com/elastic/beats/v7/x-pack/filebeat/input/httpjson	9.214s
(base)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jul 1, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jul 1, 2020

Pinging @elastic/siem (Team:SIEM)

@marc-gr marc-gr requested a review from a team July 1, 2020 10:31
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jul 1, 2020 •
edited
Loading

❕ Build Aborted

Either there was a build timeout or someone aborted the build.'}

Pipeline View Test View Changes Artifacts

Expand to view the summary

Build stats

  • Build Cause: [Started by user Marc Guasch, Replayed #23]

  • Start Time: 2020-07-10T09:52:59.479+0000

  • Duration: 79 min 44 sec

Test stats 🧪

Test Results
Failed 1
Passed 9237
Skipped 1580
Total 10818

Test errors

Expand to view the tests failures

  • Name: Build and Test / Filebeat Mac OS X / test_rotating_file_inode – test_registrar.Test

    • Age: 1
    • Duration: 5.341
    • Error Details:

Steps errors

Expand to view the steps failures

  • Name: Mage build unitTest

    • Description: mage build unitTest

    • Duration: 3 min 0 sec

    • Start Time: 2020-07-10T10:21:48.736+0000

    • log

  • Name: Mage build unitTest

    • Description: mage build unitTest

    • Duration: 11 min 49 sec

    • Start Time: 2020-07-10T10:16:58.841+0000

    • log

  • Name: Recursively delete the current directory from the workspace

    • Description: script returned exit code 1

    • Duration: 0 min 14 sec

    • Start Time: 2020-07-10T10:29:35.408+0000

    • log

  • Name: Report to Codecov

    • Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done

    • Duration: 2 min 22 sec

    • Start Time: 2020-07-10T10:51:38.064+0000

    • log

  • Name: Make -C generator/_templates/metricbeat test

    • Description: make -C generator/_templates/metricbeat test

    • Duration: 7 min 52 sec

    • Start Time: 2020-07-10T11:00:04.481+0000

    • log

  • Name: Recursively delete the current directory from the workspace

    • Description: script returned exit code 2

    • Duration: 2 min 9 sec

    • Start Time: 2020-07-10T11:07:07.627+0000

    • log

Log output

Expand to view the last 100 lines of log output 

[2020-07-10T11:11:16.275Z] + FILE=packetbeat/build/coverage/full.cov
[2020-07-10T11:11:16.275Z] + [ -f packetbeat/build/coverage/full.cov ]
[2020-07-10T11:11:16.275Z] + FILE=winlogbeat/build/coverage/full.cov
[2020-07-10T11:11:16.275Z] + [ -f winlogbeat/build/coverage/full.cov ]
[2020-07-10T11:11:16.275Z] + FILE=journalbeat/build/coverage/full.cov
[2020-07-10T11:11:16.275Z] + [ -f journalbeat/build/coverage/full.cov ]
[2020-07-10T11:11:17.304Z] Failed in branch Filebeat oss
[2020-07-10T11:11:17.527Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats
[2020-07-10T11:11:17.835Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-07-10T11:11:17.849Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Lint
[2020-07-10T11:11:17.940Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Elastic-Agent-Mac-OS-X
[2020-07-10T11:11:18.020Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-07-10T11:11:18.102Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Auditbeat-oss-Mac-OS-X
[2020-07-10T11:11:18.176Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Filebeat-x-pack-Mac-OS-X
[2020-07-10T11:11:18.263Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Auditbeat-x-pack-Mac-OS-X
[2020-07-10T11:11:18.337Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Winlogbeat-oss
[2020-07-10T11:11:18.415Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-07-10T11:11:18.527Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Dockerlogbeat
[2020-07-10T11:11:18.601Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Journalbeat-oss
[2020-07-10T11:11:18.674Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-07-10T11:11:18.746Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-07-10T11:11:18.832Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-Mac-OS-X
[2020-07-10T11:11:18.904Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-x-pack-Mac-OS-X
[2020-07-10T11:11:18.979Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Elastic-Agent-x-pack-Windows
[2020-07-10T11:11:19.052Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-07-10T11:11:19.124Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Auditbeat-oss-Windows
[2020-07-10T11:11:19.198Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-07-10T11:11:19.270Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-07-10T11:11:19.345Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Heartbeat-oss
[2020-07-10T11:11:19.419Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-07-10T11:11:19.495Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Functionbeat-Mac-OS-X-x-pack
[2020-07-10T11:11:19.567Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Libbeat-x-pack
[2020-07-10T11:11:19.638Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Winlogbeat-Windows-x-pack
[2020-07-10T11:11:19.710Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Auditbeat-x-pack-Windows
[2020-07-10T11:11:19.789Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Filebeat-Windows
[2020-07-10T11:11:19.868Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-07-10T11:11:19.941Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Auditbeat-oss-Linux
[2020-07-10T11:11:20.015Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-07-10T11:11:20.085Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Packetbeat-oss
[2020-07-10T11:11:20.163Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Heartbeat-Mac-OS-X
[2020-07-10T11:11:20.234Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-07-10T11:11:20.315Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-Windows
[2020-07-10T11:11:20.404Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-07-10T11:11:20.484Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Functionbeat-Windows
[2020-07-10T11:11:20.561Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Heartbeat-Windows
[2020-07-10T11:11:20.634Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-Python-integration-tests
[2020-07-10T11:11:20.707Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Libbeat-oss
[2020-07-10T11:11:20.800Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-07-10T11:11:20.895Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Generators-Metricbeat-Mac-OS-X
[2020-07-10T11:11:20.967Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests
[2020-07-10T11:11:21.049Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Libbeat-crosscompile
[2020-07-10T11:11:21.129Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Libbeat-stress-tests
[2020-07-10T11:11:21.201Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Filebeat-x-pack
[2020-07-10T11:11:21.276Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Filebeat-oss
[2020-07-10T11:11:21.648Z] + cat
[2020-07-10T11:11:21.648Z] + /usr/local/bin/runbld ./runbld-script
[2020-07-10T11:11:21.648Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-07-10T11:11:28.245Z] runbld>>> runbld started
[2020-07-10T11:11:28.245Z] runbld>>> 1.6.12/f45d832f2ba0aa2722ab4ec1fda8ad140f027f8b
[2020-07-10T11:11:29.199Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-19538' in order of occurrence in the config (last value wins).
[2020-07-10T11:11:30.585Z] runbld>>> Debug logging enabled.
[2020-07-10T11:11:30.585Z] runbld>>> Storing result
[2020-07-10T11:11:30.585Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-07-10T11:11:30.585Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200710111130-DE3B8E8A
[2020-07-10T11:11:30.585Z] runbld>>> Adding system facts.
[2020-07-10T11:11:31.533Z] runbld>>> Adding vcs info for the latest commit:  ba1e6133c553fc16b416a88755f70aa2e732944b
[2020-07-10T11:11:31.533Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-07-10T11:11:31.533Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-07-10T11:11:31.533Z] Processing JUnit reports with runbld...
[2020-07-10T11:11:31.533Z] + echo 'Processing JUnit reports with runbld...'
[2020-07-10T11:11:32.105Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-07-10T11:11:32.106Z] runbld>>> DURATION: 22ms
[2020-07-10T11:11:32.106Z] runbld>>> STDOUT: 40 bytes
[2020-07-10T11:11:32.106Z] runbld>>> STDERR: 49 bytes
[2020-07-10T11:11:32.106Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-07-10T11:11:32.106Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats
[2020-07-10T11:11:33.061Z] runbld>>> Storing build metadata: 
[2020-07-10T11:11:33.061Z] runbld>>> Adding test report.
[2020-07-10T11:11:33.061Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats
[2020-07-10T11:11:33.633Z] runbld>>> Found 113 test output files
[2020-07-10T11:11:34.206Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-istio.xml
[2020-07-10T11:11:34.206Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-tomcat.xml
[2020-07-10T11:11:34.206Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-iis.xml
[2020-07-10T11:11:34.206Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-openmetrics.xml
[2020-07-10T11:11:34.206Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-activemq.xml
[2020-07-10T11:11:35.599Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-graphite.xml
[2020-07-10T11:11:35.883Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-windows.xml
[2020-07-10T11:11:36.143Z] runbld>>> Test output logs contained: Errors: 0 Failures: 1 Tests: 10672 Skipped: 1343
[2020-07-10T11:11:36.143Z] runbld>>> Storing result
[2020-07-10T11:11:36.143Z] runbld>>> FAILURES: 1
[2020-07-10T11:11:36.715Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-07-10T11:11:36.715Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200710111130-DE3B8E8A
[2020-07-10T11:11:36.715Z] runbld>>> Email notification disabled by environment variable.
[2020-07-10T11:11:36.715Z] runbld>>> Slack notification disabled by environment variable.
[2020-07-10T11:11:42.798Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19538
[2020-07-10T11:11:43.023Z] [INFO] getVaultSecret: Getting secrets
[2020-07-10T11:11:43.089Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-07-10T11:11:43.954Z] + chmod 755 generate-build-data.sh
[2020-07-10T11:11:43.955Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-19538/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-19538/runs/24 ABORTED 4724211
[2020-07-10T11:11:43.955Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-19538/runs/24/steps/?limit=10000 -o steps-info.json

@marc-gr marc-gr force-pushed the backport_18956_7.x branch 3 times, most recently from 5d9377e to 8c2c4d6 Compare July 2, 2020 06:43
@marc-gr marc-gr force-pushed the backport_18956_7.x branch from 8c2c4d6 to 5c548fa Compare July 6, 2020 10:02
@marc-gr marc-gr mentioned this pull request Jul 6, 2020
Merged
6 tasks
@marc-gr marc-gr force-pushed the backport_18956_7.x branch from 5c548fa to b5d51ec Compare July 7, 2020 07:25
@marc-gr
Copy link
Copy Markdown
Contributor Author

marc-gr commented Jul 7, 2020

jenkins run tests

@marc-gr marc-gr force-pushed the backport_18956_7.x branch 5 times, most recently from 256f70d to 163818d Compare July 9, 2020 06:43
@marc-gr
Copy link
Copy Markdown
Contributor Author

marc-gr commented Jul 9, 2020

jenkins run tests

@marc-gr marc-gr force-pushed the backport_18956_7.x branch from 163818d to ba1e613 Compare July 10, 2020 06:30
@marc-gr
Add automatic retries and exponential backoff to Filebeat httpjson in…
94a0bdf 
…put (elastic#18956)

- Add automatic retries and exponential backoff to Filebeat httpjson input (elastic#18956)
- Add test cases
- Support the new OAuth2 functionality

(cherry picked from commit 0dcb3df)
@marc-gr marc-gr force-pushed the backport_18956_7.x branch from ba1e613 to 94a0bdf Compare July 13, 2020 09:18
@marc-gr marc-gr merged commit ce7ccb2 into elastic:7.x Jul 13, 2020
@marc-gr marc-gr deleted the backport_18956_7.x branch July 13, 2020 09:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewkroh andrewkroh andrewkroh approved these changes

Assignees

No one assigned

Labels

backport [zube]: Done

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@marc-gr @elasticmachine @andrewkroh