Skip to content

Cherry-pick #21290 to 7.x: Thread safe sorted set #21331

Merged
michalpristas merged 3 commits intoelastic:7.xfrom
michalpristas:backport_21290_7.x
Sep 30, 2020
Merged

Cherry-pick #21290 to 7.x: Thread safe sorted set #21331
michalpristas merged 3 commits intoelastic:7.xfrom
michalpristas:backport_21290_7.x

Conversation

@michalpristas
Copy link
Copy Markdown
Contributor

@michalpristas michalpristas commented Sep 25, 2020

Cherry-pick of PR #21290 to 7.x branch. Original message:

What does this PR do?

I ran into panic complaining about concurrent map writes so making this thread safe

Why is it important?

Less panics

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@michalpristas michalpristas self-assigned this Sep 25, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Sep 25, 2020

Pinging @elastic/ingest-management (Team:Ingest Management)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Sep 25, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Sep 25, 2020 •
edited by jenkins-beats-ci bot
Loading

💔 Build Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #21331 updated]

  • Start Time: 2020-09-29T09:11:47.769+0000

  • Duration: 97 min 55 sec

Test stats 🧪

Test Results
Failed 0
Passed 19826
Skipped 1825
Total 21651

Steps errors

Expand to view the steps failures

  • Name: Terraform Apply on x-pack/metricbeat/module/aws

    • Description:

    • Duration: 0 min 2 sec

    • Start Time: 2020-09-29T09:38:57.271+0000

    • log

  • Name: Terraform Apply on x-pack/metricbeat/module/aws

    • Description:

    • Duration: 0 min 2 sec

    • Start Time: 2020-09-29T09:39:05.579+0000

    • log

Log output

Expand to view the last 100 lines of log output 

[2020-09-29T10:47:47.000Z]  OS/Arch:      darwin/amd64
[2020-09-29T10:47:47.000Z]  Experimental: false
[2020-09-29T10:47:47.000Z]  Orchestrator: swarm
[2020-09-29T10:47:47.000Z] Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
[2020-09-29T10:47:47.000Z] It requires Docker daemon to be installed and running
[2020-09-29T10:48:16.102Z] Post stage
[2020-09-29T10:48:16.118Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats
[2020-09-29T10:48:16.134Z] Archiving artifacts
[2020-09-29T10:48:16.309Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats
[2020-09-29T10:48:16.323Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/uncategorized-1601372104566
[2020-09-29T10:48:16.376Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/libbeat-stress-tests-1601372350554
[2020-09-29T10:48:16.413Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/winlogbeat-crosscompile-1601372425947
[2020-09-29T10:48:16.443Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/auditbeat-crosscompile-1601372465411
[2020-09-29T10:48:16.472Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-elastic-agent-build-1601372476443
[2020-09-29T10:48:16.513Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-dockerlogbeat-build-1601372520864
[2020-09-29T10:48:16.551Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/journalbeat-unitTest-1601372534250
[2020-09-29T10:48:16.589Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/libbeat-crosscompile-1601372621184
[2020-09-29T10:48:16.626Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-functionbeat-build-1601372635224
[2020-09-29T10:48:16.660Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/packetbeat-build-1601372715674
[2020-09-29T10:48:16.706Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-elastic-agent-windows-windows-2019-1601372769265
[2020-09-29T10:48:16.743Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/metricbeat-unitTest-1601372881374
[2020-09-29T10:48:16.790Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/auditbeat-windows-windows-2019-1601372925974
[2020-09-29T10:48:16.827Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/heartbeat-build-1601372928072
[2020-09-29T10:48:16.866Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/heartbeat-windows-windows-2019-1601372963976
[2020-09-29T10:48:16.903Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/metricbeat-crosscompile-1601372996096
[2020-09-29T10:48:16.938Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/generator-metricbeat-test-1601372996633
[2020-09-29T10:48:16.976Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-auditbeat-windows-windows-2019-1601373001449
[2020-09-29T10:48:17.013Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-auditbeat-build-1601373003999
[2020-09-29T10:48:17.062Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-winlogbeat-build-windows-2019-1601373047594
[2020-09-29T10:48:17.101Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/auditbeat-build-1601373048795
[2020-09-29T10:48:17.136Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/generator-beat-test-1601373100657
[2020-09-29T10:48:17.167Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-filebeat-windows-windows-2019-1601373107485
[2020-09-29T10:48:17.200Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/filebeat-windows-windows-2019-1601373166842
[2020-09-29T10:48:17.235Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-libbeat-build-1601373179085
[2020-09-29T10:48:17.270Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/packetbeat-windows-windows-2019-1601373196789
[2020-09-29T10:48:17.304Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/winlogbeat-windows-windows-2019-1601373205224
[2020-09-29T10:48:17.347Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-functionbeat-windows-windows-2019-1601373237601
[2020-09-29T10:48:17.387Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/metricbeat-windows-windows-2019-1601373325938
[2020-09-29T10:48:17.433Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-metricbeat-windows-windows-2019-1601373412812
[2020-09-29T10:48:17.467Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/filebeat-build-1601374043876
[2020-09-29T10:48:17.504Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/metricbeat-pythonIntegTest-1601374334451
[2020-09-29T10:48:17.538Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/metricbeat-goIntegTest-1601374421887
[2020-09-29T10:48:17.583Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/libbeat-build-1601374484278
[2020-09-29T10:48:17.629Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-filebeat-build-1601374485904
[2020-09-29T10:48:17.667Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/generator-macos-metricbeat-macosx-1601375452093
[2020-09-29T10:48:17.697Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/filebeat-macos-macosx-1601375601706
[2020-09-29T10:48:17.731Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/heartbeat-macos-macosx-1601375647069
[2020-09-29T10:48:17.770Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-elastic-agent-macos-macosx-1601375775650
[2020-09-29T10:48:17.804Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/metricbeat-macos-macosx-1601375878018
[2020-09-29T10:48:17.839Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/packetbeat-macos-macosx-1601375917427
[2020-09-29T10:48:17.873Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-filebeat-macos-macosx-1601375959269
[2020-09-29T10:48:17.910Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/generator-macos-beat-macosx-1601375964024
[2020-09-29T10:48:17.946Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-auditbeat-macos-macosx-1601375992187
[2020-09-29T10:48:17.984Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-functionbeat-macos-macosx-1601376062559
[2020-09-29T10:48:18.022Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/x-pack-metricbeat-macos-macosx-1601376278650
[2020-09-29T10:48:18.057Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/auditbeat-macos-macosx-1601376461210
[2020-09-29T10:48:18.393Z] + cat
[2020-09-29T10:48:18.393Z] + /usr/local/bin/runbld ./runbld-test-reports --job-name elastic+beats+pull-request
[2020-09-29T10:48:18.393Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-09-29T10:48:24.984Z] runbld>>> runbld started
[2020-09-29T10:48:24.984Z] runbld>>> 1.6.12/f45d832f2ba0aa2722ab4ec1fda8ad140f027f8b
[2020-09-29T10:48:26.385Z] runbld>>> The following profiles matched the job 'elastic+beats+pull-request' in order of occurrence in the config (last value wins).
[2020-09-29T10:48:26.385Z] runbld>>> Matches in the system config:
[2020-09-29T10:48:26.385Z] runbld>>> - Matched ^elastic\+beats
[2020-09-29T10:48:26.385Z] runbld>>> - Matched ^elastic\+beats\+pull-request
[2020-09-29T10:48:27.773Z] runbld>>> Debug logging enabled.
[2020-09-29T10:48:27.773Z] runbld>>> Storing result
[2020-09-29T10:48:27.773Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-09-29T10:48:27.773Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1597739501209/t/20200929104827-8760E2CA
[2020-09-29T10:48:27.773Z] runbld>>> Adding system facts.
[2020-09-29T10:48:28.728Z] runbld>>> Adding vcs info for the latest commit:  f7d2d9915747f1835dc49e234ea6eeb06b4fab5b
[2020-09-29T10:48:28.728Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-09-29T10:48:28.728Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-09-29T10:48:28.728Z] Processing JUnit reports with runbld...
[2020-09-29T10:48:28.728Z] + echo 'Processing JUnit reports with runbld...'
[2020-09-29T10:48:29.300Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-09-29T10:48:29.300Z] runbld>>> DURATION: 22ms
[2020-09-29T10:48:29.300Z] runbld>>> STDOUT: 40 bytes
[2020-09-29T10:48:29.300Z] runbld>>> STDERR: 49 bytes
[2020-09-29T10:48:29.300Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-09-29T10:48:29.300Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats_PR-21331
[2020-09-29T10:48:31.852Z] runbld>>> Storing build metadata: 
[2020-09-29T10:48:31.852Z] runbld>>> Adding test report.
[2020-09-29T10:48:31.852Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats
[2020-09-29T10:48:32.795Z] runbld>>> Found 119 test output files
[2020-09-29T10:48:33.057Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/metricbeat-goIntegTest-1601374421887/metricbeat/build/TEST-go-integration-graphite.xml
[2020-09-29T10:48:33.057Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-21331/src/github.com/elastic/beats/metricbeat-goIntegTest-1601374421887/metricbeat/build/TEST-go-integration-windows.xml
[2020-09-29T10:48:36.363Z] runbld>>> Test output logs contained: Errors: 0 Failures: 0 Tests: 21651 Skipped: 1557
[2020-09-29T10:48:36.363Z] runbld>>> Storing result
[2020-09-29T10:48:36.363Z] runbld>>> FAILURES: 0
[2020-09-29T10:48:36.363Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-09-29T10:48:36.363Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1597739501209/t/20200929104827-8760E2CA
[2020-09-29T10:48:36.363Z] runbld>>> Email notification disabled by environment variable.
[2020-09-29T10:48:36.363Z] runbld>>> Slack notification disabled by environment variable.
[2020-09-29T10:48:41.825Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-21331
[2020-09-29T10:48:41.897Z] [INFO] getVaultSecret: Getting secrets
[2020-09-29T10:48:41.974Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-09-29T10:48:42.548Z] + chmod 755 generate-build-data.sh
[2020-09-29T10:48:42.548Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-21331/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-21331/runs/2 FAILURE 5814520
[2020-09-29T10:48:42.548Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-21331/runs/2/steps/?limit=10000 -o steps-info.json

@michalpristas michalpristas merged commit 80dc2bf into elastic:7.x Sep 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@kvch kvch kvch approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@michalpristas michalpristas

Labels

backport bug Ingest Management:beta2 Group issues for ingest management beta2

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@michalpristas @elasticmachine @kvch