Skip to content

Cherry-pick #20146 to 7.x: [Elastic Agent] Add basic protocol to control Elastic Agent.#20278

Merged
blakerouse merged 1 commit intoelastic:7.xfrom
blakerouse:backport_20146_7.x
Aug 3, 2020
Merged

Cherry-pick #20146 to 7.x: [Elastic Agent] Add basic protocol to control Elastic Agent.#20278
blakerouse merged 1 commit intoelastic:7.xfrom
blakerouse:backport_20146_7.x

Conversation

@blakerouse
Copy link
Copy Markdown
Contributor

@blakerouse blakerouse commented Jul 28, 2020

Cherry-pick of PR #20146 to 7.x branch. Original message:

What does this PR do?

Adds the protocol that Elastic Agent will expose between the CLI and daemon to control the running daemon.

Why is it important?

To provide a better UX, easy testing of re-exec and self-upgrading.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • [ ] I have added tests that prove my fix is effective or that my feature works
  • [ ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

@blakerouse
[Elastic Agent] Add basic protocol to control Elastic Agent. (elastic…
36d917f 
…#20146)

* Add protocl to control Elastic Agent.

* Add more to version response.

* Fix CI with protoc.

* Improve protocol and try and fix CI.

* Fix proto.

* Remove CI changes.

* Update go.mod.

* Add status ROLLBACK.

* Run mage fmt.

(cherry picked from commit e9f8f17)
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jul 28, 2020

Pinging @elastic/ingest-management (Team:Ingest Management)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jul 28, 2020
@blakerouse blakerouse self-assigned this Jul 28, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jul 28, 2020

💔 Tests Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #20278 opened]

  • Start Time: 2020-07-28T14:13:50.029+0000

  • Duration: 121 min 11 sec

Test stats 🧪

Test Results
Failed 13
Passed 9737
Skipped 1583
Total 11333

Test errors

Expand to view the tests failures

  • Name: Build and Test / Auditbeat x-pack / test_connected_udp_ipv4 – test_system_socket.Test

    • Age: 1
    • Duration: 60.116
    • Error Details: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Auditbeat x-pack / test_connected_udp_ipv6 – test_system_socket.Test

    • Age: 1
    • Duration: 60.119
    • Error Details: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Auditbeat x-pack / test_dns_enrichment – test_system_socket.Test

    • Age: 1
    • Duration: 60.117
    • Error Details: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Auditbeat x-pack / test_dns_long_request – test_system_socket.Test

    • Age: 1
    • Duration: 60.121
    • Error Details: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Auditbeat x-pack / test_dns_udp_ipv6 – test_system_socket.Test

    • Age: 1
    • Duration: 60.114
    • Error Details: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Auditbeat x-pack / test_dns_unidirectional_udp – test_system_socket.Test

    • Age: 1
    • Duration: 60.125
    • Error Details: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Auditbeat x-pack / test_multi_udp_upv4 – test_system_socket.Test

    • Age: 1
    • Duration: 60.125
    • Error Details: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Auditbeat x-pack / test_no_dns_enrichment – test_system_socket.Test

    • Age: 1
    • Duration: 60.125
    • Error Details: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Auditbeat x-pack / test_tcp_ipv4 – test_system_socket.Test

    • Age: 1
    • Duration: 60.123
    • Error Details: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Auditbeat x-pack / test_tcp_ipv6_disabled – test_system_socket.Test

    • Age: 1
    • Duration: 60.028
    • Error Details: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Auditbeat x-pack / test_udp_ipv4 – test_system_socket.Test

    • Age: 1
    • Duration: 60.027
    • Error Details: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Auditbeat x-pack / test_udp_ipv6 – test_system_socket.Test

    • Age: 1
    • Duration: 60.12
    • Error Details: Expected exit code to be 0, but it was 1

  • Name: Build and Test / Auditbeat x-pack / test_udp_ipv6_disabled – test_system_socket.Test

    • Age: 1
    • Duration: 60.024
    • Error Details: Expected exit code to be 0, but it was 1

Steps errors

Expand to view the steps failures

  • Name: Report to Codecov

    • Description: .ci/scripts/report-codecov.sh auditbeat filebeat heartbeat journalbeat libbeat metricbeat packetbeat winlogbeat

    • Duration: 1 min 26 sec

    • Start Time: 2020-07-28T15:44:25.766+0000

    • log

  • Name: Mage update build test

    • Description: mage update build test

    • Duration: 23 min 38 sec

    • Start Time: 2020-07-28T14:38:44.384+0000

    • log

  • Name: Make testsuite

    • Description: make -C libbeat testsuite

    • Duration: 6 min 43 sec

    • Start Time: 2020-07-28T14:38:39.841+0000

    • log

  • Name: Make test

    • Description: make -C generator/_templates/beat test

    • Duration: 7 min 29 sec

    • Start Time: 2020-07-28T16:03:54.956+0000

    • log

  • Name: Report to Codecov

    • Description: .ci/scripts/report-codecov.sh auditbeat filebeat heartbeat journalbeat libbeat metricbeat packetbeat winlogbeat

    • Duration: 1 min 27 sec

    • Start Time: 2020-07-28T16:11:33.211+0000

    • log

  • Name: Recursively delete the current directory from the workspace

    • Description: script returned exit code 2

    • Duration: 1 min 20 sec

    • Start Time: 2020-07-28T16:13:04.842+0000

    • log

Log output

Expand to view the last 100 lines of log output 

[2020-07-28T16:13:04.594Z]  Built:        Thu Apr 26 07:13:02 2018
[2020-07-28T16:13:04.594Z]  OS/Arch:      darwin/amd64
[2020-07-28T16:13:04.594Z]  Experimental: false
[2020-07-28T16:13:04.594Z]  Orchestrator: swarm
[2020-07-28T16:13:04.594Z] Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
[2020-07-28T16:13:04.594Z] It requires Docker daemon to be installed and running
[2020-07-28T16:14:25.098Z] Failed in branch Generators
[2020-07-28T16:14:26.837Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats
[2020-07-28T16:14:27.170Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-07-28T16:14:27.193Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Lint
[2020-07-28T16:14:27.344Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Winlogbeat-oss
[2020-07-28T16:14:27.493Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-07-28T16:14:27.653Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-07-28T16:14:27.804Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Dockerlogbeat
[2020-07-28T16:14:27.954Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Journalbeat-oss
[2020-07-28T16:14:28.104Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-07-28T16:14:28.253Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-07-28T16:14:28.403Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Libbeat-oss
[2020-07-28T16:14:28.556Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Elastic-Agent-x-pack-Windows
[2020-07-28T16:14:28.710Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-07-28T16:14:28.875Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-07-28T16:14:29.027Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Auditbeat-oss-Windows
[2020-07-28T16:14:29.173Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Heartbeat-oss
[2020-07-28T16:14:29.320Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Auditbeat-x-pack-Windows
[2020-07-28T16:14:29.469Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Winlogbeat-Windows-x-pack
[2020-07-28T16:14:29.618Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Libbeat-x-pack
[2020-07-28T16:14:29.766Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-07-28T16:14:29.915Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Auditbeat-oss-Linux
[2020-07-28T16:14:30.064Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Filebeat-Windows
[2020-07-28T16:14:30.218Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Packetbeat-oss
[2020-07-28T16:14:30.375Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-Windows
[2020-07-28T16:14:30.525Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-07-28T16:14:30.682Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-07-28T16:14:30.831Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-07-28T16:14:30.979Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-07-28T16:14:31.127Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Filebeat-oss
[2020-07-28T16:14:31.280Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Filebeat-x-pack
[2020-07-28T16:14:31.432Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-Python-integration-tests
[2020-07-28T16:14:31.580Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests
[2020-07-28T16:14:31.727Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Elastic-Agent-Mac-OS-X
[2020-07-28T16:14:31.881Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Filebeat-x-pack-Mac-OS-X
[2020-07-28T16:14:32.035Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Auditbeat-oss-Mac-OS-X
[2020-07-28T16:14:32.185Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-07-28T16:14:32.337Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-Mac-OS-X
[2020-07-28T16:14:32.606Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-07-28T16:14:32.929Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-x-pack-Mac-OS-X
[2020-07-28T16:14:33.232Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Heartbeat-Mac-OS-X
[2020-07-28T16:14:33.575Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Auditbeat-x-pack-Mac-OS-X
[2020-07-28T16:14:33.886Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Generators-Metricbeat-Mac-OS-X
[2020-07-28T16:14:34.176Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Functionbeat-Mac-OS-X-x-pack
[2020-07-28T16:14:34.612Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Heartbeat-Windows
[2020-07-28T16:14:34.919Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Functionbeat-Windows
[2020-07-28T16:14:35.269Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Generators-Beat-Mac-OS-X
[2020-07-28T16:14:35.869Z] + cat
[2020-07-28T16:14:35.869Z] + /usr/local/bin/runbld ./runbld-script
[2020-07-28T16:14:35.869Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-07-28T16:14:42.462Z] runbld>>> runbld started
[2020-07-28T16:14:42.462Z] runbld>>> 1.6.12/f45d832f2ba0aa2722ab4ec1fda8ad140f027f8b
[2020-07-28T16:14:43.846Z] runbld>>> The following profiles matched the job 'Beats/beats/PR-20278' in order of occurrence in the config (last value wins).
[2020-07-28T16:14:45.234Z] runbld>>> Debug logging enabled.
[2020-07-28T16:14:45.234Z] runbld>>> Storing result
[2020-07-28T16:14:45.495Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-07-28T16:14:45.495Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200728161445-97E7D4A9
[2020-07-28T16:14:45.495Z] runbld>>> Adding system facts.
[2020-07-28T16:14:46.438Z] runbld>>> Adding vcs info for the latest commit:  36d917f09462973417a2b4aea2e70b787e6efb56
[2020-07-28T16:14:46.438Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-07-28T16:14:46.438Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-07-28T16:14:46.701Z] Processing JUnit reports with runbld...
[2020-07-28T16:14:46.701Z] + echo 'Processing JUnit reports with runbld...'
[2020-07-28T16:14:46.963Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-07-28T16:14:46.963Z] runbld>>> DURATION: 38ms
[2020-07-28T16:14:46.963Z] runbld>>> STDOUT: 40 bytes
[2020-07-28T16:14:46.963Z] runbld>>> STDERR: 49 bytes
[2020-07-28T16:14:46.963Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-07-28T16:14:46.963Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats
[2020-07-28T16:14:47.915Z] runbld>>> Storing build metadata: 
[2020-07-28T16:14:47.915Z] runbld>>> Adding test report.
[2020-07-28T16:14:47.915Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats
[2020-07-28T16:14:48.859Z] runbld>>> Found 114 test output files
[2020-07-28T16:14:49.430Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-openmetrics.xml
[2020-07-28T16:14:49.430Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-iis.xml
[2020-07-28T16:14:49.430Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-istio.xml
[2020-07-28T16:14:49.430Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-tomcat.xml
[2020-07-28T16:14:49.430Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-activemq.xml
[2020-07-28T16:14:49.691Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-graphite.xml
[2020-07-28T16:14:49.692Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20278/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-windows.xml
[2020-07-28T16:14:51.077Z] runbld>>> Test output logs contained: Errors: 0 Failures: 13 Tests: 11187 Skipped: 1346
[2020-07-28T16:14:51.077Z] runbld>>> Storing result
[2020-07-28T16:14:51.077Z] runbld>>> FAILURES: 13
[2020-07-28T16:14:53.620Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-07-28T16:14:53.620Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200728161445-97E7D4A9
[2020-07-28T16:14:53.620Z] runbld>>> Email notification disabled by environment variable.
[2020-07-28T16:14:53.620Z] runbld>>> Slack notification disabled by environment variable.
[2020-07-28T16:14:59.529Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-20278
[2020-07-28T16:14:59.809Z] [INFO] getVaultSecret: Getting secrets
[2020-07-28T16:14:59.912Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-07-28T16:15:00.984Z] + chmod 755 generate-build-data.sh
[2020-07-28T16:15:00.984Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20278/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20278/runs/1 FAILURE 7270685
[2020-07-28T16:15:00.984Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20278/runs/1/steps/?limit=10000 -o steps-info.json
[2020-07-28T16:15:04.212Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20278/runs/1/tests/?status=FAILED -o tests-errors.json

michalpristas
michalpristas approved these changes Aug 3, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@michalpristas michalpristas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backport looks ok

@blakerouse blakerouse merged commit 0ac6989 into elastic:7.x Aug 3, 2020
@blakerouse blakerouse deleted the backport_20146_7.x branch August 3, 2020 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@michalpristas michalpristas michalpristas approved these changes

Assignees

@blakerouse blakerouse

Labels

backport [zube]: In Review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@blakerouse @elasticmachine @michalpristas