[Elastic Agent] Fix install with bootstrapping Fleet Server by blakerouse · Pull Request #24981 · elastic/beats

blakerouse · 2021-04-08T00:14:52Z

What does this PR do?

This fixes the install command to allow Fleet Server to be bootstrapped properly. This also adds better logging to the enroll command (that is called by install) so its clear what might be causing the bootstrap of Fleet Server to fail.

This branch also uses a new feature from Fleet Server that removes the requirement of needing to pass the --enrollment-token to either install or enroll. Now Fleet Server will tell the Elastic Agent which enrollment token to use and Elastic Agent will use it to perform enrollment. This makes the installation of Elastic Agent with Fleet Server to just the following:

$ sudo ./elastic-agent install --fleet-server http://elastic:changeme@localhost:9200

This includes removing the requirement for the Docker container to even have to talk to Kibana when bootstrapping with Fleet Server. The following command would run the container and it would not even talk to Kibana at any point.

$ docker run -it -e ELASTICSEARCH_HOST=http://host.docker.internal:9200 -e FLEET_SERVER_ENABLE=1 docker.elastic.co/beats/elastic-agent:8.0.0-SNAPSHOT

This includes some fixes to the paths that was prevent Fleet Server from starting.

Why is it important?

So bootstrapping of Fleet Server works with the install command and to simplify the bootstrapping process by removing the requirement for --enrollment-token.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
~~[ ] I have made corresponding changes to the documentation~~
~~[ ] I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Logs

% sudo ./elastic-agent install --fleet-server http://elastic:changeme@localhost:9200
The Elastic Agent is currently in BETA and should not be used in production

Elastic Agent will be installed at /Library/Elastic/Agent and will run as a service. Do you want to continue? [Y/n]:Y
2021-04-07T19:48:27.703-0400    INFO    cmd/enroll_cmd.go:289   Generating self-signed certificate for Fleet Server
2021-04-07T19:48:29.005-0400    INFO    cmd/enroll_cmd.go:579   Waiting for Elastic Agent to start Fleet Server
2021-04-07T19:48:30.007-0400    INFO    cmd/enroll_cmd.go:619   Fleet Server - Starting
2021-04-07T19:48:33.020-0400    INFO    cmd/enroll_cmd.go:593   Fleet Server - Running on default policy with Fleet Server integration; missing config fleet.agent.id
2021-04-07T19:48:34.657-0400    INFO    cmd/enroll_cmd.go:193   Successfully triggered restart on running Elastic Agent.
Successfully enrolled the Elastic Agent.

… for the enrollment token when bootstrapping with Fleet Server.

elasticmachine · 2021-04-08T00:14:54Z

Pinging @elastic/agent (Team:Agent)

blakerouse · 2021-04-08T00:17:21Z

@ruflin @EricDavisX This removes the requirement of needing the --enrollment-token when bootstrapping with Fleet Server. All that is needed is:

sudo ./elastic-agent install --fleet-server http://elastic:changeme@localhost:9200

I am not marking this as breaking, because you can still provide one if you want. Its just no longer required to provide one.

elasticmachine · 2021-04-08T00:31:41Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #24981 updated
Start Time: 2021-04-08T16:35:31.732+0000
Duration: 58 min 48 sec
Commit: 4da6d8f

Test stats 🧪

Test	Results
Failed	0
Passed	6612
Skipped	16
Total	6628

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	6612
Skipped	16
Total	6628

michalpristas · 2021-04-08T10:35:12Z

x-pack/elastic-agent/pkg/agent/application/pipeline/emitter/modifiers/fleet_decorator.go

 		kbn, ok := transpiler.Lookup(ast, "fleet.kibana")
-		if !ok {
-			return fmt.Errorf("failed to get kibana config key from fleet config")
+		if ok {


we can do inline now to reduce scope

michalpristas · 2021-04-08T10:57:46Z

x-pack/elastic-agent/pkg/agent/cmd/install.go


 	if enroll {
-		if url == "" {
+		if url == "" && fleetServer == "" {


could we just move this to enroll && fleetServer == "" and remove also from the if above
or if the goal here is to skip enroll for fleet server maybe we can just extract this up to the line 140 into a Fn and execute only when fleetServer == ""
these ifs are getting complicated

I was able to simplify this by moving it up. The goal here is to ask for URL and Token in the case that it is not provided. The complexity comes from install being interactive.

blakerouse · 2021-04-08T12:13:28Z

@michalpristas Ready for another look

michalpristas

seems ok, tested with latest kibana

blakerouse · 2021-04-08T15:23:01Z

/test

mergify · 2021-04-08T15:31:53Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b fix-install-fleet-server upstream/fix-install-fleet-server
git merge upstream/master
git push upstream fix-install-fleet-server

* Fix install command to work with Fleet Server, remove the requirement for the enrollment token when bootstrapping with Fleet Server. * Fix tests. * Add changelog entry. * Simplify if statements. * Add installation success confirmation message. * Fix leaderelection provider. * Fix fleet decorator. (cherry picked from commit a91bba5)

…24998) * Fix install command to work with Fleet Server, remove the requirement for the enrollment token when bootstrapping with Fleet Server. * Fix tests. * Add changelog entry. * Simplify if statements. * Add installation success confirmation message. * Fix leaderelection provider. * Fix fleet decorator. (cherry picked from commit a91bba5) Co-authored-by: Blake Rouse <blake.rouse@elastic.co>

blakerouse added 2 commits April 7, 2021 19:55

Fix install command to work with Fleet Server, remove the requirement…

3b622b2

… for the enrollment token when bootstrapping with Fleet Server.

Fix tests.

66cfaf7

blakerouse added Team:Elastic-Agent Label for the Agent team backport-v7.13.0 Automated backport with mergify labels Apr 8, 2021

blakerouse self-assigned this Apr 8, 2021

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Apr 8, 2021

Add changelog entry.

5e60e76

michalpristas reviewed Apr 8, 2021

View reviewed changes

Simplify if statements.

ab07f86

blakerouse requested a review from michalpristas April 8, 2021 12:13

blakerouse added 3 commits April 8, 2021 08:21

Add installation success confirmation message.

9c2d011

Merge branch 'master' into fix-install-fleet-server

cef3f5a

Fix leaderelection provider.

f502472

michalpristas approved these changes Apr 8, 2021

View reviewed changes

blakerouse added 2 commits April 8, 2021 12:31

Merge branch 'master' into fix-install-fleet-server

cd3cbe1

Fix fleet decorator.

4da6d8f

blakerouse merged commit a91bba5 into elastic:master Apr 8, 2021

blakerouse deleted the fix-install-fleet-server branch April 8, 2021 18:44

mergify bot mentioned this pull request Apr 8, 2021

[Elastic Agent] Fix install with bootstrapping Fleet Server (bp #24981) #24998

Merged

Conversation

blakerouse commented Apr 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Checklist

Related issues

Logs

Uh oh!

elasticmachine commented Apr 8, 2021

Uh oh!

blakerouse commented Apr 8, 2021

Uh oh!

elasticmachine commented Apr 8, 2021 • edited by jenkins-beats-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

💚 Flaky test report

Test stats 🧪

Uh oh!

michalpristas Apr 8, 2021

Choose a reason for hiding this comment

Uh oh!

blakerouse Apr 8, 2021

Choose a reason for hiding this comment

Uh oh!

michalpristas Apr 8, 2021

Choose a reason for hiding this comment

Uh oh!

blakerouse Apr 8, 2021

Choose a reason for hiding this comment

Uh oh!

blakerouse commented Apr 8, 2021

Uh oh!

michalpristas left a comment

Choose a reason for hiding this comment

Uh oh!

blakerouse commented Apr 8, 2021

Uh oh!

mergify bot commented Apr 8, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

blakerouse commented Apr 8, 2021 •

edited

Loading

elasticmachine commented Apr 8, 2021 •

edited by jenkins-beats-ci bot

Loading