Allow elastic agent in containers to use basic auth to get service token by michel-laterman · Pull Request #29651 · elastic/beats

michel-laterman · 2021-12-30T20:31:46Z

What does this PR do?

Allow the agent to use basic auth defined by env vars to retrieve a
service token from Elasticsearch and inject it into the config used for
the agent and fleet.

Why is it important?

Allow the agent started in a container to use basic auth to contact Elasticsearch and generate a service_token.
This should fix the tests which rely on docker-compose.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Related issues

Closes Kibana Fleet setup failed: http POST request to .... fails: Unauthorized #29603

Use cases

Allow the agent to use basic auth defined by env vars to retrieve a service token from Elasticsearch and inject it into the config used for the agent and fleet.

elasticmachine · 2021-12-30T20:46:37Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-01-04T18:12:16.783+0000
Duration: 96 min 32 sec
Commit: 8ba0d57

Test stats 🧪

Test	Results
Failed	0
Passed	7120
Skipped	16
Total	7136

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

michel-laterman · 2021-12-31T00:56:51Z

Running elastic-package stack up with this agent leads to the fleet-server agent starting successfully. However the elastic-agent (without fleet-server) fails to find it's policy. The logs contain:

Created service_token named: token-1640912065673
Error: unable to find policy named ""

It looks like using a service_token fails to list policies:

elastic-agent|retrieve-service-token-container⚡ ⇒ curl -XPOST localhost:5601/api/fleet/service-tokens -u elastic:changeme -H "kbn-xsrf: value"
{"name":"token-1640912238005","value":"AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyL3Rva2VuLTE2NDA5MTIyMzgwMDU6ak53LTZHdkFTZnFrN0dTal9SM3BiUQ"}%
elastic-agent|retrieve-service-token-container⚡ ⇒ curl localhost:5601/api/fleet/agent_policies -H "Authorization: Bearer AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyL3Rva2VuLTE2NDA5MTIyMzgwMDU6ak53LTZHdkFTZnFrN0dTal9SM3BiUQ"
{"items":[],"total":0,"page":1,"perPage":20}%
elastic-agent|retrieve-service-token-container⚡ ⇒ curl localhost:5601/api/fleet/agent_policies -u elastic:changeme
{"items":[{"id":"499b5aa7-d214-5b5d-838b-3cd76469844e","namespace":"default","monitoring_enabled":["logs","metrics"],"name":"Default Fleet Server policy","description":"Default Fleet Server agent policy created by Kibana","is_default":false,"is_default_fleet_server":true,"is_preconfigured":true,"status":"active","is_managed":false,"revision":1,"updated_at":"2021-12-31T00:54:04.727Z","updated_by":"system","package_policies":["default-fleet-server-agent-policy"],"agents":1},{"id":"2016d7cc-135e-5583-9758-3ba01f5a06e5","namespace":"default","monitoring_enabled":["logs","metrics"],"name":"Default policy","description":"Default agent policy created by Kibana","is_default":true,"is_preconfigured":true,"status":"active","is_managed":false,"revision":1,"updated_at":"2021-12-31T00:54:02.703Z","updated_by":"system","package_policies":["default-system-policy"],"agents":0}],"total":2,"page":1,"perPage":20}%

@aleksmaus, I noticed you had to adjust the permissions the service token gets in Elasticsearch, do you think we're missing another one?

simitt · 2022-01-04T07:17:01Z

Related issue: elastic/fleet-server#1048 where priveleges that are defined on the package level are missing.

aleksmaus · 2022-01-04T13:17:47Z

It looks like using a service_token fails to list policies:

@michel-laterman the fleet service token has full access to the .fleet-policies. In your case you are querying the policies through kibana API, could be a possible defect in Kibana.

Chagne from using the ES api to gather the token to the Kibana API.

michel-laterman · 2022-01-04T18:17:34Z

I can use basic auth to generate a service_token via Kibana, and use that token to post to /api/fleet/setup and /api/fleet/agents/setup.
However the token will return 0 items with /api/fleet/agent_policies (no failure status or error messages). And I have confirmed that the token returns items when searching the .fleet-policies index in ES.
@joshdover, It looks like this is an issue with the fleet api in Kibana.

joshdover · 2022-01-05T10:43:50Z

@michel-laterman I think I see the problem and I believe it should be addressed by the work in elastic/kibana#121824. Could you confirm that it works against this branch?

If so, I think we'll probably want a more focused PR to fix this for 8.0, which I can prepare.

joshdover · 2022-01-05T10:48:50Z

Went ahead and threw up a focused fix, please test against this one: elastic/kibana#122320

elasticmachine · 2022-01-05T15:28:43Z

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

michel-laterman · 2022-01-05T19:35:24Z

Thanks @joshdover, I've confirmed that a service token can now list policies

blakerouse

Nicely done! Looks good.

…ken (#29651) * Allow elastic agent in containers to use basic auth to get service token Allow the agent to use basic auth defined by env vars to retrieve a service token from Elasticsearch and inject it into the config used for the agent and fleet. * Fix linter * Use Kibana API for all requests. Change from using the ES api to gather the token to the Kibana API. (cherry picked from commit 31c0099)

…ken (#29651) (#29713) * Allow elastic agent in containers to use basic auth to get service token Allow the agent to use basic auth defined by env vars to retrieve a service token from Elasticsearch and inject it into the config used for the agent and fleet. * Fix linter * Use Kibana API for all requests. Change from using the ES api to gather the token to the Kibana API. (cherry picked from commit 31c0099) Co-authored-by: Michel Laterman <82832767+michel-laterman@users.noreply.github.com>

Allow elastic agent in containers to use basic auth to get service token

6acb0bc

Allow the agent to use basic auth defined by env vars to retrieve a service token from Elasticsearch and inject it into the config used for the agent and fleet.

michel-laterman added enhancement backport-v8.0.0 Automated backport with mergify Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team labels Dec 30, 2021

michel-laterman requested a review from blakerouse December 30, 2021 20:31

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Dec 30, 2021

Fix linter

cb286d2

michel-laterman requested review from AndersonQ, lykkin and ruflin December 30, 2021 22:00

marclop mentioned this pull request Jan 4, 2022

compose: Use service tokens for fleet-server elastic/apm-server#6959

Closed

USe Kibana API for all requests.

8ba0d57

Chagne from using the ES api to gather the token to the Kibana API.

michel-laterman marked this pull request as ready for review January 5, 2022 15:28

blakerouse approved these changes Jan 5, 2022

View reviewed changes

michel-laterman merged commit 31c0099 into elastic:master Jan 5, 2022

michel-laterman deleted the retrieve-service-token-container branch January 5, 2022 21:08

mergify bot mentioned this pull request Jan 5, 2022

[8.0](backport #29651) Allow elastic agent in containers to use basic auth to get service token #29713

Merged

michel-laterman mentioned this pull request Jan 5, 2022

Remove username/password from elastic-agent elastic/observability-docs#1423

Merged

devcorpio mentioned this pull request Jan 10, 2022

chore: enable 8.1.0-SNAPSHOT elastic/synthetics#440

Merged

blakerouse mentioned this pull request Jan 11, 2022

[Elastic Agent] ensureServiceToken should not always be called for the container #29799

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow elastic agent in containers to use basic auth to get service token#29651

Allow elastic agent in containers to use basic auth to get service token#29651
michel-laterman merged 3 commits intoelastic:masterfrom
michel-laterman:retrieve-service-token-container

michel-laterman commented Dec 30, 2021 •

edited

Loading

Uh oh!

elasticmachine commented Dec 30, 2021 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

Uh oh!

michel-laterman commented Dec 31, 2021 •

edited

Loading

Uh oh!

simitt commented Jan 4, 2022

Uh oh!

aleksmaus commented Jan 4, 2022

Uh oh!

michel-laterman commented Jan 4, 2022

Uh oh!

joshdover commented Jan 5, 2022

Uh oh!

joshdover commented Jan 5, 2022

Uh oh!

elasticmachine commented Jan 5, 2022

Uh oh!

michel-laterman commented Jan 5, 2022

Uh oh!

blakerouse left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

michel-laterman commented Dec 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Checklist

How to test this PR locally

Related issues

Use cases

Uh oh!

elasticmachine commented Dec 30, 2021 • edited by jenkins-beats-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

Uh oh!

michel-laterman commented Dec 31, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

simitt commented Jan 4, 2022

Uh oh!

aleksmaus commented Jan 4, 2022

Uh oh!

michel-laterman commented Jan 4, 2022

Uh oh!

joshdover commented Jan 5, 2022

Uh oh!

joshdover commented Jan 5, 2022

Uh oh!

elasticmachine commented Jan 5, 2022

Uh oh!

michel-laterman commented Jan 5, 2022

Uh oh!

blakerouse left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

michel-laterman commented Dec 30, 2021 •

edited

Loading

elasticmachine commented Dec 30, 2021 •

edited by jenkins-beats-ci bot

Loading

michel-laterman commented Dec 31, 2021 •

edited

Loading