[Fleet] Use fleet server indices for enrollment keys and to list agents with a feature flag by nchaulet · Pull Request #86179 · elastic/kibana

nchaulet · 2020-12-16T19:58:35Z

Summary

Introduce a feature flag to start using fleet server indice xpack.fleet.agents.fleetServerEnabled: true

If this flag is enabled we will use a different service that read and write for .fleet-enrollment-key instead of saved object.

This is a draft PR to explore what a feature flag for the development of Kibana with Fleet Server will look like

How to test that PR?

With the feature flag off

if the feature flag is off xpack.fleet.agents.fleetServerEnabled: false you should be able to create a policy, enroll agents, unenroll, all the actions as before, (this PR should not have any impact)

With the feature flag on

with the feature flag on xpack.fleet.agents.fleetServerEnabled: true you should be able to enroll agent, and rollup a policy via fleet-server, (turning the flag off again is not supported)

You should respect the following order (for now):

Start fleet-server
Create a custom kibana system user with the right permissions (see below)
start kibana with flag and the custom kibana user
Setup fleet and agent in Kibana set the kibana url to the fleet server url (by default http://localhost:8000)
enroll an agent to fleet server, you can trigger a policy change the other actions are not yet working.

Migration

During the plugin start we will transform all of the existing saved object to .fleet-* document, I discussed this kibana platform and it should work for us.

Indices creation

In this PR we rely on fleet server creating the indice, (in the future the indice should be created by an ES plugin)

So the first step is to run fleet server.

Kibana system user

The kibana_system will need to have access to .fleet* indices.

In this PR I used a custom user as the kibana system user created like

POST /_security/role/kibana_fleet_system
{
   "cluster" : [
      "all"
    ],
    "indices" : [
      {
        "names" : [
          ".fleet*"
        ],
        "privileges" : [
          "all"
        ]
      }
    ]
}



POST /_security/user/kibana_fleet_system
{
  "password" : "changeme",
  "roles" : [ "kibana_system", "kibana_fleet_system" ]
}

then in kibana.dev.yml

elasticsearch.username: 'kibana_fleet_system'
elasticsearch.password: 'changeme'

…ibana-fleet-server-feature-flag

…ibana-fleet-server-feature-flag-2

…ibana-fleet-server-feature-flag

elasticmachine · 2021-01-06T14:11:41Z

Pinging @elastic/ingest-management (Team:Ingest Management)

jen-huang

Didn't review test files or security_solution files. Left questions and comments mostly regarding the ES agents CRUD service.

x-pack/plugins/fleet/server/constants/index.ts

x-pack/plugins/fleet/server/routes/agent/unenroll_handler.ts

x-pack/plugins/fleet/server/services/agents/crud_fleet_server.ts

x-pack/plugins/fleet/server/services/agents/crud.ts

x-pack/plugins/fleet/server/services/api_keys/enrollment_api_key.ts

x-pack/plugins/fleet/server/types/index.tsx

nchaulet · 2021-01-13T23:09:44Z

Yes it's normal you cannot enroll an agent to kibana for now, you need to enroll the agent to fleet server. I will add the support for the new indices and Kibana enroll in a future PR the whole migration plan is here #87372

jen-huang · 2021-01-14T18:10:20Z

@nchaulet How can I enroll into Fleet server? Can you update the PR description with more testing steps around what is expected behavior?

…ibana-fleet-server-feature-flag

nchaulet · 2021-01-19T15:16:10Z

@jen-huang thanks for taking some time on that PR, and sorry for the missing expectations, I udpated the PR description and created a small video on what to expect when testing with the flag set to on.

What to test on that PR:

nothing break when the flag is off
you can enroll and agent via Fleet server when the flag is on.

fleet-server.mp4

jen-huang · 2021-01-20T04:49:09Z

Thanks for the updated instructions and video. I ran through the Fleet server enabled scenario again today and am seeing all of the behavior described. My agent has logs and I see data streams come in too. The only issue I see is that the status bar doesn't reflect the status of the agents, but I think that might be due to the current limitation of "can trigger a policy change the other actions are not yet working"?

Tomorrow I will run another test with Fleet server disabled and give the code another look too.

…ibana-fleet-server-feature-flag

nchaulet · 2021-01-20T13:07:32Z

@jen-huang yes the agent status with Fleet server are not yet working if it's okay with you I will fix it in a following PR as there is already a lot of change here

nchaulet · 2021-01-20T17:02:31Z

@kevinlog I would love a review from endpoint here, mostly checking if the flag set to false it's not breaking anything

jen-huang · 2021-01-20T18:02:12Z

@nchaulet Yeah no worries, just wanted to confirm that it's a known limitation right now.

kevinlog · 2021-01-20T18:21:00Z

@nchaulet

I checked it out and tested with some basic Endpoint flows with xpack.fleet.agents.fleetServerEnabled: false, Endpoints deploy correctly and I'm still able to change policies, etc.

From that perspective, the change looks good from our end - I'll leave the review of the actual code to Fleet devs.

jen-huang

Tested locally with and without Fleet server enabled. With Fleet server, ran into known limitations but overall things work as described. Without Fleet server, things worked normally and I confirmed that .fleet indices were not created.

Left some code comments but not blocking. Thanks for your patience during my reviews of this and good work on getting this first major piece completed!

jen-huang · 2021-01-20T19:03:43Z

x-pack/plugins/fleet/server/services/fleet_server_migration.ts

+  return appContextService.getInternalUserSOClient(fakeRequest);
+}
+
+async function migrateEnrollmentApiKeys() {


not a blocker but would be good to have tests for this

would agent documents need to be migrated too?

Yes agents would need to be migrated too, I am probably going to address that and add tests for that in a following PR as this PR is already complex

jen-huang · 2021-01-20T19:05:36Z

x-pack/plugins/fleet/server/plugin.ts


+    const fleetServerEnabled = appContextService.getConfig()?.agents?.fleetServerEnabled;
+    if (fleetServerEnabled) {
+      await this.licensing$.pipe(first()).toPromise();


why do we wait for licensing before starting migration?

we need licensing to be available so the SO client is correctly initialized. it probably need a comment here

nchaulet · 2021-01-20T22:19:33Z

@elasticmachine merge upstream

kibanamachine · 2021-01-21T00:06:01Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 533e506

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`fleet`	377.0KB	377.9KB	+860.0B

History

💔 Build #100158 failed 18293e9
💚 Build #100102 succeeded 74e89bc
💚 Build #99597 succeeded cbafae0
💚 Build #99466 succeeded b888fe2
💔 Build #99398 failed 60eab63

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…ts with a feature flag (elastic#86179)

…ts with a feature flag (#86179) (#88924)

nchaulet added the Team:Fleet Team label for Observability Data Collection Fleet team label Dec 16, 2020

[Fleet] Use fleet server indices for enrollment keys

4a8791b

nchaulet force-pushed the feature-poc-kibana-fleet-server-feature-flag branch from 75b2a6a to 4a8791b Compare December 16, 2020 21:12

nchaulet self-assigned this Dec 17, 2020

patrykkopycinski added 12 commits December 22, 2020 13:12

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

e704999

…ibana-fleet-server-feature-flag

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

272926f

…ibana-fleet-server-feature-flag

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

4b93480

…ibana-fleet-server-feature-flag

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

87c3452

…ibana-fleet-server-feature-flag

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

cae8106

…ibana-fleet-server-feature-flag

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

f738b37

…ibana-fleet-server-feature-flag

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

47be337

…ibana-fleet-server-feature-flag

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

d75033e

…ibana-fleet-server-feature-flag

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

5f04a27

…ibana-fleet-server-feature-flag-2

[Fleet Server] Add support for index based agents

175a379

add missing files

0989147

fix types

1fa3c49

nchaulet mentioned this pull request Jan 5, 2021

[Fleet] Migrate Agents from saved object to fleet server indices #87372

Closed

16 tasks

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

4f34ef5

…ibana-fleet-server-feature-flag

nchaulet marked this pull request as ready for review January 6, 2021 14:11

nchaulet requested a review from a team as a code owner January 6, 2021 14:11

nchaulet requested a review from a team January 6, 2021 14:11

nchaulet requested a review from a team as a code owner January 6, 2021 14:11

patrykkopycinski added v7.12.0 v8.0.0 release_note:enhancement labels Jan 6, 2021

nchaulet added the release_note:skip Skip the PR/issue when compiling release notes label Jan 6, 2021

patrykkopycinski removed the release_note:enhancement label Jan 6, 2021

nchaulet mentioned this pull request Jan 6, 2021

[Fleet] Document how to use with Kibana elastic/fleet-server#58

Merged

jen-huang reviewed Jan 6, 2021

View reviewed changes

nchaulet added 6 commits January 18, 2021 07:59

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

670a8f4

…ibana-fleet-server-feature-flag

add support for agent policy in fleet server

60eab63

Fix type issue

4647e84

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

b888fe2

…ibana-fleet-server-feature-flag

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

fe8902f

…ibana-fleet-server-feature-flag

Add forgeted file for migrations

cbafae0

Merge branch 'master' of github.com:elastic/kibana into feature-poc-k…

74e89bc

…ibana-fleet-server-feature-flag

jen-huang approved these changes Jan 20, 2021

View reviewed changes

nchaulet requested a review from paul-tavares January 20, 2021 19:20

kevinlog approved these changes Jan 20, 2021

View reviewed changes

nchaulet removed the request for review from paul-tavares January 20, 2021 19:44

fix after review

18293e9

Merge branch 'master' into feature-poc-kibana-fleet-server-feature-flag

533e506

nchaulet merged commit 3b728b7 into elastic:master Jan 21, 2021

nchaulet deleted the feature-poc-kibana-fleet-server-feature-flag branch January 21, 2021 00:29

nchaulet mentioned this pull request Jan 21, 2021

[7.x] [Fleet] Use fleet server indices for enrollment keys and to list agents with a feature flag (#86179) #88924

Merged

nchaulet added a commit to nchaulet/kibana that referenced this pull request Jan 21, 2021

[Fleet] Use fleet server indices for enrollment keys and to list agen…

b7ee47a

…ts with a feature flag (elastic#86179)

nchaulet added a commit that referenced this pull request Jan 21, 2021

[Fleet] Use fleet server indices for enrollment keys and to list agen…

b35509c

…ts with a feature flag (#86179) (#88924)

nchaulet mentioned this pull request Jan 28, 2021

[Fleet] Support Fleet server system indices #89372

Merged

jen-huang mentioned this pull request Mar 8, 2021

[Fleet] Return empty agents list when submitting a kuery with no keys #93844

Merged

Conversation

nchaulet commented Dec 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How to test that PR?

With the feature flag off

With the feature flag on

Migration

Indices creation

Kibana system user

Uh oh!

elasticmachine commented Jan 6, 2021

Uh oh!

jen-huang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nchaulet commented Jan 13, 2021

Uh oh!

jen-huang commented Jan 14, 2021

Uh oh!

nchaulet commented Jan 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jen-huang commented Jan 20, 2021

Uh oh!

nchaulet commented Jan 20, 2021

Uh oh!

nchaulet commented Jan 20, 2021

Uh oh!

jen-huang commented Jan 20, 2021

Uh oh!

kevinlog commented Jan 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jen-huang left a comment

Choose a reason for hiding this comment

Uh oh!

jen-huang Jan 20, 2021

Choose a reason for hiding this comment

Uh oh!

nchaulet Jan 20, 2021

Choose a reason for hiding this comment

Uh oh!

jen-huang Jan 20, 2021

Choose a reason for hiding this comment

Uh oh!

nchaulet Jan 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nchaulet commented Jan 20, 2021

Uh oh!

kibanamachine commented Jan 21, 2021

💚 Build Succeeded

Metrics [docs]

Page load bundle

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

nchaulet commented Dec 16, 2020 •

edited

Loading

nchaulet commented Jan 19, 2021 •

edited

Loading

kevinlog commented Jan 20, 2021 •

edited

Loading

nchaulet Jan 20, 2021 •

edited

Loading