Add tags option during enroll/install by narph · Pull Request #336 · elastic/elastic-agent

narph · 2022-04-14T13:12:19Z

What is the problem this PR solves?

Issue #149

Add ability to set tags during installation and enrolment agent
so that different identical deployments of service can be differentiated in the dataset

The specific use case is described in: https://discuss.elastic.co/t/fleet-managed-elastic-agent-environment-deployment-name-custom-field/282565/6

sudo ./elastic-agent install -f --url=https://<feet manager>:443 --enrollment-token=<token> --tag=staging01 linux
or in case, the agent is already installed via package manager

elastic-agent enroll -f --url=https://<feet manager>:443 --enrollment-token=<token> --tag=staging01 linux

Related elastic/fleet-server#1350

Desired behavior:

 .\elastic-agent.exe enroll  --url=https://192.0.2.1:8220 --fleet-server-es=http://localhost:9200  --fleet-server-service-token=AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyL3Rva2VuLTE2NTA5NTg3Mjc0OTU6TmVHbEx4WWtUaVNETHNmNmNPV09lZw  --fleet-server-policy=fleet-server-policy --insecure --tag this-is-fleet,windows

{
        "_index": ".fleet-agents-7",
        "_id": "d1c66b32-72ed-4ab3-a36c-9660a5cb7125",
        "_score": null,
        "_source": {
          "access_api_key_id": "TW7SZIAB2-L_WJKcJz9m",
          "action_seq_no": [
            -1
          ],
          "active": true,
          "agent": {
            "id": "d1c66b32-72ed-4ab3-a36c-9660a5cb7125",
            "version": "8.3.0"
          },
          "enrolled_at": "2022-04-26T07:42:15Z",
          "local_metadata": {
            "elastic": {
              "agent": {
                "build.original": "8.3.0-SNAPSHOT (build: 2312423496161dd6f05148d2df0941d2c74570c8 at 2022-04-25 13:23:01 +0000 UTC)",
                "id": "d1c66b32-72ed-4ab3-a36c-9660a5cb7125",
                "log_level": "info",
                "snapshot": true,
                "upgradeable": true,
                "version": "8.3.0"
              }
            },
            "host": {
              "architecture": "x86_64",
              "hostname": "DESKTOP-K76UDQL",
              "id": "bc395ab9-1a1a-4312-ad0b-a05cceaf78f1",
              "ip": [
                "fe80::1d58:b17:4161:9414/64",
                "169.254.148.20/16",
                "2a02:a446:f25b:1:558d:72b2:74b6:7c67/64",
                "fe80::558d:72b2:74b6:7c67/64",
                "192.168.2.26/24",
                "fe80::d135:57b1:9b7f:e10a/64",
                "169.254.225.10/16",
                "fe80::6901:1ee0:60dd:c986/64",
                "169.254.201.134/16",
                "fe80::f09a:4481:bc48:e3e3/64",
                "192.168.21.110/24",
                "fe80::fc98:6ed8:6d25:3d98/64",
                "169.254.61.152/16",
                "::1/128",
                "127.0.0.1/8",
                "fe80::b54c:5acc:d25c:75e9/64",
                "172.23.192.1/20",
                "fe80::3186:c6c:cd37:92b8/64",
                "172.18.224.1/20"
              ],
              "mac": [
                "00:ff:6d:ef:82:d6",
                "cc:48:3a:b9:87:c0",
                "9e:b6:d0:3e:a8:c4",
                "ae:b6:d0:3e:a8:c4",
                "9c:b6:d0:3e:a8:c4",
                "28:c2:1f:52:ff:bf",
                "00:15:5d:a0:47:13",
                "00:15:5d:ff:9c:07"
              ],
              "name": "DESKTOP-K76UDQL"
            },
            "os": {
              "family": "windows",
              "full": "Windows 10 Pro(10.0)",
              "kernel": "10.0.22000.613 (WinBuild.160101.0800)",
              "name": "Windows 10 Pro",
              "platform": "windows",
              "version": "10.0"
            }
          },
          "policy_id": "fleet-server-policy",
          "tags": [
            "this-is-fleet",
            "windows"
          ],
          "type": "PERMANENT"
        },
        "sort": [
          1650958935000
        ]
      }

./elastic-agent.exe container -h

....
The following environment variables are provided as a convenience to prevent a large number of environment variable to
be used when the same credentials will be used across all the possible actions above.

  ELASTICSEARCH_HOST - elasticsearch host [http://elasticsearch:9200]
  ELASTICSEARCH_USERNAME - elasticsearch username [elastic]
  ELASTICSEARCH_PASSWORD - elasticsearch password [changeme]
  ELASTICSEARCH_CA - path to certificate authority to use with communicate with elasticsearch
  KIBANA_HOST - kibana host [http://kibana:5601]
  KIBANA_FLEET_USERNAME - kibana username to enable Fleet [$ELASTICSEARCH_USERNAME]
  KIBANA_FLEET_PASSWORD - kibana password to enable Fleet [$ELASTICSEARCH_PASSWORD]
  KIBANA_CA - path to certificate authority to use with communicate with Kibana [$ELASTICSEARCH_CA]
  ELASTIC_AGENT_TAGS - user provided tags for the agent
....

mergify · 2022-04-14T13:12:25Z

This pull request does not have a backport label. Could you fix it @narph? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v./d./d./d is the label to automatically backport to the 8./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

elasticmachine · 2022-04-14T13:18:10Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-05-10T12:12:35.694+0000
Duration: 33 min 54 sec

Test stats 🧪

Test	Results
Failed	0
Passed	3853
Skipped	21
Total	3874

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages.
run integration tests : Run the Elastic Agent Integration tests.
run end-to-end tests : Generate the packages and run the E2E Tests.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

blakerouse · 2022-04-15T00:02:38Z

Does this need to also have ENV variables for the container?

ph · 2022-04-19T14:31:38Z

@blakerouse Yes, I think we need it for container

internal/pkg/agent/cmd/enroll_cmd.go

internal/pkg/agent/cmd/container.go

ph

@narph is there any unit test that could be written here? Do we have the required mechanic in place to write them?

mergify · 2022-04-28T09:35:45Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b add-tags upstream/add-tags
git merge upstream/main
git push upstream add-tags

mergify · 2022-04-28T15:30:16Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b add-tags upstream/add-tags
git merge upstream/main
git push upstream add-tags

narph · 2022-04-28T15:45:16Z

@narph is there any unit test that could be written here? Do we have the required mechanic in place to write them?

I've created a small unittest, we should update the e2e tests to validate the full flow.

elasticmachine · 2022-04-28T16:42:27Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	95.652% (`66/69`)	👍
Files	70.0% (`147/210`)	👍 0.476
Classes	69.431% (`293/422`)	👍 0.237
Methods	52.787% (`805/1525`)	👍 0.293
Lines	39.138% (`8695/22216`)	👍 0.408
Conditionals	100.0% (`0/0`)	💚

internal/pkg/agent/cmd/enroll.go

internal/pkg/agent/cmd/enroll_cmd.go

internal/pkg/agent/cmd/enroll_cmd_test.go

Co-authored-by: Michel Laterman <82832767+michel-laterman@users.noreply.github.com>

michel-laterman

lgtm

blakerouse

Looks good.

…use-orka * 'main' of github.com:elastic/elastic-agent: (23 commits) [Automation] Update go release version to 1.17.10 (elastic#432) [Automation] Update elastic stack version to 8.3.0-4149272f for testing (elastic#435) [Automation] Update elastic stack version to 8.3.0-19aba912 for testing (elastic#430) Add extra k8s resources in clusterRole (elastic#424) [Automation] Update elastic stack version to 8.3.0-8ee1196f for testing (elastic#422) [Automation] Update elastic stack version to 8.3.0-53513548 for testing (elastic#421) Add tags option during enroll/install (elastic#336) validate kubernetes templates in .CI (elastic#417) add missing kube-api resources from managed agent manifest (elastic#381) Create snyk-scan.yml (elastic#397) [Automation] Update elastic stack version to 8.3.0-d380914f for testing (elastic#414) [Automation] Update elastic stack version to 8.3.0-5c1ff35f for testing (elastic#413) [Automation] Update elastic stack version to 8.3.0-6ba9f710 for testing (elastic#410) [Automation] Update elastic stack version to 8.3.0-a1c5cfff for testing (elastic#406) [Automation] Update elastic stack version to 8.3.0-7f585873 for testing (elastic#401) [Automation] Update elastic stack version to 8.3.0-0b6ea9f2 for testing (elastic#399) ci: enable coverage (elastic#377) Remove last dependencies on beats repo (elastic#387) Remove dependency on libbeat (elastic#344) [Automation] Update elastic stack version to 8.3.0-cb2ce38c for testing (elastic#383) ...

add tags

b2d0fa6

narph requested a review from a team as a code owner April 14, 2022 13:12

narph requested review from blakerouse and lykkin and removed request for a team April 14, 2022 13:12

mergify bot assigned narph Apr 14, 2022

mergify bot added the backport-skip label Apr 14, 2022

narph marked this pull request as draft April 14, 2022 13:12

narph added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Apr 14, 2022

fmt update

febc06b

ph self-requested a review April 14, 2022 13:37

narph added the ci:end-to-end Enable the e2e-testing label Apr 14, 2022

container

2312423

jlind23 added the v8.3.0 label Apr 25, 2022

narph added 3 commits April 26, 2022 10:38

env change

d87ae48

fix lint

f5126a3

fix lint

bbb51b5

ph reviewed Apr 27, 2022

View reviewed changes

internal/pkg/agent/cmd/enroll_cmd.go Show resolved Hide resolved

ph reviewed Apr 27, 2022

View reviewed changes

internal/pkg/agent/cmd/container.go Outdated Show resolved Hide resolved

ph suggested changes Apr 27, 2022

View reviewed changes

Merge branch 'main' into add-tags

dff37b8

narph marked this pull request as ready for review April 28, 2022 11:16

add test

93c2dd6

clean up

7d63f15

Merge branch 'main' into add-tags

6b4984c

narph mentioned this pull request Apr 29, 2022

Add mapping for tags for the elastic agent elastic/elasticsearch#86298

Merged

fix lint

e3d23ae

narph requested a review from ph April 29, 2022 12:23

lint

97e3ae3

michel-laterman reviewed May 9, 2022

View reviewed changes

internal/pkg/agent/cmd/enroll.go Outdated Show resolved Hide resolved

internal/pkg/agent/cmd/enroll_cmd.go Outdated Show resolved Hide resolved

internal/pkg/agent/cmd/enroll_cmd_test.go Show resolved Hide resolved

narph and others added 4 commits May 10, 2022 09:05

Update internal/pkg/agent/cmd/enroll.go

e814fa3

Co-authored-by: Michel Laterman <82832767+michel-laterman@users.noreply.github.com>

Merge branch 'main' into add-tags

644096d

update test

480be8f

test

d50da0a

narph requested review from michel-laterman and removed request for lykkin May 10, 2022 12:13

michel-laterman approved these changes May 10, 2022

View reviewed changes

blakerouse approved these changes May 10, 2022

View reviewed changes

narph merged commit 10039f1 into elastic:main May 10, 2022

narph deleted the add-tags branch May 10, 2022 14:25

narph mentioned this pull request May 23, 2022

Add test scenario for elastic agent tags elastic/e2e-testing#2552

Merged

8 tasks

ycombinator mentioned this pull request Feb 28, 2023

Remove agent tag duplicates elastic/fleet-server#2386

Merged

8 tasks

Conversation

narph commented Apr 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is the problem this PR solves?

Uh oh!

mergify bot commented Apr 14, 2022

Uh oh!

elasticmachine commented Apr 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

Uh oh!

blakerouse commented Apr 15, 2022

Uh oh!

ph commented Apr 19, 2022

Uh oh!

Uh oh!

Uh oh!

ph left a comment

Choose a reason for hiding this comment

Uh oh!

mergify bot commented Apr 28, 2022

Uh oh!

mergify bot commented Apr 28, 2022

Uh oh!

narph commented Apr 28, 2022

Uh oh!

elasticmachine commented Apr 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌐 Coverage report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

michel-laterman left a comment

Choose a reason for hiding this comment

Uh oh!

blakerouse left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

narph commented Apr 14, 2022 •

edited

Loading

elasticmachine commented Apr 14, 2022 •

edited

Loading

elasticmachine commented Apr 28, 2022 •

edited

Loading