Skip to content

[Ingest Manager] Send snapshot flag together with metadata#21285

Merged
michalpristas merged 7 commits intoelastic:masterfrom
michalpristas:agent-send-snap-info
Oct 1, 2020
Merged

[Ingest Manager] Send snapshot flag together with metadata#21285
michalpristas merged 7 commits intoelastic:masterfrom
michalpristas:agent-send-snap-info

Conversation

@michalpristas
Copy link
Copy Markdown
Contributor

What does this PR do?

For correct Source URI resolution fleet needs to be aware whether or not agent is a SNAPSHOT build or not.
This PR includes this information in two forms in local metadata which is sent during enroll and checkin

  • elastic.agent.snapshot - additional field set to true if agent is in snapshot mode
  • elastic.agent.build.original - long version info, in case of snapshot it will contain it e.g.8.0.0-SNAPSHOT (build: 3f1e3a5d016487f4cc995b019a846f3f13104f03 at 2020-09-24 08:57:34 +0000 UTC)

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@michalpristas michalpristas added Team:Ingest Management Ingest Management:beta2 Group issues for ingest management beta2 labels Sep 24, 2020
@michalpristas michalpristas self-assigned this Sep 24, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/ingest-management (Team:Ingest Management)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Sep 24, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Sep 24, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #21285 updated]

  • Start Time: 2020-10-01T06:59:54.539+0000

  • Duration: 32 min 8 sec

Test stats 🧪

Test Results
Failed 0
Passed 1390
Skipped 4
Total 1394

@neptunian
Copy link
Copy Markdown
Contributor

Is it possible to send back the "type" of the package that was installed initially? https://www.elastic.co/guide/en/ecs/current/ecs-package.html. So I don't need to figure it out based on their environment?

@ph
Copy link
Copy Markdown
Contributor

ph commented Sep 24, 2020

@ruflin ^ FYI

@michalpristas We should add documentation around what fields is send for the elastic agent cc @dedemorton

ID: i.agentID,
Version: release.Version(),
Snapshot: release.Snapshot(),
BuildOriginal: release.Info().String(),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably a great place to put if the agent is also upgradable.

Should we add Upgradable: false for now, then replace it with the correct check once #21206 lands.

@ruflin
Copy link
Copy Markdown
Contributor

ruflin commented Sep 28, 2020

@michalpristas I'm good with this change but can you clarify if in the current upgrade implementation Kibana ships down the source URI or not? For now I think all it should do is ship down the version it wants to upgrade to and Agent makes the decision.

Copy link
Copy Markdown
Contributor

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving this PR as I think in any case it is important to know if a snapshot agent is run or not. @hbharding @mostlyjason We should also show this properly in the Agent details page.

@neptunian
Copy link
Copy Markdown
Contributor

Any reason we can't merge this now so I can test on master?

@michalpristas michalpristas merged commit 075696e into elastic:master Oct 1, 2020
michalpristas added a commit to michalpristas/beats that referenced this pull request Oct 2, 2020
…1285)

[Ingest Manager] Send snapshot flag together with metadata (elastic#21285)
v1v added a commit to v1v/beats that referenced this pull request Oct 2, 2020
* upstream/master: (27 commits)
  [Ingest Manager] Split index restrictions into type,dataset, namespace parts (elastic#21406)
  Update Filebeat module expected logs files (elastic#21454)
  Edit SQL module docs and fix broken doc structure (elastic#21233)
  [Ingest Manager] Send snapshot flag together with metadata (elastic#21285)
  Revert "[JJBB] Set shallow cloning to 10 (elastic#21409)" (elastic#21447)
  [JJBB] Use reference repo for fast checkouts (elastic#21410)
  Add initial skeleton of filestream input (elastic#21427)
  Initial spec file for apm-server (elastic#21225)
  [Ingest Manager] Upgrade Action: make source URI optional (elastic#21372)
  Add field limit check for AWS Cloudtrail flattened fields (elastic#21388)
  [Winlogbeat] Move winlogbeat javascript processor to libbeat (elastic#21402)
  ci: pipeline to generate the changelog (elastic#21426)
  [JJBB] Set shallow cloning to 10 (elastic#21409)
  docs: add link to release notes for 7.9.2 (elastic#21405) (elastic#21419)
  docs: Prepare Changelog for 7.9.2 (elastic#21229) (elastic#21403)
  fix: mark flaky tests (elastic#21300)
  fix: use a fixed version of setuptools (elastic#21393)
  Move Kubernetes events metricset to its own block in reference config (elastic#21407)
  [libbeat] Enable WriteAheadLimit in the disk queue (elastic#21391)
  docs: fix apt/yum formatting (elastic#21362)
  ...
v1v added a commit to v1v/beats that referenced this pull request Oct 2, 2020
…ne-2.0-arm

* upstream/master: (54 commits)
  [CI] Change x-pack/auditbeat build events (comments, labels) (elastic#21463)
  [CI] changeset from elastic#20603 was not added to CI2.0 (elastic#21464)
  Add new log file reader for filestream input (elastic#21450)
  [CI] Send slack message with build status (elastic#21428)
  Remove duplicated sources url in dependencies report (elastic#21462)
  Add implementation of FSWatcher and FSScanner for filestream (elastic#21444)
  [Ingest Manager] Split index restrictions into type,dataset, namespace parts (elastic#21406)
  Update Filebeat module expected logs files (elastic#21454)
  Edit SQL module docs and fix broken doc structure (elastic#21233)
  [Ingest Manager] Send snapshot flag together with metadata (elastic#21285)
  Revert "[JJBB] Set shallow cloning to 10 (elastic#21409)" (elastic#21447)
  [JJBB] Use reference repo for fast checkouts (elastic#21410)
  Add initial skeleton of filestream input (elastic#21427)
  Initial spec file for apm-server (elastic#21225)
  [Ingest Manager] Upgrade Action: make source URI optional (elastic#21372)
  Add field limit check for AWS Cloudtrail flattened fields (elastic#21388)
  [Winlogbeat] Move winlogbeat javascript processor to libbeat (elastic#21402)
  ci: pipeline to generate the changelog (elastic#21426)
  [JJBB] Set shallow cloning to 10 (elastic#21409)
  docs: add link to release notes for 7.9.2 (elastic#21405) (elastic#21419)
  ...
v1v added a commit to v1v/beats that referenced this pull request Oct 2, 2020
…ci-build-label-support

* upstream/master:
  [CI] Change x-pack/auditbeat build events (comments, labels) (elastic#21463)
  [CI] changeset from elastic#20603 was not added to CI2.0 (elastic#21464)
  Add new log file reader for filestream input (elastic#21450)
  [CI] Send slack message with build status (elastic#21428)
  Remove duplicated sources url in dependencies report (elastic#21462)
  Add implementation of FSWatcher and FSScanner for filestream (elastic#21444)
  [Ingest Manager] Split index restrictions into type,dataset, namespace parts (elastic#21406)
  Update Filebeat module expected logs files (elastic#21454)
  Edit SQL module docs and fix broken doc structure (elastic#21233)
  [Ingest Manager] Send snapshot flag together with metadata (elastic#21285)
  Revert "[JJBB] Set shallow cloning to 10 (elastic#21409)" (elastic#21447)
  [JJBB] Use reference repo for fast checkouts (elastic#21410)
  Add initial skeleton of filestream input (elastic#21427)
  Initial spec file for apm-server (elastic#21225)
  [Ingest Manager] Upgrade Action: make source URI optional (elastic#21372)
  Add field limit check for AWS Cloudtrail flattened fields (elastic#21388)
  [Winlogbeat] Move winlogbeat javascript processor to libbeat (elastic#21402)
  ci: pipeline to generate the changelog (elastic#21426)
michalpristas added a commit that referenced this pull request Oct 5, 2020
…21460)

[Ingest Manager] Send snapshot flag together with metadata (#21285) (#21460)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ingest Management:beta2 Group issues for ingest management beta2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants