Skip to content

[Ingest Manager] Download snapshot artifacts from snapshots repo#18685

Merged
michalpristas merged 7 commits intoelastic:masterfrom
michalpristas:agent-snap-download
Jun 10, 2020
Merged

[Ingest Manager] Download snapshot artifacts from snapshots repo#18685
michalpristas merged 7 commits intoelastic:masterfrom
michalpristas:agent-snap-download

Conversation

@michalpristas
Copy link
Copy Markdown
Contributor

@michalpristas michalpristas commented May 21, 2020

What does this PR do?

This PR introduces new downloader which tries to detect snapshot artifact repository in case agent is built as a snapshot.
If detection fails it just does not include snapshot downloader and proceed as usual (with disk backed up by configured downloader)

Why is it important?

More intuitive scenario when you build a snapshot without anything else and it downloads dependencies. Without this it would fail trying to download SNAPSHOT of 8.0.0 from official elastic.co

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

cc @111andre111

@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/ingest-management (Team:Ingest Management)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels May 21, 2020
@michalpristas michalpristas requested review from blakerouse, ph and ruflin May 21, 2020 09:57
@michalpristas michalpristas changed the title download from snapshot repo [Ingest Manager] Download snapshot artifacts from snapshots repo May 21, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented May 21, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #18685 updated]

  • Start Time: 2020-05-21T14:07:32.901+0000

  • Duration: 31 min 12 sec

Test stats 🧪

Test Results
Failed 0
Passed 497
Skipped 127
Total 624

@blakerouse
Copy link
Copy Markdown
Contributor

Overall code looks good. I would like to run it locally to test it out, how would I go about making agent think its a snapshot build?

@michalpristas
Copy link
Copy Markdown
Contributor Author

@blakerouse agent uses same machinery as beats so SNAPSHOT=true mage build works

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jun 2, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #18685 updated]

  • Start Time: 2020-06-10T08:17:38.865+0000

  • Duration: 38 min 54 sec

Test stats 🧪

Test Results
Failed 0
Passed 509
Skipped 127
Total 636

Copy link
Copy Markdown
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay on the review.

This worked perfectly!

@michalpristas michalpristas merged commit 3c0eeab into elastic:master Jun 10, 2020
@michalpristas michalpristas added the needs_backport PR is waiting to be backported to other branches. label Jun 12, 2020
michalpristas added a commit to michalpristas/beats that referenced this pull request Jun 12, 2020
…stic#18685)

[Ingest Manager] Download snapshot artifacts from snapshots repo (elastic#18685)
michalpristas added a commit that referenced this pull request Jun 12, 2020
) (#19155)

[Ingest Manager] Download snapshot artifacts from snapshots repo (#18685)
v1v added a commit to v1v/beats that referenced this pull request Jun 12, 2020
…ngs-archive

* upstream/master: (119 commits)
  Update filebeat input docs (elastic#19110)
  Add ECS fields from log pipeline of PostgreSQL (elastic#19127)
  Init package libbeat/statestore (elastic#19117)
  [Ingest Manager] Retryable downloads of beats (elastic#19102)
  [DOCS] Add output.console to Functionbeat doc and Functionbeat reference file (elastic#18965)
  Add compatibility info (elastic#18929)
  Set ecszap version to v0.2.0 (elastic#19106)
  [filebeat][httpjson] Fix unit test function call (elastic#19124)
  [Filebeat][httpjson] Adds oauth2 support for httpjson input (elastic#18892)
  Allow host.* fields to be disabled in Suricata module (elastic#19107)
  Make selector string casing configurable (elastic#18854)
  Switch the GRPC communication where Agent is running the server and the beats are connecting back to Agent (elastic#18973)
  Disable host.* fields by default for netflow module (elastic#19087)
  Automatically fill zube teams on backports if available (elastic#18924)
  Fix crash on vsphere module (elastic#19078)
  [Ingest Manager] Download snapshot artifacts from snapshots repo (elastic#18685)
  [Ingest Manager] Basic Elastic Agent documentation (elastic#19030)
  Make user.id a string in system/users, in line with ECS (elastic#19019)
  [docs] Add 7.8 release highlights placeholder file (elastic#18493)
  Fix translate_sid's empty target field handling (elastic#18991)
  ...
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
…stic#18685)

[Ingest Manager] Download snapshot artifacts from snapshots repo (elastic#18685)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Ingest Management:beta1 Group issues for ingest management beta1 needs_backport PR is waiting to be backported to other branches. review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants