Skip to content

packer cache support for the 7.x and 7.latestMinor branches#25091

Merged
v1v merged 5 commits intoelastic:masterfrom
v1v:feature/packer-cache-branches.sh
Apr 15, 2021
Merged

packer cache support for the 7.x and 7.latestMinor branches#25091
v1v merged 5 commits intoelastic:masterfrom
v1v:feature/packer-cache-branches.sh

Conversation

@v1v
Copy link
Copy Markdown
Member

@v1v v1v commented Apr 14, 2021

What does this PR do?

Refactor the packer-cache to support:

  • master branch
  • 7.x branch
  • 7.. <--- this might require some tuning in the future, there is a github api limitation with the number of results/

For such, it split the docker images in two different categories:

  • common ones
  • specific per branch (such as elasticsearch for 7.x differs from the one in master and so on)

Why is it important?

Cache as much as possible.

Test

I validated the packer-cache.sh was triggered correctly on a build with the commit f1f5c27

https://beats-ci.elastic.co/blue/organizations/jenkins/Beats%2Fbeats%2FPR-25091/detail/PR-25091/3/pipeline/ is the one with the packer-cache.sh call, and the analysis was done in the comment -> #25091 (comment)

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Apr 14, 2021
Comment on lines +15 to +17
function getBeatsVersion() {
grep 'defaultBeatVersion' libbeat/version/version.go | cut -d= -f2 | sed 's#"##g' | tr -d " "
}
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a make get-version goal but it requires go and mage, so I didn't want to get crazy to prepare the environment and decided to use some bash commands instead

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Apr 14, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #25091 updated

  • Start Time: 2021-04-14T21:07:29.650+0000

  • Duration: 19 min 19 sec

  • Commit: fa37937

Trends 🧪

Image of Build Times

❕ Flaky test report

No test was executed to be analysed.

@v1v
Copy link
Copy Markdown
Member Author

v1v commented Apr 14, 2021

Tests

 curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/pipelines/beats/pipelines/PR-25091/runs/3/nodes/27/log/\?start\=0 -s | grep 'Status: Downloaded newer image for docker.*SNAPSHOT'
[2021-04-14T20:43:05.962Z] Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:8.0.0-SNAPSHOT
[2021-04-14T20:43:31.359Z] Status: Downloaded newer image for docker.elastic.co/kibana/kibana:8.0.0-SNAPSHOT
[2021-04-14T20:43:55.991Z] Status: Downloaded newer image for docker.elastic.co/logstash/logstash:8.0.0-SNAPSHOT
[2021-04-14T20:44:12.891Z] Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.13.0-SNAPSHOT
[2021-04-14T20:44:34.636Z] Status: Downloaded newer image for docker.elastic.co/kibana/kibana:7.13.0-SNAPSHOT
[2021-04-14T20:45:02.900Z] Status: Downloaded newer image for docker.elastic.co/logstash/logstash:7.13.0-SNAPSHOT
[2021-04-14T20:45:17.539Z] Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.12.1-SNAPSHOT
[2021-04-14T20:45:40.348Z] Status: Downloaded newer image for docker.elastic.co/kibana/kibana:7.12.1-SNAPSHOT
[2021-04-14T20:45:52.815Z] Status: Downloaded newer image for docker.elastic.co/logstash/logstash:7.12.1-SNAPSHOT
curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/pipelines/beats/pipelines/PR-25091/runs/3/nodes/27/log/\?start\=0 -s | grep 'git checkout' 
[2021-04-14T20:37:06.348Z]  > git checkout -f ad7971818d360296e3af14564a1b149c28ad55ab # timeout=15
[2021-04-14T20:37:09.435Z]  > git checkout -f a6f836a1227404b78a4e06f9cec184a9ee09aa12 # timeout=15
[2021-04-14T20:37:51.154Z]  > git checkout -f ad7971818d360296e3af14564a1b149c28ad55ab # timeout=15
[2021-04-14T20:37:53.948Z]  > git checkout -f a76e93af197d11e3f5cbe0df692160498a9fc927 # timeout=15
[2021-04-14T20:43:58.428Z] + git checkout 7.x
[2021-04-14T20:45:05.695Z] + git checkout 7.12

@v1v v1v marked this pull request as ready for review April 15, 2021 08:22
@v1v v1v requested a review from a team as a code owner April 15, 2021 08:22
@v1v v1v self-assigned this Apr 15, 2021
@v1v v1v requested a review from urso April 15, 2021 08:23
@botelastic botelastic bot added Team:Automation Label for the Observability productivity team and removed needs_team Indicates that the issue/PR needs a Team:* label labels Apr 15, 2021
@v1v v1v merged commit ae02c5f into elastic:master Apr 15, 2021
v1v added a commit to v1v/beats that referenced this pull request Apr 15, 2021
* upstream/master:
  packer cache support for the 7.x and 7.latestMinor branches (elastic#25091)
  Remove EventFetcher and EventsFetcher interface (elastic#25093)
  Update go-structform to 0.0.8 (elastic#25051)
  Update copy_fields.asciidoc (elastic#25053)
  [elastic-agent] ensure container is backwards compatible (elastic#25092)
  Add --fleet-server-service-token. Rename --fleet-server to --fleet-server-es. (elastic#25083)
  Add cgroup.cpuacct percentages (elastic#25057)
  Add tests for truncated and symlinked files in filestream input (elastic#24425)
  Fix panic when Hearbeat monitor initialization fails twice (elastic#25073)
  [Filebeat][httpjson] Change append transform to initiate new fields as a slice (elastic#25074)
  Osquerybeat: Result values type translation (elastic#25012)
  Update Osquerybeat spec to get it downloading from the correct artifactory path (elastic#25076)
  Fix changelog (elastic#25079)
  Strip Azure EventHub connection string in debug logs (elastic#25066)
  Change googlecloud to gcp in field names (elastic#25038)
  Bump stack version to 7.12.0 for testing (elastic#24957)
  packer-cache: cache the existing docker images on ARM and some more (elastic#25068)
  Disable logstash TestFetch flaky test (elastic#25044)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Team:Automation Label for the Observability productivity team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants