Skip to content

[CI] Fix project build & deploy job#196562

Merged
delanni merged 14 commits intoelastic:mainfrom
delanni:enable-pr-builds-for-project-deploy-job
Oct 29, 2024
Merged

[CI] Fix project build & deploy job#196562
delanni merged 14 commits intoelastic:mainfrom
delanni:enable-pr-builds-for-project-deploy-job

Conversation

@delanni
Copy link
Copy Markdown
Member

@delanni delanni commented Oct 16, 2024 •
edited
Loading

Summary

In #195581 we've added the option to deploy through the clickable triggers. But in it's current state, it's broken in several aspects.

(1) It's not starting on click. Triggers was resulting in a 422 on Buildkite's side, and after digging more into it, this was the error:
Screenshot 2024-10-16 at 16 53 13
Apparently, building PRs needs to be enabled on jobs that want to be triggered through the PR bot.

(2) It is set up to run regardless of the labels

(3) There's no feedback on runs

Changes

This PR:

  • enables buildability in the pipeline's config
  • exits early if deploy labels are missing
  • adds a comment on the PR if a deploy job is started or finished
  • removes the kibana build step, it's not needed, as we have a step to build the docker image

TODO:

  • Add feedback about a started job (either through a non-required check, or a github comment)
  • Early exit if a label is missing

There are several other builds started right now, because the logic that would trigger a build on changing a draft to ready. To be fixed in https://github.com/elastic/buildkite-pr-bot/issues/78

Tested after manually by enabling the option on the UI, and triggering through the checkbox: https://buildkite.com/elastic/kibana-deploy-project-from-pr/builds/23

@delanni delanni changed the title enable build_pull_requests for the deploy serverless job [CI] Enable build_pull_requests for the deploy serverless project job Oct 16, 2024
@delanni delanni added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting skip-ci labels Oct 16, 2024
@delanni delanni changed the title [CI] Enable build_pull_requests for the deploy serverless project job [CI] Fix project build & deploy job Oct 16, 2024
@delanni delanni marked this pull request as ready for review October 18, 2024 08:34
@delanni delanni requested a review from a team as a code owner October 18, 2024 08:34
@delanni delanni added the ci:project-deploy-elasticsearch Create an Elasticsearch Serverless project label Oct 18, 2024
@delanni
Copy link
Copy Markdown
Member Author

delanni commented Oct 18, 2024

buildkite deploy project

@delanni delanni marked this pull request as draft October 18, 2024 08:48
@delanni delanni removed the skip-ci label Oct 18, 2024
@delanni delanni force-pushed the enable-pr-builds-for-project-deploy-job branch from 91b96fc to 58643c9 Compare October 18, 2024 13:54
@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Oct 18, 2024

PR Project deployment started at: https://buildkite.com/elastic/kibana-deploy-project-from-pr/builds/12

@delanni delanni removed the ci:project-deploy-elasticsearch Create an Elasticsearch Serverless project label Oct 18, 2024
@delanni delanni force-pushed the enable-pr-builds-for-project-deploy-job branch from 58643c9 to 113b4fd Compare October 18, 2024 15:29
@delanni delanni added the ci:project-deploy-elasticsearch Create an Elasticsearch Serverless project label Oct 18, 2024
@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Oct 18, 2024

PR Project deployment started at: https://buildkite.com/elastic/kibana-deploy-project-from-pr/builds/23

@delanni delanni removed the ci:project-deploy-elasticsearch Create an Elasticsearch Serverless project label Oct 18, 2024
@delanni delanni marked this pull request as ready for review October 18, 2024 16:20
@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Oct 18, 2024

Project deployed, see credentials at: https://buildkite.com/elastic/kibana-deploy-project-from-pr/builds/23

Ikuni17
Ikuni17 reviewed Oct 18, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@Ikuni17 Ikuni17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since Jon found we're hitting the Github rate limits on commenting, we may need a slightly different approach. Can we edit the main build comment? Can we utilize a status check somehow?

I can't really tell from the PR history, so I'm wondering about a few situations (maybe I missed these in the initial PR):

  1. If there is a deployment label, does a normal CI run update/overwrite the deployment when it runs?
  2. Is there any concern for multiple deployments if the deployment pipeline and CI pipeline are running concurrently?

@delanni
Copy link
Copy Markdown
Member Author

delanni commented Oct 21, 2024

Thanks for the comments, it also allows me to think the scenario through again.

Can we edit the main build comment?

I think if we edit an existing comment would deplete the same limit (since we probably deplete the limits with edits, and not the creation anyway).

Also, I think these are two different actors (at least for now). It's elasticmachine (from https://github.com/elastic/buildkite-pr-bot) that creates the build result comments, and we have kibanamachine (with the token from github.com/elastic/kibana) in this case commenting on the current build indications.

The nicer solution would probably be through the pr-bot, editing the one comment box, but we'll have to sync on the rate-limit considerations.

Can we utilize a status check somehow?

I tried using a status check through the env variables I copied from the pull-request pipeline, but it didn't appear yet, maybe after editing pull_requests.json and getting the PR merged, the pr-bot will start to manage a status check for the deployment. But that's hard to test before the PR is merged.

(1) If there is a deployment label, does a normal CI run update/overwrite the deployment when it runs?
They run the same script in the end, so it would behave as if CI ran twice (i.e.: it will get redeployed if the ci:project-redeploy label is present, or updated otherwise).

(2) Is there any concern for multiple deployments if the deployment pipeline and CI pipeline are running concurrently?
Yes, good point. In theory, there's a few seconds of overlap in the two deployment scripts where a race condition can arise. I'm not sure if it's worth the time for now to mitigate for that case. Even if it lines up perfectly, the fallout is 1 project existing for the 2 days until it gets shut down (or until the PR labels require it).

@delanni delanni requested a review from Ikuni17 October 28, 2024 08:35
@delanni delanni added the Team:Operations Kibana-Operations Team label Oct 28, 2024
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Oct 28, 2024

Pinging @elastic/kibana-operations (Team:Operations)

@delanni delanni requested a review from a team October 28, 2024 08:35
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Oct 28, 2024

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

@delanni delanni merged commit ae86b54 into elastic:main Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Ikuni17 Ikuni17 Ikuni17 approved these changes

Assignees

No one assigned

Labels

backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:Operations Kibana-Operations Team v9.0.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@delanni @kibanamachine @elasticmachine @Ikuni17