Use shallow clones for CI jobs#64657
Conversation
|
Looks like after this change, all the non-Windows checkout steps are 10 seconds and the Windows ones are 15 seconds. Before they varied a lot but were 30-90+ seconds each. |
There was a problem hiding this comment.
Pull request overview
This PR optimizes CI performance by implementing shallow git clones (fetchDepth: 1) and disabling tag fetching (fetchTags: false) across all build jobs. The optimization is correctly applied to the shared build template, with an appropriate exception for the Code_check job in ci-public.yml which requires full git history for diff and log comparisons.
Key Changes
- Added
fetchDepthandfetchTagsas configurable parameters in default-build.yml with optimized defaults - Added explicit checkout steps using these parameters in both job definitions (public and internal)
- Overrode the default with
fetchDepth: 0for the Code_check job which requires full git history
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| .azure/pipelines/jobs/default-build.yml | Adds fetchDepth and fetchTags parameters with defaults (1 and false), and adds explicit checkout steps to both public and internal job templates |
| .azure/pipelines/ci-public.yml | Overrides fetchDepth to 0 for Code_check job that needs full history for git log/diff operations |
|
/backport to release/10.0 |
|
/backport to release/9.0 |
|
Started backporting to |
|
Started backporting to |
|
@adityamandaleeka backporting to git am output$ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patch
Applying: Use shallow clones for CI jobs.
Applying: Handle code_check jobs in other definitions.
Using index info to reconstruct a base tree...
A .azure/pipelines/ci-unofficial.yml
M .azure/pipelines/ci.yml
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): .azure/pipelines/ci-unofficial.yml deleted in HEAD and modified in Handle code_check jobs in other definitions.. Version Handle code_check jobs in other definitions. of .azure/pipelines/ci-unofficial.yml left in tree.
Auto-merging .azure/pipelines/ci.yml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0002 Handle code_check jobs in other definitions.
Error: The process '/usr/bin/git' failed with exit code 128 |
Co-authored-by: adityamandaleeka <219224+adityamandaleeka@users.noreply.github.com>
* Initial plan * Add shallow clone optimization for CI jobs (backport of #64657) Co-authored-by: adityamandaleeka <219224+adityamandaleeka@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: adityamandaleeka <219224+adityamandaleeka@users.noreply.github.com>
This PR optimizes the git checkout step across all CI jobs by using shallow clones (
fetchDepth: 1) and disabling tag fetching (fetchTags: false).The current default checkout fetches all branches, tags, and full history on all ~17 jobs we run. This takes a while (on some jobs well over a minute).
Since the jobs run in parallel we won't actually reduce wall clock time per PR by 17 minutes but this should help make our infrastructure more efficient.