Skip to content

CI: try free up space in Rust / cargo test (amd64) action #18709

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Jefffrey merged 5 commits into from
Nov 15, 2025
Merged

CI: try free up space in Rust / cargo test (amd64) action #18709

Jefffrey merged 5 commits into from
Nov 15, 2025

Conversation

@Jefffrey
Copy link
Contributor

@Jefffrey Jefffrey commented Nov 14, 2025 •
edited
Loading

Closes #18692 (hopefully)

Trying to get CI to pass consistently, try various techniques.

@github-actions github-actions bot added the development-process Related to development process of DataFusion label Nov 14, 2025
@Jefffrey
Copy link
Contributor Author

Jefffrey commented Nov 14, 2025 •
edited
Loading

Oh not sure it (jlumbroso/free-disk-space) works if the action runs in a container 🤔

Edit: jlumbroso/free-disk-space#21

@Jefffrey Jefffrey changed the title Use jlumbroso/free-disk-space in CI setup-builder step CI: try free up space in Rust / cargo test (amd64) action Nov 14, 2025
@Jefffrey
Copy link
Contributor Author

Jefffrey commented Nov 14, 2025

Taking inspiration from examples:

datafusion/.github/workflows/rust.yml

Lines 365 to 377 in 1dddf03

- name: Remove unnecessary preinstalled software
run: |
echo "Disk space before cleanup:"
df -h
apt-get clean
rm -rf /__t/CodeQL
rm -rf /__t/PyPy
rm -rf /__t/Java_Temurin-Hotspot_jdk
rm -rf /__t/Python
rm -rf /__t/go
rm -rf /__t/Ruby
echo "Disk space after cleanup:"
df -h

Which usually cleans about 5gb:

Run echo "Disk space before cleanup:"
Disk space before cleanup:
Filesystem      Size  Used Avail Use% Mounted on
overlay          72G   58G   14G  81% /
tmpfs            64M     0   64M   0% /dev
shm              64M     0   64M   0% /dev/shm
/dev/root        72G   58G   14G  81% /__t
tmpfs           3.2G  1.2M  3.2G   1% /run/docker.sock
tmpfs           7.9G     0  7.9G   0% /proc/acpi
tmpfs           7.9G     0  7.9G   0% /proc/scsi
tmpfs           7.9G     0  7.9G   0% /sys/firmware
Disk space after cleanup:
Filesystem      Size  Used Avail Use% Mounted on
overlay          72G   53G   19G  74% /
tmpfs            64M     0   64M   0% /dev
shm              64M     0   64M   0% /dev/shm
/dev/root        72G   53G   19G  74% /__t
tmpfs           3.2G  1.2M  3.2G   1% /run/docker.sock
tmpfs           7.9G     0  7.9G   0% /proc/acpi
tmpfs           7.9G     0  7.9G   0% /proc/scsi
tmpfs           7.9G     0  7.9G   0% /sys/firmware

But also adding what was suggested here: jlumbroso/free-disk-space#21

Combine together, seems to clean 24gb:

Run echo "Disk space before cleanup:"
Disk space before cleanup:
Filesystem      Size  Used Avail Use% Mounted on
overlay          72G   55G   18G  77% /
tmpfs            64M     0   64M   0% /dev
shm              64M     0   64M   0% /dev/shm
/dev/root        72G   55G   18G  77% /__t
tmpfs           3.2G  1.2M  3.2G   1% /run/docker.sock
tmpfs           7.9G     0  7.9G   0% /proc/acpi
tmpfs           7.9G     0  7.9G   0% /proc/scsi
tmpfs           7.9G     0  7.9G   0% /sys/firmware
Disk space after cleanup:
Filesystem      Size  Used Avail Use% Mounted on
overlay          72G   31G   42G  43% /
tmpfs            64M     0   64M   0% /dev
shm              64M     0   64M   0% /dev/shm
/dev/root        72G   31G   42G  43% /__t
tmpfs           3.2G  1.2M  3.2G   1% /run/docker.sock
tmpfs           7.9G     0  7.9G   0% /proc/acpi
tmpfs           7.9G     0  7.9G   0% /proc/scsi
tmpfs           7.9G     0  7.9G   0% /sys/firmware

🤯

@Jefffrey Jefffrey marked this pull request as ready for review November 14, 2025 16:40
2010YOUY01
2010YOUY01 approved these changes Nov 15, 2025
View reviewed changes
Copy link
Contributor

@2010YOUY01 2010YOUY01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the fix!

I suggest we merge it after CI passes, many recent PRs failed CI due to this reason.

@Jefffrey
Copy link
Contributor Author

Jefffrey commented Nov 15, 2025

Thank you for the fix!

I suggest we merge it after CI passes, many recent PRs failed CI due to this reason.

Thanks, looks like there's been 3 consecutively successful runs so the fix seems to work.

@Jefffrey Jefffrey added this pull request to the merge queue Nov 15, 2025
Merged via the queue into apache:main with commit 302dbc8 Nov 15, 2025
28 checks passed
@Jefffrey Jefffrey deleted the fix-ci-123 branch November 15, 2025 02:42
@Jefffrey Jefffrey mentioned this pull request Nov 15, 2025
Open
jizezhang pushed a commit to jizezhang/datafusion that referenced this pull request Nov 15, 2025
@Jefffrey @jizezhang
CI: try free up space in Rust / cargo test (amd64) action (apache#1…
d61021c 
…8709)

Closes apache#18692 (hopefully)

Trying to get CI to pass consistently, try various techniques.
xudong963 pushed a commit to massive-com/arrow-datafusion that referenced this pull request Nov 17, 2025
@Jefffrey @xudong963
CI: try free up space in Rust / cargo test (amd64) action (apache#1…
c2112e2 
…8709)

Closes apache#18692 (hopefully)

Trying to get CI to pass consistently, try various techniques.
logan-keede pushed a commit to logan-keede/datafusion that referenced this pull request Nov 23, 2025
@Jefffrey @logan-keede
CI: try free up space in Rust / cargo test (amd64) action (apache#1…
06a8b8b 
…8709)

Closes apache#18692 (hopefully)

Trying to get CI to pass consistently, try various techniques.
@alamb alamb mentioned this pull request Dec 1, 2025
Merged
alamb pushed a commit to alamb/datafusion that referenced this pull request Dec 1, 2025
@Jefffrey @alamb
CI: try free up space in Rust / cargo test (amd64) action (apache#1…
f58d75e 
…8709)

Closes apache#18692 (hopefully)

Trying to get CI to pass consistently, try various techniques.
@alamb alamb mentioned this pull request Dec 1, 2025
Merged
xudong963 pushed a commit that referenced this pull request Dec 2, 2025
@alamb @Jefffrey
[branch-51] CI: try free up space in Rust / cargo test (amd64) acti…
388ed32 
…on (#18709) (#19037)

## Which issue does this PR close?

- part of #18843 
- Backports #18709 from
@Jefffrey

## Rationale for this change

@tobixdev is hitting a CI failure due to out of space on a different PR:
-
#19017 (comment)


@Jefffrey  fixed this on main with
-  #18709 

So let's backport that change to branch-51
## What changes are included in this PR?

- Backport #18709 to branch-51

## Are these changes tested?
by ci
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

no

Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com>
@alamb alamb mentioned this pull request Dec 3, 2025
Merged
alamb pushed a commit to influxdata/arrow-datafusion that referenced this pull request Dec 3, 2025
@Jefffrey @alamb
CI: try free up space in Rust / cargo test (amd64) action (apache#1…
da0cdf2 
…8709)

Closes apache#18692 (hopefully)

Trying to get CI to pass consistently, try various techniques.
erratic-pattern pushed a commit to influxdata/arrow-datafusion that referenced this pull request Dec 6, 2025
@Jefffrey @erratic-pattern
CI: try free up space in Rust / cargo test (amd64) action (apache#1…
9c877bd 
…8709)

Closes apache#18692 (hopefully)

Trying to get CI to pass consistently, try various techniques.
erratic-pattern pushed a commit to influxdata/arrow-datafusion that referenced this pull request Dec 26, 2025
@Jefffrey @erratic-pattern
CI: try free up space in Rust / cargo test (amd64) action (apache#1…
31a942f 
…8709)

Closes apache#18692 (hopefully)

Trying to get CI to pass consistently, try various techniques.
This was referenced Jan 8, 2026
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@2010YOUY01 2010YOUY01 2010YOUY01 approved these changes

Assignees

No one assigned

Labels

development-process Related to development process of DataFusion

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

CI failing on main due to storage issues

2 participants

@Jefffrey @2010YOUY01