Skip to content

Add Python snapshots upload job#4951

Merged
hoodmane merged 1 commit intomainfrom
hoodmane/python-snapshots-upload-job
Sep 8, 2025
Merged

Add Python snapshots upload job#4951
hoodmane merged 1 commit intomainfrom
hoodmane/python-snapshots-upload-job

Conversation

@hoodmane
Copy link
Copy Markdown
Contributor

@hoodmane hoodmane commented Sep 2, 2025

No description provided.

Comment on lines +15 to +27
uses: ./.github/workflows/_bazel.yml
with:
image: ubuntu-22.04
os_name: linux
arch_name: 'X64'
suffix: ''
extra_bazel_args: '--config=ci-linux'
run_tests: false
extra_bazel_args: '--config=ci-linux'
upload_binary: true
secrets:
BAZEL_CACHE_KEY: ${{ secrets.BAZEL_CACHE_KEY }}
WORKERS_MIRROR_URL: ${{ secrets.WORKERS_MIRROR_URL }}

build:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {}

Copilot Autofix

AI 7 months ago

To fix the problem, add an explicit permissions block at the root level of .github/workflows/release-python-snapshots.yml, directly after the name: and before on:. This block should enumerate only the necessary permissions for the workflow. Since the workflow involves committing and pushing changes to the repository ("Check for open PR and commit changes" step), write access to contents may be needed. Unless other access is required, start with:

permissions:
  contents: write

This limits the GITHUB_TOKEN permissions to only what's needed for interacting with repository contents. If later analysis or errors show other required permissions (e.g., for pull-requests), they can be added as needed.

Required changes:

  • Insert a permissions: block after the workflow name: at the top of .github/workflows/release-python-snapshots.yml.

Suggested changeset 1
.github/workflows/release-python-snapshots.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/release-python-snapshots.yml b/.github/workflows/release-python-snapshots.yml
--- a/.github/workflows/release-python-snapshots.yml
+++ b/.github/workflows/release-python-snapshots.yml
@@ -1,4 +1,6 @@
 name: Make Python Snapshots
+permissions:
+  contents: write
 
 on:
   workflow_dispatch:
EOF
@@ -1,4 +1,6 @@
name: Make Python Snapshots
permissions:
contents: write

on:
workflow_dispatch:
Copilot is powered by AI and may make mistakes. Always verify output.
Comment on lines +29 to +108
needs: [build-linux]
runs-on: ubuntu-22.04
name: Build and upload Python memory snapshots
steps:
- uses: actions/checkout@v4
with:
show-progress: false
- uses: actions/setup-python@v5
with:
python-version: '3.13'
- name: Setup Runner
uses: ./.github/actions/setup-runner
- name: Download workerd binary
uses: actions/download-artifact@v4
with:
name: linux-X64-binary
path: /tmp

- name: Make workerd binary executable
run: chmod +x /tmp/workerd

- name: Build and upload Python memory snapshots
env:
PROD_R2_ACCOUNT_ID: ${{ secrets.PYODIDE_CAPNP_R2_ACCOUNT_ID }}
PROD_R2_ACCESS_KEY_ID: ${{ secrets.PYODIDE_CAPNP_R2_ACCESS_KEY_ID }}
PROD_R2_SECRET_ACCESS_KEY: ${{ secrets.PYODIDE_CAPNP_R2_SECRET_ACCESS_KEY }}
TEST_R2_ACCOUNT_ID: ${{ secrets.R2_EW_SNAPSHOT_TESTS_ACCOUNT_ID }}
TEST_R2_ACCESS_KEY_ID: ${{ secrets.R2_EW_SNAPSHOT_TESTS_ACCESS_KEY_ID }}
TEST_R2_SECRET_ACCESS_KEY: ${{ secrets.R2_EW_SNAPSHOT_TESTS_SECRET_ACCESS_KEY }}
WORKERD_BINARY: "/tmp/workerd"
run: |
bazel build @workerd//src/pyodide:bundle_version_info
# boto3 v1.36.0 fails with:
# NotImplemented error occurred in CreateMultipartUpload operation: Header 'x-amz-checksum-algorithm' with value 'CRC32' not implemented
pip install 'boto3<1.36.0' requests
python3 src/pyodide/make_snapshots.py ${{ inputs.update-released == true && '--update-released' || '' }}

- name: Check for open PR and commit changes
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Commit changes to python_metadata.bzl and push to branch
# Configure git
git config --local user.email "action@github.com"
git config --local user.name "release-python-runtime.yml GitHub Action"

# Get current branch name
BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)
echo "Current branch: $BRANCH_NAME"

# Check if there are changes to python_metadata.bzl
if git diff --quiet build/python_metadata.bzl; then
echo "No changes to python_metadata.bzl"
exit 0
fi

echo "Changes detected in python_metadata.bzl"

# Check if there's an open PR for this branch
PR_COUNT=$(gh pr list --head "$BRANCH_NAME" --state open --json number --jq length)

if [ "$PR_COUNT" -eq 0 ]; then
echo "No open PR found for branch $BRANCH_NAME, skipping commit"
exit 0
fi

echo "Found open PR for branch $BRANCH_NAME"

# Commit and push the changes
git add build/python_metadata.bzl
git commit --fixup HEAD -m "Update python_metadata.bzl with new bundle info

This commit updates the backport and integrity values in python_metadata.bzl
based on the latest Pyodide bundle upload.

🤖 Generated automatically by release-python-runtime workflow"

# --no-verify because 'git merge-base origin/main HEAD' fails in pre-commit.
# There shouldn't be formatting errors anyways and if there are CI will catch it.
git push origin "$BRANCH_NAME" --no-verify
echo "Changes committed and pushed to $BRANCH_NAME"

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 7 months ago

To fix the problem, explicitly set the required permissions for the workflow or specific jobs. In this workflow, the main job (build) uses the GITHUB_TOKEN to commit and push changes, which requires contents: write. It is best practice to specify permissions at the job level if only one job needs elevated permissions. For maximum clarity and security, add a permissions block for the build job with at least contents: write; if jobs only require read, set contents: read. For the inherited job (build-linux), since it only uploads binaries and doesn’t push commits, set contents: read (if you wish to be more granular and build-linux doesn't need content writes). However, because build-linux uses a reusable workflow, the permissions block should be in this workflow at the job level for build.

Edit .github/workflows/release-python-snapshots.yml:

  • Add a permissions block to the build job, specifying contents: write.
Suggested changeset 1
.github/workflows/release-python-snapshots.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/release-python-snapshots.yml b/.github/workflows/release-python-snapshots.yml
--- a/.github/workflows/release-python-snapshots.yml
+++ b/.github/workflows/release-python-snapshots.yml
@@ -28,6 +28,8 @@
     needs: [build-linux]
     runs-on: ubuntu-22.04
     name: Build and upload Python memory snapshots
+    permissions:
+      contents: write
     steps:
       - uses: actions/checkout@v4
         with:
EOF
@@ -28,6 +28,8 @@
needs: [build-linux]
runs-on: ubuntu-22.04
name: Build and upload Python memory snapshots
permissions:
contents: write
steps:
- uses: actions/checkout@v4
with:
Copilot is powered by AI and may make mistakes. Always verify output.
@hoodmane hoodmane force-pushed the hoodmane/python-snapshots-upload-job branch 2 times, most recently from 0f30e68 to bcaebf7 Compare September 2, 2025 09:57
@hoodmane hoodmane marked this pull request as ready for review September 2, 2025 09:58
@hoodmane hoodmane requested review from a team as code owners September 2, 2025 09:58
@hoodmane hoodmane force-pushed the hoodmane/python-snapshots-upload-job branch from bcaebf7 to e3e8fbd Compare September 2, 2025 12:49
hoodmane pushed a commit that referenced this pull request Sep 2, 2025
This commit updates the snapshot info for 0.28 in python_metadata.bzl
based on the snapshots uploaded by the github action on:
#4951

This should fix the sentry errors about the missing baseline snapshot in
Pyodide 0.28.
hoodmane added a commit that referenced this pull request Sep 2, 2025
This commit updates the snapshot info for 0.28 in python_metadata.bzl
based on the snapshots uploaded by the github action on:
#4951

This should fix the sentry errors about the missing baseline snapshot in
Pyodide 0.28.
hoodmane added a commit that referenced this pull request Sep 2, 2025
This commit updates the snapshot info for 0.28 in python_metadata.bzl
based on the snapshots uploaded by the github action on:
#4951

This should fix the sentry errors about the missing baseline snapshot in
Pyodide 0.28.
@hoodmane hoodmane force-pushed the hoodmane/python-snapshots-upload-job branch from 1f82510 to 6c20d01 Compare September 8, 2025 08:14
@hoodmane hoodmane merged commit aaf67cb into main Sep 8, 2025
21 of 23 checks passed
@hoodmane hoodmane deleted the hoodmane/python-snapshots-upload-job branch September 8, 2025 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants