Skip to content

Use disk cache to run wd tests#1970

Merged
hoodmane merged 5 commits intomainfrom
ggu/disk-cache-tests
May 10, 2024
Merged

Use disk cache to run wd tests#1970
hoodmane merged 5 commits intomainfrom
ggu/disk-cache-tests

Conversation

@garrettgu10
Copy link
Contributor

This PR does a few things:

  1. It updates the pyodide bucket to one that includes a file called all_wheels.zip in its GitHub release.
  2. It uses this file to create a bazel repo that contains all the wheels
  3. It passes a path to the bazel repo as --disk-cache-dir when the test is being run.

Open questions:

  1. Does this work on Windows
  2. Does this break tests on edgeworker somehow

Why not just download the wheels from R2? The main reason is that the R2 bucket is not accessible from our build infrastructure.

@garrettgu10 garrettgu10 requested review from dom96 and hoodmane April 4, 2024 20:58
@garrettgu10 garrettgu10 requested review from a team as code owners April 4, 2024 20:58
@dom96
Copy link
Contributor

dom96 commented Apr 5, 2024

The main reason is that the R2 bucket is not accessible from our build infrastructure.

Why is that? Can't we make the R2 bucket accessible from there?

@garrettgu10 garrettgu10 force-pushed the ggu/disk-cache-tests branch from 9410f42 to 546adcf Compare April 10, 2024 16:01
@garrettgu10 garrettgu10 force-pushed the ggu/disk-cache-tests branch 2 times, most recently from 5de9c3d to b1d765c Compare April 16, 2024 17:08
@garrettgu10 garrettgu10 force-pushed the ggu/disk-cache-tests branch 3 times, most recently from f52e195 to e0ef78b Compare May 3, 2024 17:28
garrettgu10 and others added 3 commits May 7, 2024 11:41
Comment on lines 9 to 10
(name = "aiohttp", pythonRequirement = "aiohttp"),
(name = "ssl", pythonRequirement = "ssl"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
(name = "aiohttp", pythonRequirement = "aiohttp"),
(name = "ssl", pythonRequirement = "ssl"),

Copy link
Contributor

@hoodmane hoodmane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @garrettgu10!

@hoodmane hoodmane merged commit 33dce06 into main May 10, 2024
@hoodmane hoodmane deleted the ggu/disk-cache-tests branch May 10, 2024 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants