Replace relative URLs to absolute URL when freezing the lockfile#241
Conversation
for more information, see https://pre-commit.ci
|
Now we should merge this, release micropip, release the packages, update the package set, and backport the package set update. |
|
Yes. |
|
The file names are incorrect on NodeJS: |
|
@alumni Oops... thanks. |
|
@alumni I cannot reproduce the behavior. Could you provide some reproducer? |
|
I found some other issues with node anyways... |
|
@ryanking13 I think I figured it out when it happens: if Assume the folder const { mkdir, writeFile } = require('node:fs/promises');
const { join } = require('node:path');
const { loadPyodide } = require('pyodide');
const packageCacheDir = join(process.cwd(), '.cache', 'pyodide');
const lockFileURL = join(process.cwd(), '.cache', 'pyodide-lock.json');
// file structure setup:
await mkdir(packageCacheDir, { mode: 0o755, recursive: true });
const lockFile = (await import('pyodide/pyodide-lock.json', { with: { type: 'json' } })).default;
await writeFile(lockFileURL, JSON.stringify(lockFile, null, 4), { encoding: 'utf-8', flush: true });
const pyodide = await loadPyodide({ lockFileURL, packageCacheDir });
// install micropip 0.10.1 from PyPi
await pyodide.loadPackage('https://files.pythonhosted.org/packages/6a/79/3dda562c22b4978493adda6a7810f43b513b46cef82cbfd2aef48c3e29e0/micropip-0.10.1-py3-none-any.whl')
const micropip = pyodide.pyimport('micropip');
await micropip.install('simplejson');
// Console output:
// Loading simplejson
// Didn't find package simplejson-3.20.1-cp313-cp313-pyodide_2025_0_wasm32.whl locally, attempting to load from https://cdn.jsdelivr.net/pyodide/v0.28.0/full/
// Package simplejson-3.20.1-cp313-cp313-pyodide_2025_0_wasm32.whl loaded from https://cdn.jsdelivr.net/pyodide/v0.28.0/full/, caching the wheel in node_modules for future use.
// Loaded simplejson
//
// File <cwd>/.cache/pyodide/simplejson-3.20.1-cp313-cp313-pyodide_2025_0_wasm32.whl was created
JSON.parse(micropip.freeze()).packages.simplejson;
// output:
{
depends: [],
file_name: '<cwd>/.cache/simplejson-3.20.1-cp313-cp313-pyodide_2025_0_wasm32.whl',
imports: [ 'simplejson' ],
install_dir: 'site',
name: 'simplejson',
package_type: 'package',
sha256: 'c5884ab81866a337e4d797f4f269a9ad26513097d2993d09bc90e80dfa1ccbbe',
unvendored_tests: true,
version: '3.20.1'
}Actual file: Probably path functions could be used if Pyodide is running in NodeJS, instead of |
|
Oh, okay, thanks for the detailed reproducer. So the problem is that |
|
pyodide/pyodide#5763 will do the fix. I am not sure whether it will be fully compatible with the Windows path. But as far as I remember, node handles mixed slashes quite well. |
This updates
micropip.freezeto replace relative URLs (or just file names) to absolute URL.This was discussed in pyodide/pyodide#5652 (comment) but I forgot to fix it before releasing 0.28.
This also fixes pyodide/pyodide#5736