FS.syncfs with populate=true throws error on iPad (iOS generally?) #4057
Description
🐛 Bug
After writing a file to OPFS and then running pyodide.FS.syncfs with populate=true, to make the file available in pyodide, pyodide throws an exception.
This only happens after the second time you run the page. Presumably because the error is thrown when comparing existing files.
It throws even if the new file you're trying to write has a different name, so it's not exclusive to handling overwrites.
The error is: TypeError: undefined is not an object (evaluating 'e.timestamp.getTime')
I believe I've tracked it down to this line:
Line 180 in f8f026a
e["timestamp"] or e2["timestamp"] returns undefined so the .getTime() call throws.
It may be that Safari/Chrome on iPad is not following the specification when it comes to writing/reading from OPFS, and actually all objects should have a timestamp?
To Reproduce
Available to test in this CodePen here: https://codepen.io/paulcalcraft/pen/abQrroB
No error on desktop or Android Chrome, but iPad (both Chrome and Safari) throw an error the second time you run it (let it run its full output, then refresh the page and let it run again).
Apologies for the long MWE, but I believe it's all necessary because iPad only supports writing with createSyncAccessHandle, which needs to be on a web worker.
<!doctype html>
<html>
<head>
<script src="https://cdn.jsdelivr.net/pyodide/v0.23.4/full/pyodide.js"></script>
</head>
<body>
FS.syncfs Pyodide Error on iPad<br>
Logs from Pyodide and worker are shown below.<br><br>
<script type="text/javascript">
console.log = function (message) {
document.body.innerHTML += message + "<br>";
}
console.error = function (message) {
document.body.innerHTML += "<span style='color:red'>" + message + "</span><br>";
}
async function main() {
console.log = function (message) {
document.body.innerHTML += message + "<br>";
}
let pyodide = await loadPyodide();
console.log(pyodide.runPython(`
import sys
sys.version
`)); // 3.11.2 (main, Jul 7 2023, 05:19:00) [Clang 17.0.0 (https://github.com/llvm/llvm-project df82394e7a2d06506718cafa347b
const dirHandle = await navigator.storage.getDirectory();
let nativefs = await pyodide.mountNativeFS("/opfs", dirHandle);
// define worker that writes a file using createSyncAccessHandle()
// only available in web worker
// (createWritable doesn't work on iPad)
var blob = new Blob([`
console.log = function(message) {
postMessage(message);
}
console.error = function(message) {
postMessage("error:" + message);
}
self.onmessage = async function(e) {
console.log("hello from worker");
try {
const dirHandle = await navigator.storage.getDirectory();
const fileHandle = await dirHandle.getFileHandle('test.txt', { create: true });
let writable = await fileHandle.createSyncAccessHandle();
let dataToWrite = new DataView(new Int32Array(10).buffer)
// Write the file to the writable stream
await writable.write(dataToWrite);
if (writable.flush) {
await writable.flush();
console.log("flushed")
}
await writable.close();
console.log('File saved successfully');
postMessage("done");
} catch (err) {
postMessage("error:" + err);
}
}`
], { type: 'text/javascript' });
// Create a URL for the blob
var url = URL.createObjectURL(blob);
// Create a worker using the blob URL
var worker = new Worker(url);
// Set up a listener to receive messages from the worker
worker.onmessage = async function (e) {
console.log("worker:" + e.data);
if (e.data === "done") {
// Code to run after the worker has executed
console.log("Worker complete");
let prom = new Promise((resolve, reject) => {
pyodide.FS.syncfs(true, (err) => {
if (err) {
console.error(err);
reject(err);
} else {
resolve()
}
});
});
// nativefs.syncfs() is not an option because it syncs in the other direction
// await nativefs.syncfs();
await prom;
await pyodide.runPython(`
import os
print(os.listdir('/opfs')) # ['test.txt'] expected
# print length of text.txt:
print("test.txt filesize", os.stat('/opfs/test.txt').st_size) # 40 expected
`);
console.log('file system loaded')
console.log("done")
}
};
// Start the worker
worker.postMessage('start');
}
main();
</script>
</body>
</html>Expected behavior
FS.syncfs Pyodide Error on iPad
Logs from Pyodide and worker are shown below.
3.11.2 (main, Jul 7 2023, 05:19:00) [Clang 17.0.0 (https://github.com/llvm/llvm-project df82394e7a2d06506718cafa347b
worker:hello from worker
worker:flushed
worker:File saved successfully
worker:done
Worker complete
['test.txt'']
test.txt filesize 40
file system loaded
done
Environment
- Pyodide Version: v0.23.4
- Browser version: Chrome Version 115.0.5790.160 for iPad, Safari 16.5 for iPad
- Any other relevant information: It works correctly on Chrome on Windows and Android