Skip to content

Support wasm32-unknown-unknown target#1643

Closed
Spxg wants to merge 29 commits intorusqlite:masterfrom
Spxg:wasm-demo
Closed

Support wasm32-unknown-unknown target#1643
Spxg wants to merge 29 commits intorusqlite:masterfrom
Spxg:wasm-demo

Conversation

@Spxg
Copy link
Contributor

@Spxg Spxg commented Feb 10, 2025

I don't know if it can be implemented this way.

@Spxg Spxg changed the title Support wasm32-unknown-unknown target Support wasm32-unknown-unknown target Feb 10, 2025
@Spxg Spxg force-pushed the wasm-demo branch 3 times, most recently from bcab28c to 8bc5f00 Compare February 10, 2025 14:21
@Spxg Spxg marked this pull request as ready for review February 10, 2025 14:22
@gwenn
Copy link
Collaborator

gwenn commented Feb 10, 2025

I know nothing about wasm so we need to find someone kind enough to validate this PR.
And would you mind adding a dedicated job, something like https://github.com/rusqlite/rusqlite/pull/1010/files#diff-7829468e86c1cc5d5133195b5cb48e1ff6c75e3e9203777f6b2e379d9e4882b3R86
in your PR
Thanks.

@Spxg
Copy link
Contributor Author

Spxg commented Feb 11, 2025

I know nothing about wasm so we need to find someone kind enough to validate this PR. And would you mind adding a dedicated job, something like https://github.com/rusqlite/rusqlite/pull/1010/files#diff-7829468e86c1cc5d5133195b5cb48e1ff6c75e3e9203777f6b2e379d9e4882b3R86 in your PR Thanks.

Of course, I will add the tests runner to CI/CD later today, currently all the tests pass (except a couple that require std::fs)

@Spxg
Copy link
Contributor Author

Spxg commented Feb 11, 2025

(a) https://github.com/Spxg/sqlite-wasm-rs/blob/master/sqlite-wasm-rs/sqlite-wasm/sqlite3.wasm comes from https://sqlite.org/2025/sqlite-wasm-3490000.zip ? (b) You don't use https://github.com/sqlite/sqlite/blob/master/ext/wasm/api/sqlite3-wasm.c ?

(a) sqlite3.wasm is only used in wrapper feature, and the interface is exported through wasm-bindgen. The sqlite3.wasm in the project is not from the official product, see https://github.com/Spxg/sqlite-wasm-rs?tab=readme-ov-file#why-vendor-sqlite-wasm

(b) This file is used to compile sqlite3 into emscripten wasm, mainly for generating bindings in emscripten js. We don't need it, we have enabled -DSQLITE_OS_OTHER, and do not compile sqlite directly into .wasm, and we don't need emscripten js binding.

@Spxg
Copy link
Contributor Author

Spxg commented Feb 11, 2025

The tests have been added to the CI/CD (approval is required to verify whether they work). All tests have passed in local testing.
The rusqlite_test_helper library has been added to handle the differences between the wasm platform and the native platform.

@gwenn
Copy link
Collaborator

gwenn commented Feb 11, 2025

Thanks for your time and explanations.

@codecov
Copy link

codecov bot commented Feb 11, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.08%. Comparing base (44e0ef9) to head (8e53118).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1643      +/-   ##
==========================================
- Coverage   86.90%   82.08%   -4.82%     
==========================================
  Files          58       58              
  Lines       10638     7398    -3240     
==========================================
- Hits         9245     6073    -3172     
+ Misses       1393     1325      -68     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Spxg
Copy link
Contributor Author

Spxg commented Feb 11, 2025

Fixed CI for clippy and minimal versions. The reduced coverage is probably because I added two wasm-only tests and some comments.

By the way, why does the CI process require approval? It seems that it is not convenient to find problems and fix CI.

@gwenn
Copy link
Collaborator

gwenn commented Feb 12, 2025

Fixed CI for clippy and minimal versions. The reduced coverage is probably because I added two wasm-only tests and some comments.

By the way, why does the CI process require approval? It seems that it is not convenient to find problems and fix CI.

Maybe because a patched / malicious build.rs, you can do anything !

@Spxg
Copy link
Contributor Author

Spxg commented Feb 12, 2025

Maybe because a patched / malicious build.rs, you can do anything !

Reasonable. I need to rerun CI now to verify whether everything passes. Can you approve it?

@trevyn
Copy link
Contributor

trevyn commented Feb 12, 2025

By the way, why does the CI process require approval? It seems that it is not convenient to find problems and fix CI.

You can enable Actions on your fork and get CI to run inside your account instead of from the PR.

@Spxg
Copy link
Contributor Author

Spxg commented Feb 12, 2025

By the way, why does the CI process require approval? It seems that it is not convenient to find problems and fix CI.

You can enable Actions on your fork and get CI to run inside your account instead of from the PR.

Thanks!

@Spxg
Copy link
Contributor Author

Spxg commented Feb 12, 2025

CI have passed: https://github.com/Spxg/rusqlite/actions/runs/13284078817/job/37088729090, cov failed because I didn’t have a token to upload. (thanks to @trevyn )

I'm ready for the review.

@Spxg Spxg marked this pull request as draft February 13, 2025 14:26
@Spxg Spxg marked this pull request as ready for review February 14, 2025 15:32
@Spxg
Copy link
Contributor Author

Spxg commented Feb 14, 2025

Because of breaking changes, sqlite-wasm-rs is updated to 0.3

  1. Deleted wrapper feature
  2. Use bundled feature to compile sqlite by default
  3. Provided precompiled feature, see https://github.com/Spxg/sqlite-wasm-rs#why-provide-precompiled-library for details

@Spxg
Copy link
Contributor Author

Spxg commented Feb 21, 2025

we need to find someone kind enough to validate this PR.

Any suggestions?

@gwenn
Copy link
Collaborator

gwenn commented Feb 21, 2025

we need to find someone kind enough to validate this PR.

Any suggestions?

Maybe @thomcc or @trevyn ?

@lucyawrey
Copy link

This PR is exactly what I need for one of my projects, so I hope someone can validate it soon.

I can at least confirm @Spxg 's fork works without issue when used as a dependency in a project targeting wasm32-unknown-unknown. I only seem to be able to get it to work in-memory for wasm builds though, and am not sure how to get it to work with a persistent VFS.

@Spxg
Copy link
Contributor Author

Spxg commented Aug 1, 2025

@gwenn any suggestions? wasm-bindgen was unmaintained for a while, which might have been a concern for you, but the organization has changed and things are moving in a positive direction. wasm-bindgen/wasm-bindgen#4533

@Spxg Spxg closed this Aug 2, 2025
@gwenn
Copy link
Collaborator

gwenn commented Aug 7, 2025

@Spxg
Copy link
Contributor Author

Spxg commented Aug 7, 2025

Just for reference: https://blog.rust-lang.org/2025/04/04/c-abi-changes-for-wasm32-unknown-unknown/

Yes, wasm32-unknown-unknown is now C-ABI aligned, which means linking to emscripten or wasi will not be a problem.

@gwenn
Copy link
Collaborator

gwenn commented Aug 7, 2025

Just for reference: https://blog.rust-lang.org/2025/04/04/c-abi-changes-for-wasm32-unknown-unknown/

Yes, wasm32-unknown-unknown is now C-ABI aligned, which means linking to emscripten or wasi will not be a problem.

Which means the official sqlite3.wasm ?

@Spxg
Copy link
Contributor Author

Spxg commented Aug 7, 2025

Which means the official sqlite3.wasm ?

This means we can compile sqlite3 with emcc and link it into our rust wasm32-unknown-unknown program, just like sqlite-wasm-rs does.

However, this is only one step, more work is actually to implement a persistent VFS under wasm32-unknown-unknown, like opfs: https://github.com/Spxg/sqlite-wasm-rs/blob/master/sqlite-wasm-rs/src/vfs/sahpool.rs

If you are interested, you can try it out at https://sqlight.dev

@Spxg
Copy link
Contributor Author

Spxg commented Aug 7, 2025

more work

Of course, another important work is to complete some C functions:
https://github.com/Spxg/sqlite-wasm-rs/blob/master/sqlite-wasm-rs/src/shim.rs

@gwenn
Copy link
Collaborator

gwenn commented Aug 7, 2025

Ok, thanks for the details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants