Revamp duckdb-wasm extensions CI#10672
Merged
Mytherin merged 6 commits intoduckdb:mainfrom Feb 15, 2024
Merged
Conversation
2378381 to
51d3a4a
Compare
51d3a4a to
200035e
Compare
Contributor
Author
|
Note that this CI job, invoked on request via workflow, is meant only to build the duckdb-wasm compatible extensions and store them on the official repository. |
200035e to
f0859c3
Compare
f0859c3 to
8906508
Compare
Contributor
Author
|
I think this makes sense, tried various times on my CI while polishing this, seems to work as expected. Upload / signature will be properly tested once extensions are up by the duckdb-wasm CI on the branch bumping to duckdb v0.10.0. |
Contributor
Author
|
Potentially an additional test, if we want to be extra sure, would be building duckdb-wasm extensions for a random commit (say commit the first landed PR after v0.10.0, that would be duckdb_ref = "8508978c55" + s3_release = true). That should make that extensions ARE build, signed and deployed, but for an uninteresting commit hash (and we can then delete them from S3). |
samansmink
reviewed
Feb 15, 2024
Collaborator
samansmink
left a comment
There was a problem hiding this comment.
Think this looks good, added 2 small comments
Contributor
Author
|
Thanks @samansmink, comments have been addressed. |
krlmlr
added a commit
to duckdb/duckdb-r
that referenced
this pull request
Mar 15, 2024
Merge pull request duckdb/duckdb#10658 from hannes/csvpathlength Merge pull request duckdb/duckdb#10756 from Mytherin/preserveinsertionordermemory Merge pull request duckdb/duckdb#10746 from samansmink/enable-azure-autoload Merge pull request duckdb/duckdb#10747 from maiadegraaf/list_reverse_bug Merge pull request duckdb/duckdb#10748 from taniabogatsch/capi-tests Merge pull request duckdb/duckdb#10739 from peterboncz/pb/immmedate_mode_only_in_non_autocommit Merge pull request duckdb/duckdb#10688 from Tmonster/union_exclude Merge pull request duckdb/duckdb#10710 from samansmink/comment-on-column Merge pull request duckdb/duckdb#10725 from hawkfish/fuzzer-preceding-frame Merge pull request duckdb/duckdb#10723 from hawkfish/fuzzer-null-timestamp Merge pull request duckdb/duckdb#10436 from taniabogatsch/map-fixes Merge pull request duckdb/duckdb#10587 from kryonix/main Merge pull request duckdb/duckdb#10738 from TinyTinni/fix-assert-in-iscntrl Merge pull request duckdb/duckdb#10708 from carlopi/ci_fixes Merge pull request duckdb/duckdb#10726 from hawkfish/fuzzer-to-weeks Merge pull request duckdb/duckdb#10727 from hawkfish/fuzzer-window-bind Merge pull request duckdb/duckdb#10733 from TinyTinni/remove-static-string Merge pull request duckdb/duckdb#10715 from Tishj/python_tpch_regression_rework Merge pull request duckdb/duckdb#10728 from hawkfish/fuzzer-argminmax-decimal Merge pull request duckdb/duckdb#10717 from carlopi/fix_extension_deploy Merge pull request duckdb/duckdb#10694 from Mytherin/castquerylocation Merge pull request duckdb/duckdb#10448 from peteraisher/feature/use-assertThrows-for-jdbc-tests Merge pull request duckdb/duckdb#10691 from Mytherin/issue10685 Merge pull request duckdb/duckdb#10684 from Mytherin/distincton Merge pull request duckdb/duckdb#9539 from Tishj/timestamp_unit_to_tz Merge pull request duckdb/duckdb#10341 from Tmonster/tpch_ingestion_benchmark Merge pull request duckdb/duckdb#10689 from Mytherin/juliaversion Merge pull request duckdb/duckdb#10669 from Mytherin/skippedtests Merge pull request duckdb/duckdb#10679 from Tishj/reenable_window_rows_overflow Merge pull request duckdb/duckdb#10672 from carlopi/wasm_extensions_ci Merge pull request duckdb/duckdb#10660 from szarnyasg/update-storage-info-for-v0100 Merge pull request duckdb/duckdb#10643 from bleskes/duck_transaction_o11y Merge pull request duckdb/duckdb#10654 from carlopi/fix_10548 Merge pull request duckdb/duckdb#10650 from hannes/noprintf Merge pull request duckdb/duckdb#10649 from Mytherin/explicitenumnumbers
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI on this PR is not actually checking anything, given the changed files are never invoked.
More proper test for this workflow is as follow:
here I would expect it should succeed, building the extensions, signing them with a fake key, setting up aws, and invoking aws s3 cp with --dryrun
Sample of the log will be like (based on a past run):
This should (if AWS variable are properly inherited from the environment) upload the extensions, otherwise fall back to --dryrun. Please double check you have AWS keys enabled in your repository if you want to try this.
s3_release set to false is not uploading nor signing anything, so it can be safely invoked also also on the duckdb/duckdb repo as an additional test AFTER merging this into main (before that this workflow is present in a older version performing an outdated logic).
My idea is that this PR can be reviewed, eventually merged, and then when it lands on main the workflow can be invoked first passing {'duckdb_ref' : 'v0.10.0', 's3_release' : false} checking the resulting paths make sense / stuff has not exploded, and then invoking it properly passing {'duckdb_ref' : 'v0.10.0', 's3_release' : true}.
Artifacts build during the first step are available after completion in the Summary page: https://github.com/carlopi/duckdb/actions/runs/7912968453.