migrate changes to Cell Ranger module tests from UniverSC PR#2646
Merged
TomKellyGenetics merged 2 commits intonf-core:masterfrom Jan 18, 2023
Merged
migrate changes to Cell Ranger module tests from UniverSC PR#2646TomKellyGenetics merged 2 commits intonf-core:masterfrom
TomKellyGenetics merged 2 commits intonf-core:masterfrom
Conversation
PR nf-core#1706 to match UniverSC
ggabernet
requested changes
Dec 14, 2022
Member
ggabernet
left a comment
There was a problem hiding this comment.
Hi @TomKellyGenetics, thanks for the PR! I've just added some review comments
|
|
||
| withName: CELLRANGER_COUNT { | ||
| ext.args = '--chemistry SC3Pv3' | ||
| ext.args = '--chemistry SC3Pv3 --description sample-123' |
Member
There was a problem hiding this comment.
Suggested change
| ext.args = '--chemistry SC3Pv3 --description sample-123' | |
| ext.args = '--chemistry SC3Pv3' |
Member
There was a problem hiding this comment.
I did not see the param you proposed in the cellranger docs:
https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/count
Member
Author
There was a problem hiding this comment.
It's documented in the command-line help system.
$ cellranger count --help
cellranger-count
Count gene expression (targeted or whole-transcriptome) and/or feature barcode reads from a single sample and GEM well
USAGE:
cellranger count [OPTIONS] --id <ID> --transcriptome <PATH>
OPTIONS:
--id <ID> A unique run id and output folder name [a-zA-Z0-9_-]+
--description <TEXT> Sample description to embed in output files [default: ]
Member
Author
There was a problem hiding this comment.
For context this returns a string in the web_summary.html file so the reason I implemented this was to:
a. match results between Cell Ranger and UniverSC tests with 10X Genomics data.
b. enable unit testing with a "contain" check (which passes locally but was removed during review of GitHub Actions results)
As discussed here: #1706 (comment)
TomKellyGenetics
pushed a commit
to TomKellyGenetics/nf-core-modules
that referenced
this pull request
Dec 15, 2022
migrates changes from nf-core#1706 to nf-core#2646
TomKellyGenetics
added a commit
to TomKellyGenetics/nf-core-modules
that referenced
this pull request
Dec 16, 2022
ggabernet
approved these changes
Dec 20, 2022
Member
ggabernet
left a comment
There was a problem hiding this comment.
alright, looks good to me then
apeltzer
added a commit
that referenced
this pull request
Feb 14, 2023
* initialise template for new module: "universc" * update UniverSC module updates metadata, docker container source, licensing, and citations * update UniverSC module unit tests and documentation * update UniverSC module add inputs, outputs and example calls for UniverSC and Cell Ranger v3.0.2 calls versions for UniverSC and Cell Ranger * initialise template for new module: "universc" * update UniverSC module updates metadata, docker container source, licensing, and citations * update UniverSC module unit tests and documentation * update UniverSC module add inputs, outputs and example calls for UniverSC and Cell Ranger v3.0.2 calls versions for UniverSC and Cell Ranger * resolve formatting issues for UniverSC module * resolve linting errors for UniverSC module * fix test jobs to call UniverSC version without errors * correct configuration for UniverSC test jobs * correct linting errors for UniverSC module * correct docker build files for UniverSC * correct syntax errors in cellranger version call * prettier docs for UniverSC * add output to test data for UniverSC module * update UniverSC module to restructured repo #2141 remove files for restructed UniverSC module (avoids duplicate tests) * define separate outputs for Cell Ranger and UniverSC tests * remove TODO statements and update UniverSC meta.yml defines input and output variables and triggers automated tests * resolve minor linting issues with UniverSC * update paths to UniverSC module in test config * update container versions and tags for UniverSC module * simplifiy container configurations for UniverSC * update configuration for UniverSC tests (build Cell Ranger transcriptome reference first) * test UniverSC module with Cell Ranger references * update tests for UniverSC for restructured repository * update reference inputs for UniverSC module * set up references for cellranger OS test * resolve permissions errors for starting UniverSC * update input arguments for UniverSC and Cell Ranger OS (tests passing) * correct versions and checksums for UniverSC tests * resolves linting issues for UniverSC * resolves linting issues for UniverSC * update expected test outputs for UniverSC tests * migrate UniverSC tests to calling open source Cell Ranger uses Cell Ranger 3.0.2 OS implementation (MIT License) tests passing locally * migrate changes to source code to updated UniverSC container * update unit tests for UniverSC to correct output (using new container) * update output criteria for UniverSC unit tests * change output directory for UniverSC unit tests * test adding podman to GitHub actions (will revert if reviewers object to it) * correct test errors for Cell Ranger OS tests (UniverSC module) * array format for test checks (UniverSC and Cell Ranger OS) * remove unncessary files from UniverSC module * remove podman from automated testing * remove mentions of nf-core/universc container * call executable script from PATH in UniverSC container * migrate UniverSC module Cell Ranger OS count own directory * reorganise UniverSC submodules * update process names in UniverSC module for consistency * update formatting for UniverSC tests * update unit tests for UniverSC submodules running Cell Ranger OS 3.0.2 * reorganise UniverSC submodules to fit naming conventions * remove stub from UniverSC for testing * add universc/mkfastq to unit tests * correct syntax in cellranger module files * update expected output for universc and universc/count tests now consistent with cellranger/count module * correct syntax for universc/count meta.yaml (pass linting) * add stub to universc and universc/count * update unit tests for universc module * update unit tests for universc module * update unit tests for universc module * update unit tests for universc module * update universc/mkfastq test to run stub * update universc/mkfastq expected outputs when running stub * remove trailing whitespace (linting error) * restructure UniverSC main module * update configuration to run each UniverSC test once only * correct UniverSC unit test configuration * update name of tools in universc tests configuration * update expected output for Cell Ranger and UniverSC tests * updates unit tests for Cell Ranger and UniverSC uses contain for web summary HTML as suggested by @apeltzer #1706 * updates unit tests for Cell Ranger for web summary HTML * updates unit tests for Cell Ranger for web summary HTML * updates unit tests for Cell Ranger for web summary HTML to use description * correct path to Cell Ranger test output * update container options for run UniverSC with singularity runs without root priviledges in writeable container * migrate UniverSC container to mirrored image at nfcore/universc:1.2.4 adds documentation for image build configuration discussed in #1706 * remove redundant submodules from UniverSC with functions already supported by Cell Ranger module #1706 * migrate UniverSC references to generate by Cell Ranger submodules * update test configuration for universc/launch * update expected outputs for UniverSC to use Cell Ranger references * update expected outputs for UniverSC to use Cell Ranger references * remove UniverSC submodules for mkref and mkgtf (already implement in Cell Ranger module) discussed in #1706 * move universc/launch submodule to universc module * remove tests for UniverSC submodules for mkref and mkgtf (already implement in Cell Ranger module) discussed in #1706 * move tests for universc/launch submodule to universc module * migrate universc/launch submodule to universc module * update paths in unit tests from universc/launch to universc * update documentation for UniverSC module * update paths in test config from universc/launch to universc * restore cellranger module (remove changes from PR 1706) * restore cellranger module (remove changes from PR 1706) * restore cellranger module (remove changes from PR 1706) * update style of documentation to pass linting * add podman to settings and docs (passes local test) * test podman configuration * test podman configuration * restore changes to testing (removes podman discussed in #2675) * restore changes to other modules (removes cellranger discussed in #2646) * update podman settings in UniverSC docs * update podman parameters * update container version for universc to stable release 1.2.5 * remove conda tests for universc (not supported) * update container version for universc to latest release 1.2.5.1 (run tests on pushed version on personal account) * update container version for universc to use nfcore/universc:1.2.5.1 mirror * exit logic for universc module that doesn't support conda consistent with other modules exit logic for modules that dont support conda #2657 * trigger GitHub Actions test for tomkellygenetics/universc:1.2.5.1 * add log files to universc output directory (confirm running subroutines as expected) * correct UniverSC test configuration addresses singularity test issue https://github.com/nf-core/modules/actions/runs/3955706571/jobs/6774566021 * update configuration for singularity in universc tests * test running universc with singularity --fakeroot requires shadow-uidmap::newuidmap installed * update configuration for singularity in universc tests * debug GH Actions configuration for singularity in universc tests * test running singularity with —fakeroot write permissions * test singularity— * revert changes to singularity tests disables singularity for universc (image too large) * update container settings for universc allows running rootless podman or singularity using --runtime crun or --writable-tmpfs apptainer/singularity#3220 * test universc with singularity --writable-tmpfs * revert changes to singularity tests (--writable-tmpfs not supported on GH Actions) * update container settings for universc to call nfcore/universc:1.2.5.1 (pending mirrored version available) * update version in UniverSC citation --------- Co-authored-by: Simon Thomas Kelly <simonthomas.kelly@hugp.com> Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com> Co-authored-by: TomKellyGenetics <tomkellygenetics@gmail> Co-authored-by: Alexander Peltzer <apeltzer@users.noreply.github.com>
sam-white04
pushed a commit
to fulcrumgenomics/nf-core-modules
that referenced
this pull request
Feb 27, 2023
* initialise template for new module: "universc" * update UniverSC module updates metadata, docker container source, licensing, and citations * update UniverSC module unit tests and documentation * update UniverSC module add inputs, outputs and example calls for UniverSC and Cell Ranger v3.0.2 calls versions for UniverSC and Cell Ranger * initialise template for new module: "universc" * update UniverSC module updates metadata, docker container source, licensing, and citations * update UniverSC module unit tests and documentation * update UniverSC module add inputs, outputs and example calls for UniverSC and Cell Ranger v3.0.2 calls versions for UniverSC and Cell Ranger * resolve formatting issues for UniverSC module * resolve linting errors for UniverSC module * fix test jobs to call UniverSC version without errors * correct configuration for UniverSC test jobs * correct linting errors for UniverSC module * correct docker build files for UniverSC * correct syntax errors in cellranger version call * prettier docs for UniverSC * add output to test data for UniverSC module * update UniverSC module to restructured repo nf-core#2141 remove files for restructed UniverSC module (avoids duplicate tests) * define separate outputs for Cell Ranger and UniverSC tests * remove TODO statements and update UniverSC meta.yml defines input and output variables and triggers automated tests * resolve minor linting issues with UniverSC * update paths to UniverSC module in test config * update container versions and tags for UniverSC module * simplifiy container configurations for UniverSC * update configuration for UniverSC tests (build Cell Ranger transcriptome reference first) * test UniverSC module with Cell Ranger references * update tests for UniverSC for restructured repository * update reference inputs for UniverSC module * set up references for cellranger OS test * resolve permissions errors for starting UniverSC * update input arguments for UniverSC and Cell Ranger OS (tests passing) * correct versions and checksums for UniverSC tests * resolves linting issues for UniverSC * resolves linting issues for UniverSC * update expected test outputs for UniverSC tests * migrate UniverSC tests to calling open source Cell Ranger uses Cell Ranger 3.0.2 OS implementation (MIT License) tests passing locally * migrate changes to source code to updated UniverSC container * update unit tests for UniverSC to correct output (using new container) * update output criteria for UniverSC unit tests * change output directory for UniverSC unit tests * test adding podman to GitHub actions (will revert if reviewers object to it) * correct test errors for Cell Ranger OS tests (UniverSC module) * array format for test checks (UniverSC and Cell Ranger OS) * remove unncessary files from UniverSC module * remove podman from automated testing * remove mentions of nf-core/universc container * call executable script from PATH in UniverSC container * migrate UniverSC module Cell Ranger OS count own directory * reorganise UniverSC submodules * update process names in UniverSC module for consistency * update formatting for UniverSC tests * update unit tests for UniverSC submodules running Cell Ranger OS 3.0.2 * reorganise UniverSC submodules to fit naming conventions * remove stub from UniverSC for testing * add universc/mkfastq to unit tests * correct syntax in cellranger module files * update expected output for universc and universc/count tests now consistent with cellranger/count module * correct syntax for universc/count meta.yaml (pass linting) * add stub to universc and universc/count * update unit tests for universc module * update unit tests for universc module * update unit tests for universc module * update unit tests for universc module * update universc/mkfastq test to run stub * update universc/mkfastq expected outputs when running stub * remove trailing whitespace (linting error) * restructure UniverSC main module * update configuration to run each UniverSC test once only * correct UniverSC unit test configuration * update name of tools in universc tests configuration * update expected output for Cell Ranger and UniverSC tests * updates unit tests for Cell Ranger and UniverSC uses contain for web summary HTML as suggested by @apeltzer nf-core#1706 * updates unit tests for Cell Ranger for web summary HTML * updates unit tests for Cell Ranger for web summary HTML * updates unit tests for Cell Ranger for web summary HTML to use description * correct path to Cell Ranger test output * update container options for run UniverSC with singularity runs without root priviledges in writeable container * migrate UniverSC container to mirrored image at nfcore/universc:1.2.4 adds documentation for image build configuration discussed in nf-core#1706 * remove redundant submodules from UniverSC with functions already supported by Cell Ranger module nf-core#1706 * migrate UniverSC references to generate by Cell Ranger submodules * update test configuration for universc/launch * update expected outputs for UniverSC to use Cell Ranger references * update expected outputs for UniverSC to use Cell Ranger references * remove UniverSC submodules for mkref and mkgtf (already implement in Cell Ranger module) discussed in nf-core#1706 * move universc/launch submodule to universc module * remove tests for UniverSC submodules for mkref and mkgtf (already implement in Cell Ranger module) discussed in nf-core#1706 * move tests for universc/launch submodule to universc module * migrate universc/launch submodule to universc module * update paths in unit tests from universc/launch to universc * update documentation for UniverSC module * update paths in test config from universc/launch to universc * restore cellranger module (remove changes from PR 1706) * restore cellranger module (remove changes from PR 1706) * restore cellranger module (remove changes from PR 1706) * update style of documentation to pass linting * add podman to settings and docs (passes local test) * test podman configuration * test podman configuration * restore changes to testing (removes podman discussed in nf-core#2675) * restore changes to other modules (removes cellranger discussed in nf-core#2646) * update podman settings in UniverSC docs * update podman parameters * update container version for universc to stable release 1.2.5 * remove conda tests for universc (not supported) * update container version for universc to latest release 1.2.5.1 (run tests on pushed version on personal account) * update container version for universc to use nfcore/universc:1.2.5.1 mirror * exit logic for universc module that doesn't support conda consistent with other modules exit logic for modules that dont support conda nf-core#2657 * trigger GitHub Actions test for tomkellygenetics/universc:1.2.5.1 * add log files to universc output directory (confirm running subroutines as expected) * correct UniverSC test configuration addresses singularity test issue https://github.com/nf-core/modules/actions/runs/3955706571/jobs/6774566021 * update configuration for singularity in universc tests * test running universc with singularity --fakeroot requires shadow-uidmap::newuidmap installed * update configuration for singularity in universc tests * debug GH Actions configuration for singularity in universc tests * test running singularity with —fakeroot write permissions * test singularity— * revert changes to singularity tests disables singularity for universc (image too large) * update container settings for universc allows running rootless podman or singularity using --runtime crun or --writable-tmpfs apptainer/singularity#3220 * test universc with singularity --writable-tmpfs * revert changes to singularity tests (--writable-tmpfs not supported on GH Actions) * update container settings for universc to call nfcore/universc:1.2.5.1 (pending mirrored version available) * update version in UniverSC citation --------- Co-authored-by: Simon Thomas Kelly <simonthomas.kelly@hugp.com> Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com> Co-authored-by: TomKellyGenetics <tomkellygenetics@gmail> Co-authored-by: Alexander Peltzer <apeltzer@users.noreply.github.com>
anoronh4
pushed a commit
to anoronh4/add_submodule
that referenced
this pull request
Apr 27, 2023
* initialise template for new module: "universc" * update UniverSC module updates metadata, docker container source, licensing, and citations * update UniverSC module unit tests and documentation * update UniverSC module add inputs, outputs and example calls for UniverSC and Cell Ranger v3.0.2 calls versions for UniverSC and Cell Ranger * initialise template for new module: "universc" * update UniverSC module updates metadata, docker container source, licensing, and citations * update UniverSC module unit tests and documentation * update UniverSC module add inputs, outputs and example calls for UniverSC and Cell Ranger v3.0.2 calls versions for UniverSC and Cell Ranger * resolve formatting issues for UniverSC module * resolve linting errors for UniverSC module * fix test jobs to call UniverSC version without errors * correct configuration for UniverSC test jobs * correct linting errors for UniverSC module * correct docker build files for UniverSC * correct syntax errors in cellranger version call * prettier docs for UniverSC * add output to test data for UniverSC module * update UniverSC module to restructured repo nf-core/modules#2141 remove files for restructed UniverSC module (avoids duplicate tests) * define separate outputs for Cell Ranger and UniverSC tests * remove TODO statements and update UniverSC meta.yml defines input and output variables and triggers automated tests * resolve minor linting issues with UniverSC * update paths to UniverSC module in test config * update container versions and tags for UniverSC module * simplifiy container configurations for UniverSC * update configuration for UniverSC tests (build Cell Ranger transcriptome reference first) * test UniverSC module with Cell Ranger references * update tests for UniverSC for restructured repository * update reference inputs for UniverSC module * set up references for cellranger OS test * resolve permissions errors for starting UniverSC * update input arguments for UniverSC and Cell Ranger OS (tests passing) * correct versions and checksums for UniverSC tests * resolves linting issues for UniverSC * resolves linting issues for UniverSC * update expected test outputs for UniverSC tests * migrate UniverSC tests to calling open source Cell Ranger uses Cell Ranger 3.0.2 OS implementation (MIT License) tests passing locally * migrate changes to source code to updated UniverSC container * update unit tests for UniverSC to correct output (using new container) * update output criteria for UniverSC unit tests * change output directory for UniverSC unit tests * test adding podman to GitHub actions (will revert if reviewers object to it) * correct test errors for Cell Ranger OS tests (UniverSC module) * array format for test checks (UniverSC and Cell Ranger OS) * remove unncessary files from UniverSC module * remove podman from automated testing * remove mentions of nf-core/universc container * call executable script from PATH in UniverSC container * migrate UniverSC module Cell Ranger OS count own directory * reorganise UniverSC submodules * update process names in UniverSC module for consistency * update formatting for UniverSC tests * update unit tests for UniverSC submodules running Cell Ranger OS 3.0.2 * reorganise UniverSC submodules to fit naming conventions * remove stub from UniverSC for testing * add universc/mkfastq to unit tests * correct syntax in cellranger module files * update expected output for universc and universc/count tests now consistent with cellranger/count module * correct syntax for universc/count meta.yaml (pass linting) * add stub to universc and universc/count * update unit tests for universc module * update unit tests for universc module * update unit tests for universc module * update unit tests for universc module * update universc/mkfastq test to run stub * update universc/mkfastq expected outputs when running stub * remove trailing whitespace (linting error) * restructure UniverSC main module * update configuration to run each UniverSC test once only * correct UniverSC unit test configuration * update name of tools in universc tests configuration * update expected output for Cell Ranger and UniverSC tests * updates unit tests for Cell Ranger and UniverSC uses contain for web summary HTML as suggested by @apeltzer nf-core/modules#1706 * updates unit tests for Cell Ranger for web summary HTML * updates unit tests for Cell Ranger for web summary HTML * updates unit tests for Cell Ranger for web summary HTML to use description * correct path to Cell Ranger test output * update container options for run UniverSC with singularity runs without root priviledges in writeable container * migrate UniverSC container to mirrored image at nfcore/universc:1.2.4 adds documentation for image build configuration discussed in nf-core/modules#1706 * remove redundant submodules from UniverSC with functions already supported by Cell Ranger module nf-core/modules#1706 * migrate UniverSC references to generate by Cell Ranger submodules * update test configuration for universc/launch * update expected outputs for UniverSC to use Cell Ranger references * update expected outputs for UniverSC to use Cell Ranger references * remove UniverSC submodules for mkref and mkgtf (already implement in Cell Ranger module) discussed in nf-core/modules#1706 * move universc/launch submodule to universc module * remove tests for UniverSC submodules for mkref and mkgtf (already implement in Cell Ranger module) discussed in nf-core/modules#1706 * move tests for universc/launch submodule to universc module * migrate universc/launch submodule to universc module * update paths in unit tests from universc/launch to universc * update documentation for UniverSC module * update paths in test config from universc/launch to universc * restore cellranger module (remove changes from PR 1706) * restore cellranger module (remove changes from PR 1706) * restore cellranger module (remove changes from PR 1706) * update style of documentation to pass linting * add podman to settings and docs (passes local test) * test podman configuration * test podman configuration * restore changes to testing (removes podman discussed in nf-core/modules#2675) * restore changes to other modules (removes cellranger discussed in nf-core/modules#2646) * update podman settings in UniverSC docs * update podman parameters * update container version for universc to stable release 1.2.5 * remove conda tests for universc (not supported) * update container version for universc to latest release 1.2.5.1 (run tests on pushed version on personal account) * update container version for universc to use nfcore/universc:1.2.5.1 mirror * exit logic for universc module that doesn't support conda consistent with other modules exit logic for modules that dont support conda nf-core/modules#2657 * trigger GitHub Actions test for tomkellygenetics/universc:1.2.5.1 * add log files to universc output directory (confirm running subroutines as expected) * correct UniverSC test configuration addresses singularity test issue https://github.com/nf-core/modules/actions/runs/3955706571/jobs/6774566021 * update configuration for singularity in universc tests * test running universc with singularity --fakeroot requires shadow-uidmap::newuidmap installed * update configuration for singularity in universc tests * debug GH Actions configuration for singularity in universc tests * test running singularity with —fakeroot write permissions * test singularity— * revert changes to singularity tests disables singularity for universc (image too large) * update container settings for universc allows running rootless podman or singularity using --runtime crun or --writable-tmpfs apptainer/singularity#3220 * test universc with singularity --writable-tmpfs * revert changes to singularity tests (--writable-tmpfs not supported on GH Actions) * update container settings for universc to call nfcore/universc:1.2.5.1 (pending mirrored version available) * update version in UniverSC citation --------- Co-authored-by: Simon Thomas Kelly <simonthomas.kelly@hugp.com> Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com> Co-authored-by: TomKellyGenetics <tomkellygenetics@gmail> Co-authored-by: Alexander Peltzer <apeltzer@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes discussed here #1706
Updates unit tests to match UniverSC results.
PR checklist
Closes #XXX
versions.ymlfile.labelPROFILE=docker pytest --tag <MODULE> --symlink --keep-workflow-wd --git-awarePROFILE=singularity pytest --tag <MODULE> --symlink --keep-workflow-wd --git-awarePROFILE=conda pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware