Skip to content

Latest commit

 

History

History
337 lines (220 loc) · 13.5 KB

File metadata and controls

337 lines (220 loc) · 13.5 KB

Contributing

Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given. A detailed description of the Snakemake codebase and architecture can be found :ref:`here <codebase_intro>`.

You can contribute in many ways:

Types of contributions

Report bugs or suggest enhancements

Report bugs or suggest enhancements at https://github.com/snakemake/snakemake/issues

If you are reporting a bug, follow the template and fill out the requested information, including:

  • Your operating system name and version.
  • Any details about your local setup that might be helpful in troubleshooting.
  • Detailed steps to reproduce the bug and ideally a test case.

If you are proposing am enhancement or a new feature:

  • Explain in detail how it would work.
  • Keep the scope as narrow as possible, to make it easier to implement.
  • Remember that this is a volunteer-driven project, and that contributions are welcome :).

Fix Bugs

Look through the Github issues for bugs. If you want to start working on a bug then please write short message on the issue tracker to prevent duplicate work.

Implement Features

Look through the Github issues for feature/enhancement requests. If you want to start working on an issue then please write short message on the issue tracker to prevent duplicate work.

Contributing a plugin

Currently, Snakemake supports executor plugins, storage plugins, and report plugins. The Snakemake plugin catalog shows which plugins are available and how to contribute new ones.

Write Documentation

Snakemake could always use more documentation, whether as part of the official docs, in docstrings, or even on the web in blog posts, articles, and such.

Snakemake uses Sphinx for the user manual (that you are currently reading). See :ref:`project_info-doc_guidelines` on how the reStructuredText is used for the documentation.

Pull Request Guidelines

To update the documentation, fix bugs or add new features you need to create a Pull Request . A PR is a change you make to your local copy of the code for us to review and potentially integrate into the code base.

To create a Pull Request you need to do these steps:

  1. Create a Github account.
  2. Fork the repository.
  3. Clone your fork locally.
  4. Go to the created snakemake folder with cd snakemake.
  5. Create a new branch with git checkout -b <descriptive_branch_name>.
  6. Make your changes to the code or documentation.
  7. Format your changes with Snakefmt or BLACK.
  8. Run git add . to add all the changed files to the commit (to see what files will be added you can run git add . --dry-run).
  9. To commit the added files use git commit. (This will open a command line editor to write a commit message. These should have a descriptive 80 line header, followed by an empty line, and then a description of what you did and why. To use your command line text editor of choice use (for example) export GIT_EDITOR=vim before running git commit).
  10. Now you can push your changes to your Github copy of Snakemake by running git push origin <descriptive_branch_name>.
  11. If you now go to the webpage for your Github copy of Snakemake you should see a link in the sidebar called "Create Pull Request".
  12. Now you need to choose your PR from the menu and click the "Create pull request" button. Be sure to change the pull request target branch to <descriptive_branch_name>!

If you want to create more pull requests, first run git checkout main and then start at step 5. with a new branch name.

Feel free to ask questions about this if you want to contribute to Snakemake :)

AI-assisted contributions

We welcome the use of AI tools as part of the development process. They can be valuable aids for drafting, refactoring, documentation, and exploration. However, contributions to Snakemake require human judgment, contextual understanding, and familiarity with the project's structure, goals, and standards.

When using AI tools:

  • You remain the author of the contribution. Review, understand, and test all AI-assisted code or documentation before submitting it under your name. You should be able to explain and defend the changes on request.
  • Avoid fully automated submissions. Issues or pull requests generated end-to-end by automated tools, without meaningful human review or intent, are not appropriate.
  • Be respectful of reviewers' time. Ensure that both the content of the PR and its description reflect your own understanding. Reviewers should not be expected to infer authorship or unknowingly interact with an AI during review.
  • Disclose significant AI assistance. If AI tools were used for a substantial portion of the contribution, please note this in the PR description.

Contributors are responsible for the correctness, maintainability, and long-term impact of all submitted changes, regardless of whether AI tools were used.

Pull requests that do not meet these expectations may be closed without review.

Development with pixi

pixi is a tool that is designed to help you manage your development environment. It acts as a drop-in replacement for conda.

After having cloned the Snakemake repository and changed to the directory, use the following command to get pixi set up for Snakemake development:

$ pixi install

This will create a new environment with all the dependencies needed for development. Then you can make changes and run e.g.

$ pixi run snakemake --help

Testing Guidelines

To ensure that you do not introduce bugs into Snakemake, you should test your code thoroughly.

Putting these tests into repeatable test cases ensures they can be checked on multiple platforms and Python versions.

Continuous integration

For any pull request, all tests are automatically executed within Github Actions, providing feedback to you and the official development team whether the proposed changes are working as expected and do not hamper other functionality Snakemake provides. However, it is useful to be able to run the tests locally, thereby being able to quickly debug any occurring failures.

Setup to run the test suite locally

Unit tests and regression tests are written to be run by pytest.

Testing Guide using pixi

Prerequisites: Make sure you have pixi installed: See :ref:`pixi-getting_started`.

Activate your environment:

There are a few environments you can use to run the tests.

The dev environment is most useful for overall development. This environment will also install the docs and style features which will allow you to also build documentation and run black.

$ pixi shell -e dev

The py311 and py312 environments are what are used in the CI tests which isolate the Python version and the test dependencies. Use this if you want to test your code against the same environment as any failing CI tests.

$ pixi shell -e py311

Run a comprehensive, simple, or single test: The test suite defines two types of tests via pixi tasks that you can run:

test-all: This task runs the comprehensive test suite, which includes most of the tests in the tests/ directory.

$ pixi run test-all

test-simple: This task runs the main tests located in tests/tests.py.

$ pixi run test-simple

Single test: You can also run a single test by using pytest directly with the test file and the test name.

$ pixi run pytest tests/tests.py::test_log_input

Tip

This test suite is quite long, and can be run in parts similar to the CI/CD tests which run it in 1/10 parts.

To do so, you can use the --splits and --group flags to run a subset of the tests. For example, to run the first group of tests in a 10 part split:

$ pixi run test-simple \
    --splits 10 \
    --group 1 \
    --splitting-algorithm=least_duration

Marked tests

Some tests have been marked using pytest markers. These allow for running specific tests or excluding specific tests. For example, the pixi run test-simple currently excludes the needs_envmodules tests. There is also another marker for needs_s3 which will skip tests that require an S3 connection. If you are not looking to test the S3 functionality, you can modify the test command to exclude these tests.

$ pixi run test-simple -m "not needs_envmodules and not needs_s3"

For a full list of available markers, you can run:

$ pixi run pytest --markers

Warnings and oddities

You will likely see warnings related to deprecated functions in dependent libraries, especially botocore.

You may also get intermittent failures from tests that rely on external connectivity. The default test suite makes connections to multiple external services.

Tests that require singularity will be auto-skipped if no singularity or apptainer installation is available. At the time of writing neither the singularity package on conda-forge nor the apptainer package are reliable, in that there are multiple failing tests on a standard Ubuntu system. This is likely due to system security profiles that conda, being a non-root application, cannot change. The Debian/Ubuntu singularity-container DEB package, which must be installed by the system administrator, does work. The equivalent RPM package should also work on RedHat-type systems.

Depending on how the Snakemake code was downloaded and installed in the test environment, Snakemake may not be able to determine its own version and may think that it is version 0. The existing unit tests should all cope with this, and in general you should avoid writing tests that rely on explicit version checks.

Documentation Guidelines

The documentation uses Sphinx and is written in reStructuredText. For details on the syntax, see the Sphinx primer on reStructuredText and the Sphinx documentation on cross-references.

For the documentation, please adhere to the following guidelines:

  • Put each sentence on its own line, this makes tracking changes through Git SCM easier.
  • Provide hyperlink targets, at least for the first two section levels. For this, use the format <document_part>-<section_name>, for example project_info-doc_guidelines for the current section. Set the hyperlink target right above the section heading with .. _project_info-doc_guidelines:. Reference the hyperlink (i.e. link to it) with :ref:`project_info-doc_guidelines`.
  • Use the section structure recommended by Sphinx, which references the recommendations in the Python Developer's Guide. Namely, the levels are:
.. document_part-section_heading:

===============
Section heading
===============


.. document_part-subsection_heading:

Subsection heading
------------------

.. document_part-subsubsection_heading:

Subsubsection heading
^^^^^^^^^^^^^^^^^^^^^

.. document_part-paragraph_heading:

Paragraph heading
"""""""""""""""""

Documentation Setup

To get started, make sure you have pixi installed: See :ref:`pixi-getting_started`. We use pixi to manage the docs environment and tasks to streamline the developer experience.

$ ➜ pixi task list --environment docs
Tasks that can run on this machine:
-----------------------------------
build-apidocs, build-docs, docs

- build-apidocs   Build the API documentation in the apidocs/ directory
- build-docs      Build the documentation in the docs/ directory
- docs            Serve the documentation on http://localhost:8000 with live reload

Test if the docs build: To only build the documentation, you can use the build-docs task.

$ pixi run build-docs

Live server with auto-reload: To serve the documentation on a local server with live reload, use the docs task.

$ pixi run docs
[sphinx-autobuild] Starting initial build
[sphinx-autobuild] > python -m sphinx build docs/ docs/_build/html
...
The HTML pages are in docs/_build/html.
[sphinx-autobuild] Serving on http://0.0.0.0:8000
[sphinx-autobuild] Waiting to detect changes...