I created an API token for the ics.py project at test.pypi.org, but I can't add it to this project's secrets on GitHub as I have no access to its settings. I can also add you on test.pypi so that I'm no longer the sole owner - do you already have an account there?

It seems like i can't give you access as the repo is under my personal account. (It might be time to push the move to a dedicated org 👀) I do have an test account with the same username: https://test.pypi.org/user/C4ptainCrunch/

So, i have an action that kind of pushed to test pypi. But it fails as the version already exists.
I'll have to find a way to append the commit hash to the version name for the test pypi.

What is also missing:

• Only push if the test passes
• Double check the if: startsWith(github.ref, 'refs/tags') is correct
• Un comment and fix the production upload

Codecov Report

Merging #274 (1e749da) into main (c171552) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #274   +/-   ##
=======================================
  Coverage   81.28%   81.28%           
=======================================
  Files          30       30           
  Lines        2885     2885           
=======================================
  Hits         2345     2345           
  Misses        540      540           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c171552...1e749da. Read the comment docs.

This approach is stuck at the moment as we try to publish 0.8.0-dev multiple times on (dev)pypi and publishing the same version multiple times is prohibited.

We could try to include the commit hash in the version but this has multiple problems:

• PEP440 says "While they may be useful for continuous integration purposes, publishing developmental releases of pre-releases to general purpose public index servers is strongly discouraged, as it makes the version identifier difficult to parse for human readers."
• PEP440 also says "Many build tools integrate with distributed version control systems like Git and Mercurial in order to add an identifying hash to the version identifier. As hashes cannot be ordered reliably such versions are not permitted in the public version field."

So i guess we should abandon the idea to publish every commit of main to devpi.

What i'm tempted to do is:

1. Change this PR to trigger the upload with GH actions only on tags on main
2. Merge this PR while keeping the upload to devpypi
3. Release with a tag 0.8.0-dev1 and test the automatic upload on devpypi
4. If 3. worked, change the endpoint to production pypi
5. (Maybe) Publish 0.8.0-dev2 and see it it appears correctly on pypi ?
6. (Later, when ready) Release with a tag 0.8.0 and it should be uploaded magically

What do you think ?

@C4ptainCrunch I have often seen 0.8.0-aplha.x, 0.8.0-beta.x, and 0.8.0-rc.x being used in the wild.

I think 0.8.0dev0 etc. is pretty common. See, e.g., https://pypi.org/project/yaml2ics/, which is based on ics-py :) @tupui You've done this recently, any advice?

What do you think ?

Sounds good. Note that you can also delete (and not only "yank") releases on PyPi, so you could also get rid again of that dev2 "release".

So i guess we should abandon the idea to publish every commit of main to devpi.

Instead of per-commit preview releases, we could adopt nightly builds that are published to the test PyPi once per day using the date as monotonically increasing index, e.g. 0.8.0-dev20220304. One could also include a check to only publish a nightly build if anything changed in the last 24 hours.

My 2 cents after having deployed a few things both on internal PyPi and normal.

There is very little value to publish a version nightly on dev PyPi:

• You would have a complex/lengthy build process then yes totally, ex SciPy/NumPy and other such projects provide nightly/weekly wheels so that you can test without having the need to build yourself (which is hard and prone to bugs which you don't want to have to care about). In case of a pure python, there is just no benefit vs pulling from GH (users of dev versions are not the same public).
• It may also be more difficult to link a particular version with the code if you don't make a commit/tag.
• Most importantly it adds strain on PyPi's infrastructure.

For the release itself, I tend to prefer a semi automatic way with locally using bump2version (if you need to update the version in other places than pyproject.toml) and then commit the change with its tag. Then I have an action which do the release based on the tag. Which OC is the same as bumping the version with a commit and then doing a release on GH.

Point is, IMHO releases should require a manual interaction as you can potentially break all packages depending on your package. And you should have a separation of credentials on PyPi vs GH.

To add on to what tupui said which I agree with -- I'm relatively new here, but my observation is I would suggest to keep it simple: This project has had zero new feature commits in the last 20+ days (just a patch release and test improvement). The tension here is because there is a lot wrapped up in 8.0, thus a large desire to have a pypi development release.

My suggestion based on how most other python projects work is (1) to significantly cut as much scope as possible to get 8.0 out and (2) to lean into semvar and get comfortable with major version bumps indicating breaking changes, and very frequent releases of minor and patch releases. Then in the future, a major rewrite can happen in a branch and be pulled into the main line when it's ready to get on the release train, and use a major version bump when there are breaking changes.

Let's try this again, with a much less complicated route:

• On every PR, every commit on main and every tag, we run python -m build (we want to be sure it's never broken)
• If it's a tag that starts with v*, after the build, upload to Pypi.

=> No more fidling with automatic version names, bump2version, ... only the core "job to be done": being able to reproducibly build and upload packages in an isolated environment not depending on my laptop.

Notes:

• This depends on Move build system from poetry to hatch #354, there is no point in depending on poetry anymore
• Right now, it publishes to test PyPI, when this is merged, i'll release a tag and check if it really works as intended. If it does, than i'll change the token to publish on the real PyPI on the subsequent tags.

Of course, the only thing that we could not test in the PR (the actual release workflow) does not work when building a tag.

What was wrong: The artifacts from the build were not available in the upload step. I'm just building in both steps right now. It's not clean, but building twice for each tag is not a huge problem anyway.

What is still not working: Something in the pypa/gh-action-pypi-publish action refuses the upload with the following error long_description has syntax errors in markup and would not be rendered on PyPI.
Fixing the error is trivial but i'd like to first find what tool gives the error so that we can run it before the publishing step and have a heads up (If the tests are green, creating a tag should be guaranteed to upload to pypi).
It seems like it's the twine check on L35 that does this but on my laptop, it does not fail or show warnings so i'll have to dig a bit more.

It works 🎉 https://pypi.org/project/ics/0.8.0.dev0/