chore: add pypi release by kemingy · Pull Request #1931 · lycheeverse/lychee

kemingy · 2025-11-21T16:12:28Z

Add PyPI release, as listed in ❤️ Help publish lychee for common platforms #59

This is blocked by the rustls issue, tracked in #1721

❯ cargo tree --no-default-features --features rustls-tls --features email-check -i openssl
openssl v0.10.73
└── native-tls v0.2.14
    ├── async-native-tls v0.4.0
    │   ├── async-smtp v0.6.0
    │   │   └── check-if-email-exists v0.9.1
    │   │       └── lychee-lib v0.21.0 (/home/user/GitHub/lychee/lychee-lib)
    │   │           ├── archive v0.1.0 (/home/user/GitHub/lychee/examples/archive)
    │   │           ├── benches v0.0.0 (/home/user/GitHub/lychee/benches)
    │   │           ├── builder v0.1.0 (/home/user/GitHub/lychee/examples/builder)
    │   │           ├── chain v0.1.0 (/home/user/GitHub/lychee/examples/chain)
    │   │           ├── client_pool v0.1.0 (/home/user/GitHub/lychee/examples/client_pool)
    │   │           ├── collect_links v0.1.0 (/home/user/GitHub/lychee/examples/collect_links)
    │   │           ├── extract v0.1.0 (/home/user/GitHub/lychee/examples/extract)
    │   │           ├── lychee v0.21.0 (/home/user/GitHub/lychee/lychee-bin)
    │   │           └── simple v0.1.0 (/home/user/GitHub/lychee/examples/simple)
    │   └── check-if-email-exists v0.9.1 (*)
    ├── hyper-tls v0.5.0
    │   └── reqwest v0.11.27
    │       ├── check-if-email-exists v0.9.1 (*)
    │       └── pwned v0.5.0
    │           └── check-if-email-exists v0.9.1 (*)
    ├── hyper-tls v0.6.0
    │   └── reqwest v0.12.24
    │       ├── builder v0.1.0 (/home/user/GitHub/lychee/examples/builder)
    │       ├── chain v0.1.0 (/home/user/GitHub/lychee/examples/chain)
    │       ├── collect_links v0.1.0 (/home/user/GitHub/lychee/examples/collect_links)
    │       ├── lychee v0.21.0 (/home/user/GitHub/lychee/lychee-bin)
    │       ├── lychee-lib v0.21.0 (/home/user/GitHub/lychee/lychee-lib) (*)
    │       └── reqwest_cookie_store v0.9.0
    │           ├── lychee v0.21.0 (/home/user/GitHub/lychee/lychee-bin)
    │           └── lychee-lib v0.21.0 (/home/user/GitHub/lychee/lychee-lib) (*)
    ├── reqwest v0.11.27 (*)
    ├── reqwest v0.12.24 (*)
    └── tokio-native-tls v0.3.1
        ├── hyper-tls v0.5.0 (*)
        ├── hyper-tls v0.6.0 (*)
        ├── reqwest v0.11.27 (*)
        └── reqwest v0.12.24 (*)

Signed-off-by: Keming <kemingy94@gmail.com>

thomas-zahner · 2025-11-24T10:36:27Z

@kemingy Ah that's great, thanks!

So is there a linker error when using openSSL? I've started a PR to switch from openSSL to rustls. When we switch, we will also drop the rustls-tls feature as it will become either the default or maybe even the only option, dropping openSSL support.

Do you have any test runs of the workflow? Does it work already? Do you have examples on how to install and then use the package with Python?

In Publish to PyPI there is secrets.PYPI_API_TOKEN do we need to create and add such a token before we are ready to merge? And what is the name/key of the published artifact? Is it lychee-cli from the pyproject.toml?

kemingy · 2025-11-24T11:05:01Z

Hi @thomas-zahner,

So is there a linker error when using openSSL? I've #1928 to switch from openSSL to rustls. When we switch, we will also drop the rustls-tls feature as it will become either the default or maybe even the only option, dropping openSSL support.

I have tried the release on https://github.com/kemingy/lychee/actions/runs/19575668669. It failed because of the OpenSSL. This can be fixed by installing the libssl-dev, I suppose (haven't tried). I prefer not to edit the yaml file since it's auto-generated by maturin. Since we already have the plan to make rustls as the default option.

As a bin type release, pip install lychee-cli (this name TBD) will place the binary (lychee) to local bin (very likely ~/.local/bin, but depends on which pip is used).

In Publish to PyPI there is secrets.PYPI_API_TOKEN do we need to create and add such a token before we are ready to merge? And what is the name/key of the published artifact? Is it lychee-cli from the pyproject.toml?

The secret secrets.PYPI_API_TOKEN is not mandatory. I would recommend to use OIDC, then we don't need the secret.

We can discuss about the name lychee-cli, as the name lychee has been taken years ago. You can verify the name from https://pypi.org/project/

mre · 2025-11-24T11:44:23Z

lychee-cli seems fine. An alternative name might be lychee-bin, which is in alignment with the crate name. It has the added advantage that lychee-lib could be used as the canonical library name if we ever decide to release an "official" one. There already is https://github.com/jb--/lychpy, which we could adopt as well.

Signed-off-by: Keming <kemingy94@gmail.com>

kemingy · 2025-11-25T08:45:22Z

I have changed the name to lychee-bin. Tried to install OpenSSL to all the runners, but it's pretty troublesome. I would prefer to wait for the usable rustls feature :)

BTW, I have deleted the secret, so it will use the OIDC by default. Please add a trusted publisher according to the https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/

I'll change this to a draft until the rustls feature is ready.

thomas-zahner · 2025-11-25T09:14:41Z

Thanks that sounds sensible 👍

- rm sccache since maturin-action is using the legacy pypi sccache Signed-off-by: Keming <kemingy94@gmail.com>

Signed-off-by: Keming <kemingy94@gmail.com>

kemingy · 2026-02-11T09:39:11Z

Hi, since we now use the rustls by default, the PyPI release is much simpler. I have tested the action on my fork: https://github.com/kemingy/lychee/actions/runs/21899505565

thomas-zahner · 2026-02-11T16:39:01Z

+          subject-path: "wheels-*/*"
+      - name: Publish to PyPI
+        if: ${{ startsWith(github.ref, 'refs/tags/') }}
+        uses: PyO3/maturin-action@v1


Where does the published artifact end up?

How does authentication work? Why can't anybody else push the same artifact and override ours? If I run maturin generate-ci github > .github/workflows/pypi.yml I get the additional lines below. (though I also get older version such as actions/upload-artifact@v4)

+ env: + MATURIN_PYPI_TOKEN: ${{ secrets.PYPI_API_TOKEN }}

I need to delete the TOKEN to enable the PyPI OIDC. The way it verify is from the GitHub repo and workflow path configured in the PyPI. You can follow the instructions here: https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/

Ah I see, thank you. So did you follow these steps already? In other words, are we ready to merge and the workflow and deployment should be fully functional? Or should I/we follow these steps?

I think mre or you should be the one that control the PyPI package of lychee-bin. Or maybe there should be a lychee org on the PyPI. I can help maintain the PyPI part.

So as long as we have setup the OIDC in the PyPI, we are able to publish lychee on it. This PR has done everything required from the code side (if I didn't miss anything).

I've now requested the organisation lycheeverse on PyPI. I feel like you are the expert on this topic so I'd be glad if you could support us. In general we (the main lychee maintainers) don't really maintain the downstream packages. But we are of course responsible for the code in this repo and as the PyPI package seems to be fully generated from this workflow file we become responsible. So it seems like an organisation with you, Matthias and me makes sense.

I don't really understand yet how the new PyPI package would be used by users. Are bindings generated automatically for usage in Python code? What does the usage of this package look like? Could you create a minimal example repo using the new PyPI package? Is this possible before we release it?

Any updates from the org?

If there is still no update from the pypi for the organization, maybe I can add this to my pypi account, and invite the org as co-owner later?

The org was created some time ago now: https://pypi.org/org/lycheeverse/

@kemingy @mre What are your PyPI account names? I'd like to add you to the organisation.

I've now tried to add a new pending publisher over at https://pypi.org/manage/organization/lycheeverse/publishing/

Unfortunately without success. I'm getting This publisher has already been registered in your organization. See your existing pending publishers below. even though the same page tells me No pending publishers are currently configured. Publishers for projects that don't exist yet can be added below.. This really makes no sense as far as I can tell.

Below is the form I've filled.

@kemingy Maybe you will have more luck trying to add the project? Or maybe this is a bug?

Sorry, my bad, I added the project yesterday. I have removed it from my account. You shall be able to add it now @thomas-zahner. BTW my pypi account name is keming.

Oh I see, thanks! It worked!

thomas-zahner

@kemingy Thanks for your efforts! Lets see if this works 🤞

thomas-zahner · 2026-05-01T16:53:05Z

The release on PyPI worked with v0.24.2 🎉
See https://pypi.org/project/lychee-bin/

However uv tool install lychee-bin doesn't seem to work on my NixOS machine.
Could you verify if this works?

kemingy · 2026-05-03T08:29:43Z

The release on PyPI worked with v0.24.2 🎉 See https://pypi.org/project/lychee-bin/

However uv tool install lychee-bin doesn't seem to work on my NixOS machine. Could you verify if this works?

Hi @thomas-zahner I have tested on Ubuntu22.04 (x86-64) & macOS 26 (arm64), both works. But I saw the size is strange here https://pypi.org/project/lychee-bin/#files. macOS and Windows releases are normal size but all the Linux releases are huge. I think it's because the debug info is not stripped in release

lychee/Cargo.toml

Lines 8 to 9 in fabdcec

    
           [profile.release] 
        
           debug = true

What do you think? Shall we remove the debug info or just keep it there.

mre · 2026-05-03T18:17:56Z

Yeah, let's remove it. Would you like to create a PR?

kemingy · 2026-05-04T00:02:31Z

Yeah, let's remove it. Would you like to create a PR?

Sure! #2184

kemingy added 2 commits November 21, 2025 23:27

chore: add pypi release

edb0659

Signed-off-by: Keming <kemingy94@gmail.com>

use rustls to avoid openssl link issue

baf2e9c

Signed-off-by: Keming <kemingy94@gmail.com>

install libssl-dev, bump action version generated by maturin

ffa4c3b

Signed-off-by: Keming <kemingy94@gmail.com>

kemingy marked this pull request as draft November 25, 2025 08:45

kemingy added 3 commits February 10, 2026 17:48

Merge branch 'master' into pypi

5ec6a72

- rm openssl

af49105

- rm sccache since maturin-action is using the legacy pypi sccache Signed-off-by: Keming <kemingy94@gmail.com>

fix the aws-lc-sys compile error on aarch64

7481676

Signed-off-by: Keming <kemingy94@gmail.com>

kemingy marked this pull request as ready for review February 11, 2026 09:38

thomas-zahner reviewed Feb 11, 2026

View reviewed changes

cconverse711 mentioned this pull request Feb 20, 2026

pre-commit hook: use pre-commit's built-in rust boostrapping #2057

Closed

Merge branch 'lycheeverse:master' into pypi

f250b96

thomas-zahner approved these changes Apr 30, 2026

View reviewed changes

thomas-zahner merged commit 9d3b7ab into lycheeverse:master Apr 30, 2026
18 of 20 checks passed

kemingy deleted the pypi branch May 3, 2026 08:09

kemingy mentioned this pull request May 4, 2026

chore: strip the release debug info #2184

Merged

Uh oh!

Conversation

kemingy commented Nov 21, 2025

Uh oh!

thomas-zahner commented Nov 24, 2025

Uh oh!

kemingy commented Nov 24, 2025

Uh oh!

mre commented Nov 24, 2025

Uh oh!

kemingy commented Nov 25, 2025

Uh oh!

thomas-zahner commented Nov 25, 2025

Uh oh!

kemingy commented Feb 11, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thomas-zahner Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kemingy Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thomas-zahner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

thomas-zahner commented May 1, 2026

Uh oh!

kemingy commented May 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mre commented May 3, 2026

Uh oh!

kemingy commented May 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

thomas-zahner Feb 12, 2026 •

edited

Loading

kemingy Feb 12, 2026 •

edited

Loading

kemingy commented May 3, 2026 •

edited

Loading