Skip to content

lint: modernise lint tooling#34547

Open
willcl-ark wants to merge 9 commits intobitcoin:masterfrom
willcl-ark:modernise-linter
Open

lint: modernise lint tooling#34547
willcl-ark wants to merge 9 commits intobitcoin:masterfrom
willcl-ark:modernise-linter

Conversation

@willcl-ark
Copy link
Member

@willcl-ark willcl-ark commented Feb 10, 2026

Modernise our lint tooling by:

- Replacing pyenv + pip with uv for better Python environment and dependency management
- Replacing mypy with https://github.com/astral-sh/ty
- Replacing the 01_install.sh runtime install script with COPY --from multi-stage Docker image imports for uv, ruff, shellcheck, mlc, and ty
- Moving ruff lint rules from hardcoded Rust array (in lint_py.rs) into a top-level ruff.toml, and add ty.toml for the type checker
- Extracting all remaining pip dependencies into dedicated ci/lint/requirements.txt
- Add a ruff format lint which, if the original file is formatted, will fail if a change un-formats it

Extra rationale:

COPY --from pulls pre-built binaries from upstream images instead of compiling/downloading at runtime. Containerfile layer optimisations reduce rebuild frequency further.

ty is significantly faster/more modern/maintained than mypy, and configured declaratively.

Adding root-level [ty|ruff].toml config files means contributors can easily run ty check or ruff check locally without running the full linter, along with being accessible to other tooling (similarly for requriements.txt).

Pinning tool versions in the dockerfile makes it more excplicit and easier to find.

The tradeoff we make here is that there is no longer a bare install script to install tooling on a local machine. However I think this is OK, as it currently only works for apt-based OSes anyway, and I don't think running the linter outside of the container is such a valuable use-case as it is with some of the other CI jobs.

Further work can drop individual rules from ty.toml fixing up the infringing code as necessary.

I can split this up if wanted, but IMO it makes sense to do it altogether.

@DrahtBot DrahtBot added the Tests label Feb 10, 2026
@DrahtBot
Copy link
Contributor

DrahtBot commented Feb 10, 2026

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #34757 (guix: re-enable riscv exported symbol checking by fanquake)
  • #34550 (guix: update time-machine to f04dc9234c58b064507bb46ab1797f95f09b543e by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@fanquake
Copy link
Member

Tried running this locally, but it currently fails on arm64, because there is no arm64 variant for becheran/mlc:

Error: choosing an image from manifest list docker://becheran/mlc:latest: no image found in image index for architecture "arm64", variant "v8", OS "linux"

@willcl-ark willcl-ark marked this pull request as draft February 10, 2026 10:42
@willcl-ark
Copy link
Member Author

Tried running this locally, but it currently fails on arm64, because there is no arm64 variant for becheran/mlc:

Error: choosing an image from manifest list docker://becheran/mlc:latest: no image found in image index for architecture "arm64", variant "v8", OS "linux"

This should be fixed in the latest push. We currently (and still here) use an x86_64 MLC binary, but this works in an arm64 container (e.g on Mac) because the Rosetta emulation is passed in/used inside the container.

I keep this behaviour for now, but ideally MLC woudl start publishing multi-platform docker images.

https://docs.astral.sh/uv/

Install python in the linter using uv and a venv.
This is faster and more simple than building pyenv.
Align with uv tool installation method.
@willcl-ark
Copy link
Member Author

bumped to latest ruff (0.15.5) and ty (0.0.21) and added newly-needed rule ignores.

Copy link
Member

@maflcko maflcko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left some comments, but feel free to ignore them

c7e4a67~8 🥝

Show signature

Signature:

untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
trusted comment: c7e4a67086626b8832372cf461cee30c22c62ea0~8 🥝
IVhYiz19o3yQe68zl8z74QmB+pkeGtEnNKMDk40qNOxOuO20cBwlEWe2l+juySUkL1C+N5ejQa0svHEqvOa2BQ==

Add ty typechecker configuration file.

Includes all needed rules to have `ty` pass on selected files in current
codebase. This is pretty much all useful rules, but we will remove
them systematically in follow-ups.
Remove mypy and switch to `ty` for typechecking.

https://docs.astral.sh/ty/
https://astral.sh/blog/ty

It's faster and more maintained than mypy. Comes with better editor
integrations, a language server and is written in rust (tm).

As features are added we may be able to replace vulture with ty too.

Install from docker image as per `uv` and `ruff`.
willcl-ark and others added 4 commits March 13, 2026 20:10
Note: we keep MLC install via curl binary download. This is currently
hardcoded to an x86_64 binary, but this seems to run fine in an arm64
container (on MacOS), as Rosetta support is passed in (on docker and
podman) to take care of exactly this.

If upstream MLC start providing a multi-image dockerfile then this
dependency could be made it's own `FROM --...` layer as most of the
others are.
.python-version always matches the minimum supported Python version.
It's main purpose is to catch accidental use of too modern syntax
in scripts and functional tests.

We (currently) don't specify a minimum patch version, so it's not
necessary to do so here. The minor verion is enough.

This also avoids requiring users to keep a potentially unsafe old
patch version installed.
Enforce monotonic ruff formatting for Python files: if a file was
already ruff format-compliant on the base branch, it must stay
formatted in the PR. Files that aren't yet formatted are not
enforced, allowing incremental adoption without a big-bang reformat.

This follows the strategy used by nixpkgs to adopt nixfmt without
merge queues:

1. Add per-file CI check (this commit) — a no-op today since no
   files are ruff format-compliant yet.
2. Reformat inactive files (files not touched by recent PRs) in a
   batch. The check prevents regressions from that point on.
3. Reformat remaining files once old PRs have flushed through.

The per-file approach avoids the merge queue requirement: old PRs
opened before a batch reformat can still merge without breaking
master, because the check only compares against the base branch
version of each changed file, not the entire tree.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants