Skip to content

Add API scripts and output files#3682

Merged
apoelstra merged 3 commits intorust-bitcoin:masterfrom
tcharding:11-29-check-api
Dec 9, 2024
Merged

Add API scripts and output files#3682
apoelstra merged 3 commits intorust-bitcoin:masterfrom
tcharding:11-29-check-api

Conversation

@tcharding
Copy link
Copy Markdown
Member

Last time we did this there was push back by Kixunil because he did not like updating the API files. This time we only add API files for those crates we are focused on stabalizing.

As we try to do the checklists for making sure the crates conform to the Rust API guidelines one has to repeatedly audit the code. Being able to quickly get lists of things out of the code is super useful.

And the after we release 1.0 these API text files become valuable for review.

--

Add a script to generate API files using cargo check-api for the crates that we are trying to stabalise.

  • hashes
  • io
  • primitives
  • units

Add a script to query the API files. E.g., contrib/api.sh units types

Current commands are

types: Show all public structs and enums
types_no_err: Show all public structs and enums excluding errors
traits: Show all public traits

@tcharding
Copy link
Copy Markdown
Member Author

cc @jamillambert

@tcharding
Copy link
Copy Markdown
Member Author

cc @Kixunil (no mention in PR description because of our merge script.)

@yancyribbens
Copy link
Copy Markdown
Contributor

Last time we did this there was push back by Kixunil because he did not like updating the API files

I also pushed back on this last time and I am still not a fan. For one, anything with 12k line diff makes you asking isn't there a better way.

@yancyribbens
Copy link
Copy Markdown
Contributor

s/makes you asking/makes me ask

@apoelstra
Copy link
Copy Markdown
Member

In e8606d67fb3d3c8bda4b3d68f6488644e1e199d0:

Trailing whitespace in contrib/api.sh

Otherwise I think this is a great idea.

@yancyribbens yes, there is an initial 12k diff. In future the diffs should be much smaller and only green.

@yancyribbens
Copy link
Copy Markdown
Contributor

yes, there is an initial 12k diff

Yes, I understand that. It will also cause much larger commits every time by embedding this metadata for every commit, even though one could get this information by comparing the results of this script between two commits.

Otherwise I think this is a great idea.

I thought this was taken out last time with good reason. I mean, I appreciate the Github warning message when the API is being changed, even though it gets annoying when it creates the warning for every new commit.

@apoelstra
Copy link
Copy Markdown
Member

will also cause much larger commits every time by embedding this metadata for every commit

git will absolutely compress the contents of multiple commits if it is the same.

I mean, I appreciate the Github warning message when the API is being changed,

I don't care about the Github message. I want to have a git history of the API changes. It was taken out last time because Kix complained about it and he gets a veto on stuff here (though in this case I was strongly tempted to override that veto..) when he's present.

In any case, I think you'll find it much less annoying when it's applied only to crates that are near-stable and intended not to change.

@yancyribbens
Copy link
Copy Markdown
Contributor

I want to have a git history of the API changes.

This could be created from a script that goes through the commit history and compares the api of two commits and then puts the result someplace.

It was taken out last time because Kix complained about it and he gets a veto on stuff here (though in this case I was strongly tempted to override that veto..) when he's present.

I think it was added though during the last time Kix disappeared. If he comes back again, is it going to get a veto again and then taken out again. Probably not, but I thought he made compelling arguments last time which I agree with.

In any case, I think you'll find it much less annoying when it's applied only to crates that are near-stable and intended not to change.

I guess that's part of working in a community is living with the decisions of others even when you don't agree. Anyway, it might annoy me but I'll live.

@sanket1729
Copy link
Copy Markdown
Member

My main concern is that this adds an extra step to the contribution process. If every single PR requires us to update them due to a minor version upgrade in a dependency, it could become quite frustrating and potentially discourage contributions.

This depends on how often we expect these to change, which also depends on our dependancies. I am okay to try it again and revert if this gets annoying.

@apoelstra
Copy link
Copy Markdown
Member

If every single PR requires us to update them due to a minor version upgrade in a dependency, it could become quite frustrating and potentially discourage contributions.

This isn't about dependencies, but about our own API. And this won't affect every PR -- only the ones that change the API. Which for near-stable crates, we do want to discourage.

@tcharding
Copy link
Copy Markdown
Member Author

And since we have currently have no path towards 2.0, with @Kixunil even claiming that there won't be one, preventing changes to the API is almost a must.

@tcharding
Copy link
Copy Markdown
Member Author

I pulled all the CI and docs stuff out of #2713

@tcharding tcharding force-pushed the 11-29-check-api branch 2 times, most recently from f05f1b6 to 302a701 Compare December 4, 2024 03:51
@tcharding
Copy link
Copy Markdown
Member Author

tcharding commented Dec 4, 2024

Now includes idea from #2893

CONTRIBUTING.md Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In 0f28f33bfd9fdb231e814d81a16772a286b21e28:

Should clarify that this applies only to io, hashes, primitives and units.

@apoelstra
Copy link
Copy Markdown
Member

Also can you pull the actual construction of the API files into their own commit? It can be the first commit (so initially there is nothing enforcing them) if that makes things nicer for you. But they overwhelm the diff and are hard to filter out (I think you need something like git show -- ^api/ api/README.md or something).

contrib/api.sh Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In 302a7018f1e4d62b407a6ad29dd24684aec6b27e:

I would suggest just always dumping the usage after "unsupported crate". Then you can special-case -h and --help up here if you want.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cheers, the api script is pretty messy still but it shell checks cleanly.

We are about to introduce a script that generates text files for the
public API surface of various crates.

Run the soon-to-be-introduced script and commit the changes. Covers:

- `hashes`
- `io`
- `primitives`
- `units`

The script and CI setup will be done in the next patch, this is just to
make review easier.
Add a script to generate API files using `cargo public-api` for the
crates that we are trying to stabalise (the so called 'leaf crates').

- hashes
- io
- primitives
- units

We already ran the script and committed the files last patch. The fact
that this patch does not include any changes to the `api/` directory and
that CI passes is enough to convince us that last patch was valid.

Add a CI job that runs the script and checks there are no changes to
the committed text files thereby proving no API changes are made in a
PR without explicitly modifying the API text files.

Add documentation to `CONTRIBUTING.md` on what is expected of devs.
Add a simple script that allows one to query the current API. Done by
parsing the API text files and grepping for things.

This is useful as a dev tool as we try to stabalize the leaf crates.
Copy link
Copy Markdown
Member

@apoelstra apoelstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 7e0501c; successfully ran local tests; let's do it

@apoelstra apoelstra merged commit 1512d6a into rust-bitcoin:master Dec 9, 2024
@tcharding tcharding deleted the 11-29-check-api branch December 10, 2024 02:48
tcharding added a commit to tcharding/rust-bitcoin that referenced this pull request Dec 10, 2024
In a recent PR (rust-bitcoin#3682) we introduced api text files. Then in another
PR (rust-bitcoin#3711) we removed `alloc` feature gating. Possibly due to the
timing of running through CI and merging these two PRs managed to get
merged without an update to the API text files.

As would be expected; removing the `alloc` feature gate adds a bunch of
new lines to the `no-features` api text file.
apoelstra added a commit that referenced this pull request Dec 12, 2024
601a47f Add new api text file changes (Tobin C. Harding)

Pull request description:

  In a recent PR (#3682) we introduced api text files. Then in another PR (#3711) we removed `alloc` feature gating. Possibly due to the timing of running through CI and merging these two PRs managed to get merged without an update to the API text files.

  As would be expected; removing the `alloc` feature gate adds a bunch of new lines to the `no-features` api text file.

ACKs for top commit:
  jamillambert:
    ACK 601a47f
  apoelstra:
    ACK 601a47f; successfully ran local tests

Tree-SHA512: 65e22192bcc386172ec1b74651c35deef954f9f461e6dc0758140b67c62c9afa0290031ddb04d67455a558dbfb10126c042c01002057b400cd5aff7d4e38e4da
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants