Add tools to make updating zkapauthorizer and magic-folder easier. #400
Conversation
crwood
left a comment
There was a problem hiding this comment.
Thanks a lot for this. This brings some much-needed sanity to (sub)dependency-management.
One issue/regression I notice with this approach, however, is that pip installing zipfiles/tarballs directly from GitHub for Tahoe-LAFS/ZKAPAuthorizer revisions results in those packages being installed into the python environment without a meaningful version-string; running pip list I see something like this:
tahoe-lafs 0.0.0
...
zero-knowledge-access-pass-authorizer 0+unknown
Likewise, running dist/Tahoe-LAFS/tahoe --version shows tahoe-lafs/unknown.
Previously, when building Tahoe-LAFS, I would have to call a python setup.py update_version in order to to force Versioneer to generate its version-string. In the case of ZKAPAuthorizer (which, as far as I can tell, also uses Versioneer) no such additional step was required (however, I see that, even so, the version-string isn't being produced when pip installing from a tarball).
Needless to say, without accurate version information, it can be difficult for users/devs to determine which version of tahoe is in use and can therefore impede future debugging efforts, etc.
How would you feel about adding such additional steps to ensure that the proper version-strings are being set/included for these two packages -- or would you consider the lack-of-version-strings-in-archives to be better addressed upstream for the Tahoe-LAFS and ZKAPAuthorizer projects (rather than here)? In any case, what do you think is the best way to resolve this?
Also (this is totally minor but I'm curious if there's a deeper reason for this) why Tahoe-LAFS is installing from a zipfile, while ZKAPAuthorizer is installing from a tarball?
I think doing additional step here generate version numbers in the upstream projects would add a bunch of complexity. For example, we couldn't generate the pinned dependencies by just point pip-compile at the zkapauthorizer tarball[1], so we'd either need to post-process the generated dependencies, or install package twice, once with the pinned dependencies, and then again to overwrite it with one that has version information. [1] pip doesn't support hashes for git repositories, for example
Given that we pin zkapauthorizer here, and pin tahoe-lafs in zkapauthorizer, the version of gridsync is enough to recover the version of zkapauthorizer used, or we could include the pinned version of zkapauthorizer (which is now in an easily parsable format) as data in gridsync.
No deep reason, zkapauthorizer explicitly depends on a zip (here) but this PR depends explicitly on a tarball.
Note that tahoe-lafs is not using versioneer to generate its version, but something similar, hence the difference in behaviour. |
|
Agreed on all fronts -- especially re. minimizing complexity (and, in this case, for stuff that is arguably already too complex as it is).
Ah, you're right. If I recall correctly, Versioneer was actually born/inspired from the Tahoe-way of dynamically updating its version string (and not the other way around). Anyway, merging this... Thanks again for improving all of this unglamorous build/deps/hell stuff. :) |
Fixes https://whetstone.privatestorage.io/privatestorage/privatestoragedesktop/-/issues/603.
Add some scripts to make updating dependencies easier, and documentation for them.
This essentially depends on PrivateStorageio/ZKAPAuthorizer#253, but I've changed one of the generated files by hand so that the PRs can land in either order.
To deal with the platform-specific dependencies of tahoe-lafs, I've more-or-less copied the solution I implemented in tahoe-lafs/magic-folder#437 (though I've adjusted it slight to not need to modify the generated file).