We don't need to build Python wheels with both cp310 and cp311 tags; Python 3.11 (and 3.12) is fine with cp310 wheels. Since we don't actually use Python's extension ABI, that should be fine no matter what.
We just need to figure out a stable way to force always emitting a cp310 wheel. https://github.com/karellen/wheel-axle/ seems to do that but I'm a bit hesitant to add a whole new dep for that.