Add support for parsing unicode in the units module#9348
Conversation
|
I though about which representation would be best, basically there are three options: I went for the most verbose one, as it is both ascii in the code and human readable. Any other opinions on this? |
|
If it is "strictly speaking not correct," should we even support it? 🤔 |
|
Semantically correct would always be to use 'MICRO SIGN', however I think as
it's probably better to add both |
|
The PR looks OK, but I'm a bit with @pllim on whether we should support the greek letter as well. My sense would be that it is OK to interpret it on input, but that we should not expose it in |
|
I see the potential of "Why does µ work but μ not"? I don't really see a harm of supporting both as long as At closer inspection, the two characters look exactly the same in the mono font but are subtly different in the sans font on github. |
|
They do look very similar... 👀 |
|
My main hesitation is creating yet more entries in the units registry - indeed, I'm not completely sold on even having it there (but very sold on being able to parse it; I think we should roundtrip in unicode as well). How about we start simple and just allow the mu in strings? I made a PR to your PR... |
|
p.s. Off for a bit, back in about 1.5 hours. |
|
I also fixed the used character for the micro sign in the unicode output format. See, @pllim commonly used 🤣 |
|
@maxnoe - I made another PR to your branch, where superscripts are still translated, but as part of a regex that surrounds the translations with parentheses - which are recognized as powers and avoids the mix of regular and superscript numbers just working. p.s. I had hoped to do it with the lexer/parser itself, but did not manage. Might still be better... |
|
p.s. It might be fun to see if the roundtrip tests that are done now also work in unicode! p.s.2 If there are superscript slashes and periods, possibly we should add those too... |
|
@mhvk I checked the unicode superscripts table, seems there is just numbers, +, - and a few selected letters. So I think we got everything covered. |
This ensures that mixes of superscripts and normal characters fail.
|
How about something like the following (sorry, in a hurry... off for a bit). |
|
Yes, that's the SO answer I found, but where to put, is there some astropy compat module, or should we just put it in the parser file? |
|
Beyond the one last comment this all looks great! |
|
I had a Version in between that replaced the omega with Ohm. Make trans only supports single character replacements. Will fix tomorrow. |
|
This looks nice! Unfortunately I won't be able to give it a thorough review until ~Tuesday next week, as I'm away with family for the long (US) weekend. |
|
I haven't had time for a full review, but tested it under Python 3.8; from a quick overview looks good to me too - @mhvk's last commit certainly does. |
|
I think this is by design. Personally, I would be opposed to enabling prefixes for Ångrström. |
|
For |
|
@adrn - would you still be able to have a look? With @dhomeier's quicker review and @taldcroft's liking of the general idea in #9347 (comment), I think there has been enough review to merge, but a good look is always helpful. |
I am not strongly pushing for it either, but in the spirit of the earlier discussion on which µ variants to accept, I think it's worth at looking at the potential harm vs. benefits. Stuff copied from older literature might very well come in milliangstroms, too. |
|
True. Though note that this is somewhat orthogonal to the discussion here - one would just set |
|
As a co-author, I perhaps shouldn't, but if @adrn doesn't get to it, I'l, do it tomorrow. |
adrn
left a comment
There was a problem hiding this comment.
Looks reasonable to me - I don't have any comments!
|
@maxnoe - thanks for the nice work and good collaboration! |
|
@mhvk Was a pleisure, thanks for all the support! |
Description
As a first simple step in adding feature #9347, this PR adds two unicode symbols
as possible representations for the micro prefix
MICRO SIGNandGREEK SMALL LETTER MU.While the second one is strictly speaking not correct, it probably appears often enough in the wild to be useful to support.