Conversation
|
@lucyleeow I did not follow all discussion so I'll let you review, feel free to merge if you're happy. I can look deeper if it would help to have another set of eyes or you're busy though! |
drammock
left a comment
There was a problem hiding this comment.
left a few comments to hopefully speed up reviewing
sphinx_gallery/gen_rst.py
Outdated
|
|
||
| def _get_class(gallery_conf, key): | ||
| """Get a class for the given conf key.""" | ||
| from .sorting import FunctionSortKey |
There was a problem hiding this comment.
nesting required to avoid circular import
sphinx_gallery/gen_rst.py
Outdated
| if not is_builtin_alias: | ||
| val = FunctionSortKey(val) |
There was a problem hiding this comment.
just to spell out the logic of this function:
- if it's a string alias of a builtin SortKey class, convert to FQN string
- then, if it's a string, check for ".", split, and try to import
- if that worked, wrap the imported func in
FunctionSortKey(unless it's already a built-in SortKey class)
sphinx_gallery/sorting.py
Outdated
|
|
||
|
|
||
| class FunctionSortKey: | ||
| def FunctionSortKey(func, r=None): |
There was a problem hiding this comment.
opted for CamelCase on the function name, to avoid changing the API (though if you hate this, it's probably no big deal to change it to snake_case, since it wasn't really working before anyway AFAICT).
There was a problem hiding this comment.
If we allow within_subsection_order to take a class instance, then I think this can be reverted and would work on within_subsection_order ?
|
one other important note: now that |
|
Half way through review I realised that I already figured out that (also which part of the docs were you reading? They can definitely be improved, I was not sure where to put the section on custom sort keys as you can use them for several 'unrelated' configs 😩 ) I do like that this now supports sphinx-gallery/sphinx_gallery/gen_rst.py Lines 1698 to 1701 in 9947c26 (obviously we need to improve our testing) Ideally, I think I would love to get rid off
sphinx-gallery/sphinx_gallery/gen_rst.py Line 569 in 9947c26
Sorry @larsoner, have to bring you in here, you'll be able to come up with a better solution than my tired brain. |
Ah yes, I somehow missed that
-0.5 from me; this PR came about because I want to use it (on subsection orders) and couldn't get it to work. I'd be content though if passing an FQN string of a sorting function worked on subsection orders. Note that at present (in this PR)
...then maybe
I'll add a test of whether FunctionSortKey still works on |
|
I completely agree that ideally I would like all 3 configs to be able to take a custom sort key and for them to work similarly. I think a function and string FQN should work on all 3 but we've just got to get the class thing working. I thought of getting rid of We can keep Thanks for working on this! |
|
Thinking about it more, maybe the convenience classes could be a 'callable' (i.e. class instance) instead of a class. This would allow us to use The only other thing we would want to make sure works is any existing user's own classes used for I guess the other option is to loosen, so a class instance or a class are both accepted? Not sure what the best solution is here. I would like to go for getting rid of Apologies for patchiness, I am on holiday but I will try to keep on top of this. |
drammock
left a comment
There was a problem hiding this comment.
OK _get_class is now gone, replaced by a tweaked version of _get_callables. It was quite a game of whack-a-mole running tests locally (mostly because I didn't realize at first all the different ways _get_callables was being used).
Thinking about it more, maybe the convenience classes could be a 'callable' (i.e. class instance) instead of a class. This would allow us to use _get_callables and keep these CamelCased classes as classes, and not have to change them to functions? Instead of init-ing with the src_dir we can just pass it the full path?
What I ended up doing was detecting whether the "callable" was one of our built-in sorter classes, or the class returned by FunctionSortKey, and if so, instantiating them within _get_callables. I think this means that any user-created callable class instances should still work (and IIUC we don't expect users to pass in an actual class right? just instances?)
| with pytest.raises(ConfigError, match=r"Unknown string option.*when importing"): | ||
| _fill_gallery_conf_defaults(sphinx_gallery_conf) | ||
| sphinx_gallery_conf = dict(within_subsection_order=1.0) | ||
| with pytest.raises(ConfigError, match="a fully qualified.*got float"): | ||
| with pytest.raises(ConfigError, match="must be callable"): |
There was a problem hiding this comment.
these changes reflect the change from using _get_class to using _get_callables (they had similar but not identical error messages)
| sphinx_gallery_conf = { | ||
| "examples_dirs": "src", | ||
| "gallery_dirs": "ex", | ||
| "within_subsection_order": FunctionSortKey(custom_sorter), |
There was a problem hiding this comment.
I can add here
"minigallery_sort_order": FunctionSortKey(custom_sorter),
"subsection_order": FunctionSortKey(custom_sorter),and the test will still pass, but I think that's not especially informative because (IIUC) the site being built by for this test doesn't have subsections / minigalleries? Could use some guidance on how/where to test that FunctionSortKey(my_func) and FQN strings work correctly for minigalleries and subsection ordering... maybe that's already tested here?:
sphinx-gallery/sphinx_gallery/tests/test_full.py
Line 1350 in 9947c26
...but I find that test/parametrization inscrutable, so I'd appreciate some guidance on whether those cases are already covered, and if not, how/where I should add/tweak a test to cover them.
There was a problem hiding this comment.
So we could do minigalleries here.
I added a add_rst pytest mark here, which will add a index.rst file in src with the contents of file arg. (This happens in the sphinx_wrapper_app in conftest). See this test for example:
sphinx-gallery/sphinx_gallery/tests/test_gen_gallery.py
Lines 647 to 662 in 13f7e06
AFAICT we only test FQN string for subsection_order in tinybuild.
It may be a big ask, but what do you think about adding a second subsection to the sphinx_gallery/tests/testconfs/src gallery...?
It looks like at least mpl does: Does that change things? |
ah, yes it would. thanks for the tip, I'll rethink this. |
sphinx_gallery/gen_rst.py
Outdated
| # make sure built-in sorter classes get instantiated (so they become callable) | ||
| if is_custom_sorter or what in BUILTIN_SORTERS: |
There was a problem hiding this comment.
if we change this conditional to
| # make sure built-in sorter classes get instantiated (so they become callable) | |
| if is_custom_sorter or what in BUILTIN_SORTERS: | |
| # make sure classes get instantiated (so they become callable) | |
| if key == "within_subsection_order" and inspect.isclass(what): |
then MPL's custom sorter class should work too, and the test suite still passes locally.
There was a problem hiding this comment.
I think I am okay with this.
AFAICT custom within_subsection_order config values have to be a class as we previously always instantiated it, before passing it to sort.
This will however, also let you pass a class instance, like the other 2 sort configs, instead of a class that takes the srcdir on init.
Do you see an easy enough way to modify |
Done in 2316166 |
|
Sorry I got to this late. Would it work to alter I only suggest as it would be nice to have more unit tests, as tinygallery tests do get quite convoluted. |
I think I've achieved what you ask in d25c7d5 but please double-check it's what you expected, I'm still not sure I've wrapped my head around your test suite. |
|
Yes thats exactly what I was thinking. Just seems like we'd need to update a few more tests, to account for the new subgallery but otherwise seems reasonable. Thank you. |
|
ok sorting tests are fixed @lucyleeow (sorry I missed those before, thought I ran full pytest before pushing but I must not have). All green now. |
lucyleeow
left a comment
There was a problem hiding this comment.
So sorry about the delay.
Some nits, mostly around testing (and its mostly our fault, e.g., I didn't realise we hardly test subsection_order), otherwise this is looking good.
I think some doc updates should go with this, but I can add them later in a separate PR as there are a few things I want to amend and move around.
sphinx_gallery/gen_gallery.py
Outdated
|
|
||
| # classes (not pickleable so need to resolve using fully qualified name) | ||
| _get_class(gallery_conf, "within_subsection_order") # make sure it works | ||
| # callable classes (not pickleable so need to resolve using fully qualified name) |
There was a problem hiding this comment.
I am confused by this comment, maybe we could improve it.
AFAICT, within_subsection_order not does not need to be a class? If it is a class we instantiate it, but otherwise as long as it is a callable, it's fine.
The pickleable part; I think is talking about how we updated to using strings for the builtin sorters for new Sphinx (ref: #1289). I think we could move this to within _get_callables, above:
if what in builtin_aliases:
what = f"sphinx_gallery.sorting.{what}"So maybe we can just amend this comment to:
| # callable classes (not pickleable so need to resolve using fully qualified name) | |
| # check `within_subsection_order` |
| def _check_order(sphinx_app, key, expected_order=None): | ||
| """Iterates through sphx-glr-thumbcontainer divs and reads key from the tooltip. | ||
|
|
||
| Used to test that these keys (in index.rst) appear in a specific order. |
There was a problem hiding this comment.
Could we add a line here explaining what expected_order is doing?
| "backreferences_dir": "gen_modules/backreferences", | ||
| "subsection_order": f"{util_root}.noop_key", | ||
| "within_subsection_order": "FileNameSortKey", | ||
| "within_subsection_order": CustomSortKey, |
There was a problem hiding this comment.
I think it would be nice to keep the string, to make sure this works. Is it possible check the custom class in test_gen_gallery.py now that we have added a subsection?
There was a problem hiding this comment.
somehow I missed this comment before, but I think it's now moot? The string-aliases for builtin sorters are now comprehensively tested in test_example_sorting (as are FQN strings of custom funcs, and actual callable functions) so AFAICT it's actually preferable to keep this as-is (since in the parametrized test we only test callable functions, not callable classes).
There was a problem hiding this comment.
Yes totally agree, this is good as is.
| sphinx_gallery_conf = { | ||
| "examples_dirs": "src", | ||
| "gallery_dirs": "ex", | ||
| "within_subsection_order": FunctionSortKey(custom_sorter), |
There was a problem hiding this comment.
So we could do minigalleries here.
I added a add_rst pytest mark here, which will add a index.rst file in src with the contents of file arg. (This happens in the sphinx_wrapper_app in conftest). See this test for example:
sphinx-gallery/sphinx_gallery/tests/test_gen_gallery.py
Lines 647 to 662 in 13f7e06
AFAICT we only test FQN string for subsection_order in tinybuild.
It may be a big ask, but what do you think about adding a second subsection to the sphinx_gallery/tests/testconfs/src gallery...?
|
Let me know when you are done and I'll take another look. Thanks again 🙏 |
OK I think it's ready! I didn't touch the docs because you said you wanted to tackle that in a separate PR. I did a somewhat bold refactor of the example sorting tests --- there's now just one test, which is parametrized over the different ways of sorting. It's a bit wonky looking because part of the parametrization is itself in a On the plus side: one of the added tests revealed that it wasn't actually working to set subsection order with a plain list, so I've fixed that too (I checked by removing the |
lucyleeow
left a comment
There was a problem hiding this comment.
LGTM, thanks for all your work on this. Especially for fixing the subsection order plain list!
I see a few minigallery directives in the new plot example files. Did you have plans for adding minigallery sort tests? I am very okay with you not doing this though, as you've already done so much work on this PR already.
Otherwise, just a question.
I'll merge in a few days in case @larsoner wants to take a look
| The regex below extracts info from that source-of-truth line to check that the | ||
| thumbnail ordering was correct --- unless `expected_order` is provided, in which | ||
| case an alternative regex extracts the digit-part of the filename from the crossref, | ||
| and compares the order of crossrefs to `expected_order`. |
There was a problem hiding this comment.
Thanks for adding this. I actually did not realise the check was this complicated!
| 'gallery_dirs': 'ex', | ||
| 'within_subsection_order': "FileSizeSortKey", | ||
| }""" | ||
| _subsection_explicit_order_list = pytest.mark.add_conf.with_args( |
There was a problem hiding this comment.
I was confused about the with_args so I tested it without (i.e. just pytest.mark.add_conf) and the test seemed to work fine. Is it safer/better to use with_args ?
There was a problem hiding this comment.
nope, I just misunderstood the pytest docs. After re-reading, it seems you only need with_args if there's a single positional arg to the custom marker and that single positional arg is itself a callable. I'll push a commit removing the with_args part.
| "backreferences_dir": "gen_modules/backreferences", | ||
| "subsection_order": f"{util_root}.noop_key", | ||
| "within_subsection_order": "FileNameSortKey", | ||
| "within_subsection_order": CustomSortKey, |
There was a problem hiding this comment.
Yes totally agree, this is good as is.
nope, those were just copy-pasted from existing |
| subsec_order = gallery_conf.get("subsection_order") | ||
| if isinstance(subsec_order, list): | ||
| gallery_conf["subsection_order"] = ExplicitOrder(subsec_order) |
There was a problem hiding this comment.
this is the change that makes bare lists work for subsection_order (must be wrapped in ExplicitOrder before the call to _get_callables)
|
Thank you @drammock ! |
in MNE-BIDS doc/conf.py we have: ``` sphinx_gallery_conf["within_subsection_order"] = "mne_bids.utils._example_sorter" ``` But AFAICT this functionality wasn't workign prior to sphinx-gallery/sphinx-gallery#1391 , which was included in the release of sphinx-agallery v0.19 (released Feb 12 2025). https://sphinx-gallery.github.io/stable/changes.html#v0-19-0 Given that v0.19 was released less than 1 year ago, I think it is reasonable to assume that devs/contributors might have a sphinx-gallery version < 0.19 installed locally, in which case building the mne-bids docuemntation will fail, but the reason/solution won't be explicit. So I think that it is best to be explicit about the minimum required sphinx-gallery version we require to be able to successfully build our docs.
* DOC: Add html-noplot recipe to doc/Makefile And we also tweak `make view` so that the built documentation opens regardless of whether it was built with make html or make html-noplot * FIX: Pin sphinx-gallery in MNE-BIDS doc/conf.py we have: ``` sphinx_gallery_conf["within_subsection_order"] = "mne_bids.utils._example_sorter" ``` But AFAICT this functionality wasn't workign prior to sphinx-gallery/sphinx-gallery#1391 , which was included in the release of sphinx-agallery v0.19 (released Feb 12 2025). https://sphinx-gallery.github.io/stable/changes.html#v0-19-0 Given that v0.19 was released less than 1 year ago, I think it is reasonable to assume that devs/contributors might have a sphinx-gallery version < 0.19 installed locally, in which case building the mne-bids docuemntation will fail, but the reason/solution won't be explicit. So I think that it is best to be explicit about the minimum required sphinx-gallery version we require to be able to successfully build our docs.
closes #1389
I think no docs update is needed because now things work the way the docs describe them, but feel free to request changes / push commits if you think I'm wrong about that.