@maxnoe can you see if the documented changes here make sense to you?

@lucyleeow ready for review/merge from my end

subsection_order field on the conf still requires a callable which is called directly in gen_gallery and thus generates the non pickleable warning. I also went through the tests of sphinx_gallery and couldn't find any testcase covering the subsection_order field making sure having a callable in this field doesn't mess up anything.

while working on the same issue at matplotlib i made some changes in gen_gallery and gen_rst to handle the issue with subsection_order and it seemed to build without any issue.

• changed line 447 in gen_gallery to

  sortkey = _get_class(gallery_conf, "subsection_order")

• and changed the _get_class function in gen_rst at line 1503 to

  if not inspect.isclass(val) and not hasattr(val, '__class__'):
          raise ConfigError(
              f"{what} must be 1) a fully qualified name (string) that resolves "
              f"to a class or 2) or a class itself 3) or an instance of a class,"
              f" got {val.__class__.__name__} ({repr(val)})"
          )

  so that it passes with both a class definition or an instance of a class

I'll probably try modifying it just to use _get_callables instead so that a callable needs to be passed rather than a class

Pushed a commit, feel free to give it a try / another look @NirobNabil @lucyleeow !

Been waiting for this PR to go into a release so that the fix at matplotlib can be merged. Any idea when the next release window of sphinx-gallery might be?

We'll release after this PR goes in.

Edit: 🤦 wait I better investigate CI failures (#1291 (review)) first nevermind, this PR fixes them

@lucyleeow are you up for pushing directly? I think it will be faster

Can be done in separate PR but what do you think about just updating the gallery_conf value in _get_callables, so we don't have to do it twice (call _get_callables twice)?

@lucyleeow are you up for pushing directly? I think it will be faster

Done, only changed docs.

what do you think about just updating the gallery_conf value in _get_callables, so we don't have to do it twice (call _get_callables twice

I wish we could but the env (including gallery conf) is cached after SG runs so it would cause the same serializing problems

Ah I didn't realise that. Can't see a way around that. Feel free to merge!

Ah I forgot about the tests, I can update in a separate PR.

Yeah let's get this in and I can start testing with MNE-Python for example. Then maybe release Friday?

what do you think about just updating the gallery_conf value in _get_callables, so we don't have to do it twice (call _get_callables twice

I wish we could but the env (including gallery conf) is cached after SG runs so it would cause the same serializing problems

Just a question, as I was just thinking about this. When you say after SG runs, does it mean more generally after build? As I understand, we connect functions to various sphinx events, so 'after SG runs' would just mean after sphinx finishes building? Or am I mis-understanding?

I mean after the examples are run in the build-inited event, not after the sphinx-build finishes. It caches the env sometime after build-inited completes