We're offering some functionality for low level task graph rewrites/optimizations, see https://docs.dask.org/en/stable/optimize.html

I propose to deprecate and remove most, if not all, of this functionality. With #9969 most of this will break. Some/all of this could be migrated but I am not entirely convinced this is worth it.

Explicitly, I propose to remove

• cull (keep internally)
• fuse (keep internally, possibly replace with simpler task spec version)
• fuse_linear (keep internally, possibly replace with simpler task spec version)
• inline_functions (There is rudimentary support for this in task spec already)
• functions_of
• Rewrite Rules

(Re)Moving these things also means that magical kwargs to compute like fuse_keys, fast_functions, inline_functions_fast_functions would vanish.

For now, we'd likely keep the fuse functions but keep it pretty much internal. It is currently only executed for the array code so keeping it there in place still makes sense (for now) but I don't believe this is a valid function to expose.

I'm a bit on the fence regarding custom optimizers, e.g. the possibility to register an optimizer function with something like dask.config.set(array_optimize=my_optimizer), see also https://docs.dask.org/en/stable/optimize.html#customizing-optimization I suspect there are very few power users even capable of writing this kind of code. It will certainly break and I'm not entirely convinced this is worth restoring.

cc @mrocklin in case you have insights or an opinion on any of this