Follow-up on #12084

That PR fixed the "empty column specification" problem for lists and boolean arrays, but not for slices.
I commented there:

@jnothman after updating this PR to also check for all-False boolean arrays, I realized that also slices can be empty .. But to fully properly determine of they are empty, you need the X data, which I actually just removed from _iter in the previous PR (#12107).
Not difficult to add this back, but it also raises the question if this should only be done during fit and not anymore rely on the data during transform (similarly as we decided to not evaluate the function any more during transform). This is certainly all possible, but will complicate the code a bit further .. Thoughts ?

So we should further fix this fully check for empty column specifiers, including slices, and possibly only check for this during fit (and store the information somehow)