Describe the problem

Currently only IMPALA and APPO handle MultiDiscrete properly. We should fix this or raise an error telling users to use Tuple instead.

As a workaround, you can use Tuple[Discrete, Discrete, ...] instead of MultiDiscrete.

Source code / logs

This is from https://groups.google.com/forum/#!topic/ray-dev/RaqWO89Rl6E