Currently the IMPALA implementation is restricted to Discrete/MultiDiscrete, but this is just a implementation detail. Cc @michaelzhiluo who has a patch for this