Describe the problem

The parameter space noise is online adjusted to make it equivalent to action space noise with given \sigma
The problem is that, in RLLib, we provide noise_scale argument for users to scale the OU noise. Besides, we allow the noise_scale to decay.
To be equivalent, the threshold used to adjust parameter space noise should not ignore the intended noise_scale. We know that multiplying the action space noise with noise_scale is equivalent to multiply the \sigma of OU by the noise_scale. Thus, we could simply adjust the parameter space noise according to noise_scale * \sigma.

Source code / logs

the threshold used to adjust the parameter space noise: https://github.com/ray-project/ray/blob/master/python/ray/rllib/agents/ddpg/ddpg_policy.py#L61