[rllib] Auto clip actions to Box space range; deprecate squash_to_range#3426
[rllib] Auto clip actions to Box space range; deprecate squash_to_range#3426ericl merged 8 commits intoray-project:masterfrom
Conversation
|
Test FAILed. |
|
jenkins retest this please |
|
Test FAILed. |
|
jenkins retest this please |
|
Test FAILed. |
|
Test FAILed. |
|
Test FAILed. |
| # None means clip for Atari only. | ||
| "clip_rewards": None, | ||
| # Whether to np.clip() actions to the action space low/high range spec. | ||
| "clip_actions": True, |
There was a problem hiding this comment.
when would you want this to be false?
There was a problem hiding this comment.
For backwards compatibility I think it's a reasonable option to have.
Co-Authored-By: ericl <ekhliang@gmail.com>
|
Test FAILed. |
What do these changes do?
Per discussion here https://groups.google.com/forum/#!topic/ray-dev/loJOOU5LzDk, squash_to_range incorrectly zeros out gradients and can cause divergence.