[RLlib] Add Exploration API documentation.#7373
Conversation
|
Can one of the admins verify this patch? |
|
Test FAILed. |
|
Test FAILed. |
|
Test FAILed. |
|
|
||
| .. image:: custom_metric.png | ||
|
|
||
| Customized Exploration Behavior (Training and Evaluation) |
There was a problem hiding this comment.
Should add a link in rllib-toc.rst to this too (the TOC has to be manually updated).
| a deterministic action. | ||
| timestep (int): The current sampling time step. If None, the | ||
| component should try to use an internal counter, which it | ||
| then increments by 1. If provided, will set the internal |
There was a problem hiding this comment.
Is this different from timesteps_total / num_steps_sampled?? We should make sure to use the same timestep counter / terminology everywhere to avoid confusion.
There was a problem hiding this comment.
The policy always passes self.global_timestep into the call to get_exploration_action. It's the same behavior across all Policy types. This prop is updated in Policy.on_global_var_update -> global_vars["timestep"], so it should always be the total number of sample steps.
There was a problem hiding this comment.
I added that explanation to the docs as well.
| "explore": True, | ||
| "exploration_config": { | ||
| "type": "EpsilonGreedy", # <- Exploration sub-class by name or full path to module+class | ||
| # (e.g. “ray.rllib.utils.exploration.epsilon_greedy.EpsilonGreedy”) |
There was a problem hiding this comment.
This makes the code block overflow horizontally on my machine, consider trimming it down by a few words.
doc/source/rllib-training.rst
Outdated
| The following table lists all existing Exploration sub-classes and where they | ||
| are currently used (by default): | ||
|
|
||
| .. image:: images/rllib-exploration-api-table.png |
There was a problem hiding this comment.
This table is informative. Can it be shown right after introducing the exploration config, but before the large blocks of code for Exploration class?
Also,
- should include a google drawings link as a comment (shared public, view only) so that this can be edited in the future.
- can you export the table as a SVG?
|
@ericl : All requested changes have been made. |
|
Test FAILed. |
|
LINT test looks ok, merging. Maybe we can skip the C++ build if no source components have changed, it looks like the build failures are contained to |
* Add Exploration API documentation. * Add Exploration API documentation. * Add Exploration API documentation. * Update exporation docs.
How to use the new Exploration API needs to be explained in our documentation.
This PR adds it under RLlib->Training->Advanced Python APIs.
Related issue number
Checks
scripts/format.shto lint the changes in this PR.