[rllib] Enable performance metrics reporting for RLlib pipelines, add A3C#7299
[rllib] Enable performance metrics reporting for RLlib pipelines, add A3C#7299ericl merged 52 commits intoray-project:masterfrom
Conversation
| import time | ||
|
|
||
|
|
||
| class _Timer: |
There was a problem hiding this comment.
This class is just moved from rllib/utils to be accessible to ray/util.
|
Can one of the admins verify this patch? |
|
Test FAILed. |
|
Test PASSed. |
|
Test FAILed. |
|
Test FAILed. |
|
Ping @sven1977 , this should be reviewed first before the other PR. |
|
Test FAILed. |
sven1977
left a comment
There was a problem hiding this comment.
Just the docstring and the "optimizer" question.
| from_actors, ParallelIteratorWorker, LocalIterator | ||
|
|
||
|
|
||
| def test_metrics(ray_start_regular_shared): |
There was a problem hiding this comment.
Should we use Unittest.TestCase?
There was a problem hiding this comment.
It's better to use pytest style, since it has better reporting behaviour under bazel (capturing the stdout).
| @@ -0,0 +1,19 @@ | |||
| """Experimental pipeline-based impl; run this with --run='A3C_pl'""" | |||
| if hasattr(self, "workers"): | ||
| self.workers.stop() | ||
| if hasattr(self, "optimizer"): | ||
| if hasattr(self, "optimizer") and self.optimizer: |
There was a problem hiding this comment.
I was wondering about these things. If Trainer is not sure to have a prop "optimizer" or not, should this prop not be part of the Trainer class (and then sometimes be None)? I know we do this in many places, so not urgent, but we should think about fixing these.
There was a problem hiding this comment.
Probably the right solution is to make it a required prop of a subclass of Trainer.
I'll leave it alone for now, since we can probably remove .optimizer entirely once the pipeline implementation becomes the default.
Why are these changes needed?
This adds a MetricsContext concept that is used for recording and retrieving internal performance metrics of RLlib pipelines. The context is shared across all pipeline operators. Operators can record counters, info values, and timer values via the context.
This also adds support for A3C and the AsyncGradients() operator, which uses the metrics context in a couple new ways.