Skip to content

[rllib] Port DDPG to the build_tf_policy pattern#5242

Merged
ericl merged 32 commits intoray-project:masterfrom
ericl:port-ddpg
Jul 24, 2019
Merged

[rllib] Port DDPG to the build_tf_policy pattern#5242
ericl merged 32 commits intoray-project:masterfrom
ericl:port-ddpg

Conversation

@ericl
Copy link
Copy Markdown
Contributor

@ericl ericl commented Jul 21, 2019 •
edited
Loading

What do these changes do?

This ports DDPG to the policy builder pattern. This is the last major algorithm that needed to be ported.

Pendulum performance seems to be on par. @joneswong could you check if parameter noise exploration still works as expected? There was a lot of changes around handling in that code.

fyi @qxcv @gehring

Related issue number

Closes #4822
Closes #4788

Linter

  • I've run scripts/format.sh to lint the changes in this PR.

ericl added 27 commits July 5, 2019 18:03
@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Jul 21, 2019

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/15530/
Test FAILed.

@ericl ericl force-pushed the port-ddpg branch 4 times, most recently from fd50692 to 10be568 Compare July 22, 2019 00:10
@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Jul 22, 2019

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/15531/
Test FAILed.

@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Jul 22, 2019

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/15532/
Test FAILed.

@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Jul 22, 2019

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/15534/
Test FAILed.

@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Jul 22, 2019

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/15535/
Test FAILed.

@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Jul 22, 2019

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/15537/
Test FAILed.

@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Jul 22, 2019

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/15538/
Test FAILed.

@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Jul 22, 2019

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/15539/
Test PASSed.

@ericl ericl added the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Jul 22, 2019
@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Jul 22, 2019

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/1750/
Test FAILed.

@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Jul 22, 2019

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/1751/
Test FAILed.

@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Jul 23, 2019

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/15590/
Test PASSed.

@ericl ericl merged commit 60f5963 into ray-project:master Jul 24, 2019
This was referenced Sep 3, 2019
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@richardliaw richardliaw richardliaw approved these changes

Assignees

@joneswong joneswong

Labels

tests-ok The tagger certifies test failures are unrelated and assumes personal liability.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[rllib] Port DDPG, IMPALA, etc. to new build_tf_policy() pattern [rllib] Modularize policy graph and trainer construction

4 participants

@ericl @AmplabJenkins @richardliaw @joneswong