Skip to content

Add storage specification to BaseStorage class doc.#1174

Merged
hvy merged 5 commits intooptuna:masterfrom
ytsmiling:add-base-storage-class-doc
May 12, 2020
Merged

Add storage specification to BaseStorage class doc.#1174
hvy merged 5 commits intooptuna:masterfrom
ytsmiling:add-base-storage-class-doc

Conversation

@ytsmiling
Copy link
Copy Markdown
Member

Motivation

As noted in #1170, defining consistency models let storage implementations to remove their bottlenecks.

Description of the changes.

This PR adds a class doc to BaseStorage class which defines consistency models which storage classes should guarantee. Additionally, it also notes on thread-safety and data-persistence.

This PR introduces load(study_id) method. It is not supported by the current implementation and will be added by successive PRs. The method is not only required to reduce the sync cost with a remote database, but also necessary to alleviate the problem described in #1166 in multi-worker settings.

@codecov-io
Copy link
Copy Markdown

codecov-io commented Apr 28, 2020

Codecov Report

Merging #1174 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1174      +/-   ##
==========================================
- Coverage   91.04%   91.02%   -0.02%     
==========================================
  Files         142      142              
  Lines       12255    12276      +21     
==========================================
+ Hits        11157    11174      +17     
- Misses       1098     1102       +4     
Impacted Files Coverage Δ
optuna/storages/base.py 56.32% <ø> (ø)
optuna/pruners/hyperband.py 86.95% <0.00%> (-5.24%) ⬇️
optuna/storages/in_memory.py 98.02% <0.00%> (+0.16%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0589b2b...4e41726. Read the comment docs.

@sile sile mentioned this pull request Apr 28, 2020
3 tasks
@hvy hvy self-assigned this Apr 30, 2020
Copy link
Copy Markdown
Member

@hvy hvy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Content LGTM. Some thoughts and questions.

  • It's a lot of content. Grouping into sections might make it easier to comprehend and maintain. For instance, monotinic-reads and read-your-writes could have its own section. Other groupings might be less straight forward, though.
  • It might be a bit confusing since load isn't yet introduced. How about adding a TODO.
  • Are you imagining this specification to sort of grow to be more specific in on the coming days/weeks while addressing #1170. I'm curious about the intended level of detail.

@hvy hvy added document Documentation related. optuna.storages Related to the `optuna.storages` submodule. This is automatically labeled by github-actions. labels Apr 30, 2020
@ytsmiling
Copy link
Copy Markdown
Member Author

Thank you for checking the doc.

  • I forgot to add sections and will address the layout issue.
  • I agree that we should add a TODO comment. (Note: Current implementations do not require the load method, and we only need to add an empty method.)
  • I intended to fix this spec (at least within Major refactoring of storage classes. #1170) and expand it only when we need more supports from storage for future development (e.g. for new dashboard features). This spec is intended to provide a set of conditions that ensures that storage implementations won't cause the system down even in multi-worker settings. If this spec is not sufficient to ensure/implement safe storages, it is ideal to address the ambiguity or corner cases of this doc within this PR.

Copy link
Copy Markdown
Member

@sile sile left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you for the great work!


**Stronger consistency requirements for special data**

TODO(ytsmiling) Add load method to storage class implementations.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[NOTE] I'd like to discuss the name when this method is implemented (load sounds too general to me).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this comment. I added a TODO comment in #1170.

@sile sile added this to the v1.4.0 milestone May 8, 2020
@ytsmiling ytsmiling removed this from the v1.4.0 milestone May 11, 2020
Copy link
Copy Markdown
Member

@hvy hvy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@hvy hvy added this to the v1.5.0 milestone May 11, 2020
@hvy hvy merged commit 93391b8 into optuna:master May 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

document Documentation related. optuna.storages Related to the `optuna.storages` submodule. This is automatically labeled by github-actions.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants