-
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Isolation of optuna integration modules. #4484
Copy link
Copy link
Closed
Labels
contribution-welcomeIssue that welcomes contribution.Issue that welcomes contribution.good first issueGood first issue for contribution.Good first issue for contribution.
Description
Isolation of optuna integration modules.
Optuna is now isolating optuna.integration module to optuna-integration.
The following is a list of integration modules and their priorities.
| Status | Module | Priority | Note |
|---|---|---|---|
| Done | AllenNLP | high | #4579 |
| Done | BoTorch | low | #5230 |
| Done | Catalyst | high | #4644 |
| Done | CatBoost | high | #5198 |
| Done | Chainer | high | #4497 #4370 |
| Done | Dask | low | #5222 |
| Done | fast.ai | high | #5120 |
| Done | Keras | high | #4558 |
| Done | LightGBM | low | #5249 |
| Done | MLflow | high | #5246 |
| Done | Weights & Biases | high | #5237 |
| Done | MXNet | high | #4790 |
| Done | pycma | high | #5236 |
| Done | PyTorch(ignite) | high | #5213 |
| Done | PyTorch(lightning) | high | #5213 |
| Done | PyTorch(ddp) | high | #5213 |
| Done | scikit-learn | low | #5225 |
| Done | scikit-optimize | high | #5234 |
| Done | SHAP | low | #4791 |
| Done | skorch | high | #4663 |
| Done | TensorBoard | high | #5161 |
| Done | TensorFlow | high | #4666 |
| Done | tf.keras | high | #4662 |
| Done | XGBoost | high | #5239 |
Do not migrate low-priority module because it will have some complex dependencies on tests and others. (Migration of low-priority modules would be done by Committers.)
Migration procedure
Following is the transfer procedure, please follow these instructions
On the Optuna Integration side:
- Clone the optuna-integration forked repository
git clone git@github.com:{YOUR-FOR}/optuna-integration.git - Add the Optuna master branch to the local upstream.
git remote add upstream https://github.com/optuna/optuna.git - Create the new working branch
git checkout -b YOUR-BRANCH-NAME. - Check out the relevant file for integration purposes.
git fetch upstreamandgit checkout upstream/master -- <path/to/file>- The desired files are as follows
optuna/integration/<module>tests/integration_tests/<module>
- Move the file paths from
optuna/integration/<module>tooptuna_integration/<module>andtests/integration_tests/<module>totests/<module>. - Create a commit for each file, with a commit message that reads
"Fetched from https://github.com/optuna/optuna/commit/<commit-hash>".
<commit-hash>can be obtained usinggit rev-list upstream/master -- <path/to/file> | head -n 1.
- The desired files are as follows
- In the desired files, replace
optuna._importswithoptuna_integration._imports. - Update the required dependencies in pyproject.toml. Especially, please add the corresponding third party library in the
allsection of the pyproject.toml. - Modify the CI workflow file if necessary. Check the Optuna's CI workflow file of integration, i.e.,
.github/workflows/tests-integration.yml. If you find the any specific treatment for the modified integration, please add it to the optuna-integration's CI workflow file, i.e.,.github/workflows/tests.yml. - Update the imports in tests to use
optuna_integrationinstead ofoptuna.integration. - Add the new module to
docs/source/reference/index.rst(from here). - Add the new module to the relevant section in
README.md. - Change all
optuna.integrationsentences tooptuna_integration(if they exist).
On the Optuna side:
- Import the package that was migrated from optuna_integration and pass its contents to
__all__magic variable.- For example.
from optuna_integration.chainer import ChainerPruningExtension
__all__ = ["ChainerPruningExtension"]- Remove the relevant files from
test/integrationand related imports fromtest/integration/test_integration.py. - Remove any relevant tests from
pytest tests -m "integration" --ignorein.github/workflows/tests-integration.yml. - Remove the relevant documentation from
docs/source/reference/integration.rst. - Remove the integration dependencies from
setup.pyandpyproject.toml, taking care not to remove dependencies that are still needed.- Do not remove the relevant imports from
_import_structurevariable inoptuna/integration/__init__.py, as this would render thefrom optuna.integration import ...idiom unusable.
- Do not remove the relevant imports from
- Transfer any relevant issues from Optuna to Optuna Integration.
- Request that the author of any relevant PRs transfer them from Optuna to Optuna Integration.
These changes should be merged into optuna-integration at first, then merged into optuna.
On the Optuna exmaple side:
- Change the corresponding integration's ci yaml to install
optuna-integration.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
contribution-welcomeIssue that welcomes contribution.Issue that welcomes contribution.good first issueGood first issue for contribution.Good first issue for contribution.