Skip to content

Dispose SQLAlchemy's engine in storage_tests/rdb_tests/test_storage.py#6338

Merged
c-bata merged 18 commits intooptuna:masterfrom
kAIto47802:dispose-sqlalchemy-engine-test-storage
Dec 5, 2025
Merged

Dispose SQLAlchemy's engine in storage_tests/rdb_tests/test_storage.py#6338
c-bata merged 18 commits intooptuna:masterfrom
kAIto47802:dispose-sqlalchemy-engine-test-storage

Conversation

@kAIto47802
Copy link
Copy Markdown
Collaborator

@kAIto47802 kAIto47802 commented Nov 12, 2025

Motivation

This PR is the follow-up on:

and runs in parallel with::

The original PR did not cover all test cases. This PR addresses the remaining ones in storage_tests/rdb_tests/test_storage.py.

Important

With these PRs and this PR, all the tests creating RDBStorage in test_storages has been addressed.

The remaining tests creating RDBStorage is only study_tests/test_study_summary.py, which creates the RDBStorage instance for only two times and considered to be less likely to encounter the SQLAlchemy's issue.

Description of the changes

  • Make existing create_test_storage context manager so that storage.engine.dispose() is executed every time.
  • Make sure to use create_test_storage for all tests.

Note

To ensure that dispose() is always called even when an assert fails, it should be invoked inside __exit__ rather than at the end of the test function.

Important

Most of the diff comes from increased indentation after introducing the context manager.

@y0z
Copy link
Copy Markdown
Member

y0z commented Nov 14, 2025

@fusawa-yugo Could you review this PR?

@fusawa-yugo
Copy link
Copy Markdown
Contributor

could you please use contextmanager here?

create_test_storage(engine_kwargs={"pool_size": 5})

@c-bata c-bata added the CI Continuous integration. label Nov 21, 2025
@c-bata
Copy link
Copy Markdown
Member

c-bata commented Nov 21, 2025

@sawa3030 Could you review this PR?

Copy link
Copy Markdown
Collaborator

@sawa3030 sawa3030 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR. I’ve left a minor comment. PTAL

Comment on lines +152 to +166
@contextmanager
def create_test_storage(
url: str = "sqlite:///:memory:",
engine_kwargs: dict[str, Any] | None = None,
skip_compatibility_check: bool = False,
skip_table_creation: bool = False,
) -> Generator[RDBStorage, None, None]:
storage = RDBStorage(
url,
engine_kwargs=engine_kwargs,
skip_compatibility_check=skip_compatibility_check,
skip_table_creation=skip_table_creation,
)
yield storage
storage.engine.dispose()
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be slightly outside the scope of this PR, but could you move the function create_test_storage to the top of the file, before test_init? Since it’s used in the very first test (test_init), I think it would make the file a bit easier to follow.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the comment.
Indeed, I had been thinking about this as well, but I avoided including it in this PR to keep the diff minimal.
However, I completely agree that this function should be moved to the top of the file, so I will apply this change as you suggested. I also think it is unlikely that a standalone PR whose sole purpose is to move this function will be created in the future.

@kAIto47802 kAIto47802 force-pushed the dispose-sqlalchemy-engine-test-storage branch from af10f13 to e8d831e Compare November 21, 2025 08:15
@kAIto47802
Copy link
Copy Markdown
Collaborator Author

Thank you for the comment. I've update the code accordingly. PTAL :octocat:

Copy link
Copy Markdown
Collaborator

@sawa3030 sawa3030 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the update. LGTM!

@sawa3030 sawa3030 removed their assignment Nov 27, 2025
@fusawa-yugo
Copy link
Copy Markdown
Contributor

LGTM.
Thank you!

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 4, 2025

This pull request has not seen any recent activity.

@github-actions github-actions bot added the stale Exempt from stale bot labeling. label Dec 4, 2025
@c-bata c-bata removed the stale Exempt from stale bot labeling. label Dec 5, 2025
@c-bata c-bata added this to the v4.7.0 milestone Dec 5, 2025
@c-bata c-bata merged commit 181f4c1 into optuna:master Dec 5, 2025
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI Continuous integration.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants