Skip to content

chore: run tests for importance only with in-memory#6109

Merged
HideakiImamura merged 1 commit intooptuna:masterfrom
himkt:importance-init-inmemory
May 31, 2025
Merged

chore: run tests for importance only with in-memory#6109
HideakiImamura merged 1 commit intooptuna:masterfrom
himkt:importance-init-inmemory

Conversation

@himkt
Copy link
Copy Markdown
Member

@himkt himkt commented May 31, 2025 •
edited
Loading

Motivation

Some parts of importance test (test_init.py) comprehensively run on available storages in Optuna. But the logic is basically storage class agnostic. This PR makes the run happen only with in-memory storage.

Description of the changes

Before

> uv run -m pytest tests/importance_tests/test_init.py
=============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.12.0, pytest-8.3.5, pluggy-1.6.0
rootdir: /Users/himkt/work/github.com/himkt/optuna
configfile: pyproject.toml
plugins: profiling-1.8.1
collected 197 items

tests/importance_tests/test_init.py ....................................................................................................................................... [ 68%]
..............................................................                                                                                                              [100%]

================================================================================ warnings summary =================================================================================
tests/importance_tests/test_init.py: 26 warnings
  /Users/himkt/work/github.com/himkt/optuna/.venv/lib/python3.12/site-packages/fakeredis/_connection.py:179: DeprecationWarning: Call to '__init__' function with deprecated usage of input argument/s 'retry_on_timeout'. (TimeoutError is included by default.) -- Deprecated since version 6.0.0.
    super().__init__(**kwds)

tests/importance_tests/test_init.py::test_get_param_importance_target_is_none_and_study_is_multi_obj[grpc_rdb-MeanDecreaseImpurityImportanceEvaluator]
  /Users/himkt/work/github.com/himkt/optuna/.venv/lib/python3.12/site-packages/google/protobuf/runtime_version.py:98: UserWarning: Protobuf gencode version 5.28.1 is exactly one major version older than the runtime version 6.31.1 at api.proto. Please update the gencode to avoid compatibility violations in the next runtime release.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================================== 197 passed, 27 warnings in 36.10s ========================================================================

After

> uv run -m pytest tests/importance_tests/test_init.py
=============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.12.0, pytest-8.3.5, pluggy-1.6.0
rootdir: /Users/himkt/work/github.com/himkt/optuna
configfile: pyproject.toml
plugins: profiling-1.8.1
collected 41 items

tests/importance_tests/test_init.py .........................................                                                                                               [100%]

=============================================================================== 41 passed in 3.17s ================================================================================

@himkt himkt self-assigned this May 31, 2025
@himkt himkt requested a review from Copilot May 31, 2025 04:49
Copilot AI reviewed May 31, 2025
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the importance tests to run only against the in-memory storage backend, removing all parametrization over different storage modes. Key changes:

  • Deleted imports and uses of STORAGE_MODES and StorageSupplier
  • Updated all create_study calls to rely on the default (in-memory) storage
  • Removed storage_mode parameter from test function signatures and decorators
Comments suppressed due to low confidence (1)

tests/importance_tests/test_init.py:50

  • Tests now only run with in-memory storage and no longer cover other storage backends. Ensure that other storage modes are tested elsewhere or explicitly document this change to avoid gaps in storage-agnostic behavior coverage.
study = create_study(directions=["minimize", "minimize"])

Alnusjaponica
Alnusjaponica approved these changes May 31, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@Alnusjaponica Alnusjaponica left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for your contribution.

@Alnusjaponica Alnusjaponica added test Unit test. sprint-20250531 PR from the online sprint event May 31, 2025. labels May 31, 2025
@HideakiImamura HideakiImamura self-assigned this May 31, 2025
HideakiImamura
HideakiImamura approved these changes May 31, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@HideakiImamura HideakiImamura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing speedup!!! LGTM!!!

@HideakiImamura HideakiImamura merged commit 353d569 into optuna:master May 31, 2025
16 checks passed
@HideakiImamura HideakiImamura added this to the v4.4.0 milestone May 31, 2025
@himkt himkt deleted the importance-init-inmemory branch May 31, 2025 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@HideakiImamura HideakiImamura HideakiImamura approved these changes

@Alnusjaponica Alnusjaponica Alnusjaponica approved these changes

Assignees

@himkt himkt

@HideakiImamura HideakiImamura

Labels

sprint-20250531 PR from the online sprint event May 31, 2025. test Unit test.

Projects

None yet

Milestone

v4.4.0

Development

Successfully merging this pull request may close these issues.

4 participants

@himkt @HideakiImamura @Alnusjaponica