Made error message for `create_study`'s direction easier to understand `optuna.study` by sinano1107 · Pull Request #6021 · optuna/optuna

sinano1107 · 2025-03-27T14:41:48Z

Motivation

Users sometimes makes the mistake of passing the list of directions to the direction argument. The error message for this mistake is hard to understand what is the problem.

Correct

optuna.create_study(directions=["minimize", "minimize"])

Wrong

Using direction instead of directions

optuna.create_study(direction=["minimize", "minimize"])

ValueError: Please set either 'minimize' or 'maximize' to direction. You can also set the corresponding `StudyDirection` member.

Description of the changes

The error message displayed when a list object, is assigned to the direction parameter of create_study has been changed as follows.

previous

ValueError: Please set either 'minimize' or 'maximize' to direction. You can also set the corresponding `StudyDirection` member.

new

ValueError: For multi-objective optimization, using `directions` instead of `direction`.

optuna/study/study.py

sinano1107 · 2025-03-28T00:24:02Z

For some reason docs/readthedocs.org:optuna now fails.
If you know anything about this, I would appreciate it.

…imization

sinano1107 · 2025-03-28T00:54:50Z

I thought the docs/readthedocs.org:optuna might be failing because I used git push -force-with-lease, so I created a new #6022, it also has the same error.

Sorry for the mess in the history and PR section.

In the end I don't know the cause either.

sinano1107 · 2025-03-28T00:58:36Z

I don't know why, but it didn't seem to pinch this time!

nzw0301

Yeah, CI jobs sometimes fail due to an external service. Sorry for confusing you.

In terms of PR, could you add a test?

sinano1107 · 2025-03-28T06:51:56Z

I added a test in the form of an extension to test_optimize_with_direction test_create_study_with_multi_objectives, but I feel it is a little less prospective.

Let's change it to create a new function like test_create_study_with_invalid_args to test it if necessary. Which would you prefer?

HideakiImamura · 2025-03-28T06:57:51Z

@nzw0301 Could you continue to review this PR?

HideakiImamura

Thanks for the PR. I have several comments. PTAL.

HideakiImamura · 2025-03-28T06:59:08Z

tests/study_tests/test_study.py

+    assert str(e.value) == (
+        "Please set either 'minimize' or 'maximize' to direction. You can also set the "
+        + "corresponding `StudyDirection` member."
+    )


IMO, we should not test the exact contents of the error message since it is not a specification, so might be changed later. It would be great to confirm that the error is emitted.

HideakiImamura · 2025-03-28T06:59:15Z

tests/study_tests/test_study.py

+    assert (
+        str(e.value)
+        == "For multi-objective optimization, using `directions` instead of `direction`."
+    )


HideakiImamura · 2025-03-28T06:59:22Z

tests/study_tests/test_study.py

+    with pytest.raises(ValueError) as e:
        # Empty `direction` isn't allowed.
        _ = create_study(directions=[])
+    assert str(e.value) == "The number of objectives must be greater than 0."


HideakiImamura · 2025-03-28T06:59:29Z

tests/study_tests/test_study.py

-    with pytest.raises(ValueError):
+    with pytest.raises(ValueError) as e:
        _ = create_study(direction="minimize", directions=["maximize"])
+    assert str(e.value) == "Specify only one of `direction` and `directions`."


HideakiImamura · 2025-03-28T06:59:35Z

tests/study_tests/test_study.py

-    with pytest.raises(ValueError):
+    with pytest.raises(ValueError) as e:
        _ = create_study(direction="minimize", directions=[])
+    assert str(e.value) == "Specify only one of `direction` and `directions`."


optuna/study/study.py

sinano1107 · 2025-03-28T07:21:33Z

@HideakiImamura
Yes, I agree, but such a test would not detect the changes made in this PR. The code before the changes are made will still pass the test.

Do you think it is ok for us to ignore it?

nzw0301

Thanks! The added test looks perfect. Let me leave a minor comment.

tests/study_tests/test_study.py

Co-authored-by: Kento Nozawa <k_nzw@klis.tsukuba.ac.jp>

nabenabe0928

I think this part should also be modified.

optuna/optuna/study/study.py

Lines 1258 to 1261 in 896e0ca

    
           raise ValueError( 
        
               "Please set either 'minimize' or 'maximize' to direction. You can also set the " 
        
               "corresponding `StudyDirection` member." 
        
           )

Could you, @sinano1107 , please check the comment here as well?

sinano1107 · 2025-03-28T10:56:47Z

@nabenabe0928
Does this edit captured your intentions?

nzw0301

LGTM!

nabenabe0928 · 2025-03-31T03:44:34Z

@sinano1107
Yes, the change looks good to me:)
Could you add the unit test for this change as well?
Namely, could you add a test using directions="minimize", which is an invalid input.

sinano1107 · 2025-04-01T03:53:05Z

@nabenabe0928
I'm happy to hear that from you.
I have added a test, please check it out.

nabenabe0928

LGTM!

sinano1107 force-pushed the master branch from 1364be3 to 18546f2 Compare March 27, 2025 14:50

nzw0301 linked an issue Mar 27, 2025 that may be closed by this pull request

Make error message for create_study's direction easier to understand #5806

Closed

nzw0301 reviewed Mar 27, 2025

View reviewed changes

optuna/study/study.py Outdated Show resolved Hide resolved

nzw0301 added the code-fix Change that does not change the behavior, such as code refactoring. label Mar 27, 2025

sinano1107 force-pushed the master branch 2 times, most recently from 3d4ca56 to b2ba40f Compare March 28, 2025 00:37

add understandable error message for direction in multi-objective opt…

36bed38

…imization

sinano1107 force-pushed the master branch from ae1c8af to 36bed38 Compare March 28, 2025 00:40

sinano1107 mentioned this pull request Mar 28, 2025

add understandable error message for direction in multi-objective opt… #6022

Closed

sinano1107 force-pushed the master branch from aa35e1f to 36bed38 Compare March 28, 2025 00:57

sinano1107 requested a review from nzw0301 March 28, 2025 01:32

nzw0301 reviewed Mar 28, 2025

View reviewed changes

add test

6130333

HideakiImamura assigned nzw0301 and HideakiImamura Mar 28, 2025

HideakiImamura reviewed Mar 28, 2025

View reviewed changes

nabenabe0928 reviewed Mar 28, 2025

View reviewed changes

optuna/study/study.py Outdated Show resolved Hide resolved

sinano1107 added 2 commits March 28, 2025 16:18

fix test

6688910

fix error message

a2c317f

sinano1107 requested a review from nzw0301 March 28, 2025 07:45

nzw0301 requested changes Mar 28, 2025

View reviewed changes

tests/study_tests/test_study.py Outdated Show resolved Hide resolved

nzw0301 changed the title ~~Made error message for create_study's direction easier to understand optuna.study~~ Made error message for create_study's direction easier to understand optuna.study Mar 28, 2025

Update tests/study_tests/test_study.py

896e0ca

Co-authored-by: Kento Nozawa <k_nzw@klis.tsukuba.ac.jp>

sinano1107 requested a review from nzw0301 March 28, 2025 09:44

nabenabe0928 reviewed Mar 28, 2025

View reviewed changes

update other error message

8b2c631

nzw0301 approved these changes Mar 28, 2025

View reviewed changes

add test

b2e9786

nabenabe0928 approved these changes Apr 1, 2025

View reviewed changes

nabenabe0928 merged commit d31ae17 into optuna:master Apr 1, 2025
14 checks passed

nabenabe0928 added this to the v4.3.0 milestone Apr 1, 2025

	raise ValueError(
	"Please set either 'minimize' or 'maximize' to direction. You can also set the "
	"corresponding `StudyDirection` member."
	)

Uh oh!

Conversation

sinano1107 commented Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Description of the changes

Uh oh!

Uh oh!

sinano1107 commented Mar 28, 2025

Uh oh!

sinano1107 commented Mar 28, 2025

Uh oh!

sinano1107 commented Mar 28, 2025

Uh oh!

nzw0301 left a comment

Choose a reason for hiding this comment

Uh oh!

sinano1107 commented Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HideakiImamura commented Mar 28, 2025

Uh oh!

HideakiImamura left a comment

Choose a reason for hiding this comment

Uh oh!

HideakiImamura Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

HideakiImamura Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

HideakiImamura Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

HideakiImamura Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

HideakiImamura Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sinano1107 commented Mar 28, 2025

Uh oh!

nzw0301 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nabenabe0928 left a comment

Choose a reason for hiding this comment

Uh oh!

sinano1107 commented Mar 28, 2025

Uh oh!

nzw0301 left a comment

Choose a reason for hiding this comment

Uh oh!

nabenabe0928 commented Mar 31, 2025

Uh oh!

sinano1107 commented Apr 1, 2025

Uh oh!

nabenabe0928 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sinano1107 commented Mar 27, 2025 •

edited

Loading

sinano1107 commented Mar 28, 2025 •

edited

Loading