Skip to content

[MRG] Define custom markers for pytest.#16652

Merged
rth merged 9 commits intoscikit-learn:masterfrom
cmarmo:unknown-pytest-mark
Mar 23, 2020
Merged

[MRG] Define custom markers for pytest.#16652
rth merged 9 commits intoscikit-learn:masterfrom
cmarmo:unknown-pytest-mark

Conversation

@cmarmo
Copy link
Copy Markdown
Contributor

@cmarmo cmarmo commented Mar 6, 2020 •
edited
Loading

Reference Issues/PRs

Fix part of #16651: PytestUnknownMarkWarning in test_gradient_boosting.py

What does this implement/fix? Explain your changes.

Add marker custom definition in conftest.py.
For now this works only for builds using python != 3.7.
See the build log for CHECK_ALL_WARNINGS set to True.

@jnothman
Copy link
Copy Markdown
Member

jnothman commented Mar 7, 2020

(I'm glad pytest no longer silently fails with unknown marks!)

@cmarmo cmarmo changed the title [WIP] Define custom markers for pytest. [MRG] Define custom markers for pytest. Mar 10, 2020
@cmarmo
Copy link
Copy Markdown
Contributor Author

cmarmo commented Mar 10, 2020

(I'm glad pytest no longer silently fails with unknown marks!)

I'm glad you are glad... :)
Unfortunately, I'm unable to understand why the marker declaration is not taken into account in the Windows py37_conda_mkl build. May I ask for review for this one in order to fix #16651 for now? Thanks for your patience.

@jnothman
Copy link
Copy Markdown
Member

jnothman commented Mar 10, 2020

What evidence do you have that it's not taken into account? The test doesn't fall, so I'm not sure what I'll meant to be looking for. Thanks for any clues, @cmarmo!

@cmarmo
Copy link
Copy Markdown
Contributor Author

cmarmo commented Mar 11, 2020

I have created a pull request into my master version, adding to all the builds a CHECK_ALL_WARNINGS in the test suite: this makes some builds fail due to test warnings. I've chosen to proceed build by build.
#16651 corresponds to the Linux pylatest_pip_openblas_pandas warnings.
This PR is open at the same time for scikit-learn/scikit-learn:master and for cmarmo/sckit-learn:test-fails-on-warnings where the warnings throw an error. This PR silents the PytestUnknownMarkWarning in test_gradient_boosting.py for Linux pylatest_pip_openblas_pandas.
Hope this helps...

@cmarmo cmarmo mentioned this pull request Mar 13, 2020
Closed
4 tasks
@cmarmo cmarmo mentioned this pull request Mar 13, 2020
Open
NicolasHug
NicolasHug approved these changes Mar 13, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @cmarmo

I confirm that this removes the warning.

However the recommended way is to create a pytest.ini file. (see https://docs.pytest.org/en/latest/example/markers.html#registering-markers)

The current solution is recommended when implementing plugins (https://docs.pytest.org/en/latest/writing_plugins.html#registering-markers) which isn't really our use-case here.

Any thought against adding a pytest.ini? I guess we'd have to ship it in the source dist as well.

LGTM anyway

@rth
Copy link
Copy Markdown
Member

rth commented Mar 13, 2020

However the recommended way is to create a pytest.ini file. (see docs.pytest.org/en/latest/example/markers.html#registering-markers)

If it works in conftest.py I would rather keep it there. We have way too many various config files as it is. The docs doesn't say it's the recommended approach, just a simpler one. It should be equivalent I imagine.

NicolasHug
NicolasHug reviewed Mar 13, 2020
View reviewed changes
@cmarmo
Copy link
Copy Markdown
Contributor Author

cmarmo commented Mar 13, 2020

Any thought against adding a pytest.ini? I guess we'd have to ship it in the source dist as well.

The idea was not to change the dist, indeed...
modifying the setup.cfg file will work too (see the first commit) in this PR.
I'm just diving into pytest so any of those three solutions is equivalent for me.

cmarmo and others added 2 commits March 13, 2020 18:43
@cmarmo @NicolasHug
Update conftest.py
f75190c 
Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>
@cmarmo
Fix lint error.
0e75f48
@cmarmo
Copy link
Copy Markdown
Contributor Author

cmarmo commented Mar 13, 2020

For the sake of completeness this was my ref link:
https://docs.pytest.org/en/latest/mark.html

@NicolasHug
Copy link
Copy Markdown
Member

NicolasHug commented Mar 14, 2020

Is there still an issue with the windows CI?
Else I'd say we can merge

@cmarmo
Copy link
Copy Markdown
Contributor Author

cmarmo commented Mar 14, 2020

Is there still an issue with the windows CI?
Else I'd say we can merge

Sorry it took some time, Azure builds are bugging for me, I had to run pipelines manually... so yes there is still an issue with Windows py36_pip_openblas_32bit (win32 -- Python 3.6.8, pytest-5.4.1): the PytestUnknownMarkWarning is still there. If you have any idea .... Thanks.

@NicolasHug
Copy link
Copy Markdown
Member

NicolasHug commented Mar 15, 2020

Hm, no idea. It doesn't seem to be related to pytest's version because I have the same locally. Could be a pytest bug?

@cmarmo cmarmo mentioned this pull request Mar 22, 2020
Closed
rth
rth approved these changes Mar 23, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@rth rth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the reminder @cmarmo. LGTM. I opened #16750 to improve the documentation of this mechanism.

@rth rth merged commit dcfb3df into scikit-learn:master Mar 23, 2020
@cmarmo cmarmo mentioned this pull request Mar 23, 2020
Open
@cmarmo cmarmo deleted the unknown-pytest-mark branch March 23, 2020 13:22
gio8tisu pushed a commit to gio8tisu/scikit-learn that referenced this pull request May 15, 2020
@cmarmo @gio8tisu
MAINT Define custom markers for pytest (scikit-learn#16652)
a39699f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@NicolasHug NicolasHug NicolasHug approved these changes

@rth rth rth approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@cmarmo @jnothman @rth @NicolasHug