Skip to content

Handle test warnings#1313

Merged
jni merged 28 commits intoscikit-image:masterfrom
blink1073:suppress-test-warnings
Dec 27, 2014
Merged

Handle test warnings#1313
jni merged 28 commits intoscikit-image:masterfrom
blink1073:suppress-test-warnings

Conversation

@blink1073
Copy link
Copy Markdown
Contributor

Addresses #1182.

  • Adds test_setup and test_teardown functions in skimage._shared.testing to control the test environment, which is called from each test folder in its __init__.py.
  • The test_setup function imports packages known to raise warnings on import, then sets warnings.simplefilter('error) to raise errors for warnings.
  • The test_teardown removes the filter, allowing doctests to raise warnings. It was deemed unwise to skip large swaths of doctests just because they were raising warnings.
  • An attempt was made to get a majority of the doctests to not raise errors, by not removing the warning filter in test_teardown. This technique could be used in future improvement endeavours.
  • Also adds a function expected_warnings in skimage._shared._warnings that allows us to flexibly mark tests for any combination of expected warnings, instead of them causing errors.

@jni
Copy link
Copy Markdown
Member

jni commented Dec 21, 2014

@blink1073 WOW.

Looks like you figured out the checklists AND kicked the checklist's ass! =)

@blink1073 blink1073 force-pushed the suppress-test-warnings branch from dbf1cae to 187ea08 Compare December 21, 2014 15:04
@blink1073
Copy link
Copy Markdown
Contributor Author

Help @scikit-image/core, I've got the whole thing done save for one error. The test passes when I run nosetests from skimage.filters, but not when I run it from skimage:

======================================================================

FAIL: skimage.filters.rank.tests.test_rank.test_all

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/venv/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest

self.test(*self.arg)

File "/home/travis/build/blink1073/scikit-image/skimage/filters/rank/tests/test_rank.py", line 18, in test_all

check_all()

File "/home/travis/build/blink1073/scikit-image/skimage/filters/rank/tests/test_rank.py", line 27, in check_all

rank.autolevel(image, selem))

File "/usr/lib/python2.7/dist-packages/numpy/testing/utils.py", line 256, in assert_equal

return assert_array_equal(actual, desired, err_msg, verbose)

File "/usr/lib/python2.7/dist-packages/numpy/testing/utils.py", line 707, in assert_array_equal

verbose=verbose, header='Arrays are not equal')

File "/usr/lib/python2.7/dist-packages/numpy/testing/utils.py", line 636, in assert_array_compare

raise AssertionError(msg)

AssertionError:

Arrays are not equal

(mismatch 92.16%)

x: array([[ 0, 255, 37, 171, 8, 255, 0, 218, 255, 98, 255, 0, 25,

249, 0, 49, 0, 255, 225, 165, 255, 195, 126, 255, 0],

[255, 0, 241, 126, 168, 42, 255, 80, 148, 0, 196, 253, 31,...

y: array([[ 0, 201, 0, 216, 55, 0, 193, 36, 255, 0, 255, 0, 179,

0, 255, 110, 0, 255, 4, 199, 0, 216, 70, 222, 255],

[220, 255, 42, 200, 255, 99, 20, 72, 224, 11, 66, 207, 130,...

----------------------------------------------------------------------

@blink1073
Copy link
Copy Markdown
Contributor Author

Well, its fixed, but it's a head scratcher. There's a call to random.seed() at the top of the test file, but apparently the file was imported, then another file with a different random.seed() was run before actually running the tests in test_rank. When I put the random.seed() call within the check_all, all is well. Hurray for randomness!

@blink1073 blink1073 changed the title Suppress test warnings Handle test warnings Dec 22, 2014
@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage decreased (-0.36%) when pulling 1131e75 on blink1073:suppress-test-warnings into acc06d0 on scikit-image:master.

@blink1073
Copy link
Copy Markdown
Contributor Author

This is ready for review. I had one failed build in the last commit due to a network timeout, restarted with fingers crossed!

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage decreased (-0.36%) when pulling 506d12d on blink1073:suppress-test-warnings into acc06d0 on scikit-image:master.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit of a misdirection... I would prefer from skimage._shared._warnings import all_warnings. Does that make sense?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And would not with warnings.catch_warnings() work? I think it reads better.

Final thought: I don't know whether it's feasible but it would be nice to specifically discard expected warnings, not all warnings... Sorry for putting that out there... =P Thoughts?

@jni
Copy link
Copy Markdown
Member

jni commented Dec 23, 2014

@blink1073 A phenomenal amount of work! Sorry to pile more on... There's a few nitpicky things, a few clarifications that I'd like to have in the code for future us, but mostly, I'm not entirely convinced that the approach of hiding all warnings in a block of code is right (rather than just expected warnings). For example, in try-except clauses, we always specify the type of error we want to catch, so that if something truly unexpected happens, you don't suppress it silently.

Would it be possible, for example, to match the message in the warning, adding the capability to all_warnings? e.g.

with all_warnings(matching='precision'):
     img_as_ubyte(float_image)

I'm hoping this won't be too hard to fix since you've been so diligent with commenting the type of error in your context calls... What do you think?

@blink1073
Copy link
Copy Markdown
Contributor Author

@jni, you're right, it did feel like I was writing a bare try/except block everywhere. I'll beef up all_warnings and move it to _warnings.py.

@blink1073
Copy link
Copy Markdown
Contributor Author

@jni what do you think of this:

@contextmanager
def catch_warning(matching):
    """Context for use in testing to catch a known warning matching a regex

    Parameters
    ----------
    matching : str or compiled regex
        Regex for the desired warning to catch


    Examples
    --------
    >>> from skimage import data, img_as_ubyte, img_as_float
    >>> with catch_warning('precision loss'):
    ...   d = img_as_ubyte(img_as_float(data.coins()))

    Notes
    -----
    Uses `all_warnings` to ensure all warnings are raised.
    Upon exiting, it checks the recorded warnings for the desired matching
    string.

    """
    with all_warnings() as w:
        yield w
        matched = False
        for warn in w:
            if re.search(matching, str(warn.message)):
                matched = True
        if not matched:
            raise ValueError('No warning raised matching: "%s"' % matching)

@blink1073
Copy link
Copy Markdown
Contributor Author

@jni improved version:

@contextmanager
def catch_warning(matching):
    """Context for use in testing to catch a known warning matching a regex

    Parameters
    ----------
    matching : str or compiled regex
        Regex for the desired warning to catch


    Examples
    --------
    >>> from skimage import data, img_as_ubyte, img_as_float
    >>> with catch_warning('precision loss'):
    ...   d = img_as_ubyte(img_as_float(data.coins()))

    Notes
    -----
    Uses `all_warnings` to ensure all warnings are raised.
    Upon exiting, it checks the recorded warnings for the desired matching
    string.  Raises a warning if the match was not found or an Unexpected
    warning is found.

    """
    with all_warnings() as w:
        yield w
        matched = False
        for warn in w:
            if re.search(matching, str(warn.message)):
                matched = True
            else:
                raise ValueError('Unexpected warning: %s' % str(warn.message))
        if not matched:
            raise ValueError('No warning raised matching: "%s"' % matching)

@blink1073
Copy link
Copy Markdown
Contributor Author

Well that did not work, I'm not sure.

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage decreased (-0.36%) when pulling a7a84bb on blink1073:suppress-test-warnings into 8955bbf on scikit-image:master.

@blink1073
Copy link
Copy Markdown
Contributor Author

cc/ @tacaswell, any idea why we'd get this traceback after a test? I made sure we are calling fig.canvas.draw() right after we create the fig and axes. Also, the figure is explicitly added to the QMainWindow's layout. Maybe we need to use the cleanup decorator?

Traceback (most recent call last):

File "/home/travis/venv/lib/python2.6/site-packages/matplotlib/backends/backend_qt5.py", line 427, in idle_draw

self.draw()

File "/home/travis/venv/lib/python2.6/site-packages/matplotlib/backends/backend_qt5agg.py", line 149, in draw

self._priv_update()

RuntimeError: Internal C++ object (FigureCanvas) already deleted.

@jni
Copy link
Copy Markdown
Member

jni commented Dec 27, 2014

@blink1073 there's a couple of other warnings missed:

https://travis-ci.org/scikit-image/scikit-image/jobs/45190537#L6695
https://travis-ci.org/scikit-image/scikit-image/jobs/45190537#L6703
https://travis-ci.org/scikit-image/scikit-image/jobs/45190537#L6846
https://travis-ci.org/scikit-image/scikit-image/jobs/45190537#L6985

Personally I think this PR is already extremely valuable and would be happy to leave these till next time. Your call if you want to complete it!

@blink1073
Copy link
Copy Markdown
Contributor Author

@jni, all four of those are in doctests, and I'd rather run the doctest and get a warning than skip it. All of the warnings are reasonable to me.

@jni
Copy link
Copy Markdown
Member

jni commented Dec 27, 2014

@blink1073 but why not ignore the doctest output?

@blink1073
Copy link
Copy Markdown
Contributor Author

Sure, but how?

@jni
Copy link
Copy Markdown
Member

jni commented Dec 27, 2014

# doctest: +IGNORE_RESULT ?

@blink1073
Copy link
Copy Markdown
Contributor Author

That is not a built-in command, and I'm not sure if doctest.register_optionflag plays nicely with nose. I say punt.

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage decreased (-0.36%) when pulling ecc5821 on blink1073:suppress-test-warnings into 8955bbf on scikit-image:master.

@blink1073
Copy link
Copy Markdown
Contributor Author

Welp, @cleanup did not help...

@jni
Copy link
Copy Markdown
Member

jni commented Dec 27, 2014

Welp indeed. =P maybe rip out that last commit and I'll merge?

@blink1073
Copy link
Copy Markdown
Contributor Author

I say it can't hurt to have it in there, there is some leakage when you don't use it.

@jni
Copy link
Copy Markdown
Member

jni commented Dec 27, 2014

Well, you've certainly earned your say. In it goes! ;)

jni added a commit that referenced this pull request Dec 27, 2014
@jni jni merged commit 7b5dd3b into scikit-image:master Dec 27, 2014
@jni
Copy link
Copy Markdown
Member

jni commented Dec 27, 2014

Boom. Awesome.

@blink1073
Copy link
Copy Markdown
Contributor Author

Yay! That one sucked...

@blink1073
Copy link
Copy Markdown
Contributor Author

Another PR to remove the label wrapper?

@jni
Copy link
Copy Markdown
Member

jni commented Dec 27, 2014

Sucked for you, but @blink1073, it is one of the rockingest PRs ever. Super-clean builds are a great thing. And the warning catcher is excellent. Thank you!

@jni
Copy link
Copy Markdown
Member

jni commented Dec 27, 2014

@blink1073 I won't say no...! =)

@blink1073
Copy link
Copy Markdown
Contributor Author

I may be the team janitor, but I make that toilet shine!

@jni
Copy link
Copy Markdown
Member

jni commented Dec 27, 2014

ROTFL

@blink1073 blink1073 deleted the suppress-test-warnings branch December 27, 2014 03:35
@ahojnnes
Copy link
Copy Markdown
Member

The team janitor did great work! :-)

@stefanv
Copy link
Copy Markdown
Member

stefanv commented Dec 27, 2014

@jni I see another import of "filter" instead of "filters" in this PR

@jni
Copy link
Copy Markdown
Member

jni commented Dec 27, 2014

@stefanv crap! Looks like my... filters... need updating! =P

@grlee77 grlee77 mentioned this pull request Apr 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants