How do I disable and then re-enable a warning?

Question

I'm writing some unit tests for a Python library and would like certain warnings to be raised as exceptions, which I can easily do with the simplefilter function. However, for one test I'd like to disable the warning, run the test, then re-enable the warning.

I'm using Python 2.6, so I'm supposed to be able to do that with the catch_warnings context manager, but it doesn't seem to work for me. Even failing that, I should also be able to call resetwarnings and then re-set my filter.

Here's a simple example which illustrates the problem:

>>> import warnings
>>> warnings.simplefilter("error", UserWarning)
>>> 
>>> def f():
...     warnings.warn("Boo!", UserWarning)
... 
>>> 
>>> f() # raises UserWarning as an exception
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in f
UserWarning: Boo!
>>> 
>>> f() # still raises the exception
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in f
UserWarning: Boo!
>>> 
>>> with warnings.catch_warnings():
...     warnings.simplefilter("ignore")
...     f()     # no warning is raised or printed
... 
>>> 
>>> f() # this should raise the warning as an exception, but doesn't
>>> 
>>> warnings.resetwarnings()
>>> warnings.simplefilter("error", UserWarning)
>>> 
>>> f() # even after resetting, I'm still getting nothing
>>> 

Can someone explain how I can accomplish this?

EDIT: Apparently this is a known bug: http://bugs.python.org/issue4180

Answer 1

Brian Luft is correct about __warningregistry__ being the cause of the problem. But I wanted to clarify one thing: the way the warnings module appears to work is that it sets module.__warningregistry__ for each module where warn() is called. Complicating things even more, the stacklevel option to warnings causes the attribute to be set for the module the warning was issued "in the name of", not necessarily the one where warn() was called... and that's dependent on the call stack at the time the warning was issued.

This means you may have a lot of different modules where the __warningregistry__ attribute is present, and depending on your application, they may all need clearing before you'll see the warnings again. I've been relying on the following snippet of code to accomplish this... it clears the warnings registry for all modules whose name matches the regexp (which defaults to everything):

def reset_warning_registry(pattern=".*"):
    "clear warning registry for all match modules"
    import re
    import sys
    key = "__warningregistry__"
    for mod in sys.modules.values():
        if hasattr(mod, key) and re.match(pattern, mod.__name__):
            getattr(mod, key).clear()

---

Update: CPython issue 21724 addresses issue that resetwarnings() doesn't clear warning state. I attached an expanded "context manager" version to this issue, it can be downloaded from reset_warning_registry.py.

Answer 2

I've run into the same issues, and while all of the other answers are valid I choose a different route. I don't want to test the warnings module, nor know about it's inner workings. So I just mocked it instead:

import warnings
import unittest
from unittest.mock import patch
from unittest.mock import call

class WarningTest(unittest.TestCase):
    @patch('warnings.warn')
    def test_warnings(self, fake_warn):
        warn_once()
        warn_twice()
        fake_warn.assert_has_calls(
            [call("You've been warned."),
             call("This is your second warning.")])

def warn_once():
    warnings.warn("You've been warned.")

def warn_twice():
    warnings.warn("This is your second warning.")

if __name__ == '__main__':
    __main__=unittest.main()

This code is Python 3, for 2.6 you need the use an external mocking library as unittest.mock was only added in 2.7.

Answer 3

Following on from Eli Collins' helpful clarification, here is a modified version of the catch_warnings context manager that clears the warnings registry in a given sequence of modules when entering the context manager, and restores the registry on exit:

from warnings import catch_warnings

class catch_warn_reset(catch_warnings):
    """ Version of ``catch_warnings`` class that resets warning registry
    """
    def __init__(self, *args, **kwargs):
        self.modules = kwargs.pop('modules', [])
        self._warnreg_copies = {}
        super(catch_warn_reset, self).__init__(*args, **kwargs)

    def __enter__(self):
        for mod in self.modules:
            if hasattr(mod, '__warningregistry__'):
                mod_reg = mod.__warningregistry__
                self._warnreg_copies[mod] = mod_reg.copy()
                mod_reg.clear()
        return super(catch_warn_reset, self).__enter__()

    def __exit__(self, *exc_info):
        super(catch_warn_reset, self).__exit__(*exc_info)
        for mod in self.modules:
            if hasattr(mod, '__warningregistry__'):
                mod.__warningregistry__.clear()
            if mod in self._warnreg_copies:
                mod.__warningregistry__.update(self._warnreg_copies[mod])

Use with something like:

import my_module_raising_warnings
with catch_warn_reset(modules=[my_module_raising_warnings]):
    # Whatever you'd normally do inside ``catch_warnings``

Answer 4

Brian is spot on about the __warningregistry__. So you need to extend catch_warnings to save/restore the global __warningregistry__ too

Something like this may work

class catch_warnings_plus(warnings.catch_warnings):
    def __enter__(self):
        super(catch_warnings_plus,self).__enter__()
        self._warningregistry=dict(globals.get('__warningregistry__',{}))
    def __exit__(self, *exc_info):
        super(catch_warnings_plus,self).__exit__(*exc_info)
        __warningregistry__.clear()
        __warningregistry__.update(self._warningregistry)

Answer 5

Reading through the docs and few times and poking around the source and shell I think I've figured it out. The docs could probably improve to make clearer what the behavior is.

The warnings module keeps a registry at __warningsregistry__ to keep track of which warnings have been shown. If a warning (message) is not listed in the registry before the 'error' filter is set, any calls to warn() will not result in the message being added to the registry. Also, the warning registry does not appear to be created until the first call to warn:

>>> import warnings
>>> __warningregistry__
------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython console>", line 1, in <module>
NameError: name '__warningregistry__' is not defined

>>> warnings.simplefilter('error')
>>> __warningregistry__
------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython console>", line 1, in <module>
NameError: name '__warningregistry__' is not defined

>>> warnings.warn('asdf')
------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython console>", line 1, in <module>
UserWarning: asdf

>>> __warningregistry__
{}

Now if we ignore warnings, they will get added to the warnings registry:

>>> warnings.simplefilter("ignore")
>>> warnings.warn('asdf')
>>> __warningregistry__
{('asdf', <type 'exceptions.UserWarning'>, 1): True}
>>> warnings.simplefilter("error")
>>> warnings.warn('asdf')
>>> warnings.warn('qwerty')
------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython console>", line 1, in <module>
UserWarning: qwerty

So the error filter will only apply to warnings that aren't already in the warnings registry. To make your code work you'll need to clear the appropriate entries out of the warnings registry when you're done with the context manager (or in general any time after you've used the ignore filter and want a prev. used message to be picked up the error filter). Seems a bit unintuitive...