Careful with your assertRaises() and inheritance of exceptions

10 April 2013   10 comments   Python

Mind That Age!

This blog post is 5 years old! Most likely, its content is outdated. Especially if it's technical.

This took me by surprise today!

If you run this unit test, it actually passes with flying colors:

import unittest

class BadAssError(TypeError):

def foo():
    raise BadAssError("d'oh")

class Test(unittest.TestCase):

    def test(self):
        self.assertRaises(BadAssError, foo)
        self.assertRaises(TypeError, foo)
        self.assertRaises(Exception, foo)

if __name__ == '__main__':

Basically, assertRaises doesn't just take the exception that is being raised and accepts it, it also takes any of the raised exceptions' parents.

I've only tested it with Python 2.6 and 2.7. And the same works equally with unittest2.

I don't really know how I feel about this. It did surprise me when I was changing one of the exceptions and expected the old tests to break but they didn't. I mean, if I want to write a test that really makes sure the exception really is BadAssError it means I can't use assertRaises().


Matheus Gaudencio
A BadAssError IS A TypeError and also IS A Exception. There is no problem there.

Your tests were probably using a parent to check against an exception instead of the specific type.

For instance, if you change foo() to raise TypeError, the first assertRaises will fail.
Jean-Paul Calderone
It directly parallels the behavior of "except". If an exception would be handled by "except Foo", then "assertRaises(Foo, ...)" will pass. If you changed your exceptions and your unit tests kept passing, consider what that will mean for *application code* already written to handle certain exceptions. Did you want the different exception type to continue to be handled by existing code? Or did you want it to start bypassing certain "except" suites?

I think Python's idea of respecting inheritance hierarchies in the exception handling system itself is questionable, but given that's what we have, this behavior seems to make sense in `assertRaises`. Also, note that `assertRaises` will give you exact exception object if you want it, so you can make additional assertions about it - such as its exact type.
Jason P.
How is this any different than a test like

def testClass(self):
    x = ValueError(example)
    self.assertTrue(isinstance(x, StandardError))

Why should checking class membership have different semantics for assertRaises than it does for isinstance?

If you really need to validate that the exact class is being raised, and not a subclass (which you shouldn't ever need to do, since try...except clauses will still catch the subclass), you can use the context manager behavior of assertRaises in unittest2 or Python 2.7

with self.assertRaises(BadAssError) as cm:

self.assertEqual(cm.exception.__class__, BadAssError)
Peter Bengtsson
Why? In the same sense that `assertEqual` does an `==` not an `isintance`.
My point is that it's a bit surprising. I'm not pointing out that it's a bug.
Brandon Craig Rhodes
I am not sure what you mean. If you want to make sure that the exception “really is” BadAssError, then you test whether assertRaises(BadAssError,…) and you get a real, accurate answer about whether the exception raised will match the pattern “except BadAssError…” in your users' code. It is true that the check assertRaises(BadAssError,…) will *also* succeed if the exception raised is a subclass of BadAssError — but if you were afraid of *that*, then you simply wouldn't create a child-class exception of BadAssError, right?
Peter Bengtsson
I know it's a very "silly" issue. I raised this (no pun intended) because I refactored my code to use more explicit exception classes but got baffled that my tests continued to pass even though I was messing around in the code.
I don't see what the problem is: a BadAssError is a TypeError is an Exception, so it makes perfect sense that raising a BadAssError and catching a TypeError will pass: that's exactly what a standard `except` will do.

> I mean, if I want to write a test that really makes sure the exception really is BadAssError

The only case where this could be an issue is if your BadAssError was subtyped (so you had a BadAsserError(BadAssError)) and that subtype was raised and you wanted the base and not the subtype.

The chances of exactly this occuring are pretty low, and in that case you should feel free to use a standard try/except and asserting that `type(e) is BadAssError` (if that snipped does not make you squirm, there might be something wrong with you)
A. Jesse Jiryu Davis
I'm not surprised by this behavior, but sometimes you do want to check the exact error class. When I changed a function in PyMongo to raise a different error in the same hiearchy, I wrote assertRaisesExactly as an alternative to assertRaises:
Peter Bengtsson
Thank you! Clearly I'm not alone in being anal about testing *exactly* which exception is raised.
Iñaki Silanes Cristóbal
For a enlightening metaphor, imagine that in your example, "Exception" is "animal", "TypeError" is "dog", and "BadAssError" is "poodle". You have some function that returns a poodle, which naturally must make the answer to those questions "Yes":

* Did the function return a poodle?
* Did the function return a dog?
* Did the function return an animal?

If you want to test for dog, regardless of breed, test for dog. If you want to test for poodle, and beagle or labrador won't do, test for poodle. But why would you want to test for dog and fail if it is a poodle? If you want a test that would pass for any dog except a poodle, then do two tests:

* assert it is a dog
* assert it is not a poodle

If you were always testing for dogs and suddenly you realize some functions should return a general "dog", but some others should return specifically "poodle" or "labrador", there is no reason why all old tests should not still pass. All functions are still returning dogs. If you want to test some of them for poodle, you can make that specific test for those specific functions.
Thank you for posting a comment

Your email will never ever be published

Related posts

Recruiters: if you're going to lie, do it properly 07 April 2013
Local jed settings 19 April 2013
Related by Keyword:
Chainable catches in a JavaScript promise 05 November 2015
Who was logged in during a Django exception 15 April 2010
Mocking os.stat in Python 08 November 2009
To assert or assertEqual in Python unit testing 14 February 2009
String comparison function in Python (alpha) 22 December 2007
Related by Text:
HTML Tree on Hacker News 18 May 2014
YouTube - Nigella's XXXmas 19 December 2008
How and why to use django-mongokit (aka. Django to MongoDB) 08 March 2010
How to track Google Analytics pageviews on non-web requests (with Python) 03 May 2016
hashin 0.7.0 and multiple packages 30 August 2016