Nasty surprise of Django cache

09 December 2008   10 comments   Django

Powered by Fusion×

Caching in Django absolutely rocks. Much so because of its simplicity which lowers the threshold to the height of a snowflake. However the simplicity can cause some unexpected surprises.

I ran my site and clicked around until I found a bug I wanted to fix. Then I wrote the test and ran the testrunner but I was getting some really weird behavior. The view looked like this:

def club_page(request, clubname):
   club = _find_club(clubname)
   if club is None:
       raise Http404('Could not find the club')

   classes = cache.get('club_page_classes')
   if classes is None:
       classes = ClubClass.objects.filter(club=club).order_by('start_time')
       cache.set('club_page_classes', classes, CACHE_TIMEOUT)

What happened (and what took me a while to figure out) was that the memcache was still active and being used when running the tests since it's only within the hour that I started running the tests with completely different data. Here's how I solved the problem:

class ViewsTestCase(unittest.TestCase):
    def test_club_page(self):
       """ test rendering the club page """

There must be an easier way to patch the testrunner to reset all cache before running the tests. Anybody?


/etc/init.d/memcached restart
Stas Shtin
Why not use locmem backend for tests?
Peter Bengtsson
Unlike memcache, is locmem reset when you start django with ./
Jeethu Rao
Yes its reset, actually its reset whenever the code reloads as well.
I usually do something like this in

if DEBUG :
CACHE_BACKEND = 'locmem:///'
else :
CACHE_BACKEND = 'memcached://'
Peter Bengtsson
That's actually a pretty good idea because I want the cache to be reset every time the dev server is restarted. Thanks.
Eric Moritz
You can monkey patch your cache settings in your test module. Stick this on the top of your test module:

from django.conf import settings
settings.CACHE_BACKEND = 'dummy:///'

A better solution is to have a dev settings module that has no cache and a production settings module that does have cache.

When it comes to caching I like abstracting the "if whatever is None" to a decorator so that it can be reused.

For instance that decorator could be like this:

That way you don't have to worry about the cache being a factor in your tests when it's disabled because you can set up a unittest to test the decorator in isolation and can assume it's functioning correctly in any dependent code.
Peter Bengtsson
Didn't know that was even possible. I'll try that.
I think I want the cache to work in the tests. Otherwise they wouldn't test the cache.
Best would be if I could delete everything from the cache before the tests begin.
Eric Moritz
Yeah that's why I like to abstract the cache functionality out of the function. That way you can test the functionality without caching and test the caching mechanism in two seperate tests.

That way you can reuse the decorator and rest assured that any function that uses it will cache correctly.

The code above was a simple example, it won't work with dynamic cache keys but I'm sure you could figure out a more complex decorator if you need it.
Peter Bengtsson
All good stuff. I'm going to use your little decorator tip too to clean up my code and use locmem (as Stas Shtin pointed out) as the backend. That way cache does come into play in the tests too but reset each time I start the testrunner.
Stas Shtin
Local memory cache exists only in memory space of python process, so when you terminate it old data is gone.
Thank you for posting a comment

Your email will never ever be published

Related posts

bool is instance of int in Python 05 December 2008
Wing IDE versus Jed 11 December 2008
Related by keywords:
To readline() or readlines() 12 March 2004
bool is instance of int in Python 05 December 2008
Reciprocal lesson about gender perspectives 02 September 2011
Nginx vs. Squid 17 March 2009
IssueTrackerProduct now officially abandoned 30 March 2012
How and why to use django-mongokit (aka. Django to MongoDB) 08 March 2010
On the command line no one can hear you screen. Or can they? 03 May 2012
Random ID generator for Zope 02 September 2005
Google Calendar, iCalendar Validator but not bloody Apple iCal 09 April 2009
tempfile in Python standard library 07 February 2006
In Django, how much faster is it to aggregate? 27 October 2010
Google and Python code 22 February 2006