Do you train Kung Fu?
Or know someone who does?
Then check out KungFuPeople.com
Mobile version of this pageLaunching Kwissle.com
Next:
Chinese tea sampler pack now on sale
Related blogs
Optimized stylesheetsPersistent caching with fire-and-forget updates
Mocking DBRefs in Mongoose and nodeunit
Correction: running Django tests with MongoDB is NOT slow
mongoengine vs. django-mongokit
MongoUK 2011 - London conference all about MongoDB
How I made my MongoDB based web app 10 times faster
Speed test between django_mongokit and postgresql_psycopg2
Optimization of getting random rows out of a PostgreSQL in Django
DoneCal homepage now able to do 10,000 requests/second
More optimization of Peterbe.com - CSS sprites
Fastest way to uniqify a list in Python
Date formatting in Python or in PostgreSQL (part II)
Quick PostgreSQL optimization story
Python optimization anecdote
The problem with CSS
Related by category
Python
MongoDB
Optimization story involving something silly I call "dict+"
http://https://gist.github.com/102177712th of June 2011
Here's a little interesting story about using MongoKit to quickly draw items from a MongoDB
So I had a piece of code that was doing a big batch update. It was slow. It took about 0.5 seconds per user and I sometimes had a lot of users to run it for.
The code looked something like this:
if play.winner == user:
bla()
elif play.draw:
ble()
else:
blu()
Because the model PlayedQuestion contains DBRefs MongoKit will automatically look them up for every iteration in the main loop. Individually very fast (thank you indexes) but because of the number of operations very slow in total. Here's how to make it much faster:
The problem with this is that you get dict instances for each which is more awkward to work with. I.e. instead of `play.winner` you have use `play['winner'].id`.
Here's my solution that makes this a lot easier:
def __init__(self, *args, **kwargs):
if 'collection' in kwargs: # excess we don't need
kwargs.pop('collection')
dict.__init__(self, *args, **kwargs)
self._wrap_internal_dicts()
def _wrap_internal_dicts(self):
for key, value in self.items():
if isinstance(value, dict):
self[key] = dict_plus(value)
def __getattr__(self, key):
return self[key]
...
for play in db.PlayedQuestion.collection.find({'user.$id': user._id}):
play = dict_plus(play)
if play.winner.id == user._id:
bla()
elif play.draw:
ble()
else:
blu()
Now, the whole thing takes 0.01 seconds instead of 0.5. 50 times faster!!
