Following on from yesterday's blog about How and why to use django-mongokit I extended the exampleproject which is inside the django-mongokit project with another app called exampleapp_sql which does the same thing as the exampleapp but does it with SQL instead. Then I added a very simple benchmarker app in the same project and wrote three functions:

1. One to create 10/100/500/1000 instances of my class
2. One to edit one field of all 10/100/500/1000 instances
3. One to delete each of the 10/100/500/1000 instances

Here I'm going to explain how to combine Django and MongoDB using MongoKit and django-mongokit.

MongoDB is a document store built for high speed and high concurrency with a very good redundancy story. It's an alternative to relational databases (e.g. MySQL) that is what Django is tightly coupled with in it's ORM (Object Relation Mapping) and what it's called now is ODM (Object Document Mapping) in lack of a better acronym. That's where MongoKit comes in. It's written in Python and it connects to the MongoDB database using a library called pymongo and it turns data from the MongoDB and turns it into instances of classes you have defined. MongoKit has nothing to do with Django. That's where django-mongokit comes in. Written by yours truly.

Which came first?

Does it matter?

A friend of mine found a nice car she on www.enterprisecarsales.com so she copied the current URL from the address bar and emailed that to me. The URL was: http://www.enterprisecarsales.com/carsales/vehicleDetails.do?carIndex=1&vin=1GCHC24U36E112452 which by the look of it (notice the /vehicleDetails.do part of the URL) takes you to a page that says "Sorry, we are unable to complete your page request since the page you are trying to access no longer is available." How stupid is that? Come on, web developers made those kind of mistakes in 2001. Not 2010.

This means that people can't talk to each other about found matches on the site and this is something people want to do. Especially if you're going to spend $thousands on a car.

Come on Enterprise web team: install Django or something and give users what they want not your excuses.

IssueTrackerMassContainer is a simple Zope product that is used to put a bunch of IssueTrackerProduct instances into. It doesn't add much apart from a nice looking dashboard that lists all recent issues and then with an AJAX poll it keeps updating automatically.

But what it was doing was it recursively put together all issues across all issue trackers, sorting them and then returning only the first 20. Fine, but once the numbers start to add up it can become a vast sort operation to deal with.

In my local development copy of 814 issues, by the use of pympler and time() I was able to go from 7 Mb taking 2 seconds down to using only 8 Kb and taking 0.05 seconds.

It's of course a simplification but is it not true? Come on, admit it, isn't there something to it?

What do you think? As a man I must admit that it does feel like that. I don't know about the girls perspective so I'd love to hear what you think (keeping it mind that it's an exaggeration)

Check out these all free icons

Last month I marveled over the cute LastGraph which shows you a visualization of what artists you've been listening to on Last.fm lately in a neat way. Today I discovered this cute visualization: Tube Tags

I think it's only for paying Last.fm users since it says "VIP" next to the link. My understanding is that it shows the genres of music you've been listening too recently in a London Underground type of map way. Very neat.

Here's mine (31Kb PDF)

If I've understood it correctly, this is a mashup made on Guardian's API made by someone not at the Guardian. More photos here

What's interesting about this chart, which I've never seen like this before, is the last one where they show how much money various countries pledged versus how much they actually delivered. It's something I've also wanted to see because it opens up a whole new dimension of truth to the equation. For example, Canada has pledged USD 130m to the Haiti Earthquake but they only delivered 51% of what they pledged to the 2004 Tsunami crisis.

Spotted this website called Healing Food Reference which is what the name says; a reference of food and their healing "power". From the home page:

"This site is part of a public education project created by Mike Adams, the Health Ranger, and hosted by Truth Publishing. Its purpose is to educate and empower consumers with information they can use to prevent and even help reverse degenerative disease. There are no commercial sponsors of this site, and neither Mike Adams nor Truth Publishing was paid anything to create this site."

Basically click on a type of food, for example tofu and you get a list of things it can help with such as fever and then from the fever page you can more foods that help.

At first I was a bit taken aback by the home page and huge amounts of text on it but once you're in it is so easy to use. Perhaps the reason the home page is jam packed with content is for his search engine optimization.

I've been a big fan of Yakuake for a long time. It's a terminal you have open all the time in Linux that is shown and hidden, over any other windows, by a simply hit on the F12 button.

But as of more recent versions of Yakuake it has become really slow. It sometimes take 2-3 seconds from F12 press till you can type on the terminal. So I uninstalled it and tried Yeahconsole but I uninstalled it equally fast as I understood it was broken and didn't work at all despite being in the Xubuntu apt repositories.

Last but not least I ended up using Guake which not only works but also works really really fast. Screenshots here

If you want to just remember one single password and you want to use different passwords for different site then Oplop might be right for you.

This cost me some hair-pulling today as I was trying to write a custom test runner for a Django project I'm working on that creates a test Xapian database just for running the tests. Basically, you can't do this:

Because if you do you end up getting these strange DatabaseOpeningError exceptions. So, here's how you do it:

Hopefully by blogging about this some other poor coder will save some time.

If you, like me, have various projects that do things like OAuth on Twitter or Google or you have a development site that goes to PayPal. So you're doing some Django development on http://localhost:8000/foo and click, for example, to do an OAuth on Twitter with an app you have there. Then Twitter will redirect you back to the live site with which you've set it up. But you're doing local development so you want to go back to http://localhost:8080/... instead.

Add this bookmarklet: to localhost:8000 to your browser Bookmarks toolbar and it does exactly that.

Here's its code in more verbose form:

For those who use last.fm and like to visualize stuff with pretty graphs. For the really curious, here's mine of the last year (15 Mb)

"Copenhagen was much worse than just another bad deal, because it illustrated a profound shift in global geopolitics. This is fast becoming China's century, yet its leadership has displayed that multilateral environmental governance is not only not a priority, but is viewed as a hindrance to the new superpower's freedom of action. I left Copenhagen more despondent than I have felt in a long time. After all the hope and all the hype, the mobilisation of thousands, a wave of optimism crashed against the rock of global power politics, fell back, and drained away."

From the brilliantly informed and well articulated article How do I know China wrecked the Copenhagen deal? I was in the room

Just so you know, I love China, but there's a lot of things I hate about it too. Admittedly, any country in their position would perhaps do the same as the force of people getting rich is more powerful than almost anything else. Please do take the time to read this article as it helps to give an interesting perspective on the post-Copenhagen-conference talks.

I have a Django model that looks something like this:

Retroactively now I wanted to add a field called add_date which uses the auto_now_add=True trick. The migration used in this project is South which is great but doesn't work very well with the auto_now_add=True because the field doesn't have a straight forward default. So, first I changed the field to this:

Notice the null=True which is important. Then I used startmigration to generate the code for the forward and backward to which I added a this stuff:

That way all old records will have the date (not entirely accurate but good enough) and all new records will automatically get a date. Is there a better way? I bet, but I don't know how to do it.

You'll have to believe it when you see it. Here's the orignal if you want to know who they are.

Orphaned Land is a great metal band from Israel who I've known about for some time but never researched much.

"Israel's ORPHANED LAND is probably the only band from this country that has managed to succeed at building up a huge following among Muslims and Arabian people"

I've never taken the time to listen to their lyrics and figure out what they're singing about. It's not a surprise that they're not pro one side only but I didn't know that they appeal to both sides which is quite fantastic.

YUI Compressor apparently supports compressing CSS. Cool! I had to try it and what's even more cool is that it's the only other CSS minifier/compressor that doesn't choke on CSS hacks (the ones I tried). The only other CSS minifier/compressor is my very own slimmer. So, let's see what the difference is.

Running the YUI Compressor 10 times on a relatively small file takes 0.3 seconds on average. Running the same with python 2.6 and slimmer.css_slimmer takes 0.1 seconds on average. I think most of this time is spent loading the jar file than the actual time of running the compression.

My friend Chris West has built a great new site called Those Crazy Chinese which describes itself like this:

"Chinese (Mandarin) is a beautiful and highly literal language - directly translated, many words have entertaining and occasionally logical meanings."

It's built in Django and it integrates to Twitter so if you're on Twitter just follow it there to get the latest additions of new interesting and amusing literal translations. This website is mainly geared towards people who are, like Chris, learning Mandarin.

I have some code that checks that a file is treated differently if some time has passed. In other words, it reacts if the modification time is more than it was before. The code uses os.stat(filename)[stat.ST_MTIME] to do this. The challenge was to mock os.stat so that I can pretend some time has passed without having to wait. Here was my first solution which made running many tests sloooow:

So, here's how I mock os.stat to avoid having to do the time.sleep(1) in the test:

I hope this helps someone else who's trying to do the same. It took me some time to figure out that os.stat is used by lots of various sub routines in the os module so it's important to only mock the relevant argument otherwise you might get unexpected problems.

I've just read Frank Zappa: The Biography by Barry Miles. It's a detailed book on 380 pages about virtually every year of Frank Zappa's life. From his parents to his death.

I'm a huge Frank Zappa fan and have been for more than a decade. It's probably the most listened to artist ever in my life in terms of number of listened to songs. Actually not probably; definitely. I adore his music and his personality and this is the second book I read about him. The other book I read was Real Frank Zappa Book written by Peter Occhiogrosso based on biography interviews with Frank for the purpose of writing this book. That book was much more bland and emphasized particularly his early political work and also very much emphasizes on his work as a orchestral conductor/business man.

The detail work in this book is really fantastic. It's thanks to Barry's in-depth understanding of music and the music industry that you get deep down to the nitty-gritty details of Zappa's work. As always with books like this, it's not till you read about the lyrics that you fully understand the lyrics even if you have listened to them many a times. Some of these lyrics I'm actually kind of sad to have understood now as of reading about them in this book. For example, I now understand that the song We're Turning Again which is an up-yours to his old band members Mothers of Invention.

If anything bad can be said about the book it's that it sort of ends on a bad note (no pun intended), as it ends on the sad last few years when Frank was really sick and up to the point of his death. And also, I would have liked to find out more about Barry's own personal relationship with Frank because he couldn't possibly have written this book had he not admired the guy too.

There is no doubt in my mind that Frank Zappa is one of the most innovative and inspirational characters in twentieth-century music history. And probably show-biz too for that matter. Even though the book reveals some truths about Frank as a bit of "douche bag" I'm still firmly one of his biggest fans. If you wanna find out more about Frank Zappa this is most like the book to get.

Following is an extract from a quoted interview by Gail Sloatman who later became Gail Zappa:

"And I remember thinking, Oh my God! Here's this guy, I think he's extraordinary, it's such a different sensation! I know he hasn't taken a bath in four months and his moustache smells like peanut butter..."

She sums it up nicely in her own very personal words so well. There is something amazing about this guy beyond the less appealing facade.

Bruno Renié has written a nifty app:iPhone push notifications for Twitter with Prowl

With the power of Prowl it pushes a notification to your iPhone when someone mentions you on Twitter. You first need to install the Prowl app on your iPhone and then go to their website to get your notification key. Then, go to Bruno's site, sign in with your Twitter account (OAuth, so no password give-away) and badabing! you get instant notifications on your phone, for free, when someone mentions you.

If you're one of those people who use Twitter instead of instant messaging and have lots of mentions this might be excessive for you but if not it can be very useful if you are like me who is very slow to spot that someone has replied or mentioned your name.

Great work Bruno!

Pagetest web page performance test is a great tool for doing what Firebug does but not in your browser. Pagetest can do repeated tests to iron out any outliers. An alternative is Pingdom tools which has some nifty sorting functions but is generally the same thing.

So I ran the homepage of my website on it and concluded that: Wow! Half the time is spent on DNS lookup!

The server it sits on is located here in London, UK and the Pagetest test was made from a server also here in the UK. Needless to say, I was disappointed. Is there anything I can do about that? I've spent so much time configuring Squid, Varnish and Nginx and yet the biggest chunk is DNS lookup.

In a pseudo-optimistic fashion I'm hoping it's because I've made the site so fast that this is what's left when you've done all you can do. I'm hoping to learn some more about this "dilemma" without having to read any lengthy manuals. Pointers welcomed.

One really annoying thing about PIL is that it's importable as ImageandPIL. It leads me and other newbies to think if it's different. I don't want choices:

 Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
 [GCC 4.3.3] on linux2
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import PIL
 >>> import Image
 >>> ?

When PIL/Image is put into standard lib, can we call the module: imaging?

In case this bites someone else like it bit more and chewed off many many minutes of debugging time.

If you ever get weird columns in your Django Administration interface, I now know why that happens. See this screenshot example:

This happens when you've defined a TEMPLATE_STRING_IF_INVALID in your settings.py. I always put in my local_settings.py this line:

 TEMPLATE_STRING_IF_INVALID = '{{ %s }}'

So that I can quickly see which variable references in template code is potential typos. I'm not a big fan of the implicit magic of equating absence to False/None so I try to avoid the confusion altogether.

The current project I'm working has at the time of writing 20 different forms (90% model forms) instantiated in different scenarios. Django doesn't automatically strip whitespace in text based fields. So instead of doing this:

Instead I wrote a common class for all of my form classes to use:

Now all text inputs and textareas are automatically whitespace stripped. Perhaps useful for other Djangonauts.

When you enable TinyMCE you can either choose theme="simple" or theme="advanced". If you go for the simple you get one lovely little bar of buttons but it's missing the link button which is more important than most other buttons altogether. When you enable "advanced" you get three rows of buttons that makes you dizzy. I mean, should really be editing advanced tables in a WYSIWYG editor or mathematical equations?

Here's a config that I think works great. It's all in one row and it's got the bare minimum in terms of additional plugins (no extra downloads required). It's in Python but translates quite easily into Javascript:

The Fail Blog is funny but sometimes disgusting and borderline sad. Fail Dogs is almost better. A lot of the headlines they put on the pictures just makes it twice as fun. For example this one or this one

Enjoy!