Lesson learned: Unicodifying request variables in Zope

16 April 2009   3 comments   Zope

Mind That Age!

This blog post is 8 years old! Most likely, it's content is outdated. Especially if it's technical.

Powered by Fusion×

This cost me a good hour of debugging so I thought I'd share it in case anybody else stumbles across the same problem. In the end, to solve my problem I had to add debug statements to StringIO.py to be able to find out where in my Zope Page Template a non-unicode string with non-ascii characters appeared and messed things up.

The error I was getting was this, which I suspect several Zope developers have encountered before:

UnicodeDecodeError: \
'ascii' codec can't decode byte 0xc3 in position 46: ordinal not in range(128)

The traceback only mentions files in the innards of ZPT of which none you can really do anything about. We all know that the key to avoid Unicode error is to be consistent. You can do this:

>>> '\xc3' + 'string'
>>> u'\xc3' + u'string'

But you can't do this:

>>> '\xc3' + u'string'
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: 
ordinal not in range(128)

So, how did I get these non-unicode strings into my application in first place. Simple, I have a search parameter q and end up with a URL like this:


And the template had this little innocent piece of code:

<input tal:attributes="value request/q"/>

That's what f'ed everything up. So, I ended up having to add this:

<input tal:attributes="value python:context.unicodify(request['q'])"/> 

With this little helper function in the base class:

def unicodify(self, s):
   if isinstance(s, str):
       return unicode(s, 'utf8')
   return s

So, hopefully by writing this it will help someone else making the same trivial mistake and not wasting their evening with sporadic print statements all over their frameworks code.


Wouldn't bla/bla?q:ustring:utf8=régime be a simpler solution ?
Peter Bengtsson
Sure, ...but uglier. I often use, but rarely rely on Zope's magic type casting. In the same sense I often use but don't rely on Javascript
Hello, finally found your blog :) and it is very interesting. I got question, what is better Python or PHP in web development?
Thank you for posting a comment

Your email will never ever be published

Related posts

Fujitsu FLEPia 14 April 2009
Git + Twitter = Friedcode 22 April 2009
Entering multilanguage data in Zope 26 November 2006
Zope compared to PHP 22 June 2005
Emacs html-mode for .zpt and .dtml 15 April 2004