Playing with filestream_iterator
30 May 2007
There are several ways of serving static files from Zope. The simplest way is to just do something like this:
size = os.stat(file_path)[stat.ST_SIZE] REQUEST.RESPONSE.setHeader('Content-Type','image/jpeg') REQUEST.RESPONSE.setHeader('Content-length',int(size)) return open(file_path, 'rb').read()
The disadvantage with this is if the file is say >1Mb, the whole 1Mb needs to be loaded into RAM before it can be sent. This is surely garbage collected afterwards but if you serve many of these at the same time, there's obviously the risk that the RAM gets too full.
The alternative is to use
filestream_iterator from the
ZPublisher which is an iterator that sends out chunks of the file at a time. Note that to use the
filestream_iterator you must set the
Content-Length header first. Here's how to use it:
from ZPublisher.Iterators import filestream_iterator ... size = os.stat(file_path)[stat.ST_SIZE] REQUEST.RESPONSE.setHeader('Content-Type','image/jpeg') REQUEST.RESPONSE.setHeader('Content-length',int(size)) return filestream_iterator(file_path, 'rb')
I did a little unofficial benchmark to test if there's any speed difference between the two approaches. I'm currently only interested in serving photographs (jpg) in this app and I'm sure other results would be found if you did bigger filesystem files. These jpgs are in average about 200 Kb. During the course of writing this I tought I'd also try using sendfile as well to see if that makes it any faster:
Serving 526 jpgs with... function average total time (5 tests) open() 5.07750701904 filestream_iterator 4.32704296112 sendfile 4.431672620774
Interesting. Using filestream_iterator shaves off a few milliseconds but it's over many many servings. Conclusion, the benefit of
filestream_iterator is not speed, it's RAM (un)usage.
Note. From reading the LocalFS source code I noticed that they only bother with the
filestream_iterator if the filesize is greater than 128Kb. I copied that too into my app now but don't understand why. Is there a disadvantage of serving tiny files with
Worth noting that
sendfile wasn't any faster but I suppose you only see a speed boost with
sendfile on much larger files.