Gzip and Slimmer optimization anecdote

30 January 2007   7 comments   Zope

Powered by Fusion×

I've wanted to Gzip the static content (CSS & Javascript) on my sites for a long time but never found a good enough solution. mod_gzip is out of the question because as far as I've understood it it does a compression on the fly, every time you request the file.

Other solutions have disappointed me because enabling gzip compression has been for all content. I don't want my HTML files gzipped because they're rendered on the fly based on business logic plus by compressing the HTML files. Long story short my yet-to-be-released app now serves the following files from Zope but are only compressed and whitespace slimmed once per server restart:

screen.css             15224      2738   556%
print.css               2633       885   298%
jquery-latest.js*      57712     18130   318%
jquery-latest.pack.js  20461     10513   195%
common.js               3803      1131   336%
complete-expense.js    18184      2847   639%
Total                 118017     36244   326%

* only used in debug mode

Yes that's right. The static files are now 326% smaller in file size and since the complexity is O(1) the CPU overhead is virtually none. What's cool about this is as an application developer I don't have to worry about it once the files have been set on the class. I never actually see the slimmed or compressed files. The code is also clever enough to serve the uncompressed version of the file if the server doesn't accept gzip using HTTP_ACCEPT_ENCODING.

To make things even more optimized; add the fact that I'm running this Zope behind Squid and with this command:

# nice -n 20 ab2 -n1000 -c10 http://XXX/misc_/MExpenses/screen.css

I get this lovely result:

Concurrency Level:      10
Time taken for tests:   0.372630 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      3168000 bytes
HTML transferred:       2738000 bytes
Requests per second:    2683.63 [#/sec] (mean)
Time per request:       3.726 [ms] (mean)
Time per request:       0.373 [ms] (mean, across all concurrent requests)
Transfer rate:          8300.46 [Kbytes/sec] received

Bare in mind that this is also proxied by Apache2 to get the extra comfort that Apache2 gives us.

Doing the same benchmark on the screen.css file WITH Squid but without gzipping and slimming it I get this result:

Concurrency Level:      10
Time taken for tests:   1.471274 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      15613552 bytes
HTML transferred:       15224000 bytes
Requests per second:    679.68 [#/sec] (mean)
Time per request:       14.713 [ms] (mean)
Time per request:       1.471 [ms] (mean, across all concurrent requests)
Transfer rate:          10363.13 [Kbytes/sec] received

(Without Squid caching, my Zope in this instance does 242.4 [#/sec] for the same file)

Apart from making Zope does less hard work for stupid static files, I've managed to go from 242 requests per second up to 2684 requests per second. Not bad :)


If the files are truly static it would be an option to place them on the filesystem, then use "multiviews" and .gz files along with them. AFAIR mod_gzip can do that automatically (create the .gz files on the filesystem). Don't know if this could be made to work for files served through Zope.

Looking forward to see your solution! :-)
Peter Bengtsson
The solution isn't very clever. Just particular. If you want to I can email you a file that does it but even though I'd like to Open Source the package it's part of that would be much more work packaging than it's worth at the moment.
i'm a really big fan of your tremedous work. can i bring my 47 cromosomes friend to your mansion tomorrow. bye, i love you.
Nothing can be more than 100% smaller in size, i.e. the whole. What you should say is actually 69% reduction in size, instead of 326%.
I'm confused if the files are gzipped then how do they get uncompressed when getting to the client? What is used to decompress the files once at the client?
Peter Bengtsson
The client will know how to uncompress it. As part of sending the server what type of client it is (user agent) it also sends if it can accept gzip compressed content.

It's basically unpacked on-the-fly by your computer.
Tomas Nielsen
If you are on a java enabled server I have a servlet that can gzip your dynamic HTML as well.

Check out my code at:

As for the css; setting a good header caching will give you only one hit per user since the page will not be requested again until the cache expires.

Your email will never ever be published

Related posts

Canon and Sony Ericsson rule Flickr 27 January 2007
A Flash interface that doesn't suck 04 February 2007
Related by keywords:
Fastest way to uniqify a list in Python 14 August 2006
Comparing Google Closure with UglifyJS 10 July 2011
Gzip rules the world of optimization, often 09 August 2014
The awesomest way possible to serve your static stuff in Django with Nginx 24 March 2010
Optimization of getting random rows out of a PostgreSQL in Django 23 February 2011
How much faster is Nginx+gunicorn than Apache+mod_wsgi? 22 March 2012
A Django base class for all your Forms 16 November 2013
Local Django development with Nginx 11 October 2010
HTML whitespace "compression" - don't bother! 11 March 2013
Automatically strip whitespace in Django forms 12 October 2009
Optimize Plone.org with slimmer.py 15 February 2005
ztar - my wrapper on tar -z 29 June 2005