I've wanted to Gzip the static content (CSS & Javascript) on my sites for a long time but never found a good enough solution. mod_gzip is out of the question because as far as I've understood it it does a compression on the fly, every time you request the file.

Other solutions have disappointed me because enabling gzip compression has been for all content. I don't want my HTML files gzipped because they're rendered on the fly based on business logic plus by compressing the HTML files. Long story short my yet-to-be-released app now serves the following files from Zope but are only compressed and whitespace slimmed once per server restart:


FILE               ORIG SIZE  NEW SIZE   REDUCTION
screen.css             15224      2738   556%
print.css               2633       885   298%
jquery-latest.js*      57712     18130   318%
jquery-latest.pack.js  20461     10513   195%
common.js               3803      1131   336%
complete-expense.js    18184      2847   639%
Total                 118017     36244   326%

* only used in debug mode

Yes that's right. The static files are now 326% smaller in file size and since the complexity is O(1) the CPU overhead is virtually none. What's cool about this is as an application developer I don't have to worry about it once the files have been set on the class. I never actually see the slimmed or compressed files. The code is also clever enough to serve the uncompressed version of the file if the server doesn't accept gzip using HTTP_ACCEPT_ENCODING.

To make things even more optimized; add the fact that I'm running this Zope behind Squid and with this command:


# nice -n 20 ab2 -n1000 -c10 http://XXX/misc_/MExpenses/screen.css

I get this lovely result:


Concurrency Level:      10
Time taken for tests:   0.372630 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      3168000 bytes
HTML transferred:       2738000 bytes
Requests per second:    2683.63 [#/sec] (mean)
Time per request:       3.726 [ms] (mean)
Time per request:       0.373 [ms] (mean, across all concurrent requests)
Transfer rate:          8300.46 [Kbytes/sec] received

Bare in mind that this is also proxied by Apache2 to get the extra comfort that Apache2 gives us.

Doing the same benchmark on the screen.css file WITH Squid but without gzipping and slimming it I get this result:


Concurrency Level:      10
Time taken for tests:   1.471274 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      15613552 bytes
HTML transferred:       15224000 bytes
Requests per second:    679.68 [#/sec] (mean)
Time per request:       14.713 [ms] (mean)
Time per request:       1.471 [ms] (mean, across all concurrent requests)
Transfer rate:          10363.13 [Kbytes/sec] received

(Without Squid caching, my Zope in this instance does 242.4 [#/sec] for the same file)

Apart from making Zope does less hard work for stupid static files, I've managed to go from 242 requests per second up to 2684 requests per second. Not bad :)

Comments

Post your own comment
betabug

If the files are truly static it would be an option to place them on the filesystem, then use "multiviews" and .gz files along with them. AFAIR mod_gzip can do that automatically (create the .gz files on the filesystem). Don't know if this could be made to work for files served through Zope.

Looking forward to see your solution! :-)

Peter Bengtsson

The solution isn't very clever. Just particular. If you want to I can email you a file that does it but even though I'd like to Open Source the package it's part of that would be much more work packaging than it's worth at the moment.

Anonymous

hello!
i'm a really big fan of your tremedous work. can i bring my 47 cromosomes friend to your mansion tomorrow. bye, i love you.

Pete

Nothing can be more than 100% smaller in size, i.e. the whole. What you should say is actually 69% reduction in size, instead of 326%.

Wayne

I'm confused if the files are gzipped then how do they get uncompressed when getting to the client? What is used to decompress the files once at the client?

Peter Bengtsson

The client will know how to uncompress it. As part of sending the server what type of client it is (user agent) it also sends if it can accept gzip compressed content.

It's basically unpacked on-the-fly by your computer.

Tomas Nielsen

If you are on a java enabled server I have a servlet that can gzip your dynamic HTML as well.

Check out my code at:
http://www.dominoexperts.com/de/forum.nsf/0/A4573078C76401BBC12573330054B72A

As for the css; setting a good header caching will give you only one hit per user since the page will not be requested again until the cache expires.

Your email will never ever be published.

Related posts