Optimize Plone.org with slimmer.py

15 February 2005   13 comments   Python


Mind That Age!

This blog post is 13 years old! Most likely, its content is outdated. Especially if it's technical.

If you do a speed report on Plone.org you notice that the three biggest files it services are plone.css, plone_javascripts.js and index.html. If you run these through my little slimmer whitespace optimizing program in Python you get the following results:

$ python slimmer.py --test http://plone.org/plone.css
Took 0.016 seconds
Bytes before: 29355
Bytes after:  20265
Bytes saved:  9090  (69.0% of original size)

$ python slimmer.py --test http://plone.org/plone_javascripts.js
Took 0.057 seconds
Bytes before: 27251
Bytes after:  19974
Bytes saved:  7277  (73.0% of original size)

$ python slimmer.py --test http://plone.org/ 
Took 0.029 seconds
Bytes before: 37186
Bytes after:  26466
Bytes saved:  10720 (10K)  (71.0% of original size)

Using an on-the-fly whitespace optimizer isn't always the right thing to do but in the case of this particular site you prevent serving: 26Kb in 0.1 seconds (on this computer).

26Kb isn't a lot by looking at their traffic summary you see that they could serve 3.3Gb less per month (before gzip compression). That's a big save.


Ian Bicking
Wouldn't mod_gzip be far more effective? Or if not mod_gzip, then something else to gzip the output (when possible). It doesn't obscure the source at all, can potentially save a greater amount of bandwidth, and is highly orthogonal to other parts of the system and the output that is produced.
Or how about both? Wouldn't that be ideal?

This particular page isn't about HOW plone.org should do their job but what could be done to their output as an alternative that works out of the box.
Ian Bicking
"Or how about both?"

I suspect that you'd save very little space by removing whitespace then gzipping, compared to gzipping alone -- whitespace is very compressable. I suppose that would be of interest to add to --test, to do gzipped bytes before and after. I'd try it, but I can't find a slimmer.py that works like you show...?
That would be cool! gzip on the fly in the slimmer module. However, that's not the point. The point is that slimmer.py removes whitespace from html, css and js. At least it does _that_ well :)
Andrew Arnott
I downloaded the CheckoutableTemplates to get slimmer.py, and it doesn't support --test, and doesn't do anything. It looks more like a library. Good design, but I don't know enough Python (0% actually) to utilize it to compress my .css files. Has anyone written a command-line processor frontend for slimmer.py?
Peter Bengtsson
1. you need python installed
2. you need to run it on the command line
3. there is no front end

If you don't know how to run commands on the command line (windows user?) I suggest you try to catch some ideas from some other website that explains how to do it.
Andrew Arnott
I was trying to run the Python script from the Linux command-line, actually. The command I was using was:

python slimmer.py --test http://www.byu.edu

Upon inspection of the slimmer.py file that I obtained as part of the CheckoutableTemplates, I found that there were only functions defined, and no code to check for parameters. That's why I said it looks like it was written as a library.

However, I found a more recent version (0.1.16) of slimmer.py and that has all the parameter-handling code that makes it work. Thanks for the fast response however.
Since the slimmer filter doesn't really increase the document entropy, you still get roughly the same percentages for zip(slimmer(text)) vs. zip(text) as you get for slimmer(text) vs. text.

(but all zip() variants are of course a lot smaller than the others, so you save a lot less bytes. in this case, it's just over 4k, if my calculations are correct).
Torsten Will
Yes, exactly my thought. slimmer.py wont gain much if compression is applied anyway.

But what about translation to uppercase and removing all vowels... ;-)
(peter, your preview function seems to lose the "hide my email address" setting. naughty).
Thank you. I've fixed that glitch now.
Alexander Limi
Plone by default serves up gzipped content, this would not make any noticable difference.
Cool! I thought it was something configured in apache/squid. Need to learn from that.

Your email will never ever be published

Related posts

Python optimization anecdote 11 February 2005
Debian popular in Sweden 18 February 2005
Related by Keyword:
Msgpack vs JSON (with gzip) 19 December 2017
Concurrent Gzip in Python 13 October 2017
Fastest way to download a file from S3 29 March 2017
Crash-stats just became a whole lot faster 25 August 2015
How to test if gzip is working on your site or app 20 August 2015
Related by Text:
jQuery and Highslide JS 08 January 2008
I'm back! Peterbe.com has been renewed 05 June 2005
Anti-McCain propaganda videos 12 August 2008
I'm Prolog 01 May 2007
Ever wondered how much $87 Billion is? 04 November 2003