Neat idea. I noticed that the counts were low for a number of pages, which appears to be due to deferred loading logic. https://gist.github.com/9191642 is a refactored version which uses setInterval to see if all pending requests have completed and waits at least 200ms after that time to allow JavaScript to issue new requests after the first round of load events has fired.
This is definitely a tedious exercise in detecting the weird things developers try. I wonder how many pages would change if we simply triggered a scroll event after window.load - I just updated https://gist.github.com/9191642 with longer timeouts and something which triggers scrolling a couple of times, which basically doubles the number of requests and domains for http://www.washingtonpost.com.
But I'm not sure doing a scroll in the script is fair on the site. It's kinda like going to the URL then clicking on another page. Except of a click it's a scroll.
Comment
I'll try your script.
Another thing I can imagine is sites that hang on to a onScroll event and load more crap when you've scrolled down a bit.
Parent comment
Neat idea. I noticed that the counts were low for a number of pages, which appears to be due to deferred loading logic. https://gist.github.com/9191642 is a refactored version which uses setInterval to see if all pending requests have completed and waits at least 200ms after that time to allow JavaScript to issue new requests after the first round of load events has fired.
Replies
This is definitely a tedious exercise in detecting the weird things developers try. I wonder how many pages would change if we simply triggered a scroll event after window.load -
I just updated https://gist.github.com/9191642 with longer timeouts and something which triggers scrolling a couple of times, which basically doubles the number of requests and domains for http://www.washingtonpost.com.
But I'm not sure doing a scroll in the script is fair on the site. It's kinda like going to the URL then clicking on another page. Except of a click it's a scroll.