When we were developing SPDY, we wanted to know if SPDY was always faster or just sometimes faster than HTTP. The following chart is what convinced us that SPDY is indeed almost always faster. I didn’t publish this at that time because I didn’t like looking at web performance as a single number. “Is it faster” includes many variables, such as the network simulated (bandwidth & latency), the packet loss rate, the content chosen, and the measurement (total PLT, first PLT, second PLT, or time-to-first-render). You’d really want a whole stack of these charts, rather than just a single combination.
What I like about this chart is how easy it is to compare two protocols. If the two protocols are identical in performance, all points would be on the midline (red). If one protocol is slower, then the points will fall closer to that axis.
Notes about this test:
- Used a static copy of the Alexa Top-300 websites (excluded porn sites), full content, unchanged
- Simulated 2Mbps download, 386Kbps upload, 100ms RTT, 0% packet loss
- No SSL
- The average speedup was ~40% on this test
- This test predates CWND changes in the kernel
- Server was linux, client was chrome on Windows 7
Cool Graph (worth a thousand words).A few questions for my own clarification:
(a) How about the same graph with HTTPS as thats the default mode of operation that most folks use (dont know of anyone using it with http)?
(b) What did you use for the emulation? (Dummynet methinks!)
(c) Does the static copy of alexa also serve the images,css and js from the same domain or the CDN that the site used? In other words, are they all served from same domain or multiple domains
(d) Do we have the distribution of the size of these sites?
Two notes: SPDY used to be called “FLIP”. I think you meant “If one protocol is SLOWER, then the points will fall closer to that axis.”
@Paddy – would love to run more tests. Network was simulated with dummynet (see http://code.google.com/p/web-page-replay/) everything was statically served from my own server (no CDN, but multiple domains were maintained), I don’t have the dataset from this test anymore.
@Steve – correct & correct! fixed, thanks!