One aspect of SSL which many people are not aware of is that SSL is capable of compressing the entire SSL stream. The authors of SSL knew that if you’re going to encrypt data, you need to compress it before you encrypt it, since well-encrypted data tends to look pretty random and non-compressible. But even though SSL supports compression, no browsers support it. Except Chrome 6 & later.
Generally, stream-level compression at the SSL layer is not ideal. Since SSL doesn’t know what data it is transporting, and it could be transporting data which is already compressed, such as a JPG file, or GZIP content from your web site. And double-compression is a waste of time. Because of this, historically, no browsers compressed at the SSL layer – we all felt certain that our good brothers on the server side would solve this problem better, with more optimal compression.
But it turns out we were wrong. The compression battle has been waging for 15 years now, and it is still not over. Attendees of the Velocity conference each year lament that more than a third of the web’s compressible content remains uncompressed today.
When we started work on SPDY last year, we investigated what it would take to make SSL fast, and we noticed something odd. It seemed that the SSL sites we tested (and these were common, Fortune-500 companies) often were not compressing the content from their web servers in SSL mode! So we asked the Web Metrics team to break-out compression statistics for SSL sites as opposed to unsecure HTTP sites. Sure enough, they confirmed what we had noticed anecdotally – a whopping 56% of content from secure web servers that could be compressed was sent uncompressed!
Saddened and dismayed, the protocol team at Chromium decided to reverse a decade long trend, and Chrome became the first browser to negotiate compression with SSL servers. We still recognize that compression at the application (HTTP) layer would be better. But with less than half of compressible SSL content being compressed, optimizing for the minority seems like the wrong choice.
So how do you know if your browser compresses content over SSL? It’s not for the faint of heart. All it takes is your friendly neighborhood packet tracer, and a little knowledge of the SSL protocol. Both the client and the server must agree to use compression. So if a server doesn’t want to use it (because it may be smart enough to compress at the application layer already), that is no problem. But, if your server uses recent OpenSSL libraries, it can. And you can detect this by looking at the SSL “Client Hello†message. This is the first message sent from the client after the TCP connection is established. Here is an example from Chrome, viewed with Wireshark.
So, does this help? Sites that use compression at the application layer don’t need this, and it has no effect (those are in the minority). For servers that opt out, it has no effect. But for servers that opt in, HTTP headers do get compressed a little better, and any uncompressed data stream gets compressed much better.
I did find that https://www.apache.org/ is a showcase example for my cause. Apache.org runs a modern SSL stack and advertises compression support on the server, but they forgot to compress jquery.js. If you load this site with Chrome, you’ll only download 77,128 bytes of data. If you use IE8 (firefox will be similar), you’ll download 146,595 bytes of data. Most of this difference is just plain old compression.
For the 50% of site owners out there that can’t configure their servers to do compression properly, don’t feel too bad – even apache.org can’t get it right! (Sorry Apache, we still love you, but can we turn mod_gzip on by default now? 🙂
I am not an expert in the field of compression across the internet (or SSL, et. al.), but I have encountered cases in the past where I got much better overall throughput by not compressing. That is, it was cheaper to send twice as much data over the wire as it was to spin the CPUs on each end to compress and uncompress the data.
In my case I was sending significant quantities of XML over the internet (which included a two-way transition across a large corporate firewall/proxy setup) as fast as I could (WFC calls…effectively RPC). My “study” was far less than scientific in terms of spreading across many machines, or over various speed of internet links and such. But the difference was very clear in my case: it was cheaper to use bandwidth than cpu/ram.
Not saying you should draw conclusions from my one specific experience…but just saying that compression may not always be the best default setup – i.e. in your example, maybe Apache already figured this out for themselves.
Hey Dave –
In cases where you’ve got low RTT and high bandwidth, compressing absolutely can make things worse. I have studied this and should writeup some results, but basically for the low end guys (think average DSL 2-3Mbps,100ms RTT), the win from compression is huge (like 50% faster) and for the high end guys (1Gbps/5ms RTT) the degradation due to compression is small like 2-3%.
You can take a look at any number of make-your-website-faster books or tools, and they *ALL* recommend gzip compression as part of the strategy for making your site faster.
But you are right, an even better approach would be to intelligently compress or not compress based on network characteristics to the client.
So a few questions..
1) Is SSL compression possible and easy to enable on IIS?
2) Is SSL compression recommended?
3) SSL compression only works for chrome 6 and up?
Any thoughts if fully appreciated.
Pingback:Speculation on CRIME « SSL Blog - Entrust Insights
Yup, the article referenced above points out a potential security flaw when using SSL & Compression: http://security.stackexchange.com/questions/19911/crime-how-to-beat-the-beast-successor/19914#19914
The good news is that it is a difficult attack (you need to be able to sniff packets and also lure the victim to an attack site), but the bad news is that the attacker can steal a cookie, even if transferred over SSL.
The lesson to learn here is that you should never combine user-controlled content and sensitive content into the same compression context (e.g. the cookie value and something that the attacker can control).