91

UPDATE Feb 10 2012:

zOompf has completed some very thorough research on this very topic here. It trumps any findings below.

UPDATE Sept 11 2010:

A testing platform has been created for this here



HTTP 1.1 definitions of GZIP and DEFLATE (zlib) for some background information:

" 'Gzip' is the gzip format, and 'deflate' is the zlib format. They should probably have called the second one 'zlib' instead to avoid confusion with the raw deflate compressed data format. While the HTTP 1.1 RFC 2616 correctly points to the zlib specification in RFC 1950 for the 'deflate' transfer encoding, there have been reports of servers and browsers that incorrectly produce or expect raw deflate data per the deflate specification in RFC 1951, most notably Microsoft products. So even though the 'deflate' transfer encoding using the zlib format would be the more efficient approach (and in fact exactly what the zlib format was designed for), using the 'gzip' transfer encoding is probably more reliable due to an unfortunate choice of name on the part of the HTTP 1.1 authors." (source: http://www.gzip.org/zlib/zlib_faq.html)

So, my question: if I send RAW deflate data with NO zlib wrapper (or gzip, for that matter) are there any modern browsers (e.g., IE6 and up, FF, Chrome, Safari, etc) that can NOT understand the raw deflate compressed data (assuming HTTP request header "Accept-Encoding" contains "deflate")?

Deflate data will ALWAYS be a few bytes smaller than GZIP.

If all these browsers can successfully decode the data, what downsides are there to sending RAW deflate instead of zlib?



UPDATE Sept 11 2010:

A testing platform has been created for this here

EricLaw
  • 54,427
  • 7
  • 140
  • 182
David Murdoch
  • 82,194
  • 38
  • 141
  • 186
  • 1
    Would you mind expanding on why System.IO.Compression.DeflateStream sucks compared to zlib.net? Google is not showing me much that's relevant except one person mentioning it "doesn't have particularly good compression ratio." – Joel Mueller Oct 18 '09 at 05:30
  • Right, the compression ratio for both .net's gzip and deflate methods don't seem to be anywhere near what they should be. However, I haven't done any speed benchmarks between the two (zlib.net vs. native .net). – David Murdoch Oct 19 '09 at 13:52
  • Why don’t you just log the results of your test case? – Gumbo Oct 22 '09 at 21:44
  • @Gumbo: How can I do that? There is no way to know if the page renders correctly on the browser end, is there? I guess there may be an AJAX solution; but even then I may run into some cases where XMLHttpRequest doesn't work correctly. – David Murdoch Oct 23 '09 at 13:22
  • 1
    I've been poking around in the System.IO.Compression lib, and it appears to use a static/pre-defined tree- thus the compression is not optimized for the specific stream. Should be the fastest method, but definitely will yield poor compression ratios. – Brady Moritz Sep 13 '10 at 15:22
  • Also, it doesn't buffer, you can easily handle that yourself, but you need to know about it. If you keep writing bytes to the .NET compression implementations, one at a time, or in small chunks, you get horrible compression ratios. – Lasse V. Karlsen Feb 17 '11 at 20:08
  • I have deflated xml data, how I can see it in my browser? – John Feb 01 '12 at 23:57
  • 2
    @JoelMueller This might explain: http://www.virtualdub.org/blog/pivot/entry.php?id=335 – Nayuki Mar 10 '12 at 00:45
  • @NayukiMinase - thanks, that's a good explanation. – Joel Mueller Mar 12 '12 at 16:35
  • was there any update after 2012 ? – ihebiheb Jun 10 '19 at 15:57

4 Answers4

37

UPDATE: Browsers have been dropping support for raw deflate. zOompf has completed some very thorough research on this very topic here. Unfortunately, it appears that raw deflate is NOT safe to use.

Check http://www.vervestudios.co/projects/compression-tests/results for more results.

Here are the browsers that have been tested:

/*  Browser                       DEFLATE      ZLIB     */
    XP Internet Explorer 6        PASS         FAIL
    XP Internet Explorer 7        PASS         FAIL
    XP Internet Explorer 8        PASS         FAIL
    Vista Internet Explorer 8     PASS         FAIL
    XP Firefox 3.6.*              PASS         PASS
    XP Firefox 3.5.3              PASS         PASS
    XP Firefox 3.0.14             PASS         PASS
    Win 7 Firefox 3.6.*           PASS         PASS
    Vista Firefox 3.6.*           PASS         PASS
    Vista Firefox 3.5.3           PASS         PASS
    XP Safari 3                   PASS         PASS
    XP Safari 4                   PASS         PASS     
    XP Chrome 3.0.195.27          PASS         PASS
    XP Opera 9                    PASS         PASS
    XP Opera 10                   PASS         PASS
    XP Sea Monkey 1.1.8           PASS         PASS
    Android 1.6 Browser (v4)*     N/A          N/A
    OS-X Safari 4                 PASS         PASS
    OS X Chrome 7.0.517.44        PASS         PASS
    OS X Opera 10.63              PASS         PASS
    iPhone 3.1 Safari             PASS         PASS

* Android Sends HTTP request header "Accept-Encoding: gzip". Deflate is not permitted.


I conclude that we can always send raw DEFLATE (when the HTTP request header "Accept-Encoding" contains "deflate") and the browser will be able to correctly interpret the encoded data. Can someone prove this wrong?

note: .NET's native implementation of DEFLATE (System.IO.Compression.DeflateStream) is raw DEFLATE. It also sucks. Please use zlib.net for all of your .NET deflating needs.

All Іѕ Vаиітy
  • 18,901
  • 11
  • 65
  • 90
David Murdoch
  • 82,194
  • 38
  • 141
  • 186
  • 1
    Can you be more specific on using zlib.net for deflating? How does that match the above chart, where it says raw deflate works but zlib doesn't in some XP IE cases? – David Eison Aug 04 '11 at 22:08
  • Android supports deflate compression since API 9. see: http://developer.android.com/reference/java/util/zip/DeflaterInputStream.html for more information – Stuart Blackler Sep 30 '11 at 09:51
  • 1
    @DavidMurdoch the only sane result from your tests seems to **never use deflate**, http://www.vervestudios.co/projects/compression-tests/results I think this answer should be edited to reflect that – Sam Saffron Mar 24 '12 at 12:38
  • Yup, I agree. Updating it now. – David Murdoch Mar 24 '12 at 13:27
6

The Android 1.6 browser (v4) fails both the zlib and the deflate test on your page. I've added it to your list.

Josef Pfleger
  • 72,585
  • 15
  • 95
  • 99
1

Isn't it the case that AddOutputFilterByType DEFLATE using mod_deflate sends by gzip by default?

Paul Irish
  • 43,185
  • 21
  • 92
  • 125
  • 1
    Hey Paul, I feel like I'm talking to a celebrity...you are everywhere. :-) Anyway, `AddOutputFilertByType DEFLATE` gzips the response instead of deflating it by default (as far as I know). `Gzip` is `deflate` + a 10 byte header + 8 byte footer - which means that `Gzip` will ALWAYS be larger than `deflate`...so why should we ever use gzip? (see http://en.wikipedia.org/wiki/Gzip#File_format for a breakdown of what gzip is made of). With that said, I'm not sure how to go about setting `deflate` as the preferred compression method in Apache. – David Murdoch Aug 26 '10 at 19:16
-1

as far as i know, yes - pretty much you "can always send raw DEFLATE and everything would be okay"... there is not "always", but most of all cases. if not, this is the browser's problem.

Letterman
  • 3,766
  • 4
  • 26
  • 38
  • I'm trying to find the cases when raw deflate fails. According to spec it should fail in all browsers. – David Murdoch Apr 14 '10 at 13:35
  • Raw `deflate` (i.e. not **zlib**, no headers at all) will only work in **IE7** if `encoding:gzip` and (only tested in chrome v24) `encoding:deflate` in **chrome**. – Scotty.NET Feb 28 '13 at 14:58