215

What advantages do either method offer for html, css and javascript files served by a LAMP server. Are there better alternatives?

The server provides information to a map application using Json, so a high volume of small files.

See also Is there any performance hit involved in choosing gzip over deflate for http compression?

Community
  • 1
  • 1
Ken
  • 71,088
  • 29
  • 81
  • 100

9 Answers9

318

Why use deflate instead of gzip for text files served by Apache?

The simple answer is don't.

RFC 2616 defines deflate as:

deflate The "zlib" format defined in RFC 1950 in combination with the "deflate" compression mechanism described in RFC 1951

The zlib format is defined in RFC 1950 as :

     0   1
     +---+---+
     |CMF|FLG|   (more-->)
     +---+---+

       0   1   2   3
     +---+---+---+---+
     |     DICTID    |   (more-->)
     +---+---+---+---+

     +=====================+---+---+---+---+
     |...compressed data...|    ADLER32    |
     +=====================+---+---+---+---+

So, a few headers and an ADLER32 checksum

RFC 2616 defines gzip as:

gzip An encoding format produced by the file compression program "gzip" (GNU zip) as described in RFC 1952 [25]. This format is a Lempel-Ziv coding (LZ77) with a 32 bit CRC.

RFC 1952 defines the compressed data as:

The format presently uses the DEFLATE method of compression but can be easily extended to use other compression methods.

CRC-32 is slower than ADLER32

Compared to a cyclic redundancy check of the same length, it trades reliability for speed (preferring the latter).

So ... we have 2 compression mechanisms that use the same algorithm for compression, but a different algorithm for headers and checksum.

Now, the underlying TCP packets are already pretty reliable, so the issue here is not Adler 32 vs CRC-32 that GZIP uses.

Turns out many browsers over the years implemented an incorrect deflate algorithm. Instead of expecting the zlib header in RFC 1950 they simply expected the compressed payload. Similarly various web servers made the same mistake.

So, over the years browsers started implementing a fuzzy logic deflate implementation, they try for zlib header and adler checksum, if that fails they try for payload.

The result of having complex logic like that is that it is often broken. Verve Studio have a user contributed test section that show how bad the situation is.

For example: deflate works in Safari 4.0 but is broken in Safari 5.1, it also always has issues on IE.

So, best thing to do is avoid deflate altogether, the minor speed boost (due to adler 32) is not worth the risk of broken payloads.

Arend
  • 3,668
  • 2
  • 24
  • 37
Sam Saffron
  • 121,058
  • 74
  • 309
  • 495
  • Shouldn't there be a new standard that combines adler32 with gzip? – Pacerier Jul 04 '12 at 02:14
  • 1
    @Sam Saffron, does this mean if the web browser is not in the picture, I can use deflate over gzip? For instance, if I'm going to upload a compressed file to my FTP server. – Xegara Nov 03 '16 at 12:57
  • 1
    Another very minor difference is that the zlib wrapper is six bytes vs. 18 bytes for gzip. So for very small packets, there may be an advantage to sending 12 fewer bytes. The conclusion does not change however, which is that due to Microsoft screwing it up for everyone by misinterpreting what "deflate" meant in what they delivered on their IIS servers, it is easier to just use the gzip format. – Mark Adler Jan 24 '18 at 22:16
  • But how could the payload possibly be broken, if it is transmitted using TCP? The whole idea of TCP is to transmit unbroken payloads. – user1095108 Nov 23 '18 at 17:13
  • This answer date from 2012. So does modern browsers still suffer from the issue of the incorrect implementation of the deflate algorithms or is it safe to use it now ? Is this part of the answer still up to date ? – ihebiheb Jun 10 '19 at 14:09
173

GZip is simply deflate plus a checksum and header/footer. Deflate is faster, though, as I learned the hard way.

gzip vs deflate graph

Community
  • 1
  • 1
Jeff Atwood
  • 60,897
  • 45
  • 146
  • 152
  • Seems to be mainly the checksum, which would match with the smaller difference on faster systems. – schnaader Dec 23 '08 at 11:17
  • On the Intel chips it doesn't make a bit of difference, since they get that fancy SSE4.2 "CRC32" instruction... –  Feb 22 '09 at 18:01
  • only the very very newest Intel chips have that, though -- Core i7 or better – Jeff Atwood May 05 '09 at 02:36
  • 13
    Not to mention that zlib doesn't have support for the extension, and even if it did, the CRC32 instruction in SSE 4.2 uses the polynomial 1EDC6F41, and the gzip format uses the polynomial EDB88320 - totally different algorithms, effectively. – Jack Lloyd Oct 20 '09 at 15:59
  • 7
    And since deflate is faster, why is SO using gzip? – David Murdoch Sep 16 '10 at 12:51
  • Deflate is actually the zlib format, which is raw deflate data with a much smaller header and the computationally cheaper Adler32 checksum on the end. – Randy the Dev Sep 27 '10 at 11:06
  • 40
    Well, this answer turns out to be incorrect ... see: http://zoompf.com/blog/2012/02/lose-the-wait-http-compression ... in particular client have 2 ways they can "interpret" deflate, headerless/checksumless and with zlib header. The implementation across browsers of a correct deflate is bad. deflate should be avoided. – Sam Saffron Mar 24 '12 at 23:36
  • 4
    @sam additionally I just re-ran the benchmarks and on a modern Intel chip, I get gzip 1441/692 and deflate 1286/531. Second number is decompress, first is compress. So deflate *is* still faster, do your benchmarks show otherwise? (I agree it may not be useful for other reasons, but the answer *is correct*, deflate is faster..) – Jeff Atwood Mar 27 '12 at 07:04
  • 6
    @JeffAtwood but the question wasn't faster? – Ken May 15 '12 at 21:44
  • 4
    @Ken Your question was: `What advantages ... for ... files served by LAMP server?`. And **performance is one of *the* issues on the server side** for anyone who's successful in attracting lots of requests. So Jeff's performance comparison - the main point of his answer - is pertinent. Coincidentally Sam's comment does not address it at all. I guess he's just criticizing the `GZip is simply deflate plus a checksum and header/footer` intro, which might be technically wrong. – Evgeniy Berezovsky Jun 27 '14 at 20:59
  • Sorry but you seem to be comparing apples to oranges. Deflate is a compression algorithm while GZip is a file format. The only difference is that GZip computes checksum (CRC-32). – mar Jul 23 '17 at 18:07
  • CRC-32 is slow, Adler32 is slow as well. These can easily account for those extra 20-40%. So naturally, no checksum is faster. But then you also can't detect corrupt streams. – mar Jul 23 '17 at 18:13
16

You are likely not able to actually pick deflate as an option. Contrary to what you may expect mod_deflate is not using deflate but gzip. So while most of the points made are valid it likely is not relevant for most.

Konrad Viltersten
  • 28,018
  • 52
  • 196
  • 347
Amblyopius
  • 431
  • 3
  • 2
4

The main reason is that deflate is faster to encode than gzip and on a busy server that might make a difference. With static pages it's a different question, since they can easily be pre-compressed once.

Joachim Sauer
  • 278,207
  • 54
  • 523
  • 586
  • presumably with gzip you can't start transmitting the header until you've obtained, stored and compressed *all* the data? (because you need the checksum to create the header) – OJW Jun 30 '09 at 11:25
  • 8
    In the gzip format, the checksum comes at the end of the file, specifically so one can start writing deflate blocks as they are processed without having to hold everything up. – Jack Lloyd Oct 20 '09 at 16:00
4

I think there's no big difference between deflate and gzip, because gzip basically is just a header wrapped around deflate (see RFCs 1951 and 1952).

schnaader
  • 46,785
  • 9
  • 98
  • 132
2

mod_deflate requires fewer resources on your server, although you may pay a small penalty in terms of the amount of compression.

If you are serving many small files, I'd recommend benchmarking and load testing your compressed and uncompressed solutions - you may find some cases where enabling compression will not result in savings.

Dave R.
  • 7,106
  • 3
  • 27
  • 51
  • For whoever is wondering, with deflate my text files go from 30KB to 10KB - so the files have to be even smaller than that to not get any savings. I am guessing less than 1KB or something similar. – GChuf Dec 20 '19 at 21:28
0

There shouldn't be any difference in gzip & deflate for decompression. Gzip is just deflate with a few dozen byte header wrapped around it including a checksum. The checksum is the reason for the slower compression. However when you're precompressing zillions of files you want those checksums as a sanity check in your filesystem. In addition you can utilize commandline tools to get stats on the file. For our site we are precompressing a ton of static data (the entire open directory, 13,000 games, autocomplete for millions of keywords, etc.) and we are ranked 95% faster than all websites by Alexa. Faxo Search. However, we do utilize a home grown proprietary web server. Apache/mod_deflate just didn't cut it. When those files are compressed into the filesystem not only do you take a hit for your file with the minimum filesystem block size but all the unnecessary overhead in managing the file in the filesystem that the webserver could care less about. Your concerns should be total disk footprint and access/decompression time and secondarily speed in being able to get this data precompressed. The footprint is important because even though disk space is cheap you want as much as possible to fit in the cache.

Steven
  • 29
  • 2
-1

On Ubuntu with Apache2 and the deflate module already installed (which it is by default), you can enable deflate gzip compression in two easy steps:

a2enmod deflate
/etc/init.d/apache2 force-reload

And you're away! I found pages I served over my adsl connection loaded much faster.

Edit: As per @GertvandenBerg's comment, this enables gzip compression, not deflate.

aidan
  • 8,576
  • 8
  • 65
  • 78
-5

if I remember correctly

  • gzip will compress a little more than deflate
  • deflate is more efficient
JimmyJ
  • 3,903
  • 3
  • 23
  • 25