Cant cache resource when having both gzip and Etag

Question

I am trying to cache a (javascript) resource in the browser and have properly set all of Cache-control:max-age, Expires, and Etag in the response headers (as is seen from the screenshot).

The browser requests with "if-none-match" and "if-modified-since", and in both cases the conditions are met:

• if-modified-since = last-modified (the file has never been changed)
• if-none-match = Etag (again, the files has never been changed)

So I should get response 304, right? But no, I keep getting 200 OK, which means that apache keeps serving the file (albeit compressed) every time. Tested with Firefox, Chrome, curl -- no use. Server always serves the whole file, even if I am not asking it to...

Using curl, I have traced the problem to gzip & Etag:

• if I remove the gzip (and cut the suffix -gzip from the request Etag) -- all is good: 304
• if I keep the gzip and remove the request Etag altogether -- all is good: 304
• but if I keep both 'accept-encoding:gzip' and the Etag, even though both request and response Etags are the same (this time with '-gzip' at the end), the server returns the wrong 200. It feels like apache compares the etag before gzipping the fail, decides it doesnt match, and then serves the file gzipped, even though after the gzip the Etag matches.

Here is the request/response:

• Request Method: GET
• Status Code: HTTP/1.1 200 OK

Request Headers 00:09:12.000

• User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:36.0) Gecko/20100101 Firefox/36.0
• If-None-Match: "24e55-51138062ce6c0-gzip"
• If-Modified-Since: Sat, 14 Mar 2015 04:26:43 GMT
• Connection: keep-alive
• Cache-Control: max-age=0
• Accept-Language: en-US,en;q=0.5
• Accept-Encoding: gzip, deflate
• Accept: /

Response Headers Δ1100ms

• Vary: Accept-Encoding
• Server: Apache/2.4.7 (Ubuntu)
• Last-Modified: Sat, 14 Mar 2015 04:26:43 GMT
• Keep-Alive: timeout=5, max=100
• Expires: Wed, 25 Mar 2015 16:09:13 GMT
• Etag: "24e55-51138062ce6c0-gzip"
• Date: Wed, 18 Mar 2015 16:09:13 GMT
• Content-Type: application/javascript
• Content-Length: 53331
• Content-Encoding: gzip
• Connection: Keep-Alive
• Cache-Control: max-age=604800

Answer 1

Apache mod_deflate is creating unique Etag for each entity as these identify the specific entity variant of the URL. Each negotiated variant needs to have unique ETag:s. For mod_deflate it's as simple as adding the encoding to the already computed ETag.

One workaround is to remove the encoding from the Etag:

<Location /js>
  RequestHeader  edit "If-None-Match" "^(.*)-gzip$" "$1"
  Header  edit "ETag" "^(.*[^g][^z][^i][^p])$" "$1-gzip"
</Location>

If you are using Apache 2.5 with the mod_deflate module, you can use the directive DeflateAlterETag to specifies how the ETag hader should be altered when a response is compressed.

DeflateAlterETag AddSuffix|NoChange|Remove

Source: https://httpd.apache.org/docs/trunk/mod/mod_deflate.html#deflatealteretag

This blog post suggest to remove Etags altogether and to rely on the Cache-Control header.

To do that in httpd.conf:

<IfModule mod_headers.c>
    Header unset ETag
</IfModule>

FileETag None

Note that if entities gzip:ed by mod_deflate still carries the same ETag as the plain entiy, this may cause inconsistency in ETag aware proxy caches.

More info here:

• https://bz.apache.org/bugzilla/show_bug.cgi?id=45023

Answer 2

A workaround not yet reported is, you can just apply this configuration:

RequestHeader edit "If-None-Match" '^"((.*)-gzip)"$' '"$1", "$2"'

(Originally suggested by Joost Dekeijzer, see https://bz.apache.org/bugzilla/show_bug.cgi?id=45023#c22, and still working nowadays on version 2.4)

Answer 3

There also seems to be a problem with gzipped resources like .js .css and the Vary: Accept-encoding Header with Chrome.

Please check my Anwser given here: https://stackoverflow.com/a/40726246/135785

This solved the problem for me:

<FilesMatch "(\.js\.gz|\.css\.gz)$">
 # Serve correct encoding type.
 Header set Content-Encoding gzip
 # Force proxies to cache gzipped & non-gzipped css/js files separately.
  BrowserMatch "Chrome" ChromeFound
 Header append Vary Accept-Encoding env=!ChromeFound
</FilesMatch>

Check your Apache Config for "Header append Vary Accept-Encoding"

Answer 4

I strongly suspect the Cache-Control: max-age=0 is at fault.

Quoting this post and its excellent answer: What's the difference between Cache-Control: max-age=0 and no-cache?

On the other hand, sending a request with Cache-Control: no-cache (aka. "end-to-end reload") doesn't revalidate and the server MUST NOT use a cached copy when responding.