0

I recently saw a GET call which had the Transfer-Encoding header set to Chunked. What we noticed from this call is a delay then a 500 socket timeout.

Digging deeper, we showed this behavior of the 500 coming from ELB's and Apache web servers.

Of note, if we make the GET call with Transfer-Encoding as Chunked and include an empty payload, the ELB in our case allows the request to go through as normal.

Given this ELB's and Apache web servers exihibit this behaviour, my question is sending Transfer-Encoding as Chunked on a HTTP GET call is valid or not?

Bellini
  • 322
  • 1
  • 5
  • 11

1 Answers1

2

The Transfer-Encoding header is now under authority of RFC 7230, section 3.3.1 (see also the IANA Message Header Registry. You will find that this header is part of the HTTP message framing, so it is valid for both, requests and responses.

Keeping in mind that GET requests may carry bodies, the reaction of the server(s) is absolutely correct. What is causing the delay followed by the 500 is probably the following: The server would expect a body but fails to find one, since the chunk-encoded representation of an empty string is not an empty string but a zero-sized chunk. In consequence, the server is running into a timeout.

Community
  • 1
  • 1
DaSourcerer
  • 5,318
  • 4
  • 26
  • 52
  • 1
    Keep in mind also that the spec prohibits any semantics being assigned to the meaning of the request body on GET. Consequently a lot of agents (e.g. proxies) may strip or block GET requests with bodies, so it's recommended to avoid this. – Adrien Feb 12 '17 at 20:43
  • @Adrien while true, this is a bit out of the scope of this question. I wanted to show that `GET` requests are allowed to have a body *at all* and thus the `Transfer-Encoding` header not being misplaced here. – DaSourcerer Feb 12 '17 at 21:04
  • Understood, just like Content-Length, the existence of the Transfer-Encoding: chunked request header indicates there is a body, and it needs to be therefore completed. – Adrien Feb 13 '17 at 21:42
  • @Adrien I actually had to check just now. Seems you are [not entirely correct](https://tools.ietf.org/html/rfc7230#section-3.3.2): A `Content-Length` header with a value > 0 can be valid together with an empty body in some cases. I must admit, I didn't know this; I just assumed anything but `Content-Length: 0` were invalid here. – DaSourcerer Feb 13 '17 at 21:52
  • It's more clear and restrictive for request messages. Since you have to maintain the connection (and therefore framing) to receive the response. If you have Content-Length > 0 and don't send a body, you break framing. You can't send invalid (wrong value) C-L in a request and expect things to work. You can send C-L wrong if you also send T-E:chunked, since the C-L then is supposed to be ignored, but I wouldn't rely on this either. – Adrien Feb 13 '17 at 21:55
  • @Adrien You are wrong here, I am afraid. I see your reasoning and as I wrote earlier, I would have expected things to work exactly as you described. However, the sections [3.3.2](https://tools.ietf.org/html/rfc7230#section-3.3.2) and [3.3.3](https://tools.ietf.org/html/rfc7230#section-3.3.3) are very clear in that (1) Any `Content-Length` header with a value >= 0 is valid, (2) the presence of a `Content-Length` header does not necessarily imply the presence of a message body, and (3) `Content-Length` may appear w/o ramifications of message framing. – DaSourcerer Feb 14 '17 at 00:09
  • I'll submit it as an errata to the WG then, since without valid C-L there is no framing possible, and no possibility of multiple requests on a single connection, which is the basic tenet of HTTP/1.1. I admit the language in the RFC is pretty non-commital about C-L being correct, but I've seen so many discussions on the WG about it, that I can guarantee you if you get C-L wrong in a request it won't get through a proxy for instance, or at least the next request wont. Incorrect C-L also opens up smuggling attack vulnerabilities, so this has security implications. – Adrien Feb 14 '17 at 20:39
  • P.s. are you just going off your reading of these sections, or some other source? – Adrien Feb 14 '17 at 20:40
  • Actually I believe your reading of the section is wrong. It clearly states C-L is needed for delineating the start and finish of the message body and is required for framing. Just because it says any value is valid, should be taken to mean you can have a body of any size. It doesn't mean you can put any value in here that doesn't match the actual body size. – Adrien Feb 14 '17 at 20:46
  • Section 3.3.3 para 4 "4. If a message is received without Transfer-Encoding and with either multiple Content-Length header fields having differing field-values or a single Content-Length header field having an invalid value, then the message framing is invalid and the recipient MUST treat it as an unrecoverable error." – Adrien Feb 14 '17 at 20:55
  • It is a bit tedious to argue in the comments. So let me just quote something from section [3.3.2](https://tools.ietf.org/html/rfc7230#section-3.3.2): "For messages that do not include a payload body, the Content-Length indicates the size of the selected representation" The section makes it also clear that responses to HEAD requests or responses with a 304 status code may contain a Content-Length header. So in quintessence it is not strictly tied to message framing. You are, however, correct that one cannot simply make up the content of it. – DaSourcerer Feb 14 '17 at 21:06
  • RESPONSE messages to HEAD is the only exception, where it's prohibited from actually having a body, and the C-L refers to what would be transferred if the method were to be changed to GET. We were talking about request messages, which this does not apply to. There's no exception. If your request has a body, it must either have a correct C-L value, or T-E chunked. There's no other way to maintain framing, which is necessary. http://lists.w3.org/Archives/Public/ietf-http-wg/2017JanMar/ – Adrien Feb 14 '17 at 21:31