2

I understand an HTTP request will result in a response with a code and optional body.

If we call the originator of the request the 'client' and the recipient of the request the 'server'.

Then the sequence is

  1. Client sends request
  2. Server receives request
  3. Server sends response
  4. Client receive response

Is it possible for the Server to complete step 3 but step 4 does not happen (due to dropped connection, application error etc).

In other words: is it possible for the Server to 'believe' the client should have received the response, but the client for some reason has not?

Mike Q
  • 21,350
  • 19
  • 80
  • 124
  • 1
    probably not, because of TCP – Asu Oct 20 '16 at 09:50
  • @bolov no http://stackoverflow.com/questions/12871760/packet-loss-and-packet-duplication – Asu Oct 20 '16 at 09:53
  • 2
    @Asu, absolutely yes, actually. Because of TCP. – Jan Hudec Oct 20 '16 at 09:54
  • 1
    It depends on the reliability of the underlying transport, which varies considerably across the net, even among endpoints supporting TCP. Haven't you ever tried to visit a busy website and never gotten a response to a request because it got dropped, only to come back later and discover that your change actually did stick? – Tom Barron Oct 20 '16 at 09:54
  • yes to what? TCP guarentees a sent packet to be received, right? – Asu Oct 20 '16 at 09:55
  • 1
    The TCP standard guarantees that a sent packet will be received or the sender will get an error, but not all implementations are created equal and they don't all implement the standard perfectly. – Tom Barron Oct 20 '16 at 09:56
  • @TomBarron well, in your example the client knows that the server got the request and is waiting for a response. So there's no breach in the TCP protocol here.. – Fabian Bettag Oct 20 '16 at 09:59
  • @FabianBettag: My point was that it's possible for the server to believe incorrectly that the client received a response when in fact it did not. – Tom Barron Oct 20 '16 at 10:05
  • @TomBarron your example says nothing about the Server side, but only about the clients view. – Fabian Bettag Oct 20 '16 at 10:06
  • @TomBarron, TCP can guarantee a packet will be received *by the peer TCP stack* or the sender will get an error. That does not, however, imply that the peer *application* actually got it. Application layer needs its own acknowledge for that. That is the reason TCP stacks don't, generally, propagate information about what is acknowledged to the application: it is not really useful anyway (and note that HTTP does not have any way to acknowledge the response). – Jan Hudec Oct 20 '16 at 10:17
  • @JanHudec: Thanks, Jan. Your answer is clear and helpful. Voting up. – Tom Barron Oct 20 '16 at 10:20

2 Answers2

5

Network is inherently unreliable. You can only know for sure a message arrived if the other party has acknowledged it, but you never know it did not.

Worse, with HTTP, the only acknowledge for the request is the answer and there is no acknowledge for the answer. That means:

  • The client knows the server has processed the request if it got the response. If it does not, it does not know whether the request was processed.
  • The server never knows whether the client got the answer.

The TCP stack does normally acknowledge the answer when closing the socket, but that information is not propagated to the application layer and it would not be useful there, because the stack can acknowledge receipt and then the application might not process the message anyway because it crashes (or power failed or something) and from perspective of the application it does not matter whether the reason was in the TCP stack or above it—either way the message was not processed.

The easiest way to handle this is to use idempotent operations. If the server gets the same request again, it has no side-effects and the response is the same. That way the client, if it times out waiting for the response, simply sends the request again and it will eventually (unless the connection was torn out never to be fixed again) get a response and the request will be completed.

If all else fails, you need to record the executed requests and eliminate the duplicates in the server. Because no network protocol can do that for you. It can eliminate many (as TCP does), but not all.

Jan Hudec
  • 65,662
  • 12
  • 114
  • 158
  • Thanks Jan. I agree idempotent would solve this. To use a trivial example if the request was to increment a counter on the server by 1 then there are side-effects. In that case do you agree de-duplication on the server is the only way to guard against duplicate message processing? – Mike Q Oct 20 '16 at 10:27
  • 1
    @MikeQ, yes, in that case only deduplication on the server, at application layer, is the only solution. And for that, you want suitably derived unique IDs for the request (either uuid, client id + unique part from client, or id obtained with separate request in a way that unused ids don't matter). – Jan Hudec Oct 20 '16 at 10:35
0

There is a specific section on that point on the HTTP RFC7230 6.6 Teardown (bold added):

(...)

If a server performs an immediate close of a TCP connection, there is a significant risk that the client will not be able to read the last HTTP response.

(...)

To avoid the TCP reset problem, servers typically close a connection in stages. First, the server performs a half-close by closing only the write side of the read/write connection. The server then continues to read from the connection until it receives a corresponding close by the client, or until the server is reasonably certain that its own TCP stack has received the client's acknowledgement of the packet(s) containing the server's last response. Finally, the server fully closes the connection.

So yes, this response sent step is a quite complex stuff.

Check for example the Lingering close section on this Apache 2.4 document, or the complex FIN_WAIT/FIN_WAIT2 pages for Apache 2.0.

So, a good HTTP server should maintain the socket long enough to be reasonably certain that it's OK on the client side. But if you really need to acknowledge something in a web application, you should use a callback (image callback, ajax callback) asserting the response was fully loaded in the client browser (so another HTTP request). That means it's not atomic as you said, or at least not transactional like you could expect from a relational database. You need to add another request from the client, that maybe you'll never get (because the server had crash before receiving the acknowledgement), etc.

regilero
  • 27,883
  • 6
  • 54
  • 94