20

I am creating a test Go HTTP server, and I am sending a response header of Transfer-Encoding: chunked so I can continually send new data as I retrieve it. This server should write a chunk to this server every one second. The client should be able to receive them on demand.

Unfortunately, the client(curl in this case), receives all of the chunks at the end of the duration, 5 seconds, rather than receiving one chunk every one second. Also, Go seems to send the Content-Length for me. I want to send the Content-Length at the end, and I want the the header's value to be 0.

Here is the server code:

package main

import (
    "fmt"
    "io"
    "log"
    "net/http"
    "time"
)

func main() {
    http.HandleFunc("/test", HandlePost);
    log.Fatal(http.ListenAndServe(":8080", nil))
}

func HandlePost(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Connection", "Keep-Alive")
    w.Header().Set("Transfer-Encoding", "chunked")
    w.Header().Set("X-Content-Type-Options", "nosniff")

    ticker := time.NewTicker(time.Second)
    go func() {
        for t := range ticker.C {
            io.WriteString(w, "Chunk")
            fmt.Println("Tick at", t)
        }
    }()
    time.Sleep(time.Second * 5)
    ticker.Stop()
    fmt.Println("Finished: should return Content-Length: 0 here")
    w.Header().Set("Content-Length", "0")
}
kimura
  • 223
  • 1
  • 2
  • 5
  • Andrew Gerrand of the Go team posted something like this the other day... I will try and find it for you. – Simon Whitehead Nov 06 '14 at 00:26
  • Apologies, its not quite the same - its from the client end of the request. Still, you might learn something from it: https://github.com/nf/dl/blob/master/dl.go – Simon Whitehead Nov 06 '14 at 00:28
  • 3
    FUTURE READERS: If this isn't working for you, a mistake I made was to use `w.WriteHeader()` _before_ setting the _`Connection: Keep-Alive`_ and _`Transfer-Encoding: chunked`_ headers. Using `w.WriteHeader()` pushes the headers, after which you can no longer set additional headers, so the client never gets the _`Connection: Keep-Alive`_ and _`Transfer-Encoding: chunked`_ headers and treats the response as not-chunked – TheEnvironmentalist Jun 14 '18 at 02:01

2 Answers2

31

The trick appears to be that you simply need to call Flusher.Flush() after each chunk is written. Note also that the "Transfer-Encoding" header will be handled by the writer implicitly, so no need to set it.

func main() {
  http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    flusher, ok := w.(http.Flusher)
    if !ok {
      panic("expected http.ResponseWriter to be an http.Flusher")
    }
    w.Header().Set("X-Content-Type-Options", "nosniff")
    for i := 1; i <= 10; i++ {
      fmt.Fprintf(w, "Chunk #%d\n", i)
      flusher.Flush() // Trigger "chunked" encoding and send a chunk...
      time.Sleep(500 * time.Millisecond)
    }
  })

  log.Print("Listening on localhost:8080")
  log.Fatal(http.ListenAndServe(":8080", nil))
}

You can verify by using telnet:

$ telnet localhost 8080
Trying ::1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.1

HTTP/1.1 200 OK
Date: Tue, 02 Jun 2015 18:16:38 GMT
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked

9
Chunk #1

9
Chunk #2

...

You might need to do some research to verify that http.ResponseWriters support concurrent access for use by multiple goroutines.

Also, see this question for more information about the "X-Content-Type-Options" header.

maerics
  • 133,300
  • 39
  • 246
  • 273
  • 1
    Your flusher.Flush() version worked for me, but only after I added the nosniff header from the OP's example. Your version works fine in telnet and wget, but I guess the browsers are trying to be smarter about buffering content, so unless you specify nosniff, Chrome and Firefox both seem to refuse to process the data until it all arrives. – johnbr Apr 28 '16 at 16:46
  • 8
    For those wondering, see [this answer](http://stackoverflow.com/questions/18337630/what-is-x-content-type-options-nosniff). Basically what happens is that some browsers will try to infer the Content-Type of the page, even *after* the server explicitly tells the client it's, for instance, a text/html. Using `X-Content-Type-Options: nosniff` will disable this feature from said browsers. – morganbaz Jun 12 '16 at 16:09
0

It looks like httputil provides a NewChunkedReader function

themihai
  • 6,579
  • 11
  • 32
  • 53