How to determine if I've reached the size limit via Go's MaxBytesReader

Question

I am new to Go, and using Mux to accept HTTP POST data. I would like to use the MaxBytesReader to ensure a client does not overwhelm my server. According to the code, there is a requestBodyLimit boolean which indicates whether that limit has been hit.

My question is: when using MaxBytesReader, how can I determine whether I have actually hit the maximum when processing a request?

Here is my code:

package main

import (
        "fmt"
        "log"
        "html/template"
        "net/http"

        "github.com/gorilla/mux"
)

func main() {
        r := mux.NewRouter()
        r.HandleFunc("/handle", maxBytes(PostHandler)).Methods("POST")
        http.ListenAndServe(":8080", r)
}

// Middleware to enforce the maximum post body size
func maxBytes(f http.HandlerFunc) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
            // As an example, limit post body to 10 bytes
            r.Body = http.MaxBytesReader(w, r.Body, 10)
            f(w, r)
    }
}

func PostHandler(w http.ResponseWriter, r *http.Request) {
    // How do I know if the form data has been truncated?
    book := r.FormValue("email")
    fmt.Fprintf(w, "You've requested the book: %s\n", book)
}

How can I:

Determine that I have reached the max POST limit (or have access to requestBodyLimit
Have my code be able to branch on this condition?

Thank you for this question. I've been using `io.LimitReader` for this without realizing there is a specific `http` version for this specific purpose. Sorry if that was an off topic comment. — RayfenWindspear, Oct 18 '18 at 17:35
Incidentally, the default that will trip `requestBodyLimit` is 256k (`256 << 10`) and is apparently tied specifically to `http.MethodPost`. https://github.com/golang/go/blob/master/src/net/http/server.go#L1068 — RayfenWindspear, Oct 18 '18 at 17:46

score 5 · Accepted Answer · edited Oct 18 '18 at 19:58

5

Call ParseForm at the beginning of the handler. If this method returns an error, then the size limit was breached or the request body was invalid in some way. Write an error status and return from the handler.

There's not an easy way to detect if the error is a result of the size limit breach or some other error.

func PostHandler(w http.ResponseWriter, r *http.Request) {
    if err := r.ParseForm(); err != nil {
        http.Error(w, "Bad Request", http.StatusBadRequest)
        return
    }

    book := r.FormValue("email")
    fmt.Fprintf(w, "You've requested the book: %s\n", book)
}

Depending on your needs, it may be better to place the check in the middleware:

func maxBytes(f http.HandlerFunc) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
            r.Body = http.MaxBytesReader(w, r.Body, 10)
            if err := r.ParseForm(); err != nil {
                http.Error(w, "Bad Request", http.StatusBadRequest)
                return
            }
            f(w, r)
    }
}

edited Oct 18 '18 at 19:58

poundifdef

16,490
19
82
126

answered Oct 18 '18 at 17:14

Cerise Limón

88,331
8
164
179

I feel like I should revoke my upvote, because this could return a false negative in the case where the cutoff is exactly where it would cut off one of the `Form` values, which would not cause any error to be thrown. – RayfenWindspear Oct 18 '18 at 20:48
@RayfenWindspear In the code above, the call to ParseForm unconditionally returns an error when the size limit is breached. The form parser does not contain code to ignore read errors between form values as you might be implying. – Cerise Limón Oct 18 '18 at 22:56
Oh, my bad, you are right. I was omitting something in my micro test. It does properly return a `request body too large` error as in this play link. https://play.golang.org/p/hq9P8uG5KR6 One thing that might be important to note. Internally this uses an `io.LimitReader` but that doesn't return such an error if the size is larger. It's important for me at least because I didn't know about `http.MaxBytesReader` until now and I have a line of code using `io.LimitReader` instead that I now need to fix. – RayfenWindspear Oct 18 '18 at 23:16

sieberts · Answer 2 · 2018-10-18T18:58:09.167

1

You can determine whether or not the limit was exceeded by checking if the length of the read data is bigger than (or equals) the MaxBytesSize:

maxBytesSize := 10
r.Body = http.MaxBytesReader(w, r.Body, maxBytesSize)

// check if request body is not too large
data, err := ioutil.ReadAll(r.Body)
if err != nil {
    if len(data) >= maxBytesSize {
         //exceeded
    }
    // some other error
}

edited Oct 18 '18 at 18:58

answered Oct 18 '18 at 18:49

sieberts

452
5
15

It is unnecessary to check the length as long as you are using `http.MaxBytesReader` instead of simply using `io.LimitReader`. Not only will the length never go over `maxBytesSize`, but the former will return an error if there are more bytes than `maxBytesSize`. – RayfenWindspear Oct 18 '18 at 23:20
You could still get a different error returned by `ioutil.ReadAll()`. By Checking the length you can properly return status `http.StatusRequestEntityTooLarge`. – sieberts Oct 19 '18 at 07:50

How to determine if I've reached the size limit via Go's MaxBytesReader

2 Answers2