Unable to unmarshal golang response

Question

I've been trying to extract some JSON by unmarshalling my json response into structs, but I have no idea why it's not doing it properly. I've also tried gjson but same result. Am I missing something here?

JSON Result:

{"availabilities":[{"pickup":{"status":"OnlineOnly","purchasable":false},"shipping":{"status":"InStockOnlineOnly","purchasable":true},"sku":"12341231","sellerId":"438178","saleChannelExclusivity":"OnlineOnly","scheduledDelivery":false,"isGiftCard":false,"isService":false}]}

Code:

// Inventory ...
type Inventory struct {
    Availabilities []Availability `json:"availabilities"`
}

// Availability ...
type Availability struct {
    Sku                    string   `json:"sku"`
    SellerID               string   `json:"sellerId"`
    SaleChannelExclusivity string   `json:"saleChannelExclusivity"`
    ScheduledDelivery      bool     `json:"scheduledDelivery"`
    IsGiftCard             bool     `json:"isGiftCard"`
    IsService              bool     `json:"isService"`
    Pickup                 Statuses `json:"pickup"`
    Shipping               Statuses `json:"shipping"`
}

// Statuses ..
type Statuses struct {
    Status      string `json:"status"`
    Purchasable bool   `json:"purchasable"`
}

func (pr *Program) checkInventory() {
    url := fmt.Sprintf("https://www.bestbuy.ca/ecomm-api/availability/products?accept-language=en-CA&skus=%s", pr.Sku)
    log.Infof("URL %s", url)
    resp, err := http.Get(url)
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()
    bodyBytes, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        log.Fatal(err)
    }

    log.Info(string(bodyBytes))

    var inv Inventory
    json.Unmarshal(bodyBytes, &inv)
    log.Infof("%+v", inv)
}

Console:

INFO[2020-04-07T03:01:10-07:00] URL https://www.bestbuy.ca/ecomm-api/availability/products?accept-language=en-CA&skus=12341231 
INFO[2020-04-07T03:01:10-07:00] {"availabilities":[{"pickup":{"status":"OnlineOnly","purchasable":false},"shipping":{"status":"InStockOnlineOnly","purchasable":true},"sku":"12341231
 ,"sellerId":"438178","saleChannelExclusivity":"OnlineOnly","scheduledDelivery":false,"isGiftCard":false,"isService":false}]}
INFO[2020-04-07T03:01:10-07:00] {Availabilities:[]}

You have to pass bodyBytes as `[]byte`. modify your line `json.Unmarshal(bodyBytes, &inv)` for `json.Unmarshal([]byte(bodyBytes), &inv)` Can you try it? — pcampana, Apr 07 '20 at 11:09
Don't discard the error from `json.Unmarshal` - just in general, don't discard errors, but *especially* when that's the thing you're having a problem with. — Adrian, Apr 07 '20 at 13:39

score 3 · Accepted Answer · answered Apr 07 '20 at 12:07

The problem is in the json.Unmarshall call. It is returning an error: "invalid character 'ï' looking for beginning of value” from json.Unmarshal

As it is explained here: The server is sending you a UTF-8 text string with a Byte Order Mark (BOM). The BOM identifies that the text is UTF-8 encoded, but it should be removed before decoding.

This can be done with the following line (using package "bytes"):

body = bytes.TrimPrefix(body, []byte("\xef\xbb\xbf"))

So the resulting working code is:

// Inventory ...
type Inventory struct {
    Availabilities []Availability `json:"availabilities"`
}

// Availability ...
type Availability struct {
    Sku                    string   `json:"sku"`
    SellerID               string   `json:"sellerId"`
    SaleChannelExclusivity string   `json:"saleChannelExclusivity"`
    ScheduledDelivery      bool     `json:"scheduledDelivery"`
    IsGiftCard             bool     `json:"isGiftCard"`
    IsService              bool     `json:"isService"`
    Pickup                 Statuses `json:"pickup"`
    Shipping               Statuses `json:"shipping"`
}

// Statuses ..
type Statuses struct {
    Status      string `json:"status"`
    Purchasable bool   `json:"purchasable"`
}

func (pr *Program) checkInventory() {
    url := fmt.Sprintf("https://www.bestbuy.ca/ecomm-api/availability/products?accept-language=en-CA&skus=%s", pr.Sku)
    log.Infof("URL %s", url)
    resp, err := http.Get(url)
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()
    bodyBytes, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        log.Fatal(err)
    }
    body := bytes.TrimPrefix(bodyBytes, []byte("\xef\xbb\xbf"))
    log.Info(string(body))

    var inv Inventory
    err = json.Unmarshal([]byte(body), &inv)
    if err != nil {
        log.Fatal(err)
    }
    log.Infof("%+v", inv)
}

Which means the API is technically returning invalid JSON, as BOM is not permitted. See https://stackoverflow.com/questions/4990095/json-specification-and-usage-of-bom-charset-encoding/38036753 — Adrian, Apr 07 '20 at 13:40

Unable to unmarshal golang response

1 Answers1