JSON Unmarshal does not work with dynamic struct created by relect

Question

I'm trying to parse JSON files by using dynamically created structs, but apparently I'm doing something wrong. Can somebody please tell we what am I doing wrong here:

structured := make(map[string][]reflect.StructField)
structured["Amqp1"] = []reflect.StructField{
    reflect.StructField{
        Name: "Test",
        Type: reflect.TypeOf(""),
        Tag:  reflect.StructTag(`json:"test"`),
    },
    reflect.StructField{
        Name: "Float",
        Type: reflect.TypeOf(5.5),
        Tag:  reflect.StructTag(`json:"float"`),
    },
    reflect.StructField{
        Name: "Connections",
        Type: reflect.TypeOf([]Connection{}),
        Tag:  reflect.StructTag(`json:"connections"`),
    },
}

sections := []reflect.StructField{}
for sect, params := range structured {
    sections = append(sections,
        reflect.StructField{
            Name: sect,
            Type: reflect.StructOf(params),
        },
    )
}

parsed := reflect.New(reflect.StructOf(sections)).Elem()
if err := json.Unmarshal([]byte(JSONConfigContent), &parsed); err != nil {
    fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
    os.Exit(1)
}

https://play.golang.org/p/C2I4Pduduyg

Thanks in advance.

mkopriva · Accepted Answer · 2020-05-12T08:25:36.800

You want to use .Interface() to return the actual, underlying value, which should be a pointer to the concrete anonymous struct.

Note that the reflect.New function returns a reflect.Value representing a pointer to a new zero value for the specified type. The Interface method, in this case, returns that pointer as interface{} which is all you need for json.Unmarshal.

If, after unmarshaling, you need a non-pointer of the struct you can turn to reflect again and use reflect.ValueOf(parsed).Elem().Interface() to effectively dereference the pointer.

parsed := reflect.New(reflect.StructOf(sections)).Interface()
if err := json.Unmarshal([]byte(JSONConfigContent), parsed); err != nil {
    fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
    os.Exit(1)
}

https://play.golang.org/p/Bzu1hUyKjvM

score 0 · Answer 2 · answered May 07 '20 at 22:54

reflect.New returns a value representing a pointer. Change line 69:

fmt.Printf(">>> %v", &parsed)

Result:

>>> <struct { Amqp1 struct { Test string "json:\"test\""; Float float64 "json:\"float\""; Connections []main.Connection "json:\"connections\"" } } Value>

GoForth · Answer 3 · 2020-05-07T23:17:00.273

I would recommend something very different. I generally avoid reflection where possible. You can accomplish what you are trying to do by simply providing structs for each type of config you expect and then do an initial "pre-unmarshalling" to determine what type of config you should actually use by name (which, in this case is a key of your JSON object):

package main

import (
    "encoding/json"
    "fmt"
    "os"
)

//Amqp1 config struct
type Amqp1 struct {
    Config struct {
        Test        string       `json:"test"`
        Float       float64      `json:"float"`
        Connections []Connection `json:"connections"`
    } `json:"Amqp1"`
}

//Connection struct
type Connection struct {
    Type string `json:"type"`
    URL  string `json:"url"`
}

//JSONConfigContent json
const JSONConfigContent = `{
    "Amqp1": {
        "test": "woobalooba",
        "float": 5.5,
        "connections": [
            {"type": "test1", "url": "booyaka"},
            {"type": "test2", "url": "foobar"}
        ]
    }
}`

func main() {
    configMap := make(map[string]interface{})
    if err := json.Unmarshal([]byte(JSONConfigContent), &configMap); err != nil {
        fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
        os.Exit(1)
    }

    //get config name
    var configName string
    for cfg := range configMap {
        configName = cfg
        break
    }

    //unmarshal appropriately
    switch configName {
    case "Amqp1":
        var amqp1 Amqp1
        if err := json.Unmarshal([]byte(JSONConfigContent), &amqp1); err != nil {
            fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
            os.Exit(1)
        }
        fmt.Printf("%s >>\n", configName)
        fmt.Printf("Test: %s\n", amqp1.Config.Test)
        fmt.Printf("Float: %v\n", amqp1.Config.Float)
        fmt.Printf("Connections: %#v\n", amqp1.Config.Connections)

    default:
        fmt.Printf("unknown config encountered: %s\n", configName)
        os.Exit(1)
    }
}

The initial "pre-unmarshalling" happens on the 2nd line of main(), to a plain map where the key is a string and the value is interface{} because you don't care. You're just doing that to get the actual type of config that it is, which is the key of the first nested object.

You can run this on playground as well.

Yeah this you can do when you know the structure of the config file at implementation time. Sadly this is not true in my case. — paramite, May 12 '20 at 08:20
You still have to know the structure to some degree when using reflection. Even in the example you gave you provided the name of the config. In my solution, the name of the config is all you need. — GoForth, May 14 '20 at 22:40

JSON Unmarshal does not work with dynamic struct created by relect

3 Answers3