Dictionary in protocol buffers

Question

Is there any way to serialize a dictionary using protocol buffers, or I'll have to use Thrift if I need that?

duplicate http://stackoverflow.com/questions/3874024/how-does-protobuf-net-support-for-dictionary-keyvaluepair-works — Aravind Yarram, Nov 16 '10 at 14:00
Not really duplicate. That question was about the .net-bindings. OP doesn't specify language environment. — JesperE, Nov 17 '10 at 12:29

score 57 · Answer 1 · answered Nov 14 '16 at 09:46

57

For future answer seekers, ProtoBuf now supports Maps natively:

message MapMessage
{
    map<string, string> MyMap = 1;
}

answered Nov 14 '16 at 09:46

Flassari

928
7
13

3

Please note that Maps still can't be repeated. – mattes Mar 05 '18 at 20:06
9

Maps are collections, they're already repeated by definition. – Flassari Jul 10 '18 at 15:24
The link is for proto2 syntax, but maps are also available in proto3 - https://developers.google.com/protocol-buffers/docs/proto3#maps – Sergey Kolodyazhnyy Dec 09 '20 at 18:02

score 52 · Accepted Answer · edited Dec 11 '20 at 01:33

52

Protobuf specification now supports dictionaries (maps) natively.

Original answer

People typically write down the dictionary as a list of key-value pairs, and then rebuild the dictionary on the other end.

message Pair {
   string key = 1;
   string value = 2;
}

message Dictionary {
   repeated Pair pairs = 1;
}

edited Dec 11 '20 at 01:33

Sergey Kolodyazhnyy

686
5
16

answered Nov 17 '10 at 12:33

JesperE

59,843
19
133
192

2

Is there a way to do the same thing with a dynamic type? I have a `Dictionary` that I need to serialize. =/ Was trying to investigate to see if protocol buffers could do this without a huge effort. – mpontillo Mar 09 '11 at 20:18
3

Well, no. Protobuf is not a general object serialization protocol. You need to define protobuf messages for all the data you want to serialize. (Honestly, if you have a map, you should probably start by refactoring your code.) – JesperE Mar 10 '11 at 06:53
@Mike You would have to create a field for every type it *could* be, which is a bug just waiting to happen. – b1nary.atr0phy Aug 09 '15 at 01:01

score 0 · Answer 3 · answered Nov 15 '15 at 22:12

You can check the ProtoText package.

Assume you want to serialize a dict person_dict to a pre-defined PersonBuf protobuf object defined in personbuf_pb2 module.

In this case, to use ProtoText,

import ProtoText
from personbuf_pb2 import PersonBuf

obj = PersonBuf()
obj.update(person_dict)

score 0 · Answer 4 · answered Jan 13 '20 at 14:47

I firstly comment the @Flassari 's answer as it is really convenient.

However, in my case, I needed map<Type, repeated AnyModel> where :

enum Type {
    Undefined = 0;
    Square = 1;
    Circle = 2;
}

message AnyModel {
    string Name = 1;
}

Here I just want to return a dictionary that, for each type, contain a list of AnyModel

However, I didn't find a better workaround than the one proposed by @JesperE so I did the following: (as you can't use enum as key in map)

message MyRPCBodyCall {
    map<string, AnyModels> Models = 1;
}

enum Type {
    Undefined = 0;
    Square = 1;
    Circle = 2;
}

message AnyModel {
    string Name = 1;
}

message AnyModelArray {
    repeated AnyModel AnyModels = 1;
}

Here I convert from/to string my enum using my chosen code languages from both server/client side

So both approaches are actually valid answers IMO, depends on your requirements.

Dictionary in protocol buffers

4 Answers4

Original answer

Linked