Convert mongodb return object to dictionary

Question

I'm using the bottle framework together with mongoengine. I have an orders model :

class OrderDetail(Option):
    orderDetailsQty = FloatField()

    def to_dict(self):
        return mongo_to_dict_helper(self)


class Order(Document):
    userName = StringField(required=True)
    orderDate = DateTimeField()
    orderStatus = ListField(EmbeddedDocumentField(Status))
    orderDetails = ListField(EmbeddedDocumentField(OrderDetail))
    orderComments = ListField(EmbeddedDocumentField(Comment))
    isActive = BooleanField()

    def to_dict(self):
        orderObj = mongo_to_dict_helper(self)
        orderDetailList = []
        for orderDetail in orderObj["orderDetails"]:
            orderDetailList.append(orderDetail.__dict__)
        orderObj["OrderDetails"] = orderDetailList
        return (self)

When mongodb is queried I get an object which is then converted in to a dict by using the following function :

def mongo_to_dict_helper(obj):
    return_data = []
    for field_name in obj._fields:
        if field_name in ("id",):
            continue
        data = obj._data[field_name]

        if isinstance(obj._fields[field_name], StringField):
            return_data.append((field_name, str(data)))
        elif isinstance(obj._fields[field_name], FloatField):
            return_data.append((field_name, float(data)))
        elif isinstance(obj._fields[field_name], IntField):
            return_data.append((field_name, int(data)))
        elif isinstance(obj._fields[field_name], ListField):
            return_data.append((field_name, int(data)))
        else:
            # You can define your logic for returning elements
            pass
    return dict(return_data)

I found this function after a long search in the internet. Later found out that this function also fails while defining a member as the ListField(EmbeddedDocumentField(obj)).

I also tried writing a condition for catching the specific case of EmbeddedDocumentField :

elif isinstance(obj._fields[field_name], EmbeddedDocumentField):
    return_data.append(mongo_to_dict_helper(data))

but that didn't do any good either.

Anyone have a workaround for this issue ?

You are trying to convert whole queryset to a dict or just one document? — alexvassel, Nov 05 '12 at 10:42
I was trying to convert one object to json I was unaware of the to_mongo() function... — mahesmohan, Nov 05 '12 at 18:43

score 52 · Accepted Answer · answered Nov 05 '12 at 10:47

52

What about just using to_mongo method of an object to convert it to a dict?

object.to_mongo()

answered Nov 05 '12 at 10:47

alexvassel

9,442
2
27
30

3

Thats exactly what I was looking for. But still I get an error TypeError that the ObjectId is not serializable. – mahesmohan Nov 05 '12 at 19:16
5

Pasting a link to to_mongo() documentation will be greatly appreaceated as I cant find one anywhere. Thanks @alexvassel – mahesmohan Nov 06 '12 at 06:38
Actually, i did not find this method in the official documentation) So it is only looking into a source) – alexvassel Nov 12 '12 at 10:25
And about your error, try to serialize the str(ObjectId). Actually, i don't know what are you trying to do, so it is just a guess. – alexvassel Nov 12 '12 at 10:29
Is possible dereference all reference fields recursively using to_mongo()? – Jonathan Simon Prates Jun 11 '13 at 12:59
2

@alexvassel how to use this method?? from this `inbox = Inbox.objects.filter(messages__to_users__in=[username]).to_mongo()` i got this `'BaseQuerySet' object has no attribute 'to_mongo'` – holms Jun 01 '14 at 07:52
4

@holms you can use a comprehension: `[ob.to_mongo() for ob in query_set]` – z0r Oct 01 '14 at 22:53
1

Is there a `from_mongo()` as well? Does the `to_mongo` store metainfo to restore the object? – Eduardo Pignatelli Mar 22 '19 at 12:26
1

```object.to_mongo()``` returns [SON object](https://api.mongodb.com/python/current/api/bson/son.html) to get a dictionary you need to call ```to_dict()``` method: ```object.to_mongo().to_dict()``` – Mustapha-Belkacim Nov 08 '19 at 10:57

score 28 · Answer 2 · answered Jan 09 '15 at 06:09

28

Expanding on @alexvassel's and @z0r's answers, calling .to_mongo() converts the object to a SON instance. Once you have it, you can call its .to_dict() method to convert it to a dictionary.

For example... (qset is a queryset that's returned from mongoengine, after e.g. Posts.objects.all()).

sons = [ob.to_mongo() for ob in qset]
for son in sons:
    print str(son.to_dict())

answered Jan 09 '15 at 06:09

aralar

2,594
4
25
41

1

`to_mongo` accepts `use_db_field` argument, which should be False if you have a field with a custom `db_field`. – warvariuc Mar 24 '21 at 09:22

score 10 · Answer 3 · answered May 15 '15 at 16:23

10

import json
json.loads(yourobj.to_json())

answered May 15 '15 at 16:23

roberto el boberto

111
1
2

This is my most favorite answer in this thread, however, it returns a `list` of `dicts` and not a `dict` directly - even though there is just one result. In order to access one `dict` you have to access the `lists` objects like `list[i]`. Otherwise this is a great solution! – Yannik Suhre May 29 '20 at 08:50

score 5 · Answer 4 · answered Nov 08 '19 at 11:12

5

Extending on @alexvassel's answer, to_mongo() method returns SON object, which you can convert to dict by calling its to_dict() method

object.to_mongo().to_dict()

answered Nov 08 '19 at 11:12

Mustapha-Belkacim

904
1
9
14

score 1 · Answer 5 · answered Sep 10 '19 at 06:31

1

you can custom method to convert object to dict

class Order(Document):
    userName = StringField(required=True)
    orderDate = DateTimeField()
    orderStatus = ListField(EmbeddedDocumentField(Status))
    orderDetails = ListField(EmbeddedDocumentField(OrderDetail))
    orderComments = ListField(EmbeddedDocumentField(Comment))
    isActive = BooleanField()

    def as_dict(self):
        return {
            "user_name": self.userName,
            "order_date": self.orderDate.strftime("%Y-%m-%d %H:%M:%S"),
        }

now you can use obj.as_dict() to dict

orders = Order.objects.all()
datas = [each.as_dict() for each in orders]

answered Sep 10 '19 at 06:31

joeycheng

51
1

so how about `orderStatus`,`orderDetails` and `orderComments` which is a `EmbeddedDocumentField` ,and in some case is a `ReferenceField`,in such case,how can it be done?? – ken Jan 05 '20 at 07:07
1

@ken,you can define as_dict() at ReferenceField, and` "order_status":self.orderStatus.as_dict()`; or, you can use aggregate,`obj.objects.aggregate([{"$lookup":{"localField":"orderStatusId","foreignFiled":"_id","form":"orderStatus","as":"order_status"}}])` – joeycheng Jan 06 '20 at 08:17

score 0 · Answer 6 · answered Jun 14 '18 at 14:43

0

combining all other answers,

import json
dict = {'data':[json.loads(ob.to_json()) for ob in qset]}

answered Jun 14 '18 at 14:43

cegprakash

2,317
26
52

score 0 · Answer 7 · edited Apr 06 '21 at 14:34

0

There can be two scenario.

when query returns CommandCursor object

    **records = list(CursorObject)**

    ex - Class.objects().aggregate({...})

when query returns BaseQuerySet object

   **import json**

   **records = json.loads(BaseQuerySetObject.to_json())**

   ex - Class.objects().filter(..)

edited Apr 06 '21 at 14:34

jtlz2

4,913
6
43
84

answered Mar 05 '21 at 06:20

chandra kishor

19
2

Convert mongodb return object to dictionary

7 Answers7

Linked