Mongoose ignores $ne while updating document

Question

I'm building a persistent html fields framework using Node, Mongoose, and Socket.io for the backend. I've run into what seems like a bug in Mongoose's update method.

The documents in mongo look like this:

{
    "_id" : "pickable_qty",
    "currently_connected" : [ ],
    "locks" : [
        {
            socket_id: 'eQYVyz1z28rJZRPpAAAB',
            unique_values:{
                merchant_warehouse_id: 11.1,
                product_item_id: 5555
            }
        },
        {
            socket_id: 'eQYVyz1z28rJZRPpAAAB',
            unique_values:{
                merchant_warehouse_id: 11.2,
                product_item_id: 5555
            }
        },
        {
            socket_id: 'eQYVyz1z28rJZRPpAAAB',
            unique_values:{
                merchant_warehouse_id: 11.1,
                product_item_id: 1234
            }
        }
    ],
    "definition" : {
        "ajax_url" : "/persistent-fields/pickable_qty",
        "unique_keys" : [
            "product_item_id",
            "merchant_warehouse_id"
        ],
        "max_idle_seconds" : 30,
        "field_type" : "text"
    },
    "__v" : 0
}

The problem arises when I start inserting subdocuments into the locks array.

I insert them using Mongoose's update method like so:

FieldSchema.update(
    {
        _id: 'pickable_qty',
        'locks.unique_values': { 
            '$ne': { 
                merchant_warehouse_id: 11.1, 
                product_item_id: 5334 
            } 
        },
        'definition.unique_keys': { 
            '$all': [ 
                'merchant_warehouse_id', 
                'product_item_id' 
            ] 
        } 
    },
    {
        '$push': {
            locks: { 
                socket_id: 'eQYVyz1z28rJZRPpAAAB', 
                unique_values: {
                    merchant_warehouse_id: 11.1, 
                    product_item_id: 5334 
                } 
            } 
        }
    },
    function(err, count, res){
        console.log('err:', err, 'count:', count, 'res:', res);
        //err: null count: 1 res: { ok: true, n: 1, updatedExisting: true }
    }
);

The first insert works exactly as expected, there were no locks in the array containing the unique values {merchant_warehouse_id: 11.1,product_item_id: 5334} so the document was found and the lock subdocument was inserted.

However running this same update the second time should not insert a new locks subdocument because one already exists with the same unique_values and the $ne part of the query should cause it to return no matches to update.

I have confirmed that doing a find with the same query returns no documents both on the MongoDB command line and using Mongoose's own Schema.find method, but the Schema.update method still finds a document and inserts a duplicate lock subdoc.

Am I just going crazy or is there a reason why Mongoose's find method won't retrieve the document while update will?

score 0 · Answer 1 · answered Feb 06 '15 at 22:13

0

The problem is that you are querying whether or not an object matches a subdocument, and this can be tricky. What you want to do use a combination of $elemMatch and $ne in your query.

var query = {
    _id: 'pickable_qty',
    'locks': { 
        $elemMatch: {
            merchant_warehouse_id: {
                $ne: 11.1
            },
            product_item_id: {
                $ne: 5334
            }
        }
    },
    'definition.unique_keys': { 
        '$all': [ 
            'merchant_warehouse_id', 
            'product_item_id' 
        ] 
    } 
}

It's basically a query for a document inside of an array.

answered Feb 06 '15 at 22:13

Brian Noah

2,844
15
27

Thanks for the response, but I can't seem to get that query to retrieve any documents, even with empty `locks` and omitting the `$all`. It just seems strange that the original query works except in the context of Mongoose's `update` method – Raz Varren Feb 06 '15 at 22:35
@RazVarren Can you try this inside of your mongo shell? – Brian Noah Feb 06 '15 at 22:43
Yes, I tried it both in the mongo shell and on mongoose's `update` method. I was not able to retrieve any documents with empty `locks` using it. – Raz Varren Feb 06 '15 at 22:53
What about using locks: {$ne: []} as well? – Brian Noah Feb 06 '15 at 22:55
using `locks:{$ne:[]}` does match if there is at least one subdocument in the array, but what I am trying to prevent is having duplicate `locks`. I've update the original post to show a list of what I consider unique `locks` in the database. Basically I want to prevent mongoose from inserting a second `locks` subdocument with the same `unique_values` in it. The `"locks.unique_values":{$ne: {merchant_warehouse_id: 11.1, product_item_id: 5334}}` does exactly this in the mongo shell's `find` and `update`. Using mongoose's `find` also works correct but mongoose's `update` seems to ignore the `$ne` – Raz Varren Feb 06 '15 at 23:20
Unfortunately `$addToSet` for documents, cause that would fix this whole issue. :( – Brian Noah Feb 06 '15 at 23:24

score 0 · Accepted Answer · answered Feb 07 '15 at 00:33

0

After using MongoDB's query profiler, I figured out that Mongoose was switching the order of the keys in the lock's unique_values object while doing the update query. Then it was inserting the duplicate subdocuments with the keys in the correct order. So apparently MongoDB's $ne operator will match nested subdocument arrays only if the keys in the subdocument are in the same order as your query.

answered Feb 07 '15 at 00:33

Raz Varren

1
2

In general, equality testing for subdocuments in MongoDB depends on key order, see [exact match on embedded documents](http://docs.mongodb.org/manual/tutorial/query-documents/#exact-match-on-the-embedded-document) from the MongoDB Manual. It's probably not what one expects. Mongoose should know this and take it into account - I'd consider filing an issue in the Mongoose Github project to see if that behavior is intentional. – wdberkeley Feb 07 '15 at 05:00
I think the problem is that because I'm using Node and Node is Javascript, the object keys actually [don't have an order](http://stackoverflow.com/questions/5525795/does-javascript-guarantee-object-property-order) – Raz Varren Feb 07 '15 at 07:33

Mongoose ignores $ne while updating document

2 Answers2