Why even after using async await for multiple calls still got empty response?

Question

Here is simple node route in which calling an asynchronous api.

What needed is to return data after the looping. But It is returning blank object.

try {
  const array = ["brunch", "lunch", "crunch"]
  const data = {}
  array.map(async(d) => {
    const venue = await Venue.find({ "category": { "$in": [d] }})
    data[d] = venue
  })
  return data
} catch(err) {
  throw err
}

Please help me to achieve this

@T.J.Crowder How do I put dynamic keys as well to my data object? I don't think so it is a duplicate — Dark Knight, Sep 27 '18 at 10:24
Thank you for the fiddle I will check and let you know. **Can you please tell me one thing why need of `Promise.all()` if we are already using async await?** — Dark Knight, Sep 27 '18 at 10:37
because array.map returns an array of promises - you need to wait for all of them to complete ... note, your code (and the fiddle) doesn't do the `Venue.find` serially, it does all of them in parallel - if it needs to be one after the other (some API's don't like multiple requests at once) then you'd use different code - i.e. [this fiddle](https://jsfiddle.net/6fx195bu/) — Jaromanda X, Sep 27 '18 at 10:41
Possible dupe https://stackoverflow.com/questions/50735135/dynamic-keys-after-group-by — chridam, Sep 27 '18 at 10:58
@chridam - This is clearly a duplicate of the question I identified. Why did you reopen it? — T.J. Crowder, Sep 27 '18 at 11:28
@T.J.Crowder Opened because there are alternative solutions to the question — chridam, Sep 27 '18 at 11:29
@T.J.Crowder If you mark some answer as duplicate then you should leave some comment how it can be. Just as [this](https://stackoverflow.com/users/2313887/neil-lunn) user used to do. — Dark Knight, Sep 27 '18 at 11:45
@DarkKnight - I do, when I think there's any real chance clarification is required. (Sometimes quite extensive comments.) I didn't (and don't) here. I do try to be a fairly useful person. :-) — T.J. Crowder, Sep 27 '18 at 12:00
@JaromandaX Last question. Async await introduced to replace `Promises` So why we cannot use `await` here something like this `await array.map(async(d) => {})` ??? — Dark Knight, Sep 27 '18 at 16:15
no async await is syntax sugar over Promises - Promises are an integral part of async await. Array.map doesn't return a promise, in this case it returns an array of promises (because of async(d)) but await doesn't look into the returned value to see if there's any promises inside — Jaromanda X, Sep 27 '18 at 22:40

chridam · Accepted Answer · 2018-09-27T11:23:45.800

There is a better way to get the desired result with MongoDB and no need to loop, use the aggregation framework where you can run the following pipeline which uses $facet as

try {
    const array = ["brunch", "lunch", "crunch"]
    const facet = array.reduce((acc, cur) => {
        acc[cur] = [{ "$match": { "category": cur } }]
        return acc
    }, {})
    const pipeline = [
        { "$match": { "category": { "$in": array } } },
        { "$facet": facet }
    ]
    const results = await Venue.aggregate(pipeline).exec()
    const data = results[0]

    return data
} catch(err) {
    throw err
}

You can also group the documents by the category key and $push the documents per group and then convert into keys of a document in a $replaceRoot with $arrayToObject

try {
    const array = ["brunch", "lunch", "crunch"]
    const pipeline = [
        { "$match": { "category": { "$in": array } } },
        { "$group": { 
            "_id": "$category",
            "data": { "$push": "$$ROOT" }
        } },
        { "$group": {
            "_id": null,
            "venues": {
                "$push": {
                    "k": "$_id",
                    "v": "$data"
                }
            } 
        } },
        { "$replaceRoot": {
            "newRoot": { "$arrayToObject": "$venues" }
        } }
    ]
    const results = await Venue.aggregate(pipeline).exec()
    const data = results[0]

    return data
} catch(err) {
    throw err
}

Thank you chridam for the answer. But I know this trick. https://stackoverflow.com/questions/50735135/dynamic-keys-after-group-by. ***It is not possible to $push "millions of documents" into an array in a $group stage*** this comment scares me. — Dark Knight, Sep 27 '18 at 10:51
Either approach works, that's the limitation with the aggregation framework; breaking the 16MB limit when pushing documents to an array. The only performance takeaway from this is you are just making one call to the server but since you are iterating a small array, the performance costs are negligible when using the for loop and would advocate using the approach in the other answer by @RaghavGarg — chridam, Sep 27 '18 at 10:58
I didn't see your first answer with `$facet`. That's perfect!!! thank you once again — Dark Knight, Sep 27 '18 at 13:40

Raghav Garg · Answer 2 · 2018-09-27T12:54:54.677

2

Although @chridam approach is quite unique and maybe more efficient, in case you want to stick with a loop.

There are two approaches. You want all your operation to be run in parallel or series.

If parallel, you will have to use Promise.all.

try {
  const array = ["brunch", "lunch", "crunch"]
  const data = {}
  await Promise.all(array.map(async(d) => {
    data[d] = await Venue.find({ "category": { "$in": [d] }})
  }))
  return data
} catch(err) {
  throw err
}

If series, you will have to use simple for loop.

array.map(async(d) => {}) is making the internal db call asynchronous and not waiting for operation. normal for loop will be synchronous.

try {
  const array = ["brunch", "lunch", "crunch"]
  const data = {}
  for (d of array) {
    data[d] = await Venue.find({ "category": { "$in": [d] }})
  }
  return data
} catch(err) {
  throw err
}

edited Sep 27 '18 at 12:54

answered Sep 27 '18 at 10:46

Raghav Garg

3,059
2
18
29

your parallel code returns an ARRAY rather than an OBJECT - so, not at all an answer – Jaromanda X Sep 27 '18 at 12:23
or you could do it neater with just map, and no reduce in one loop - https://jsfiddle.net/kqsa3upb/ :p – Jaromanda X Sep 27 '18 at 12:47
@JaromandaX, thanks again, buddy. Never thought of doing it like that. – Raghav Garg Sep 27 '18 at 12:57

score 0 · Answer 3 · answered Sep 27 '18 at 11:47

Problem is the way you are using async-await. It's inside a Array.prototype.map which made .map function asynchronous and thus main thread never waits for loop to complete and move forward to next statement and returned data which is just {}.

try {
  const array = ["brunch", "lunch", "crunch"]
  const data = {};

  // here's the issue async is callback function to map, due to which never waited for map function to finished.
  array.map( async(d) => {
    const venue = await Venue.find({ "category": { "$in": [d] }})
    data[d] = venue;
  });

  return data;
} catch(err) {
  throw err
}

Change code to this:

(async function () {
    try {
      const array = ["brunch", "lunch", "crunch"]
      const data = {};

      array.map( d => {
        const venue = await Venue.find({ "category": { "$in": [d] }})
        data[d] = venue;
      });

      return data;

    }
    catch(err) {
      throw err
    }
})();

What you did is something similar to this

function main () {
    return new Promise ( resolve => {
        setTimeout( () => {
            console.log("First");
            resolve(true);
        },5000 );
    });
}

(async function () {
    await main();

    console.log("Second")
})();

console.log("Third");
// Third
// First
// Second

this code is completely wrong ... using await inside a function that **isn't** async will fail to parse, let alone run!!! and before you think changing `array.map( d => {` to `array.map( await d => {` will somehow fix the issue, then consider that would make your code identical to the code in the question — Jaromanda X, Sep 27 '18 at 12:20
No im asking to change other way `array.map( await d => {` to `array.map( d => {` — NAVIN, Sep 27 '18 at 17:28

Why even after using async await for multiple calls still got empty response?

3 Answers3

Linked