2

I found myself hitting the issue of iterating through an array, triggering async ops for each item that call my callbacks with arrays that I'm iterating again... Of course before going to the next item, you need to wait for the previous to finish. And if an error occurs while processing an item, you have to stop and exit all enclosing loops.

The code I write to handle these situations, is easy to write but not easy to read/maintain. For completeness of this question I'm currently using a method I wrote that looks like this:

iterate(items, item_callback, ended_callback);

where the item_callback looks like this:

function(item, next_item, stop) {

    //do your stuff with `item` here...

    //return next_item() to go forward
    //return stop() to break
}

I'm wondering that maybe I'm missing a better technique of iterating in node.js.

Example time:

User.find(function(err, users) {
    //iterate through users: how?
    //first attempt: for
    for(var i in users) {
        //simple stuff works here... but if I call something async... what do I do?
        //like iterating through friends
        User.find({friend:users[i]._id}, function(err, friends) {
            if(err) {
                 //handle error: how is best to do this?
            }
            //this might end after the next user is selected... and might break things
        });
    }
});

TL;DR: How to iterate in node.js so that it works with both async code and the classic sync code?

P.S.: I'm sure some users will answer something along the use the async module. I know about it. I don't like how my code looks like when using it. It's even less maintainable than with my current solution. I need something better.

sqreept
  • 4,646
  • 3
  • 18
  • 26
  • This is a very good discussion on flow control with series, parallel and limited parallel patterns: http://book.mixu.net/ch7.html – jabbermonkey Jan 11 '13 at 21:36
  • BTW, probably duplicate of http://stackoverflow.com/questions/8413857/whats-the-smartest-cleanest-way-to-iterate-async-over-arrays-or-objs – broofa Jan 11 '13 at 21:37
  • Did find that one. Certainly duplicate. – sqreept Jan 11 '13 at 21:45

2 Answers2

1

There are a ton of node.js flow control libraries for dealing with async operations. They all have slightly different approaches and syntactic sugar, but async is probably the most popular. I prefer Step for it's power-to-weight ratio. :)

broofa
  • 35,170
  • 11
  • 65
  • 70
  • Thanks for the link. `each` seems to be closer to what I'd consider maintainable code. But that is subjective. But your answer prompted another smaller question. The node.js community didn't find its ITERATOR yet? – sqreept Jan 11 '13 at 21:24
  • Yeah. This is what I'm talking about. Now have to wait for coroutines to become part of node.js. You can see why we need to REALLY WAIT here: http://news.ycombinator.com/item?id=1549168 – sqreept Jan 11 '13 at 21:50
0

Your example if I get correctly is mongoose usage. See following solution that's based on promises: Node.js deferred promisify + mongoose

Community
  • 1
  • 1
Mariusz Nowak
  • 28,556
  • 4
  • 32
  • 37