Async.js design pattern

Question

I am using async.js for my node.js app. I need help solving the following problem.

Let's say I have a following aysnc js series function.

async.waterfall([
    function getDataFromDB(request, response, callback){ ... },
    function someOperationOnDBData(dbData, response, callback){ ... },
    function renderSomeFlow(evaluatedData, response, callback){ ... }
]);

I have three functions called in the order mentioned above. I am getting data from getDataFromDB and passing to someOperationOnDBData and so on.

Suppose I need one more operation in between getDataFromDB and someOperationOnDBData but still pass DBData forward. Eg:

async.waterfall([
    function getDataFromDB(request, response, callback){ ... },
    function extraOperation(dbData, response, callback) {...}
    function someOperationOnDBData(dbData, extraOperationData, response, callback){ ... },
    function renderSomeFlow(evaluatedData, response, callback){ ... }
]);

Here adding a single step in the middle changes the function definitions and also I need to pass dbData in extraOperation to just forward it to someOperationOnDBData. Also, if I am calling a different module in the middle, it might not be possible to change it's parameter to forward some data.

How to solve this problem of passing data between functions in async.js without forwarding data in middle functions? Refactoring functions every time a new step is included is not possible. What is the design pattern for solving this type of problem?

@Hitmands could you please elaborate more? how will Promises solve this problem? — dejavu, Jun 22 '16 at 10:12
@dejavu Promises would require refactoring of your methods as you can't waterfall with native Promises. What you can do however is send a single object through the chain which you manipulate and add to. Then you don't change function signatures in every method in that chain. So long as you don't have conflicting property names that is. Plus it makes complex flow far easier than callbacks and flattens out the chain. No more callback hell. I would choose promises over callbacks every time. — ste2425, Jun 22 '16 at 10:43
@ste2425 is passing a single object and manipulating/adding new value to it regarded as good practice? I don't have much experience in node but I am not entirely convinced by it. Could you point to some blog post etc to read more about it? — dejavu, Jun 22 '16 at 12:56
About NodeJS or ES6 Promises? Google and SO will throw up tons of good info as to why they both are brilliant. Obviously if your from a .NET background NodeJS can be unpleasant as most JavaScript is for someone like that. — ste2425, Jun 22 '16 at 13:09
@ste2425 ah. no not nodejs or promises. about the design pattern of passing a single object and adding new value to it during the flow. — dejavu, Jun 22 '16 at 13:10

MrWillihog · Answer 1 · 2016-06-22T10:07:58.233

Using waterfall you have to pass the data through - you can't stop a waterfall in the middle :). There are alternatives that you might like to use, for example, auto which lets you specify which functions depend on the results of other functions and async will determine the best order to run them.

I find the syntax of auto a little awkward but it does what you need. Here is an example:

async.auto({
    db: function getDataFromDB(callback){ ... },
    extra: ['db', function extraOperation(results, callback) {...}],
    some: ['db', function someOperationOnDBData(results, callback){ ... }],
    render: ['some', 'db', function renderSomeFlow(results, callback){ ... }]
});

score 0 · Answer 2 · answered Nov 23 '17 at 14:16

I've found that currying your functions with arrow functions is a way arround this problem:

const someOperationOnDBData = extraOperationData => (dbData, response, callback) => {
  // do your function work
};

async.waterfall([
  function getDataFromDB(request, response, callback){ ... },
  function extraOperation(dbData, response, callback) {...}
  someOperationOnDBData(extraOperationData)(dbData, response, callback),
  function renderSomeFlow(evaluatedData, response, callback){ ... }
]);

Many good examples on how to curry javascript functions on Medium.com

Async.js design pattern

2 Answers2