How to wait for the internal process of a function to execute completely before returning?

Question

I'm trying to fetch webpages using axios and cheerio but i'm failing to retrieve all the data as a function is returning even before the intennal process is completely done. I'm using ReactJS and and when i click a button in the page, this function gets called :

analyse = async(e) => {

      this.a = await axios.get('xxx');
      this.s = await axios.get('yyy');
      this.f = await axios.get('zzz');

      let aReturn = this.fetchA(this.a.data);

      let sReturn = this.fetchS(this.s.data);

      let fReturn = this.fetchF(this.f.data);

      if(await fReturn === true && await aReturn === true && await sReturn === true){
            anotherFunction();
      }
}

In the above code, the axios works fine and the functions fetchA() and fetchS() are simple functions without any further internal function calls :

fetchA(html){
    const $ = cheerio.load(html);

    //some process

    return true;
}

fetchS(html){
    const $ = cheerio.load(html);

    //some process

    return true;
}

The function that is returning without complete execution is fetchF() :

fetchF(html){
    const $ = cheerio.load(html);

    $('div._someClassName').children().each((i,elem) => {

    let link = $(elem).find('a').first().attr('href');

    axios.get(link)
    .then(response => {

        this.deepF(response.data);
    })
    .catch(err => {
        console.log("Error : ",err);
     })

    };
    return true;
}

In the above function there is a unique link in every child that i need to axios.get() again and call another function deepF() to grab the data of that page. The internal function deepF is again a small function :

deepF(html){
    const $ = cheerio.load(html);

    //some process

    return true;
}

In this whole code the fetchF() is returning true without completely executing all the deepF() functions internally.

Note : All the deepF() functions are executed slowly later after the fetchF() returns true

How can i wait for fetchF() to execute all the internal deepF() before returning true in fetchF()?

jfriend00 · Accepted Answer · 2019-12-14T09:59:25.243

Change the implementation of fetchF() to properly return a promise that is linked to all the ajax calls. Accumulate the ajax promises into an array and then use Promise.all() to know when they are all done. Then, you can await the result of that function.

Change from this:

fetchF(html){
    const $ = cheerio.load(html);

    $('div._someClassName').children().each((i,elem) => {

    let link = $(elem).find('a').first().attr('href');

    axios.get(link)
    .then(response => {

        this.deepF(response.data);
    })
    .catch(err => {
        console.log("Error : ",err);
     })

    };
    return true;
}

to this:

fetchF(html){
    const $ = cheerio.load(html);
    const promises = [];
    $('div._someClassName').children().each((i,elem) => {

        let link = $(elem).find('a').first().attr('href');

        promises.push(axios.get(link).then(response => {
            return this.deepF(response.data);
        }).catch(err => {
            console.log("Error : ",err);
            throw err;
        }));
    });
    return Promise.all(promises).then(() => true).catch(() => false);
}

This makes fetchF() into a function that returns a promise that resolves only when all the axios.get() calls inside it are done and resolves with a true value. If there are any errors, it catches those errors and resolves with a false value (which seems to be the logic you wanted, though it's a bit unusual to not just let the rejected promise propagate and catch it at a higher level to signify errors).

Then, you can use:

let fReturn = await this.fetchF(this.f.data);

And, then fReturn will be true or false.

If any of your other functions have asynchronous operations in them, they will also need to return a promise that is directly linked to your asynchronous operations.

How to wait for the internal process of a function to execute completely before returning?

1 Answers1