Javascript - Infinite scroll loop

Question

Short story :

I want to parse a website that has infinite scroll on it, I don't want to implement infinite scroll. So I want to create a script that automatically scroll the page from the browser console wait for the data to appears then repeat until the end.

---

Long story :

I'm trying to scroll down an infinite scroll using javascript console into my browser. When we scroll to the bottom, an Ajax call is made and fill a <div> element so we have to wait for this to occurs then resume our process.

My main problem here is that all of this is done too quickly, it doesn't wait the ajax to be completed before resuming the process.

To have a concrete example, when I go to AngelList jobs page (need to be logged in) and when I put this in my browser console :

function sleep(µs) {
     var end = performance.now() + µs/1000;
     while (end > performance.now()) ; // do nothing
}

var loading = true;

$(window).load(function () {
   loading = false;
}).ajaxStart(function () {
   loading = true;
}).ajaxComplete(function () {
   loading = false;
});

function waitAfterScroll() {
    if(loading === true) {
        console.log("Wait the Ajax to finish loading");
        setTimeout(waitAfterScroll, 1000);
        return;
    }

    console.log("Ajax DONE");
}

var X= 0;
while(X < 100){
  sleep(1000000);
  X = X + 10;
  console.log(X);
  window.scrollTo(0,document.body.scrollHeight);

  waitAfterScroll();
}

I've got this result :

10
Wait the Ajax to finish loading
20
Wait the Ajax to finish loading
30
Wait the Ajax to finish loading
40
Wait the Ajax to finish loading
50
Wait the Ajax to finish loading
60
Wait the Ajax to finish loading
70
Wait the Ajax to finish loading
80
Wait the Ajax to finish loading
90
Wait the Ajax to finish loading
100
Wait the Ajax to finish loading
Wait the Ajax to finish loading
Ajax DONE

What I would like is :

10
Wait the Ajax to finish loading // one or multiple times.....
Ajax DONE
20
Wait the Ajax to finish loading // one or multiple times.....
Ajax DONE
30
Wait the Ajax to finish loading // one or multiple times.....
Ajax DONE
40
Wait the Ajax to finish loading // one or multiple times.....
Ajax DONE
50
Wait the Ajax to finish loading // one or multiple times.....
Ajax DONE
60
Wait the Ajax to finish loading // one or multiple times.....
Ajax DONE
70
Wait the Ajax to finish loading // one or multiple times.....
Ajax DONE
80
Wait the Ajax to finish loading // one or multiple times.....
Ajax DONE
90
Wait the Ajax to finish loading // one or multiple times.....
Ajax DONE
100
Wait the Ajax to finish loading // one or multiple times.....
Ajax DONE

I hope this is clear.

In other words, I would like to be able to scroll down, stop the execution of the javascript or at least wait for the ajax to finish loading, then repeat.

Answer 1

I am pretty sure that I misunderstood what you wanted, but anyway. Why not using Promises?:

var promise = new Promise(function(resolve, reject) {
  var req = new XMLHttpRequest();
  req.open('GET', url);

  req.onload = function() {
    if (req.status == 200) {
       resolve(req.response);
  }
};
});

promise.then(function(result) {
  window.scrollTo(0,document.body.scrollHeight);
})

Or, you could make a synchronous ajax request to stop javascript execution.

 jQuery.ajax({
    url: "someurl.html",
    async: false, 
    success: function(html){ 
      window.scrollTo(0,document.body.scrollHeight);
    }
  });

But, for performance reasons, I would not recommend that. Instead you could:

// Start off with a promise that always resolves
var sequence = Promise.resolve();
arrayOfWebsiteContentUrls.forEach(function(url) {
  sequence = sequence.then(function() {
    return fetchTheContent(url);
  }).then(function(content) {
    addContent(content);                 
    window.scrollTo(0,document.body.scrollHeight);
  });
});

I took that last bit from this excelent article on Promises.

Please, let me know if there is something that you don't get.