5

I am using Mango query in couch db for almost every query to find the documents. Here I am facing a problem in fetching all the documents matching the given conditions. The problem is that the default limit of mango query is 25 (means fetch 25 documents per query) and there are lot many documents in my database and I don’t have the exact count of documents. I cannot hard code the limit in mango query as I don’t know the upper limit of documents and I don’t think hard coding the limit is a good idea. Can anyone please help me with this issue? How can I make the limit as unlimited or is there any other way to handle this situation?

Vandana Jain
  • 51
  • 5
  • 1
    With CouchDB 2.1, you can change the default limit. Therefore, I don't think that you can avoid this limit. Anyway, you should have a limit because the more you have results, the longer it takes to be processed and send over http. You should always design your code to handle paging of requests. – Alexis Côté Aug 11 '17 at 10:30
  • @AlexisCôté Thanks Alexis!! :) I'll go with the pagination approach as it seems to be a better and the only option to go with at this point. – Vandana Jain Aug 12 '17 at 03:41
  • My bad, the pagination is not supported yet (it looks like it will be part of the next release), I guess you can temporarly put a high number for the limit. – Alexis Côté Aug 12 '17 at 14:45
  • Yeah, Pagination is not supported yet. Putting a high number for the limit won't work in my situation. I'll have to add custom pagination using Node JS (backend code). Any idea how to achieve that? We are using Node JS for backend and Angular JS for Frontend. I am thinking of calling Node js API from frontend for each Page request. Is it a good approach? Or is there any better idea to do this? – Vandana Jain Aug 13 '17 at 14:14
  • 1
    Depending on how your database is structured, you could simulate pagination by adding another selector to your mango query to find documents were the ID is greater than the previous query. – Jeff Barnes Aug 17 '17 at 19:20

1 Answers1

1

I countered the same issue and resolved it with a recursive function

const batchSize = 25;
let batchCount = 0;
let searchResults = [];

//  in my case I want to find docs with a certain date, hardcoded here as example
let selectedDate = '2017/10/30'; 

// the recursive function
let search = function (count, limit){
    return db.find({
        selector: {
            date: selectedDate  
        },
        limit: batchSize,
        skip: batchCount*batchSize 
    }).then(function(batch){
        if (batch.docs.length === 0){
            // there are no (more) search results, so we can return what we have

            return searchResults

        } else {
            // there may be more search results, so we add this batch to the result and call the function again to ask more. We increase the counter so that the first batch(es) are skipped

            for (var i = 0; i < batch.docs.length; i++) {
                searchResults.push(batch.docs[i])
            }
            batchCount ++
            return search(batchCount, batchSize) 
        }
    })
}

search(batchCount, batchSize).then(function(result){
    // you can handle the searchresults

})
Erik
  • 159
  • 1
  • 6