3

I've been searching all over on this one. I'm running CouchDB 2.0 and understand I have a choice to make between using traditional views or the newer Mango query when retrieving a set of data.

So I'm currently using the Mango query syntax and getting the results I need - however I now need to implement pagination. When researching pagination in CouchDB 2.0 I found this excellent discussion around the topic:

http://docs.couchdb.org/en/2.0.0/couchapp/views/pagination.html

It suggests that the best way to paginate large data sets is not to use skip but instead to use startkey and perform a kind of linked list pagination from one page to the next.

So this makes sense to me and works for my application, but when I then turn to the Mango/_find API I can't see any way to pass in startkey:

http://docs.couchdb.org/en/2.0.0/api/database/find.html

Confusingly enough, it does accept a skip parameter, but there is no startkey.

Is anybody able to explain what is happening here? Are the performance characteristics much different in Mango/_find such that we can safely use skip on large data sets? Or should we be using views with startkey when traversing larger collections of data?

This particular question doesn't seem to get answered in any recent documentation AFAIK. Any help would be greatly appreciated.

Dan Ballance
  • 71
  • 5

2 Answers2

0

You could perhaps work around the lack of startkey/endkey support by including a constraint in the selector:

"selector": {
    "_id": { "$gte": "myStartKey", "$lte": "myEndKey"}
}

(just my two cents; maybe somebody else has a more complete answer)

Gerben
  • 193
  • 1
  • 4
0

The CouchDB documented approach of pagination applies only to map-reduce/views kind of queries and cannot be applied to Mango queries. Primarily becos, for views, there is a single key field which is used for sorting, hence it's easy to skip previous doc by using that 'startkey' and in case of non-unique key by adding a startkey_docid.

For selector queries, to effectively skip previous records you will have to look at the sort keys specified in the original query and add another condition(s) to skip those docs which are already processed. For example, if you sorted (in asc) on a numeric field and processed till value = 10, then you could add a { "field" : { "$gte" : 10 } } as $and logic within original selector. This becomes complicated if you have multiple sort fields. A skip/limit might be an easier approach to pagination for selector queries.

Giskard
  • 1