5

I store time series data in bigtable with a rowKey of userId#timestamp. Given query parameters of (userId, startTime, endTime) how can I support pagination i.e return 'limit' records starting from 'offset' ?

note that userId#startTime rowKey may not exist in bigtable but there will some datapoints before and after startTime/EndTime. Bigtable Go client seems to support ReadRows with a prefixRange argument. I could use a prefixRange of userId and 'seek' to the startTime as I iterate using ReadRows but this seems very inefficient if starTime/endTime is way in the past. is there a better way ??

Maxim
  • 3,172
  • 9
  • 21
kiran Pendyala
  • 53
  • 5
  • This post may have what you're looking for, take a look and let me know it works for you: https://stackoverflow.com/questions/49691823/bigtable-node-client-how-to-set-read-offset-for-pagination – Maxim Dec 18 '18 at 11:34
  • thank you maxim but bigtable Go client doesn't have the readStream interface like the Node client. – kiran Pendyala Dec 19 '18 at 21:16

1 Answers1

5

You can start a ReadRows operation from userId#startTime to userId#endTime with a NewRange and set a limit on the number of rows returned with a LimitRows read option.

err = tbl.ReadRows(ctx, NewRange("<userId>#<startTime>", "<userId>#<endTime>"), func(r Row) bool {
    fmt.Println("Got a row")
    return true
}, LimitRows(100))
Dan
  • 165
  • 7
Gary Elliott
  • 694
  • 3
  • 5