3

OpenTSDB is super fast. KairosDB is known as re-write of OpenTSDB and as claimed that it's even faster than OpenTSDB (see here). However I did some tests with a pseudo-distributed cluster (1 master, 1 slave, locally) for OpenTSDB and 1-node cluster for KairosDB on my VirtualBox (5 GB RAM, 3 cores). The insertion speed was around 100,000 records / sec for OpenTSDB and 30,000 records / sec for KairosDB. Did I configure something wrong with KairosDB or OpenTSDB is actually faster?

duong_dajgja
  • 3,878
  • 1
  • 28
  • 53
  • You have to be somewhat careful how you measure throughput. Ideally you want to make to sure you force data on disk before you gauge inserts/second. Unless you're measuring how fast you're able to pass data from the client to the TCP receive buffer of the ingestion controller or a file buffer in front of the database. – Sergei Rodionov Jul 01 '15 at 12:06
  • Just FYI, there is new generation of time series databases, which can accept more than 10 millions of data points per second on a single computer. See [this benchmark results](https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae). – valyala Oct 12 '19 at 21:23

1 Answers1

1

I don't have measurements on OpenTSDB. We use KairosDB and it's quite fast.

What database did you use for KairosDB? H2 is for test only and is desperately slow. What interface did you use for pushiong data? And if you use the REST API how did you build your queries?

On a single bare metal node we were above 50,000 samples per second using Telnet (limited by the acquisition agent), and about 3 times this speed for gzipped JSON using the REST API (this is for batch inserts of historical data, and the JSON is build to insert data with one array of datapoints per series, and with up to 10,000,000 samples per document).

Maybe VirtualBox slows everything down too much (guest VMs have very poor performances).

On another hand the last time I spoke with OpenTSDB they were a lot under the 100,000 points per seconds in insertion... So they may have improved performances.

Brian Tompsett - 汤莱恩
  • 5,195
  • 62
  • 50
  • 120
Loic
  • 1,010
  • 6
  • 19