Time series database for java?

Question

I want to store millions of time series entries (long time, double value) with Java. (Our monitoring system is currently storing every entry in a large mySQL table but performance is very bad.)

Are there time series databases implemented in java out there?

Performance is bad in what sense? INSERTing? SELECTing? Unless you provide some use cases (i.e. problematic queries) nobody will ever be able to give a meaningful answer... — CAFxX, Jan 11 '11 at 08:43
Can you please what do you really mean by "Performance is really bad". Millions of inserts should not be a problem for mysql. What are you really trying to do with the data is that table? — Fawad Nazir, Nov 10 '11 at 22:10

score 6 · Accepted Answer · answered Oct 07 '11 at 12:30

6

checkout http://opentsdb.net/ as used by StumbleUpon?
checkout http://square.github.com/cube/ as used by square

I hope to see additional suggestions in this thread.

answered Oct 07 '11 at 12:30

ericslaw

797
1
7
16

score 3 · Answer 2 · answered Nov 16 '11 at 12:55

The performance was bad because of wrong database design. I am using mysql and the table had this layout:

+-------------+--------------------------------------+------+-----+-------------------+-----------------------------+
| Field       | Type                                 | Null | Key | Default           | Extra                       |
+-------------+--------------------------------------+------+-----+-------------------+-----------------------------+
| fk_category | smallint(6)                          | NO   | PRI | NULL              |                             |
| method      | enum('min','max','avg','sum','none') | NO   | PRI | none              |                             |
| time        | timestamp                            | NO   | PRI | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| value       | float                                | NO   |     | NULL              |                             |
| accuracy    | tinyint(1)                           | NO   |     | 0                 |                             |
+-------------+--------------------------------------+------+-----+-------------------+-----------------------------+

My fault was an inapproriate index. After adding a multi column primary key all my queries are lightning fast:

+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| job   |          0 | PRIMARY  |            1 | fk_category | A         |          18 |     NULL | NULL   |      | BTREE      |         |               |
| job   |          0 | PRIMARY  |            2 | method      | A         |          18 |     NULL | NULL   |      | BTREE      |         |               |
| job   |          0 | PRIMARY  |            3 | time        | A         |   452509710 |     NULL | NULL   |      | BTREE      |         |               |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

Thanks for all you answers!

score 1 · Answer 3 · answered Jan 11 '11 at 08:50

1

You can take a look at KDB. It's primarily used by financial companies to fetch market time series data.

answered Jan 11 '11 at 08:50

dogbane

242,394
72
372
395

score 0 · Answer 4 · answered Jan 11 '11 at 08:35

0

What do you need to do with the data and when?

If you are just saving the values for later, a plain text file might do nicely, and then later upload to a database.

answered Jan 11 '11 at 08:35

Thorbjørn Ravn Andersen

68,906
28
171
323

Time series database for java?

4 Answers4