0

I have a bunch of data that's clearly relational: object A has a many:1 relationship with B, and B has a 1:many relationship with C, and so on. I can write a bunch of code to maintain this without insert/update/delete anomalies, but it feels like reinventing the wheel: relational DBs do a great job at this.

SQLite would actually be a great fit here, if not for one thing...

The data, once updated, has to be stored in an SCM system, and changes to the data need to be auditable by humans, which (in my estimation) requires cleartext storage.

Is there an engine out there that will give me relational semantics but store the data in cleartext?

Recapping, what I think my requirements are:

  • Stores data in cleartext
  • Maintains/enforces referential integrity
  • (nice to have) implemented as a library rather than a standalone executable, a la SQLite
  • (nice to have) bindings for the Go language

And here are some things that are not requirements:

  • SQL; any interface is fine
  • High performance; I don't care if access is somewhat slow, it'll be dealing with megabytes of data / hundreds of thousands of records at most.
SAyotte
  • 278
  • 2
  • 9
  • Since when does MySQL store data in something other than clear text without you explicitly telling it to? – evanmcdonnal Sep 28 '16 at 18:19
  • Do you also store the compiled Go code in the SCM? Why can't you use the SQL source? – CL. Sep 28 '16 at 18:26
  • @evanmcdonnal, the MySQL on-disk format is built around b-trees as far as I know. To every SCM system I've seen that will look like a binary blob-- it won't calculate a line-by-line diff, it'll just say "these blobs are different". The fact that there are cleartext strings within that blob doesn't help-- they are separated by non-printable characters. – SAyotte Sep 28 '16 at 20:29
  • @CL we do not store the compiled Go code in the SCM, no. I'm not sure I understand your question "Why can't you use the SQL source". – SAyotte Sep 28 '16 at 20:29
  • I see... What about building a service between MySQL and SCM that just reads data from MySQL and makes it available to SCM in a format that it works with? I mean, does SCM try to read the raw data files off disk? Why doesn't it query the database like a normal application? – evanmcdonnal Sep 28 '16 at 20:39
  • Dump the database as SQL text; store that in the SCM. – CL. Sep 28 '16 at 20:56
  • @evanmcdonnal SCM stands for "source code management", e.g. Git, SVN, Perforce, so on. They are themselves repositories for versioned text; they don't query, we push and pull data into/out of them. – SAyotte Sep 29 '16 at 00:44
  • @CL that's actually a pretty good suggestion. Submit it as an answer and I'll upvote it, and accept it if nothing better comes up. – SAyotte Sep 29 '16 at 00:44
  • Ah ok, yeah didn't get that acronym. I googled it and found some very different results. Yea, CL's suggestion is about right and similar to what I was alluding to with a service reading the data then writing it in a format that the SCM can work with. – evanmcdonnal Sep 29 '16 at 00:56
  • Also see [What's a good way (or tool) to version control a SQLite database (schema only)?](http://stackoverflow.com/q/33809881/11654) – CL. Sep 29 '16 at 07:12

0 Answers0