Tigerbeetle Is a Most Interesting Database

Yes, this is by design. SQL is a great general purpose query language for read-heavy variable-length string workloads, but TigerBeetle optimizes for write-heavy transaction processing workloads (essentially debit/credit with fixed-size integers) and specifically with power law contention, which kills SQL row locks.

I spoke about this specific design decision in depth at Systems Distributed this year:

1000x - https://m.youtube.com/watch?v=yKgfk8lTQuE

Transactions atomically process one or more Transfers, keeping Account balances correct. Accounts are also records, their core fields (debits_posted, credits_posted, etc).

This gives a good idea of what TigerBeetle might be good for and what it might not be. For anything where latency/throughput and accuracy really, really matters, it could be worth the effort to make your problem fit.

a) what Tigerbeetle data looks like in practice? Assuming it doesn't look like a regular table

b) how you use it, if you can't write sql queries?

c) separately, curious what double-entry would look like for stocks, tickets etc. E.g. I'm a venue and I have 1000 tickets in inventory & deferred revenue.. each time I sell a ticket I turn that inventory to cash and the deferred into a performance liability? Or no entries at all until a ticket is sold? Something else?

Within a single machine, yeah, relational dbs still work like a charm.

Without much sweat for general purpose workloads.

But transaction processing tends to have power law contention that kills SQL row locks (cf. Amdahl’s Law).

We put a contention calculator on our homepage to show the theoretical best case limits and they’re lower than one might think: https://tigerbeetle.com/#general-purpose-databases-have-an-o...

It still holds up basically whole of internet.

in most cases SQL is good enough for 90% of workloads.

Sometimes I feel like we software engineers have the worst memory of any engineers.

The relational model has shown itself to be exactly the flexible and powerful model that Codd said it was, even in its relatively-debased from in SQL.

In fact the potential of it as a universal and flexible data model that abstracts away storage still remains to be fully unlocked.

Now as for existing SQL databases, yes, many of them were built on foundational assumptions about the nature of memory and secondary storage that no longer hold true.

Many of us in this industry still have our heads stuck in the past world of spinny spinny magnetic disks on platters with heads and cylinders and grinding noises. Real world hardware has moved on. We have a couple orders of magnitude higher IOPS generally available than we did just 10-15 years ago.

So I'm excited about products like CedarDB and others (Umbra before it, etc) that are being built on a foundation from day one of managing with hybrid in-memory/disk.

Throwing out SQL is not really the recipe for "performance" and the lessons of separating the storage system from the data model is still key, since the 1960s.

I am willing to grant that a specialized transaction/ledger system like TigerBeetle might have its place as an optimization strategy in a specific industry/domain, but we should not make the mistake of characterizing this as a general problem with the relational model and data storage broadly.

The only reason is Apple. They liked the product that was released in 2013 so much they bought the whole company, and all other FoundationDB users were abandoned and were forced to drop it.

Who would trust a database that can be yanked out of you at any moment? Though a lot of products have license terms like this only a handful were ever discontinued so abruptly. It's under Apache license now but the trust is not coming back.

Interesting that they didn't release it with an SQL client, is there no way to make it compatible? Even with extensions to SQL, I imagine it would be great for a lot of use cases.

Edit: ah, it's more of a key-value store.

I started writing this comment:

> It seems interesting, but considering what it's for, why aren't the hyperscalers using it?

And while writing it I started searching for FoundationDB and found this:

> https://github.com/<<apple>>/foundationdb

Ah, all right :-p

* spawning 1000 workers all opening a connection to a db,

* solved by service/proxy in front of db,

* proxy knows how to reach db anyway, let's do private network and not care about auth

C'mon folks, the least you can do is put a guide for adding an auth proxy or auth layer on your site.

Particularly since you don't use HTTP (cant easily tell from the docs, I'm assuming), then folks are going to be left wondering: "well how the hell do I add an auth proxy without HTTP" and just put it on the open internet...

But nowadays an extra comparison is no biggie, especially if the compiler can hint at which way it's likely to go.

Rather, I would say that the most interesting database is the fastest and safest.

Or as Jim Gray always put it: “correct and fast”.

cf. Durability and the Art of Consensus: https://www.youtube.com/watch?v=tRgvaqpQPwE

https://en.wikipedia.org/wiki/Optical_proximity_correction

We typically hear from companies that TigerBeetle is the easier part of their stack. But to be fair, they may have a cleaner architecture.

Did you contact our solutions team for assistance with your business logic modelling? If not, please get in touch! solutions@tigerbeetle.com

edit: from the horses mouth is better https://news.ycombinator.com/item?id=45437046