Beyond the Sqlite Single-Writer Limitation with Concurrent Writes

why not? Turso is a fully local database, you can just download the shell and use it as you would use sqlite.

> Can't use it locally

I'm imagining some insane replication behind the scenes, where every write is happening concurrently on a different SQLite DB, and then merged together sequentially into some master DB.

I get where you're coming from, but isnt the whole idea of open source "if you dont like the approach, you're free to fork the code and do it the way you think is right?"

As long as the fork doesnt violate trademark (turso vs sqlite) it is working-as-intended?

I, for one, encourage this kind of behavior. We should have more forks. More forks = more competition = better results for everyone.

---

To make an analogy. Would you say the same thing if this were a for-profit company?

"I cant believe someone else is competing the same space as $x. $x is hugely successful, and so many people use it. I dont know why there's an alternative"

> Forking his project while using the name

The don't use the name. They use Turso. Even the HN title is wrong - the article title doesn't mention SQLite.

They refer to SQLite, but how could you not if that's what you forked from, and that's what has the functionality you're changing. That would be a very weird article if we didn't have that context.

I'd still give them the benefit of the doubt as most (all?) of the contributors are Finns who you can almost guarantee to have a "no bullshit" type of mentality to essentially everything. And what I'd guess, the team most probably has quite an academic background because of the local culture.

Author of Turso here. Couple of points

* You are right not to rush. You should keep using SQLite until Turso matures. Some use cases are more tolerant to new tech than others. It will take time for us to reach the level of trust SQLite has for broad use cases, but we are hoping to add value for some use cases right away. Never rush, tech matters!

* I have never met Hipp, but only heard great things about him.

* We never had a fight with SQLite over their contribution model (or about anything for that matter, I never even met Hipp or anybody else from SQLite). We just disagree with it - in the sense that we believe in different things. We don't think what they do is fundamentally wrong. Different projects take different paths.

* We are not using the SQLite name. We compare ourselves to SQLite because we are file and API compatible, and we do aspire to raise the very high bar they have set. It is hard to do this without drawing the comparison, but we are a different project and state it very clearly. I am not a lawyer (and neither you seem to be), but we believe we are doing is okay. If we ever have any valid reason to believe we crossed a line here, we will of course change course.

* We are not "startup bros". We spent 20+ years of our lives building databases and operating systems.

One ugly thing I remember was the insinuation that SQLite was somehow immoral. Glauber Costa attacked SQLite[1] with lines like:

> We take our code of conduct seriously, and unlike SQLite, we do not substitute it with an unclear alternative. We strive to foster a community that values diversity, equity, and inclusion. We encourage others to speak up if they feel uncomfortable.

Sometime after the HN backlash[2], that paragraph was removed. Eventually, Turso released https://github.com/tursodatabase/turso without a code of conduct and hasn't had one since.

For me, the saddest part of the affair is that Richard Hipp and Glauber are both committed Christians. There should be no reason for envy, enmity, or memory games. But when business and money was on the table, it seems only one of them followed Jesus.

[1] https://github.com/tursodatabase/libsql/commit/3ac3ad263c0f0... https://web.archive.org/web/20221004144141/https://glauberco...

[2] https://news.ycombinator.com/item?id=33081159

SQLite against S3 can work with some clever tricks. The neatest version of that I've seen is still this WebAssembly one: https://phiresky.github.io/blog/2021/hosting-sqlite-database...

I also got sqlite-s3vfs working from Python a few months ago: https://simonwillison.net/2025/Feb/7/sqlite-s3vfs/

Both of these are very much read-only mechanisms though.

SQLite directly against S3 is workable if you mean querying a read-only database.

For example, from Go, you could use my driver, and point it to a database file stored in S3 using this: https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/readerv...

For read-write it's a terrible idea. Object storage assumes objects are immutable. There may be some support for appends, but modifying the middle of an object in place involves copying the entire thing.

What is on the verge of becoming viable is to use Litestream to do asynchronous replication to S3, and have read replicas that stream the data directly from S3. But what's stored in S3 isn't a database file, but a format created for the purpose called LTX.

> As a secondary, I wonder if it's possible to actively use a SQLite interface against a database file on S3, assuming a single server/instance is the actual active connection.

You could achieve this today using one of the many adapters that turn S3 into a file system, without needing to wait for any SQLite buy in.

Joins and Transactions are a pretty big part of SQL. I'm no expert, but if my quick search results are right, both are lost in the separate file per table scenario.

> You can shard your SQLite tables into multiple database files and query across all of them from a single connection.

You mean using ATTACH statement, right? If you use WAL mode, then you cannot get transaction safety / ACID with ATTACH [0]

> If the main database is ":memory:" or if the journal_mode is WAL, then transactions continue to be atomic within each individual database file. But if the host computer crashes in the middle of a COMMIT where two or more database files are updated, some of those files might get the changes where others might not.

Moreover, ATTACH do not support more than 125 databases, so that limits the shards to 125. [1]

ATTACH does not solve the concurrency problems. That's why SQLite also has a BEGIN CONCURRENT experimental branch

[0] - https://www.sqlite.org/lang_attach.html

[1] - https://www.sqlite.org/limits.html

TursoDB aims to be fully compatible with sqlite, so files you create with tursodb can be read by sqlite and vice-versa. Like sqlite, turso is in-process, so it runs alongside your application, which is quite different from postgres' client-server architecture.

The only think turso has in common with postgres is mvcc, which is a rather standard concurrency control model across modern databases. Idk if I answered your question :)

Looks like that branch is still under actively development - seven new commits this month: https://sqlite.org/hctree/timeline

Agreed, Sqlite docs specifically state that its not designed for this, or rather that the solution isn't appropriate, that database as a service is appropriate when you want multiple clients writing.

The entire "problem" is a side-effect of using the wrong tool for the job.

I'll give you an example of what I'm doing with SQLite.

I have an app that organizes my movie collection. I use IMDB public datasets as data source. Because versioning is hard, data changes over time (remember who directed Matrix?) it's just easier to rebuild the database every time.

A naive import of datasets is ~131M inserts. It takes 10 minutes (of which 100M takes exactly 6 minutes). Another 10 minutes to further clean up the data, set up indexes, optimize and vacuum.

It's fine for the use case, but still it would be nicer to be able to achieve that in half the time.

The app has no dependencies, database is a single file I can store on S3 and with a little bit of magic use it as a real database - meaning I can run a complex queries over http on that whole database and exchange 50kb.

The client is a standalone HTML file with some JS. No need of server for client, no need for hosting for server except of that S3 bucket.

Sure, it's definitely a niche application, but, at least for me, it makes perfect sense. Start your app with zero deps and add only what you really need. There's too much projects that start with google-esque architecture, with plenty of microservices, only to reach MVP with technical debt the size of US treasury.

sqlite is embedded. I understand that there might be scenarios in which multi-threaded sqlite is beneficial when an application has many concurrent writers. But taking a look at the company's website makes me wonder what the project's motivation is. The company offers a database service, which is a completely different scenario from embedded dbs. If the intention is to offer a cloud service, evolving from "sqlite" seems odd. The only benefit I can think of is that the new db service helps existing sqlite users migrate. My issue is that if I choose sqlite to store data locally in my browser or my cell phone, why do I suddenly want to store it in the cloud?

Why not take something that’s great and make it support more use cases? People are doing it the other way around too: https://pglite.dev/

> SQLite is built around a file that stores data for a single application

A single application can need multiple concurrent writes

It's very easy to do given the async model. It is one of the new features that is working the best so far.