Go and Sqlite Best Practices

The backwards compatibility also means that the frustration over concurrency and synchronization is largely a waste of time. Most SQLite builds are created such that all activity is serialized through a single mutex by default.

> In serialized mode, API calls to affect or use any SQLite database connection or any object derived from such a database connection can be made safely from multiple threads.

https://www.sqlite.org/threadsafe.html

Many libraries get this wrong and make it unsafe to use from multiple threads despite the underlying provider being capable. I think these are effectively bugs that should be resolved.

In my C# applications, I use System.Data.SQLite and share a single SQLiteConnection instance across the entire app. This connection instance typically gets injected as the first service, so I can just take a param on it any time I need to talk to SQL. Absolutely no synchronization occurs in my code. I've tried Microsoft.Data.Sqlite but it seems to have rare problems with sharing connections between threads.

Thanks for pointing this out. I don't think it's something this little helper library should try to do. This kind of thing belongs in whatever database abstraction you're building/using, if any. It's also a little trickier than it seems at first glance (RETURNING, etc).

And for many no/low concurrency situations there's no/little benefit.

Added this example to the README.

  rwdb, err := sqlitebp.OpenReadWriteCreate("app.db")
  if err != nil {
    log.Fatal(err)
  }
  rwdb.SetMaxOpenConns(1)
  rwdb.SetMaxIdleConns(1)


  // Read-only with default adaptive pool size (2-8 based on GOMAXPROCS)
  rodb, err := sqlitebp.OpenReadOnly("app.db")
  if err != nil {
    log.Fatal(err)
  }

To do this, it's best if the user opens two pools themselves.

I've tried to create something that does this transparently (give you a pool that magically routes writes to a single connection), but it's hard.

It starts simple: Exec is a write, Query is a read. Unless it contains the word RETURNING, then it can be an UPDATE…RETURNING. BeginTx can be read-only otherwise assume it's a write.

The problem is that attached DBs throw a wrench and make it even more confusing.

So only the user can really know if it's a write or a read. But then this makes it harder to use an ORM or even something like SQLc. So I don't know if the “driver” should do anything about it really.

Thanks for the comments. I'm a fan.

Yeah, I think "PRAGMA optimize" here on startup is basically a no-op and only costs microseconds but I should just remove it.

Seems like the ideal thing is to spawn a goroutine and run optimize periodically. Many Go daemons I run will have uptime measured in months so running on shutdown isn't super useful there.

Another one I might add is running `PRAGMA wal_checkpoint(PASSIVE)` periodically from a goroutine, which is something I've done in some of my daemons. Because otherwise a random write suffers from increased latency due to having to checkpoint.

Probably makes sense to open a dedicated read/write connection just for running `PRAGMA wal_checkpoint(PASSIVE)` at some regular interval to keep ahead of any autocheckpointing?