Using the Expand and Contract Pattern for Schema Changes

This is the same pattern as versioning, but with an extremely short sunset for the old version.

Evens between odds. Where even schema versions are migrations and 'divisible between two'.

But green/blue and A/AB/B I've used before to discuss the same.

To me is just a blue-green type deployment for schemas. You have an old and a new thing, you split and merge as traffic replays to the new thing and shows that its viable and not breaking, you swap over as you can.

I’ve always paired this with the strangler pattern.

“In programming, the strangler fig pattern or strangler pattern is an architectural pattern that involves wrapping old code, with the intent of redirecting it to newer code.”

https://en.wikipedia.org/wiki/Strangler_fig_pattern

I have usually heard it called "A–AB–B migrations". As in, you support version A, then you support both version A and version B, then you support just version B.

The rest of the sequencing details follow from this idea.

> What’s in the article, I know as the Strangler Fig Pattern.

Strangler fig pattern is mostly concerned with migrating from an old software to a new software, from example from a monolith to microservices. But I guess you can also apply it to database schemas.

What you describe is what they describe as well:

> For column-level changes, this often means adding new columns to a table that have the characteristics you want while leaving the current columns as-is.

I think what makes it confusing is that their diagrams depict a completely separate schema, but what they describe is really just altering the existing schema.

We always called these "four-phase migrations". An old Stripe article used similar naming[0].

[0]: https://stripe.com/blog/online-migrations

In the step where writes are added, from the client to the new schema instance, how does this work when writes depend on existing data, when the existing data hasn’t even been copied? Constrains like foreign keys will prevent this from working, no?

Expand the interface contract and then contract the interface contract? ;)

In my experience, this process typically spans multiple deploys. I would say the key insight that I have taken away from decades of applying this approach, is that data migrations need to be done in an __eventually consistent__ approach, rather than as an all-or-nothing, stop-the-world, global transaction or transformation.

Indeed, this pattern, in particular, is extremely useful in environments where you are trying to making changes to one part of a system while multiple deploys are happening across the entire system, or where you are dealing with a change that requires a large number of clients to be updated where you don't have direct control of those clients or they operate in a loosely-connected fashion.

So, regardless of AWS RDS as your underlying database technology, plan to break these steps up into individual deployment steps. I have, in fact, done this with systems deployed over AWS RDS, but also with systems deployed to on-prem SQL Server and Oracle, to nosql systems (this is especially helpful in those environments), to IoT and mobile systems, to data warehouse and analysis pipelines, and on and on.

It should be this way. Clients should have some protocol to communicate the schema they expect to the database probably with some versioning scheme. The database should be able to serve multiple mutually compatible views over the schema (stay robust to column renames for example). The database should manage and prevent the destruction of in use views of that schema. After an old view has been made incompatible, old clients needing that view should be locked out.

On the Rails side, Gitlab has an extensive set of helpers for this that a lot of Rails projects have adopted—I would love to see them pulled out into a Gem or adopted into Rails core proper: https://gitlab.com/gitlab-org/gitlab-foss/blob/master/lib/gi...

I'm hearing you out, but how is this going to affect the part of this that is client behavior rather than database behavior? If there is some kind of sdk that actually captures the interface here (that is, that the client needs to be compatible with both versions of the schema at once for a while) and pushes that back to the client, that could be interesting, like a way to define that column "name" and columns "first name", "last name" are conceptually part of the same thing and that the client code paths must provide handling for both at once.

If you solve that "verifying" step, you will already revolutionize software development.

Lack of reasonable support for migrations turned me off to Prisma when I first encountered it (in a KCDodds Remix app circa 2021-ish). I'm surprised that's still unaddressed.

What do you find good about Prisma?