"you Don't Need Kafka, Just Use Postgres" Considered Harmful
Posted2 months agoActiveabout 2 months ago
morling.devTechstory
heatedmixed
Debate
80/100
Apache KafkaPostgresqlMessage QueuesDatabase Design
Key topics
Apache Kafka
Postgresql
Message Queues
Database Design
The article argues against using Postgres as a message queue replacement for Kafka, sparking a debate among commenters about the trade-offs between the two approaches.
Snapshot generated from the HN discussion
Discussion Activity
Very active discussionFirst comment
1h
Peak period
25
0-6h
Avg / period
4.7
Comment distribution42 data points
Loading chart...
Based on 42 loaded comments
Key moments
- 01Story posted
Nov 2, 2025 at 3:37 PM EST
2 months ago
Step 01 - 02First comment
Nov 2, 2025 at 4:55 PM EST
1h after posting
Step 02 - 03Peak activity
25 comments in 0-6h
Hottest window of the conversation
Step 03 - 04Latest activity
Nov 5, 2025 at 11:14 AM EST
about 2 months ago
Step 04
Generating AI Summary...
Analyzing up to 500 comments to identify key contributors and discussion patterns
ID: 45793216Type: storyLast synced: 11/20/2025, 6:30:43 PM
Want the full context?
Jump to the original sources
Read the primary article or dive into the live Hacker News thread when you're ready.
There are plenty of choices between PSQL & Kafka. It's not like you take one step north and you're in the "oh no you better know what you're doing" territory.
Postgres land is a comfy place filled with transactions across all your data at once, one backup solution that you (hopefully) have had running for months or years and has been thoroughly tested, and ACID compliance. You have a single host, probably, which means that you are neither Available nor Partion-tolerant, but at least you are Consistent.
The moment you expand beyond a single database host you now have a distributed system, and woe unto you if you don't understand what that means.
You can work around this, but to the best of my knowledge you can't have consistency (between your existing Postgres database and your separate queue or event log) and simplicity.
But with PSQL you have even less built on top for a use case it wasn't meant for. Now you are in the "you better know what you're doing" territory.
When people use Redis/whatever, are you surprised that speed is just a side benefit and ergonomics is what they're looking for? Those are the same ergonomics you'd have to rebuild, as opposed to not build at all because it's already there.
If you want to rebuild Redis in PSQL you are very much in the "you better know what you're doing" territory, and you're also very much in the "are you sure this is your core business value" territory as you rediscover how much nice stuff was packed into the Redis ecosystem. And how am I supposed to uncover the surface or ergonomics of your Redis replacement?
Redis is fine as-is and I'm not suggesting anyone should rebuild the whole thing in Postgres, only that if they want a specific thing that Redis can do and they already use Postgres, then they might be able to do the same thing in Postgres with less effort and thus avoid losing atomicity and consistency and any other properties of Postgres that they find desirable.
[0] Which was mentioned by the original "You Don't Need Kafka" article
Do I really need to discuss why BullMQ is non-trivial software, or Redis Streams with consumer groups?
And I asked but you didn't answer. How am I supposed to discover the ergonomics of your custom Redis or BullMQ replacement? Do I read your hand-written docs?
LinkedIn have moved onto Northguard... but no GitHub yet
https://boringtechnology.club/
> Managed services make running Kafka a very uneventful experience (pun intended) and should be the first choice
Confluent, you say?
Good idea; this is stated in the bio on my web site, but I've just added the same info again to the end of the post.
It might be worth adding a more direct call-out to posts like this one. Many may not go as far as reading the Bio page. That may be on them technically speaking, but still.
In any case, thank you for writing and sharing your considered opinion!
The author is a an employee for Confluent/Kafka so because his paycheck and equity grant depends on it and CFLT stock price, obviously whatever he writes is going to be heavily slanted in favor of Kafka. This isn't something that is a footnote at the bottom, it should be right up at the front.
I think that shouldn't matter but I still have a lot to disagree with the article.
feels like overengineering has become the standard for some people, and I quite dislike it personally.
Example: Someone writes a software that could use something simple like SQLite, and they switched to Postgres for performance reasons. Now unless what Kafka beings is the core reason they switched to Postgres not pulling in another dependency and adding a nother piece to the puzzle, can be a total legitimate engineering decision. And that renders the "considered harmful" utterly ridiculous.
Use a system like Kafka if you need what it brings (a distributed event streaming platform). If that isn't what you need or a very simple postgres solition suffices, go for that. Maybe you need event streaming but distributing it is overkill. Maybe you just need some sort of queue. Who knows? Not the author of this post.
Indeed that is the point I am trying to make in the article. Postgres oftentimes absolutely is the right tool to use, and oftentimes it's not. The thing I'm advocating to be wary of is "if all you have is a hammer...". This is to what "considered harmful" refers.
Nailed it. I read the original post earlier this week and was very impressed with its technical detail. But the point of the the post was incongruent with the post's title. But the post got way more attention because of that title.
But if you think about the effort it took to write that post, the title was a really good bet on ROI.
> Nailed it.
Worked for the confluent marketing fluff as well.
Trying to explain the distinction between an event streaming platform and a distributed message queue to your enterprise architect is an exercise that no one should have to go through.
If you have to explain that distinction, are you really speaking to an "enterprise architect" ?
Disclaimer: I'm the author of this post.
Take the approach that appeals to you and feel happy about it without big open source telling you "you're holding it wrong!"
1 more comments available on Hacker News