AI Helps Ship Faster but It Produces 1.7× More Bugs

Coderabbit is an LLM code review company so their incentives are the opposite. AI is terrible and you need more AI to review it.

fwiw, I agree. LLM-powered code review is a lifesaver. I don't use Coderabbit but all of my PRs go through Copilot before another human looks at it. It's almost always right.

Blindly copying and pasting from StackOverflow until it kinda sorta works is basically vibe coding

AI just automates that

I agree but I'd draw a different comparison. That is vibe coding has accelerated the type of developers who relied on stack overflow to solve all their problems. The kind of dev who doesn't try to solve problems themselves. It has just accelerated this type of working, but is less reliable than before.

"on error resume next" has been the first line of many vba scripts for years

  1. if it won't compile you'll give up on the tool in minutes or an hour.
  2. if it won't run you'll give up in a few hours or a day.
  3. if it sneaks in something you don't find until you're almost - or already - in production it's too late.

One of my frustrations with AI, and one of the reasons I've settled into a tab-complete based usage of it for a lot of things, is precisely that the style of code it uses in the language I'm using puts out a lot of things I consider errors based on the "middle-of-the-road" code style that it has picked up from all the code it has ingested. For instance, I use a policy of "if you don't create invalid data, you won't have to deal with invalid data" [1], but I have to fight the AI on that all the time because it is a routine mistake programmers make and it makes the same mistake repeatedly. I have to fight the AI to properly create types [2] because it just wants to slam everything out as base strings and integers, and inline all manipulations on the spot (repeatedly, if necessary) rather than define methods... at all, let alone correctly use methods to maintain invariants. (I've seen it make methods on some occasions. I've never seen it correctly define invariants with methods.)

Using tab complete gives me the chance to generate a few lines of a solution, then stop it, correct the architectural mistakes it is making, and then move on.

To AI's credit, once corrected, it is reasonably good at using the correct approach. I would like to be able to prompt the tab completion better, and the IDEs could stand to feed the tab completion code more information from the LSP about available methods and their arguments and such, but that's a transient feature issue rather than a fundamental problem. Which is also a reason I fight the AI on this matter rather than just sitting back: In the end, AI benefits from well-organized code too. They are not infinite, they will never be infinite, and while code optimized for AI and code optimized for humans will probably never quite be the same, they are at least correlated enough that it's still worth fighting the AI tendency to spew code out that spends code quality without investing in it.

[1]: Which is less trivial than it sounds and violated by programmers on a routine basis: https://jerf.org/iri/post/2025/fp_lessons_half_constructed_o...

[2]: https://jerf.org/iri/post/2025/fp_lessons_types_as_assertion...

I'd call some null-pointer-lint-with-automatic-fixes tools "vibe coding" tbh. I've run across a couple that do a pretty good job of detecting possible nulls and add annotations about it and that's great... but then the fix is "if null, return null", in practice it's frequently applied completely blindly without any regards to correctness.

If you lean on tools like that, you can rapidly degrade your codebase into "everything might be null and might short circuit silently and it can't tell you about when it happens", leaving you with buggy software that is next to impossible to understand or troubleshoot because there aren't "should not be null" hints or stack traces or logs or anything that would help figure out causes.

Totally agree. I see it all the time: https://bower.sh/death-by-thousand-existential-checks

My experience with AI coding is mixed.

In some cases I feel like I get better quality at slightly more time than usual. My testing situation in the front end is terribly ugly because of the "test framework can't know React is done rendering" problem but working with Junie I figured out a way to isolate object-based components and run them as real unit test with mocks. I had some unmaintainable Typescript which would explode with gobbledygook error messages that neither Junie or I could understand whenever I changed anything but after two days of talking about it and working on it it was an amazing feeling to see that the type finally made sense to me at Junie at the same time.

In cases where I would have tried one thing I can now try two or three things and keep the one I like the best. I write better comments (I don't do the Claude.md thing but I do write "exemplar" classes that have prescriptive AND descriptive comments and say "take a look at...") and more tests than I would on my on my own for the backend.

Even if you don't want Junie writing a line of code it shines at understanding code bases. If I didn't understand how to use an open source package from reading the docs I've always opened it in the IDE and inspected the code. Now I do the same but ask Junie questions like "How do I do X?" or "How is feature Y implemented?" and often get answers quicker than digging into unfamiliar code manually.

On the other hand it is sometimes "lights on and nobody home", and for a particular patch I am working on now it's tried a few things that just didn't work or had convoluted if-then-else ladders that I hate (even if I told it I didn't like that) but out of all that fighting I got a clear idea of where to put the patch to make it really simple and clean.

But yeah, if you aren't paying attention it can slip something bad past you.

> I actually think a few folks like this can be valuable around the edges of software but whole systems built like this are a nightmare to work on. IMO AI vibe coding is an accelerant on this style of not knowing why something works but seeing what you want on the screen.

I would correct that: it's not an accelerant of "seeing what you want on the screen," it's an accelerant of "seeing something on the screen."

Things like habitual, unthoughtful null-checks are a recipe for subtle data errors that are extremely hard to fix because they only get noticed far away (in time and space) from the actual root cause.

Yeah. There are times when silently swallowing nulls is the proper answer. I've found myself doing it many times in C# to trap events that get triggered during creation. But you should never do so unless you've traced where they're coming from!

I've written before about my experience at a shop like this. The null check would swallow the exception and do nothing about the failure so everything just errored silently. Many high fives and haughty remarks about how smart the team was for doing this were had at the expense of lesser teams that didn't. The whole operation ran on an architecture from a Learning Tree class a guy took in 2008 and snippets stolen from StackOverflow and passed around on a USB key. Deviation from this bible was heresy and was met with sharp, unprofessional behavior. It was not a good place to work for people that value independent thought.

> AI vibe coding is an accelerant on this style of not knowing why something works but seeing what you want on the screen.

I've been saying this exact thing for years now. It also does the whole CRUD app "copy, paste, find, replace from another part of the application" workflow for building new bounded contexts very well. If you can bootstrap a codebase with good architectural practices and tests then Claude Code is a productivity godsend for building business apps.

If you are in the industry for enough time you certainly crossed with a boss who said that it needs to be fixed in 5 minutes or else even if the problem was not caused by you and the solution clearly needs more than 5 minutes. (The root cause was because someone had only 5 minutes to do something too)

I once had a job that my boss ordered (that's the word he used) me to do the wrong thing. Me and the rest of the team refused except for one guy who did it because he was certain that 9 out 10 people were wrong while he was the only right one) The company spend 2M USD in returns, refunds and compensations in a project that probably didn't cost that. It was just a patch! How he could've possibly know - said the dismissed manager.

(Now he works for oracle, why not right)

"ship fast, break things"

It doesn't generate good tests by default though.

I worked on a team where we had someone come in and help us improve our tests a lot.

The default LLM generated tests are bit like the ones I wrote before that experience.

I've been using Claude sonnet 4.5 lately and I've noticed a tendency for it to create tests that prove themselves. Rather than calling the function we're hoping to test, it re-implements the code in the test and then tests it there. It's helpful, and it usually works very well if you have well defined inputs and outputs, I much prefer it over writing tests manually, but you have to be very careful.

The "future" is a really long time.

I'll take the other side of your bet for the next 10 years but I won't take it for the next 30 years.

In that spirit, I want my fusion reactor and my flying car.

I'm seeing parallels between this and factory-assembled houses.

Input costs are lower and velocity is higher. You get a finished product out the door quicker, though maintenance is more expensive. Largely because the product is no longer a collection of individual parts made to be interfaced by a human. It is instead a machine-assembled good that requires a machine to perform "the work". Therefore, because the machine is only designed to assemble the good, your main recourse is to have the machine assemble a full replacement.

With that framing, there seems to be a tradeoff to bear in mind when considering fit for the problem we're meaning to solve. It also explains the widespread success of LLMs generating small scripts and MVPs. Which are largely disposable.

Couldn't this be true in the other direction as well? Anecdotally I see developers putting a lot more scrutiny into vibe coded PRs, while AI code tends to be highly commented (by the AI) and potentially easier to read.

I've seen way more human comments of "I don't know what this does but if I remove it everything breaks" in systems.

It's been suggested that code reviews shouldnt be about finding bugs, because they do not really work well for that goal.

https://www.microsoft.com/en-us/research/wp-content/uploads/...

> I could ship significantly faster if I ignored any error handling...

time to switch to erlang/elixir?

The problem with this black or white framing is it implies competency is a failure. Finding it not useful doesn’t make you wrong, anymore than finding it useful doesn’t make you inexperienced.

I think you need a balance. I’ve seen products fall apart due to high error rate.

I like to think of intentionalists—people who want to understand systems—and vibe coders—people who just want things to work on screen expediently.

I think success requires a balance of both. The current problem I see with AI is that it accelerates the vibe part more than the rationalist part and throws the system out of balance.

When a team says that a "trivial" feature takes months to ship is not because of the complexity of the algorithm. It's because of the infrastructure and coordination work required for the feature to properly work. It is almost aways a failure of the technical infrastructure previously created in the company. An AI will solve the trivial aspects of the problem, not the real problem.

> I’ve certainly seen a sharp increase on execs calling BS on dev teams saying they need months to develop some basic thing.

Some of the teams I worked with in the years right before AI coding went mainstream had become really terrible about this. They would spend months forming committees, writing documents, getting sign-offs and approvals, creating Gantt charts, and having recurring meetings for the simplest requests.

Before I left, they were 3 months deep into meetings about setting up role based access control on a simple internal CRUD app with a couple thousand users. We needed about 2-3 roles. They were into pros and cons lists for every library and solution they found, with one of the front runners involving a lot of custom development for some reason.

Yet the entire problem could have been solved with 3 Boolean columns in the database for the 3 different roles. Any developer could have done it in an afternoon, but they were stuck in a mindset of making a big production out of the process.

I feel like LLMs are good at getting those easy solutions done. If the company really only needs a simple change, having an LLM break free from the molasses of devs who complicate everything is a breath of fresh air.

On the other hand, if the company had an actual complicated need with numerous and changing roles over time, the simple Boolean column approach would have been a bad idea. Having people who know when to use each solution is the real key.

And you think people who don't understand the software telling people who do they're doing it wrong is an outright positive?

More important than code quality is a joint understanding of the business problem and the technical solution for it. Today, that understanding is spread across multiple parties (eng, pm, etc).

Code quality can be poor as long as someone understands the tradeoffs for why it's poor.

> if shipping faster allows for faster iterations across an idea then the silly errors are worth it.

There is more to the consideration here...

Maintenance costs compound over time, as does complexity.

If you're building something that doesn't need to last long or is really simple, sure it's worth it... But a lot of stuff does need to last a long time and continuously change.

Vibe coding is a trade off that only works up to a certain distance from the existing system. If you don't need to go far, it's great. But if you need long range then it's the wrong tool for the job at the moment.

The “it’s no worse” argument seems to ignore the headline, which literally reports that it is worse.

This attitude just furthers our race to the bottom. I agree with iteration, but software quality is getting really laughable. I know we're still on the better side of what existed in the hands of consumers in the 90s, but anyway... Execs calling BS is further evidence of that race to the bottom.

Tell that to your customers. And tell them how much longer the bugs generated by AI will take to fix by humans. Or tell them that you'll never fix the bugs because you're too busy vibe coding new ones.

It's a lost cause. "It's two times faster!"

I see that your release is GPL 3.0. Are you worried about LLM's effectively laundering your source code a year from now? I've become hesitant about releasing source code since LLM's, though I do use Claude heavily while programming to make suggestions and look for issues etc.., but I'd be interested in hearing your perspective.

are you Indian? don't lie

Are you trying to assure others or reassure yourself?

It produces more more bugs but the count goes down?!

> These tools are here to stay.

I see this stranded claim get trotted out in the corporate world all the time as a refutation to "The AI broke". I fail to understand who the invisible audience for this is supposed to be.

Also, the FOMO argument for why one should use X as much as they can comes across as scammy. Not that I believe you're trying to scam anyone. Though the person you originally got this from may have been!

> These tools are here to stay.

I don't think that these specific tools are here to stay. Categorically, yes, but I expect there to be big changes in interfaces and how they work in the next five years.

I don't think spin up time on LLM technology requires as much investment as the hype claims, nor do I think that the current methodology will be as long lived as they think. Sitting out may be detrimental in the now, but I expect that developers that do so will be able to catch up just fine.

I've used it in some repos. It doesn't catch all code review issues, especially around product requirements and logic simplification, and occasionally produces irrelevant comments (suggesting a temporary model downgrade).

But it's well worth it. It has saved me some considerable time. I let it run first, even before my own final self-review (so if others do the same, the article's data might be biased). It's particularly good at identifying dead code and logical issues. If you tune it with its own custom rules (like Claude.md), you can also cut a lot of noise.

It’s AI helps you ship faster. Ignore everything else.