What Happens When Coding Agents Stop Feeling Like Dialup?

I've used LLMs for code gen at work as well as for personal stuff.

At work primarily for quick and dirty internal UIs / tools / CLIs it's been fantastic, but we've not unleashed it on our core codebases. It's worth noting all the stuff we've got out of out are things we'd not normally have the time to work on - so a net positive there.

Outside of work I've built some bespoke software almost entirely generated with human tweaks here and there - again, super useful software for me and some friends to use for planning and managing music events we put on that I'd never normally have the time to build.

So in those ways I see it as massively increasing productivity - to build lower stakes things that would normally just never get done due to lack of time.

"You're doing AI wrong" is the new "you're doing agile wrong" which was the new "you're doing XP wrong".

I use Claude, Codex, and the Gemini CLI (all "supervised command line agents"). I write Go. I am certain that agents improve my productivity in a couple common scenarios:

1. Unsticking myself from stalling at the start of some big "lift" (like starting a new project, adding a major feature that will pull in some new dependency). The LLM can get things very wrong (this happens to me maybe 20% of the time), but that doesn't matter, because for me the motion of taking something wrong and making it "righter" is much less effort than getting past a blank page, assembling all the resources I need to know how to hello-world a new dependency, that kind of thing. Along with "wrestling for weeks with timing-dependent bugs", this kind of inertia is one of like two principal components of "effort" in programming for me, so clearing it is a very big deal. I'm left at a point where I'm actually jazzed about taking the wheel and delivering the functionality myself.

2. Large mechanical changes (like moving an interface from one component to another, or major non-architectural refactors, or instrumentation). Things where there's one meta-solution and it's going to be repeated many times across a codebase. Easy to review, a lot of typing, no more kidding myself for 20 minutes that I can make just the right Emacs macro to pull it off.

3. Bug hunting. To be clear: I am talking here about code I wrote, not the LLM. I run it, it does something stupid. My first instinct now is to drop into Claude or Gemini, paste the logs and an @-reference to a .go file as a starting point, and just say "wtf". My hit rate on it spotting things is very good. If the hit rate was even "meh borderline" that would be a huge win for the amount of effort it takes, but it isn't, it's much better.

I'm guessing a lot of people do not have these 3 experiences with LLM agents. I'm sorry! But I do think that if you stipulate that I'm not just making this up, it's hard to go from here to "I'm kidding myself about the value this is providing". Note that these are three cases that look nothing at all like vibe-coding.

Of course there are going to be discussions what is real programming (like I'm sure there were discussions what is "real" accounting with the onset of a calculator)

The moment we stop treating LLMs like people and see them as big calculators, it all clicks.

For me is the exact opposite. When not using AI, while coding you notice various things that could be improved, you can think about the architecture and what features you want next.

But AI codes so fast, that it's a real struggle keeping up to it. I feel like I need to focus 10 times harder to be able to think about features/architecture in a way that AI doesn't wait after me most of the time.

"How to harness it" is very clearly the difference between users right now, and I'd say we're currently bottom heavy with poor users stuck in a 'productivity illusion'.

But there is the question of "what is productivity?"

I'm finding myself (having AI) writing better structured docs and tests to make sure the AI can do what I ask it to.

I suspect that turns into compounding interests (or lack of technical debt).

For an industry where devs have complained, for decades, about productivity metrics being extremely hard or outright bullshit, I see way too many of the same people now waving around studies regarding productivity.

I don't see why. If anything there's more opportunity for documentation because the code was generated from a natural language prompt that documents exactly what was generated, and often why. Recording prompts in source control and linked to the generated code is probably the way to go here.

"AI" might be a good thing or a bad thing especially depending on the context and implementation, but just generally saying something is about maximizing comfort as some inherently bad goal seems off to me.

Just a sideways thing. Cognitive offloading is something humans do with each other quite a lot. People offload onto colleagues, underlings and spouses all the time.

People engage with AI through the prism of reducing engagement with the thing they dont like, and increasing engagement with the thing they do like.

It isnt a straight up productivity boost, but its more like going from being a screenwriter to a director.

In that regard, nothing will change.

increases the perception of productivity?

> AI systems do not currently speed up many or most software developers

> AI systems in the near future will not speed up developers in our exact setting

> There are not ways of using existing AI systems more effectively to achieve positive speedup in our exact setting

>programmers

Don't Look Up

Whenever I experiment with agent frameworks that spawn subagents with scoped subtasks and restricted context, things go off the rails very quickly. A subagent with reduced context makes poorer choices and hallucinates assumptions about the greater codebase, and very often lacks a basic sense the point of the work. This lack of situational awareness is where you are most likely to encounter js scripts suddenly appearing in your Python repo.

I don’t know if there is a “fix” for this or if I even want one. Perhaps the solution, in the limit, actually will be to just make the big-smart models faster and faster, so they can chew on the biggest and most comprehensive context possible, and use those exclusively.

eta: The big models have gotten better and better at longer-running tasks because they are less likely to make a stupid mistake that derails the work at any given moment. More nines of reliability, etc. By introducing dumber models into this workflow, and restricting the context that you feed to the big models, you are pushing things back in the wrong direction.

I they don't see a viable path forward without specialty hardware

Also, cursor is both overpriced and very mediocre as an agent. Codex is the way to go.

But when it doesn’t output what you want you spend mental energy and extra time reorienting to get back on track.

About 50% of the time it works every time.

This is the magic sauce compared to copy-pasting snippets to a web browser.

It can automatically read code, search through it and modify multiple files automatically. Then it can compile it, run tests, run the actual application to check it works etc.