Andrej Karpathy – It Will Take a Decade to Work Through the Issues with Agents
Posted3 months agoActive3 months ago
dwarkesh.comTechstoryHigh profile
skepticalmixed
Debate
85/100
AgiArtificial IntelligenceLarge Language Models
Key topics
Agi
Artificial Intelligence
Large Language Models
Andrej Karpathy estimates it will take a decade to work through the issues with AI agents, sparking a lively discussion on the feasibility and timeline of achieving Artificial General Intelligence (AGI).
Snapshot generated from the HN discussion
Discussion Activity
Very active discussionFirst comment
19m
Peak period
130
0-6h
Avg / period
22.9
Comment distribution160 data points
Loading chart...
Based on 160 loaded comments
Key moments
- 01Story posted
Oct 17, 2025 at 1:24 PM EDT
3 months ago
Step 01 - 02First comment
Oct 17, 2025 at 1:43 PM EDT
19m after posting
Step 02 - 03Peak activity
130 comments in 0-6h
Hottest window of the conversation
Step 03 - 04Latest activity
Oct 20, 2025 at 7:07 AM EDT
3 months ago
Step 04
Generating AI Summary...
Analyzing up to 500 comments to identify key contributors and discussion patterns
ID: 45619329Type: storyLast synced: 11/27/2025, 3:36:14 PM
Want the full context?
Jump to the original sources
Read the primary article or dive into the live Hacker News thread when you're ready.
way more pogress toward fusion than AGI. Uncontrolled runaway fusion reactions were perfected in the 50s (iirc) with the thermonuclear bombs. Controllable fusion reactions have been common for many years. A controllable, self-sustaining, and profitable fusion reaction is all that is left. The goalposts that mark when AGI has been reached haven't even been defined yet.
What does he know about that?
If you train it on a bunch of paintings whose quality ranges from a toddler's painting to Picasso's, it's not going to make one that's better than Picasso's, it's going to output something more comparable to the most average painting it was trained on. If you then adjust your training data to only include world's best paintings ever since we began to paint, the outcome is going to improve, but it'll just be another better-than-human-average painting. If you then leave it running 24/7, it'll churn out a bunch of better-than-human-average paintings, but there's still an easily-identifiable ceiling it won't go above.
An oracle that always returns the most average answer certainly has its use cases, but it's fundamentally opposed to the idea of superintelligence.
Yes, I agree, it's not high quality stuff it produces exactly, unless the person using it already is an expert and could produce high quality stuff without it too.
But there is no denying it that those things were regarded as "far-near future maybe" for a long time, until some people put the right pieces together.
Humans have since adapted to identify content differences and assign lower economic value to content created by programs, i.e. the humans being "impersonated" and "fooled" are themselves evolving in response to imitation.
This isn’t a bad thing, and I think LLMs are very impressive. But I do think we’d hesitate to call their behavior human-like if we weren’t predisposed to anthropomorphism.
second a definition is obviously not a prerequisite as evidenced by natural selection
I don't think he should stop, because I think he's right. We lack a definition of intelligence that doesn't do a lot of hand waving.
You linked to a paper with 18 collective definitions, 35 psychologist definitions, and 18 ai researcher definitions of intelligence. And the conclusion of the paper was that they came up with their own definition of intelligence. That is not a definition in my book.
> second a definition is obviously not a prerequisite as evidenced by natural selection
right, we just need a universe, several billions of years and sprinkle some evolution and we'll also get intelligence, maybe.
And real AI is probably like fusion. Its always 10 years away.
You and also everyone since the beginning of AI. https://quoteinvestigator.com/2024/06/20/not-ai/
The common thread between those who take things as "AI is anything that doesn't work yet" and "what we have is still not yet AI" is "this current technology could probably have used a less distracting marketing name choice, where we talk about what it delivers rather than what it's supposed to be delivering".
From where I sit, the generative models provide more flexibility but tend to underperform on any particular task relative to a targeted machine learning effort, once you actually do the work on comparative evaluation.
You appear to be comparing apples to oranges. A generation task is not a categorization task. Machine learning solves categorization problems. Generative AI uses model trained by machine learning methods, but in a very different architecture to solve generative problems. Completely different and incomparable application domain.
If “machine learning” is taken to be so broad as to include any artificial neural network, all of which are trained with back propagation these days, then it is useless as a term.
The term “machine learning” was coined in the era of specialized classification agents that would learn how to segment inputs in some way. Thing email spam detection, or identifying cat pictures. These algorithms are still an essential part of both the pre-training and RLHF fine tuning of LLM models. But the generative architectures are new and very essential to the current interest in and hype surrounding AI at this point in time.
> “I mean in 2035, that, like, graduating college student, if they still go to college at all, could very well be, like, leaving on a mission to explore the solar system on a spaceship in some completely new, exciting, super well-paid, super interesting job, and feeling so bad for you and I that, like, we had to do this kind of, like, really boring old kind of work and everything is just better."
Which should be reassuring to anyone having trouble finding an entry-level job as an illustrator or copywriter or programmer or whatever.
edit: Oh. Solar system. Nvm. Totally reasonable.
If nothing else it's been a sci-fi topic for more than a century. There's connotations, cultural baggage, and expectations from the general population about what AI is and what it's capable of, most of which isn't possible or applicable to the current crop of "AI" tools.
You can't just change the meaning of a word overnight and toss all that history away, which is why it comes across as an intentionally dishonest choice in the name of profits.
More to the point, the history of AI up through about 2010 talks about attempts to get it working using different approaches to the problem space, followed by a shift in the definitions of what AI is in the 2005-2015 range (narrow AI vs. AGI). Plenty of talk about the various methods and lines fo research that were being attempted, but very little about publicly pushing to call commercially available deliverables as AI.
Once we got to the point where large amounts of VC money was being pumped into these companies there was an incentive to redefine AI in favor of what was within the capabilities and scope of machine learning and LLMs, regardless of whether that fit into the historical definition of AI.
I took an AI class in 2001. We learned all sorts of algorithms classified as AI. Including various ML techniques. Under which included perceptrons.
Ten years ago you'd be ashamed to call anything "AI," and say machine learning if you wanted to be taken seriously, but neural networks have really have brought back the term--and for good reason, given the results.
From skimming the conversation it seems to mostly revolve around LLMs (transformer models) which is probably not going to be the way we obtain AGI to begin with, frankly it is too simple to be AGI, but the reason why there's so much hype is because it is simple to begin with so really I don't know.
As for abstract reasoning, if you look at ARC-2 it is barely capable though at least some progress has been made with the ARC-1 benchmark.
The Turing Test is whether it can fool a human into thinking it is talking to another human not an intelligent machine. And ironically this is becoming less true over time as people become more used to spotting the tendencies LLMs have with writing such as its frequent use of dashes or "it's not just X it is Y" type of statements.
Then I got it. :) Something so mundane that maybe the AIs can help prevent it.
I mean sure I now "control" the AI, but I still think these no AGI for 2 decades claims are a bit rough.
If you’re correct, there’s not much reward aside from the “I told you so” bragging rights, if you’re wrong though - boy oh boy, you’ll be deemed unworthy.
You only need to get one extreme prediction right (stock market collapse, AI taking over, etc ), then you’ll be seen as “the guru”, the expert, the one who saw it coming. You’ll be rewarded by being invited to boards, panels and government councils to share your wisdom, and be handsomely paid to explain, in hindsight, why it was obvious to you, and express how baffling it was that no one else could see what you saw.
On the other hand, if predict an extreme case and you get it wrong, there’s virtually 0 penalties, no one will hold that against you, and no one even remembers.
So yeah, fame and fortune is in taking many shots at predicting disasters, not the other way around.
cause elon musk says FSD is coming in 2017?
If ChatGPT is not AGI, somebody has moved goalposts.
it doesn't make him one
Assisted coding has been incredibly useful. I have been using Claude Code daily.
But if you let it take over completely without review and let it write whole features... which I take to be the meaning of "vibe" in some people's definitions... you're in for a world of long-term pain.
I've coded professionally for 40 years. I'm hugely excited about vibe coding. I use it every single day to create little tools and web apps to help me do my job.
The debate about AGI is interesting from a philosophical perspective, but from a practical perspective AI doesn't need to get anywhere close to AGI to turn the world upside down.
I feel like GPT 3 was AGI, personally. It crossed some threshold that was both real and magical, and future improvements are relying on that basic set of features at their core. Can we confidently say this is not a form of general intelligence? Just because it’s more a Chinese Room than a fully autonomous robot? We can keep moving the goalposts indefinitely, but machine intelligence will never exactly match that of humans.
Thats not a period, it's a full stop. There is no debate to be had here.
IF an LLM makes some sort of breakthrough (and massive data collation allows for that to happen) it needs to be "re trained" to absorb its own new invention.
But we also have a large problem in our industry, where hardware evolved to make software more efficient. Not only is that not happening any more but we're making our software more complex and to some degree less efficient with every generation.
This is particularly problematic in the LLM space: every generation of "ML" on the llm side seems to be getting less efficient with compute. (Note: this isnt quite the case in all areas of ML, yolo models working on embedded compute is kind of amazing).
Compactness, efficiency and reproducibility are directions the industry needs to evolve in, if it ever hopes to be sustainable.
We are approaching situation, where AI will make most decisions, and people will wear it as a skin suit, to fake competency!
Otherwise we would have to say that pre-literacy societies lacked intelligence, which would be silly since they are the ones that invented writing in the first place!
Obviously this quote would be well applied if we were at a stage where computers were better at everything humans can do and some people were saying "This is not AGI because it doesn't think exactly the same as a human". But we aren't anywhere near this stage yet.
Sure, and the question of whether AI can safely perform a particular task is interesting.
> Obviously this quote would be well applied if we were at a stage where computers were better at everything humans can do and some people were saying "This is not AGI because it doesn't think exactly the same as a human".
Why would that be required?
I used the quote primarily to point out that discussing the utility of AI is wholly distinct from discussing the semantics of words like "think", "general intelligence", or "swim". Knowing whether we are having a debate about utility/impact or philosophy/semantics seems relevant regardless of the current capabilities of AI.
2029: Human-level AI
2045: The Singularity - machine intelligence 1 billion times more powerful than all human intelligence
Based on exponential growth in computing. He predicts we'll merge with AI to transcend biological limits. His track record is mixed, but 2029 looks more credible post-GPT-5. The 2045 claim remains highly speculative.
The overwhelming majority of all gains in human life expectancy have come due to reductions in infant mortality. When you hear about things like a '40' year life expectancy in the past it doesn't mean that people just dropped dead at 40. Rather if you have a child that doesn't make it out of childhood, and somebody else that makes it to 80 - you have a life expectancy of ~40.
If you look back to the upper classes of old their life expectancy was extremely similar to those of today. So for instance in modern history, of the 15 key Founding Fathers, 7 lived to at least 80 years old: John Adams, John Quincy Adams, Samuel Adams, Jefferson, Madison, Franklin, John Jay. John Adams himself lived to 90. The youngest to die were Hamilton who died in a duel, and John Hancock who died of gout of an undocumented cause - it can be caused by excessive alcohol consumption.
All the others lived into their 60s and 70s. So their overall life expectancy was pretty much the same as we have today. And this was long before vaccines or even us knowing that surgeons washing their hands before surgery was a good thing to do. It's the same as you go back further into history. A study [1] of all men of renown in Ancient Greece was 71.3 [1], and that was from thousands of years ago!
Life expectancy at birth is increasing, but longevity is barely moving. And as Kurzweil has almost certainly done plentiful research on this topic, he is fully aware of this. Cognitive dissonance strikes again.
[1] - https://pubmed.ncbi.nlm.nih.gov/18359748/
Example: 20ish years ago, stage IV cancer was a quick death sentence. Now many people live with various stage IV cancers for many years and some even "die of sending else" these advancements obviously skew towards helping older people.
The reason humans die of 'old age' is not because of any specific disease but because of advanced senescence. Your entire body just starts to fail. At that point basically anything can kill you. And sometimes there won't even be any particular cause, but instead your heart will simply stop beating one night while you sleep. This is how you can see people who look like they're in great shape for their age, yet the next month they're dead.
But the results remain modest. The biggest breakthrough was in the 80s when somebody was able to roughly double their life expectancy from 2 months to 4 through artificial selection. But the context there is that fruit flies are a textbook 'quantity over quality' species, meaning that survival is not generally selected for, whereas humans are an equally textbook 'quality over quantity' species meaning that survival is one of the key things we select for. In other words, there was likely a lot more genetic low hanging fruit for survivability with fruit flies than there is for humans.
So I don't know. We need some serious acceleration and I'm not seeing much of anything when looked at with a critical eye.
The merge with a machine 1 million times more intelligent than us is the same as letting AI use our bodies. I'd rather live in cave. Iirc, the 7th episode of Black Mirror starts with this plot line.
Hegel thought history ended with the Prussian state, Fukuyama thought it ended in liberal America, Paul thought judgement day was so close you need not bother to marry, the singularity always comes around when the singularians get old. Funny how that works
I was a lot disappointed when he went to work for Tesla, and I think that he had some achievement there, butnot nearly the impact I believe he potentially has.
His switch (back?) to OpenAI was, in my mind, much more in keeping with where his spirit really lies.
So, with that in mind, maybe I've drunk too much kool aid, maybe not. But I'm in agreement with him, the LLMs are not AGI, they're bloody good natural language processors, but they're still regurgitating rather than creating.
Essentially that's what humans do, we're all repeating what our education/upbringing told us worked for our lives.
But we all recognise that what we call "smart" is people recognising/inventing ways to do things that did not exist before. In some cases its about applying a known methodset to a new problem, in others its about using a substance/method in a way that other substances/methodsets are used, but the different substance/methodset produces something interesting (think, oh instead of boiling food in water, we can boil food in animal fats... frying)
AI/LLMs cannot do this, not at all. That spark of creativity is agonisingly close, but, like all 80/20 problems, is likely still a while away.
The timeline (10 years) - it was the early 2010s (over 10 years ago now) that the idea of backward propagation, after a long AI winter, finally came of age. It (the idea) had been floating about since at least the 1970s. And that ushered in the start of our current revolution, that and "Deep Learning" (albeit with at least another AI winter spanning the last 4 or 5 years until LLMs arrived)
So, given that timeline, and the restraints in the currrent technology, I think that Andrej is on the right track, and it will be interesting to see where we are in ten years time.
Edit: This also demonstrates that people think (erroneously) that AI pumping out code, or content, or even essays, is inventive, but it's not.
This is merely a description and reduction, both of which AI can do, but neither of which are an invention.
Can you show me one single thing you did in your life that was truly creative and not regurgitated?
That's why people are conflating LLMs for AGI.
For now, I think that the key difference between me, and an LLM is that an LLM still needs a prompt.
It's not surveying the world around it determining what it needs to do.
I do a lot of something that I think an LLM cannot get do, look at things and try to find what attributes they have and how I can harness those to solve problems. Most of the attributes are unknown by the human race when I start.
So if I make an ai with an a prompt and tell him to re prompt itself every day for the rest of his life means is smart now? Or just because I give him the first prompt is invalid? I doubt your first prompt was given by yourself. Was probably in your mums belly your first prompt.
—-
I could give an initial prompt to my ai to survey the server and act accordingly… and he can re prompt every day himself.
——
> I do a lot of something that I think an LLM cannot get do, look at things and try to find what attributes they have and how I can harness those to solve problems. Most of the attributes are unknown by the human race when I start.
Any examples? An ai can look at a conversation and extract insights better than most people. Negotiate better than most people.
—-
I heard nothing that you can do more than a llm. Self prompting yourself to do something I don’t think is a differentiator.
You also self prompt yourself based on Previous feedback. And you do this since you’re a baby. So someone also gave you the source prompt. Maybe dna.
I don't believe you have the capacity to understand why AGI hasn't been realised yet, and, frankly, I doubt you ever will.
But, the fact that you missed that does present a case for you being an LLM.
Comparing LLMs trained on reddit comments and people who learn to speak as a byproduct of actually interacting with people and the world is nuts.
That includes anyone reading this message long after the lives of those reading it on its post date have ended.
Which of course raises the interesting question of how I can make good on this bet.
Why is growth over the last 3 years completely flat once you remove the proverbial AI pickaxes sellers?
What if all the slop generated by llms counterbalance any kind of productivity boost? 10x more bad code, 10x more spam emails, 10x more bots
AI has now been revealed to the masses. When AGI arrives most people will barely notice. It will just feel like slightly better LLMs to them. They will have already cemented notions of how it works and how it affects their lives.
But nothing will make grifters richer than promising it's right around the corner.
However, don't let the bandwagon ( from either side ) cloud your judgment. Even warm fusion or any fusion at all is still very useful and it's here to stay.
This whole AGI and "the future" thing is mostly a VC/Banks and shovel sellers problem. A problem that has become ours too because the ridiculous amounts of money "invested", so even warm fusion is not enough from an investment vs expectations perspective.
They are already playing musical money chairs, unfortunately we already know who's going to pay for all of this "exuberance" in the end.
I hope this whole thing crashes and burns as soon as possible, not because I don't "believe" in AI, but because people have been absolutely stupid about it. The workplace has been unbearable with all this stupidity and amounts of fake "courage" about every single problem and the usual judgment of the value of work and knowledge your run-of-the-mill dipshit manager has now.
955 more comments available on Hacker News