Five Years of Tinygrad
Key topics
The tech world is abuzz with the five-year milestone of tinygrad, a project that's got everyone wondering: could it potentially dethrone PyTorch and TensorFlow? Some commenters see huge potential, particularly for edge deployments and specialized hardware like comma.ai's openpilot. However, others raise eyebrows at the project's trajectory, with some speculating it's more of an "inside deal" given its ties to the founder of comma.ai. As the discussion unfolds, a heated sub-thread emerges, with some commenters poking fun at the project's perceived ties to Elon Musk, prompting a sharp rebuke from one user who argues the value lies in the project's directness, not its implied origins.
Snapshot generated from the HN discussion
Discussion Activity
Very active discussionFirst comment
54m
Peak period
48
24-36h
Avg / period
13.3
Based on 160 loaded comments
Key moments
- 01Story posted
Dec 29, 2025 at 12:12 PM EST
13 days ago
Step 01 - 02First comment
Dec 29, 2025 at 1:06 PM EST
54m after posting
Step 02 - 03Peak activity
48 comments in 24-36h
Hottest window of the conversation
Step 03 - 04Latest activity
Jan 5, 2026 at 9:13 PM EST
5d ago
Step 04
Generating AI Summary...
Analyzing up to 500 comments to identify key contributors and discussion patterns
Want the full context?
Jump to the original sources
Read the primary article or dive into the live Hacker News thread when you're ready.
Why reference <VILE_HUMAN>?
A simple search shows this "process" has been around as long as programming.
I'd bet Ada even said that shit at least once too.
He only has to spend a couple thousand a month to influence comment ranking on HN.
not everyone cares about playing voldemort
https://news.ycombinator.com/newsguidelines.html
https://news.ycombinator.com/newsguidelines.html
Is it really so difficult to parse out different aspects of a person's existence, such that one could simultaneously appreciate some of the positives (the existence and success of SpaceX, making Tesla a success against the odds, [popularisation of] some engineering leadership concepts) while also acknowledging the negatives (of which, more latterly, there are many)?
Or must the sheer emotion always cloud out rational judgement?
I remember when defcon ctf would play Geohot's PlayStation rap video every year on the wall.
https://en.wikipedia.org/wiki/TRIZ
https://upload.wikimedia.org/wikipedia/commons/b/bf/40_princ...
No wonder the soviet union collapsed!
When you need Moscow to explain the hammer sign on your corded drill, its too late for damage control.
"A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away."
- Antoine de Saint-Exupéry
Time will tell.
History has not so far been kind to projects which attempt to supplant cPython, whether they are other Python variants such as PyPy, or other languages such as julia.
Python has a lot of detractors, but (despite some huge missteps with the 2-3 transition) the core team keeps churning out stuff that people want to use.
Mojo is being positioned "as a member of the Python family" but, like Pyrex/Cython, it has special syntax, and even worse, the calling convention is both different than Python, and depends on the type of variable being passed. And the introspection is completely missing.
What about numpyro?
Disclaimer: I contribute to numpyro occasionally.
If you don't mind me asking, what's the deal with NumPyro why you chose to work it?
https://danluu.com/julialang/
But the first thing that gave me pause about Julia? They sort of pivoted to say "we're general purpose" but the whole index-starting-at-one thing really belies that -- these days, that's pretty much the province of specialty languages.
It resonates with me, as does little-endian. There are evergreen arguments about both of these, but they stand out to me as two of the few places where I appear to be in sync with the majority.
> Pascal is using 1-ord, Fortran, R, Mathematica. If anything, it seems there's a longer tradition of 1-ord in scientific computing.
Actually, Pascal let you define the starting point, but, historically, COBOL (business) and FORTRAN started at one, and these days, it's Matlab (engineering), as well as R, Mathematica, and Julia. But, really, my point was that Julia is hyped to take over from Python, and it seems if that's the focus, then you should think long and hard about what features of Python to change, and, well, obviously that wasn't really the direction they came from.
> semantics like that, without unnecessarily sacrificing performance—I don't think they get enough credit for that from programming guys.
It's good work, but at the same time, a bit overblown IMO. For example, the creators claim homoiconicity, but that stretches the definition of that word to where Python would be homoiconic as well, and it's obviously not.
As far as the multiple dispatch goes, that's actually not that difficult of a puzzle piece when you are compiling ahead of time with unboxed objects.
Long term it can find applications outside of ML-specific hardware.
What is the status of the project? What can it do? What has it achieved in 5 years?
As a side note, whenever someone incessantly focuses on lines of code as a metric (in either direction), I immediately start to take them less seriously.
It doesn't work as well when you start mixing languages, or generating code.
The third sentence is "The codebase is 18,935 lines not including tests." You can go to the repo and see what it does. Look at issues, PRs, etc. He wasn't saying it to brag about its size, but the opposite.
The mission is to commoditize the petaflop. Basically allow LLMs to be trained efficiently on commodity non-NVIDIA GPUs. Would you prefer some bullshit mission like Meta of "connecting people" or whatever the hell it is?
He said he has a hardware division that makes $2m a year. You can click on the link and buy a computer. He tells you the revenue.
He said he has a deal with AMD which is also public and on X.
The "Elon process" relies specifically to the goal of getting rid of all dependencies. Musk has spoken extensively about building things from the ground up and not relying on other vendors (in this example complex software dependencies). He says he wouldn't be able to build SpaceX competitively if he had just bought rockets or components.
I wish more people focused on public code and shipping publicly. Can I see Toyotas stack? Why does the touch screen perform worse than a 2012 iPad. What the hell is going on
I don't understand this hate someone like Hotz, a true engineer running an ambitious very open company, receives on a ... checks notes ... engineering forum? The whole setup is like a wet dream for engineers from a decade ago.
Almost completely open source
Interview is completing a paid bounty
CEO is engineer
Has a clear mission, goals, and timeline
What is your issue?
That I cannot believe. He might have shifted the make-or-buy decisions, but both Tesla and SpaceX do a lot of outsourcing.
Note the "goal" there. SpaceX's only in flight explosion came after a strut (3rd party sourced) failed on S2, on the CRS-7 mission in 2015. They in sourced that, and haven't had many issue on ascent since then. They've also launched and landed some 500 rockets since then (165 this year) so ... at least they're walking the walk?
https://www.tomshardware.com/pc-components/gpus/tiny-corp-su...
—
Check tinygrad’s twitter account for specifics if you want to catch up on progress
"When we can reproduce a common set of papers on 1 NVIDIA GPU 2x faster than PyTorch. We also want the speed to be good on the M1. ETA, Q2 next year."
[0] https://tinygrad.org/#tinybox
I find this organizational structure compelling, probably the closest to reaching 100% productivity in a week as you can get.
“International scientists rethink U.S. conference attendance“
https://www.science.org/content/article/international-scient...
It's not about (in)ability to obtain a visa.
Few people with no innate interest in the project are likely to ramp up and start contributing just for a shot at a job. Whereas if you’re Facebook or whoever you are much more likely to get people who don’t care about the product and just have decided they can make a lot of money by jumping through the right hoops.
I'm shocked that interviewing still works how it does in large companies; the Sybil attacks and DDOS are just getting started.
You get a stipend in return if the contribution is accepted and you get to contribute to the world corpus of knowledge.
(Love that talk tho)
The fact that it competes with PyTorch in so few lines speaks to the incredibly low incidental complexity imposed by Tinygrad.
What's the margin on that? Do 5 software engineers really subsist on the spread from moving $2M/yr in hardware?
That’s not Elon. See Russian TRIZ
https://en.wikipedia.org/wiki/TRIZ
Either way, super cool project and I wish them the best.
Is this like voila or something else?
The SV is a unique place where you can meet Geo and get $5M, maintain a bunch of hardware, build a framework in 20,000 LOC, and everything works well.
Also half-joked how the good food went away.
—
George is many things but not a quitter (see commai for example).
If someone could pull this, it’s him due to “never give up, never surrender” attitude.
The shit with nvidia just needs to stop
https://github.com/tinygrad/tinygrad/blob/master/tinygrad/co...
One of the cases why I think obsession with lines of code is one of the most counterproductive metrics, it always produces code like this.
The far more interesting part is the order of magnitude. If they can pull off a 20k LOC with zero dependencies (implying a pretty concise project size) and it still works well on meaningful applications, that’s pretty neat. A 1000x reduction in code size and matching/exceeding in perf is worth looking at. Probably also implying a better architecture as code golf isn’t gonna get you 1000x less code. Again - their claims not mine, so we’ll see.
But at that point they can triple the LOC to 60k with nothing but white space, new lines, and comments, for all I care. It won’t even add a zero.
I often find myself wishing this was more ergonomic in languages.
Declarations flow down the page, definitions flow across.
I'm going to guess that you're used to codebases that use an autoformatter.
There's definitely a case to be made for autoformatters in projects with a very large (>100) number of developers, but the price you pay for that is much poorer information density per screenful of code.
For things like tinygrad or mysql with a small dev team it's possible to use whitespace, style, and formatting to communicate important information, just like mathematicians do. The AMS journals don't autoformat mathematicians' equations!
But if you have hundreds of developers on a project it does in fact make sense to sacrifice that signal in order to keep them from fighting with each other over silly formatting choices.
I'm used to Python code being "Pythonic" - which is one of those "I know it when I see it" terms.
I think there is an opportunity there to have a tag in comments for tests.
Have a hotkey to generate an id so you could have
and it creates a test that exists away from the main codebase. A process that runs tests could scan the code for testID's, intelligently extract the relevant part of the code into the test harness and runs the tests. There would need to be some infrastructure needed to handle placing the code to be tested into the right context but you could haveI agree. Seems to be a personal preference, but still feels odd.
I think if I dealt with this on a daily basis I might end up finding if vastly more preferable, albeit at the risk of alienating fellow developers. In that respect it seems much like learning to use Emacs or vi
It helps that a single person can read all related bits on one screen.
I agree with some poster above, the code looks hard simply because the problem is very hard, not because they want to write it that way.
Funny to me that it is called simplify.py
You can look at the PyTorch code base and understand a small local function instantly, but if I would have a task to have a deep understanding of either PyTorch with all kernel code and low level code, whole CUDA code base + LLVM compilation code or Tinygrad I would pick Tinygrad in an instant.
The code looks hard because what it is doing is hard.
If the goal is to not abstract anything away, and always look at the implementation itself to be able to change it / rewrite it, keep it tiny and it is able to beat PyTorch+CUDA+LLVM with 1000x smaller codebase, who am I to judge?
LLMs have done two things for us: made lines of code a commodity, but arguably even more important: made understanding an arbitrary amount of lines of code a commodity too.
with this in hand, large codebases that were designed in the primordial era of requiring human cognition to grok are emabarassingly antiquated.
"umm what does this mean?" -> right click -> explain with AI -> "got it".
I literally reddit-soyjacked when i saw "tackle LLVM removal". I would argue that LLVM was/is the most well thought out and well-architected piece of software that exists today... but in the age of AI... who could possibly care now? when I say "well thought out" and "well-architected", that is in the context of human eyeballs. But who needs human eyeballs when we have much more powerful computer eyeballs? who grok code way better and way faster than we do.
now, the massive line count of projects like LLVM, Linux, and Chrome become much harder to defend in the context of code re-use. Why drag in millions of code when you can get an LLM to write just the bits you care about? LLVM has a ton of optimisations that are all well and good, but how many of them actually move the needle on HPC platforms that do an awful lot of instruction re-ordering and re-scheduling anyway?
Could we get away with 5% of the codebase to do "just the bits we care about"? in the past the answer might be, "well yes but its still a lot of (manual) coding", but now - why not? Let an LLM loose on the design doc and fill in the gaps!
No code golf! While low line count is a guiding light of this project, anything that remotely looks like code golf will be closed. The true goal is reducing complexity and increasing readability, and deleting \ns does nothing to help with that.
https://github.com/tinygrad/tinygrad?tab=readme-ov-file#cont...
https://geohot.github.io/blog/jekyll/update/2025/04/22/a-way...
We can't just call people we disagree with "absolute nuts" all the time.
https://geohot.github.io/blog/jekyll/update/2023/04/11/wireh...
https://geohot.github.io/blog/jekyll/update/2025/10/19/solut...
That doesn't sound like a "bonker level insane" in any way.
Maybe you disagree with him that Elon isn't the only adult in the room but that's just disagreement. People disgreeing with you doesn't make them insane.
An example of bonker level insane would be something like: the SF DA described the murder of an elder asian as "temper tantrum" because the DA wanted to downplay the crime. Now that is truly insane: https://www.foxnews.com/us/san-francisco-da-under-fire-after... ... Bruh, it was a brutal murder. Use other words please.
> As is calling the Ds a racist party.
I clicked on the 2 links and search for the word "racist" using the Chrome find functionality. I couldn't find that word.
Not saying you are lying, but if you don't reply. You probably are.
Can you reference a more specific link? because I have follow-up questions like: is he calling every single person in the Ds racist? Is he calling only a certain group? Is he calling a certain action?
> I clicked on the 2 links and search for the word "racist"
It is in the very first link, which makes it clear you didn't even read it before arguing with me about my assessment of the writer
Ok, let me get it straight. You don't care about the DA not wanting to prosecute a brutal murder?
Wow.
Not caring about the justice of a brutal murder is quite insane.
> It is in the very first link
It is not. But maybe I missed it. Can you quote?
The fact that you are unable to give a direct quote is a telling. It would take 1 minute to do so. Yet you are playing this evasive game.
You are straight up using fake news.
This is amazing. You are tripling down despite being unequivocally wrong. Just incredible
"We’ll see how it shapes up in the next few years. But if the racists or the other racists are still running the show, we really are just cooked."
Ready to retract everything now? I gave you multiple chances to correct your error.
Why? You just said you didn't care about the justice of a brutal murder. That's pretty extremist.
Anything to help AMD (and potentially other GPU/NPU/IPU etc. chip makers) catch up with NVidia/CUDA is potentially worth money, potentially worth a lot of money, potentially worth up to Billion$...
Why?
If we have
a) Market worth Billion$
and
b) A competitive race in that Market...
then
c) We have VALUE in anything (product, service, ?, ???) that helps any given participant capture more of that market than their competitors...
(AMD (and the other lesser known GPU/NPU/IPU etc. chip vendors) are currently lagging behind NVidia's CUDA AI market dominance -- so anything that helps the others advance in this area should, generally speaking, be beneficial for all technology users in general, and be potentially profitable (if the correct deals could be struck!) by those that have the skills to do such assisting...)
Anyway, wishing you well in your endeavors, Tinygrad!