Are OpenAI and Anthropic Losing Money on Inference?
Original: Are OpenAI and Anthropic losing money on inference?
Key topics
The debate rages on: are OpenAI and Anthropic hemorrhaging money on inference costs? As commenters dissect the issue, some argue that even if they are, the companies can still turn a profit by boosting conversion rates or introducing ads, with one commenter suggesting that LLM-generated ads could be a game-changer. However, others counter that inserting ads into chatbot output is a non-starter, likening it to ad-laden email, and that costs will eventually skyrocket to unsustainable levels, pricing out consumers. Amidst the discussion, a consensus emerges that the true costs of operating these AI models go far beyond just inference compute.
Snapshot generated from the HN discussion
Discussion Activity
Very active discussionFirst comment
59m
Peak period
126
3-6h
Avg / period
16
Based on 160 loaded comments
Key moments
- 01Story posted
Aug 28, 2025 at 6:15 AM EDT
4 months ago
Step 01 - 02First comment
Aug 28, 2025 at 7:14 AM EDT
59m after posting
Step 02 - 03Peak activity
126 comments in 3-6h
Hottest window of the conversation
Step 03 - 04Latest activity
Aug 29, 2025 at 7:32 PM EDT
4 months ago
Step 04
Generating AI Summary...
Analyzing up to 500 comments to identify key contributors and discussion patterns
Want the full context?
Jump to the original sources
Read the primary article or dive into the live Hacker News thread when you're ready.
https://nerdynav.com/chatgpt-statistics/
If they insert stealth ads, then after the third sponsored bad restaurant suggestion people will stop using that feature, too.
Hell they could even just add affiliate tracking to links (and not change any of the ranking based on it) and probably make enough money to cover a lot of the inference for free users.
This article is like saying an apartment complex isn’t “losing money” because the monthly rents cover operating costs but ignoring the cost of the building. Most real estate developments go bust because the developers can’t pay the mortgage payment, not because they’re negative on operating costs.
If the cash flow was truly healthy these companies wouldn’t need to raise money. If you have healthy positive cash flow you have much better mechanisms available to fund capital investment other than selling shares at increasingly inflated valuations. Eg issue a bond against that healthy cash flow.
Fact remains when all costs are considered these companies are losing money and so long as the lifespan of a model is limited it’s going to stay ugly. Using that apartment building analogy it’s like having to knock down and rebuild the building every 6 months to stay relevant, but saying all is well because the rents cover the cost of garbage collection and the water bill. That’s simply not a viable business model.
Update Edit: A lot of commentary below re the R&D and training costs and if it’s fair to exclude that on inference costs or “unit economics.” I’d simply say inference is just selling compute and that should be high margin, which the article concludes it is. The issue behind the growing concerns about a giant AI bubble is if that margin is sufficient to cover the costs of everything else. I’d also say that excluding the cost of the model from “unit economics” calculations doesn’t make business/math/economics since it’s literally the thing being sold. It’s not some bit of fungible equipment or long term capital expense when they become obsolete after a few months. Take away the model and you’re just selling compute so it’s really not a great metric to use to say these companies are OK.
See the recent reactions to AWS pricing on Kiro where folks had a big WTF reaction on pricing after, it appears, AWS tried to charge realistic pricing based on what this stuff actually costs.
If you’re applying the same pricing structure to Kiro as to all AWS products then, yeah, it’s not particularly hobbyist accessible?
The economics are awful and local model performance is pretty lackluster by comparison. Never mind much slower and narrower context length.
$6,000 is 2.5 years of a $200/mo subscription. And in 2.5 years that $6k setup will likely be equivalent to a $1k setup of the time.
The $20 subscription is far more capable than anything i could build locally for under $10k.
Unit economics is mostly a manufacturing concept and the only reason it looks OK here is because of not really factoring in the cost of building the thing into the cost of the thing.
Someone might say I don’t understand “unit economics” but I’d simply argue applying a unit economics argument saying it’s good without including the cost of model training is abusing the concept of unit economics in a way that’s not realistic from a business/economics sense.
The model is what’s being sold. You can’t just sell “inference” as a thing with no model. Thats just selling compute, which should be high margin. The article is simply affirming that by saying yes when you’re just selling compute in micro-chunks that’s a decent margin business which is a nice analysis but not surprising.
2. “Open source” is great but then it’s just a commodity. It would be very hard to build a sustainable business purely on the back of commoditized models. Adding a feature to an actual product that does something else though? Sure.
Compare the cost of tweeting to the cost of submitting a question to ChatGPT. The fact that ChatGPT rate limits (and now sells additional credits to keep using it after you hit the limit) indicates there are serious unit economic considerations.
We can't think of OpenAI/Anthropic as software businesses. At least from a financial perspective, it's more similar to a company selling compute (e.g. AWS) than a company selling software (e.g. Twitter/X).
> That would be like saying the unit economics of selling software is good because the only cost is some bandwidth and credit card processing fees. You need to include the cost of making the software
Unit economics is about the incremental value and costs of each additional customer.
You do not amortize the cost of software into the unit economics calculations. You only include the incremental costs of additional customers.
> just like you need to include the cost of making the models.
The cost of making the models is important overall, but it’s not included in the unit economics or when calculating the cost of inference.
The models as is are still hugely useful, even if no further training was done.
Exactly. The parent comment has an incorrect understanding of what unit economics means.
The cost of training is not a factor in the marginal cost of each inference or each new customer.
It’s unfortunate this comment thread is the highest upvoted right now when it’s based on a basic misunderstanding of unit economics.
And whether companies can survive in that scenario depends almost entirely on their unit economics of inference, ignoring current R&D costs
This talent diffusion guarantees that OpenAI and Anthropic will have to keep sinking in ever more money to stay at the bleeding edge, or upstarts like DeepSeek and incumbents like Meta will simply outspend you/hire away all the Tier 1 talent to upstage you.
The only companies that'll reliably print money off AI are TSMC and NVIDIA because they'll get paid either way. They're selling shovels and even if the gold rush ends up being a bust, they'll still do very well.
IF.
If you do stagnate for years someone will eventually decide to invest and beat you. Intel has proven so.
I don’t understand why people like you have to call this stuff out? Like most of HN thinks the way I do and that’s why the post was upvoted. Why be a contrarian? There’s really no point.
How can you possibly say this if you know anything about the evolution of costs in the past year?
Inference costs are going down constantly, and as models get better they make less mistakes which means less cycles = less inference to actually subsidize.
This is without even looking at potential fundamental improvements in LLMs and AI in general. And with all the trillions in funding going into this sector, you can't possibly think we're anywhere near the technological peak.
Speaking as a founder managing multiple companies: Claude Code's value is in the thousands per month /per person/ (with the proper training). This isn't a flash in the pan, this isn't even a "prediction" - the game HAS changed and anyone telling you it hasn't is trying to cover their head with highly volatile sand.
The title of the article directly says “on inference”. It’s not a mistake to exclude training costs. This is about incremental costs of inference.
You don’t include fixed costs in the unit economics. Unit economics is about incremental costs.
Spending hundreds of millions of dollars on training when you are two guys in a garage is quite significant, but the same amount is absolutely trivial if you are planet-scale.
The big question is: how will training cost develop? Best-case scenario is a one-and-done run. But we're now seeing an arms race between the various AI providers: worst-case scenario, can the market survive an exponential increase in training costs for sublinear improvements?
Why do you think they will mindlessly train extremely complicated models if the numbers don’t make sense?
Nobody is going to pay the same price for a significantly worse model. If your competitor brings out a better model at the same price point, you either a) drop your price to attract a new low-budget market, b) train a better model to retain the same high-budget market, or c) lose all your customers.
You have taken on a huge amount of VC money, and those investors aren't going to accept options A or C. What is left is option B: burn more money, build an even better model, and hope your finances last longer than the competition.
It's the classic VC-backed startup model: operate at a loss until you have killed the competition, then slowly increase prices as your customers are unable to switch to an alternative. It worked great for Uber & friends.
Is that actually true in 2025? Presumably you have to make coupon payments on a bond(?), but shares are free. Companies like Meta have shown you can issue shares that don't come with voting rights and people will buy them, and meme stocks like GME have demonstrated the effectiveness of churning out as many shares as the market will bear.
These companies are behaving the same way. Folks are willing to throw endless money into the present pit so on the one hand I can’t blame them for taking it.
Reality is though that when the hype wears off it’s only throwing more gasoline on the fire and building a bigger pool of investors that’s will become increasingly desperate to salvage returns. History says time and time again that story doesn’t end well and that’s why the voices mumbling “bubble” under their breath are getting louder every day.
You're arguing that maybe the big companies won't recoup their investment in the models, or profitably train new ones.
But that's a separate question. Whether a model - which now exists! - can profitably be run is very good to know. The fact that people happily pay more than the inference costs means what we have now is sustainable. Maybe Anthropic of OpenAI will go out of business or something, but the weights have been calculated already, so someone will be able to offer that service going forward.
This question came up and Sam said they were profitable if you exclude training and the COO corrected him
So at least for OpenAI, the answer is “no”
They did say it was close
And that’s if you exclude training costs which is kind of absurd because it’s not like you can stop training
https://archive.is/wZslL
They quote him as saying inference is profitable and end it at that.
Are you saying that the COO corrected him at the dinner, or on the podcast? Which podcast was it?
“I think that tends to end poorly because as demand for your service grows, you lose more and more money. Sam Altman actually addressed this at dinner. He was asked basically, are you guys losing money every time someone uses ChatGPT?
And it was funny. At first, he answered, no, we would be profitable if not for training new models. Essentially, if you take away all the stuff, all the money we're spending on building new models and just look at the cost of serving the existing models, we are sort of profitable on that basis.
And then he looked at Brad Lightcap, who is the COO, and he sort of said, right? And Brad kind of like squirmed in his seat a little bit and was like, well, we're pretty close.
We're pretty close. We're pretty close.
So to me, that suggests that there is still some, maybe small negative unit economics on the usage of ChatGPT. Now, I don't know whether that's true for other AI companies, but I think at some point, you do have to fix that because as we've seen for companies like Uber, like MoviePass, like all these other sort of classic examples of companies that were artificially subsidizing the cost of the thing that they were providing to consumers, that is not a recipe for long-term success.”
From Hard Fork: Is This an A.I. Bubble? + Meta’s Missing Morals + TikTok Shock Slop, Aug 22, 2025
Uber doesn't really compare, as they had existing competition from taxi companies that they first had to/have to destroy. And cars or fuel didn't get 10x cheaper over the time of Uber's existence, but I'm sure that they still can optimize a lot for efficiency.
I'm more worried about OpenAIs capability to build a good moat. Right now it seems that each success is replicated by the competing companies quickly. Each month there is a new leader in the benchmarks. Maybe the moat will be the data in the end, i.e. there is barriers nowadays to crawl many websites that have lots of text. Meanwhile they might make agreements with the established AI players, maybe some of those agreements will be exclusive. Not just for training but also for updating wrt world news.
It’s therefore interesting that they claimed it was close: this supports the theory inferencing from paid users is a (big) money maker if it’s close to covering all the free usage and their payroll costs?
https://x.com/FinHubIQ/status/1960540489876410404
the short of it: if you do the accounting on a per-model basis, it looks much better
The leaked OpenAI financial projections for 2024 showed about equal amount of money spent on training and inference.
Amortizing the training per-query really doesn't meaningfully change the unit economics.
> Fact remains when all costs are considered these companies are losing money and so long as the lifespan of a model is limited it’s going to stay ugly. Using that apartment building analogy it’s like having to knock down and rebuild the building every 6 months to stay relevant. That’s simply not a viable business model.
To the extent they're losing money, it's because they're giving free service with no monetizaton to a billion users. But since the unit costs are so low, monetizing those free users with ads will be very lucrative the moment they decide to do so.
You would need to figure out what exactly they are losing money on. Making money on inference is like operating profit - revenue less marginal costs. So the article is trying to answer if this operating profit is positive or negative. Not whether they are profitable as a whole.
If things like cost of maintaining data centres or electricity or bandwidth push them into the red, then yes, they are losing money on inference.
If the things that make them lose money is new R&D then that's different. You could split them up into a profitable inference company and a loss making startup. Except the startup isn't purely financed by VC etc, but also by a profitable inference company.
One thing that makes me suspect inference costs are coming down is how chatty the models have become lately, often appending encouragement to a checklist like "You can check off each item as you complete them!" Maybe I'm wrong, but I feel if inference was killing them, the responses would become more terse rather than more verbose.
Think of the model as an investment.
Exactly, or a factory.
And it's a relevant question because people constantly say these companies are losing money on inference.
To me that more or less settles both "which one is best" and "is it subsidized".
Can't be sure, but anything else defies economic gravity.
Also that's not accounting for free riders.
I have probably consumed trillions of free tokens from openai infra since gpt 3 and never spent a penny.
And now I'm doing the equivalent on Gemini since flash is free of charge and a better model than most free of charge models.
Hoping for something net profitable including fixed costs from day 1 is a nice fantasy, but that’s not how any business works or even how consumers think about debt. Restaurants get SBA financing. Homeowners are “net losing money” for 30 years if you include their debt, but they rightly understand that you need to pay a large fixed cost to get positive cash flow.
R&D is conceptually very similar. Customer acquisition also behaves that way
If this were true, the stock market would have no reason to exist.
Exactly the analogy I was going to make. :)
So there are two answers: for the model providers, it's because they're spending it all on training the next model. For the API users, it's because they're spending it all on expensive API usage.
Given the analysis is based on R1, Deepseek's actual in-production numbers seem highly relevant: https://github.com/deepseek-ai/open-infra-index/blob/main/20...
(But yes, they claim 80% margins on the compute in that article.)
> When established players emphasize massive costs and technical complexity, it discourages competition and investment in alternatives
But it's not the established players emphasizing the costs! They're typically saying that inference is profitable. Instead the false claims about high costs and unprofitability are part of the anti-AI crowd's standard talking points.
I think the cache hit vs miss stuff makes sense at >100k tokens where you start getting compute bound.
> Each H800 node delivers an average throughput of ~73.7k tokens/s input (including cache hits) during prefilling or ~14.8k tokens/s output during decoding.
That's a 5x difference, not 1000x. It also lines up with their pricing, as one would expect.
(The decode throughputs they give are roughly equal to yours, but you're claiming a prefill performance 200x times higher than they can achieve.)
The only people who thought this were non-practitioners.
I know there is lots of bearish sentiments here. Lots of people correctly point out that this is not the same math as FAANG products - then they make the jump that it must be bad.
But - my guess is these companies end up with margins better than Tesla (modern manufacturer), but less than 80%-90% of "pure" software. Somewhere in the middle, which is still pretty good.
Also - once the Nvidia monopoly gets broken, the initial build out becomes a lot cheaper as well.
Expect the trend to pick up as the pool of engineers who can create usable LLMs from scratch increases through knowledge/talent diffusion.
If OpenAI didn't come along with ChatGPT, we would probably just now be getting Google Bard 1.0 with an ability level of GPT-3.5 and censorship so heavy it would make it useless for anything beyond "Tell me who the first president was".
> “If we didn’t pay for training, we’d be a very profitable company.”
There’s also a lot of comments in this thread who want LLM companies to fail for different reasons, so they’re projecting that wish on to imagined unit economics.
I’m having flashbacks to all of the conversations about Uber and claims that it was going to collapse as soon as the investment money ran out. Then Uber gradually transitioned to profitability and the critics moved to using the same shtick on AI companies.
Sam Altman also said this:
https://xcancel.com/sama/status/1876104315296968813
Uber (and Lyft) didn't starve the alternatives: they were already severely malnourished. Also, they found a loophole to get around the medallion system in several cities, which taxi owners used in an incredibly anticompetitive fashion to prevent new competition.
Just because Uber used a shitty business practice to deliver the killing blow doesn't mean their competition were undeserving of the loss, or that the traditional taxis weren't without a lot of shady practices.
And lifetime profits for Uber are still at best break even which means that unless you timed the market perfectly, Uber probably lost you money as a shareholder.
Uber is just distorted in valuation by its presence in big US metro areas (which basically have no realistic transportation alternative).
Advertising is now a very very locked in market and will take over a decade to shift even a significant minority it into OpenAIs hands. This is not likely the first or even second monetization strategy imo.
But I’m happy to be wrong.
Can you elaborate? You’ve sparked my curiosity.
"No you're not, WE are digging a trench!"
Yes fine, but "I am as well".
Sheesh. Also I, personally, do and lead the work of taking the wallet share. So I will stick with "I" and would accept any of my team saying the same.
He’s in the habit of lying, so it would be remiss to take his word for it.
I think most folks understand that pure inference in a vacuum is likely cash flow positive, but that’s not why folks are asking increasingly tough questions on the financial health of these enterprises.
If they weren’t losing money, they wouldn’t be spending enough on R&D. This isn’t some gotcha. It’s what the investors want right now.
WeWork’s investors didn’t want them to focus on business fundamentals either and kept pumping money elsewhere. That didn’t turn out so well.
This sounds incorrect, you only process all tokens once, and later incrementally. It's an auto-regressive model after all.
From that point on every subsequent tokens is processed sequentially in autoregressive way, but because we have the KV cache, this becomes O(N) (1 token query to all tokens) and not O(N^2)
Or are these costs just insignificant compared to inference?
> $20/month ChatGPT Pro user: Heavy daily usage but token-limited
ChatGPT Pro is $200/month and Sam Altman already admitted that OpenAI is losing money from Pro subscriptions in January 2025:
"insane thing: we are currently losing money on openai pro subscriptions!
people use it much more than we expected."
- Sam Altman, January 6, 2025
https://xcancel.com/sama/status/1876104315296968813
Saying that is the equivalent of him saying "our product is really valuable! use it!"
> We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.
Source: https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat...
His possible incentives and the fact OpenAI isn't a public company simply make it hard for us to gauge which of these statements is closer to the truth.
Profitable on inference doesn't mean they aren't losing money on pro plans. What's not compatible?
The API requests are likely making more money.
The question is then whether SaaS companies paying for GPT API pricing are profitable if they charge their users a flat rate for a time period. If their users trigger inference too much, they would also lose money.
This sort of thing used to be called fraud, but there's zero chance of criminal prosecution.
it is comical that something like this was even uttered in the conversation. It really shows how disconnected the tech sector is from the real world.
Imagine Intel CEO saying "If we didn't have to pay for fabs, we'd be a very profitable company." Even in passing. He'd be ridiculed.
As a counterpoint, if OpenAI were actually profitable at this early stage that could be a bad financial decision - it might mean that they aren't investing enough in what is an incredibly fierce and capital-intensive market.
I don’t buy the logic that he will “scam” his investors and run away at some point.
If OpenAI goes down tomorrow, he will be just fine. His incentive is to sell the stock, not actually build and run a profitable business.
Look at Adam Neumann as an example of how to lose billions of investor dollars and still walk out of the ensuing crash with over a billion.
https://en.wikipedia.org/wiki/Adam_Neumann
His strategy is to sell OpenAI stock like it was Bitcoin in 2020, and if for some reason the market decides that maybe a company that loses large amounts of cash isn't actually a good investment... he'll be fine, he's had plenty of time to turn some of his stock into money :)
I guess if other people buying stock in your company is what enables you to have a super high salary (+ benefits like company plane, etc), you are still kinda selling stock though, and honestly, having considered the "start a random software company aligned with the present trend (so ~2015 DevOps/Cloud, 2020 cryptocurrency/blockchain, 2024 AI/ML), pay myself a million dollar a year salary and close shop after 5 years because 'no market lol'" route to riches myself, I still wouldn't consider Altman to be completely free of perverse incentives here :)
Still, very glad you pointed that out, thanks for sharing that information ^^
The cheap usecase from this article is not a trillion dollar industry and absolutely not the usecase hyped as the future by AI companies, that is coming for your job.
318 more comments available on Hacker News