In a First, Google Has Released Data on How Much Energy an AI Prompt Uses

Me sipping coffee with 1000 times more embodied energy while reading the article.

Same could be said for anything: human operator, google search algos pre-AI (where indexing = training), etc.

Right, and the scraping, the extra storage required, the manufacture of all the extra GPUs, etc. This is them hoping people don't understand that the query is only one part of it and aren't curious enough to ask further questions.

One random preprint I found (https://arxiv.org/html/2505.09598v2) estimates inference is 90% of the total energy usage. From some googling around, everything seems to agree that inference dominates (or is at least comparable to) training on the large commonly-used models.

I was less surprised that inference dominates training after I read that chatgpt is serving billions of requests per day.

Many papers on large models make claims about the energy used. Google has consistently said that training and inference use about the same fraction of their data center resources.

> an LLM would have linked to the original study

There is a non-trivial chance that the LLM would've added a link to _something_, but links/references seem like a very common thing to hallucinate, no?

> Since a human uses ~100W of power, the .24Watt-hours of energy > for an AI prompt is about 40human-seconds of energy.

.24 Watt-hours is 864 Watts for one second, so a 100W human takes ~9 seconds for that output.

>And unlike the human who spent multiple hours writing that article, an LLM would have linked to the original study

Or made up a fake citation, complete with fake or unrelated author names, on the spot

>And unlike the human who spent multiple hours writing that article, an LLM would have linked to the original study:

This is why journalists are nearly universally hostile towards AI.

Great, now consider the human's marginal power consumption for a task instead of their baseline consumption.

Unless your point is that we can kill a bunch of humans to save energy...?

I bet the human had LLM help. I bet it didn't take hours to put together.

Unless you're arguing that people should kill themselves to save their 100W of power, the future energy use of every alive human is already committed and doesn't matter.

Not to mention that the energy should also include all the extra energy spent on making converting energy into a form that is usable by humans (ie. food). There is probably at least an order of magnitude.

The issue is that now we're over-utilizing prompts everywhere. Every search engine query. Every keypress in an AI editor. Every new website makes queries in the back. Analyze emails. etc

So it's not just about "the one query you ask ChatGPT about what you should write your mum to say you're not coming for Thanksgiving"

It's rather that an AI query is 0.24Wh, but that we are now using thousands of those per users per days, and that we globalize it at the scale of the planet, so 7 billion users... and this becomes huge

Consider this your second upvote from me, for the second paragraph.

Thanks, I was scouring the article looking for the original study, could not believe that they didn't have it linked.

Did you use an LLM to do that arithmetic?

Attempts to reduce human experience down to a single number always seem so wrong-headed. A common one is when articles say "billions of dollars of productivity wasted to X". As if you can just replace any human activity for any human activity at any time and get the same results. I know companies would like to imagine that every employee is a fungible cog, but it seems pretty dismal as a world view.

14 years of progress on energy efficiency might also have an impact here...

> the takeaway is that ordinary Google searching is also quite energy-intense.

A related takeaway should be that machine inference is pervasive and has been for years, and that defining "AI" to mean just chatbots is to ignore most of the iceberg.

At that point was Google already using deep learning for search? I'd guess the number fluctuated a bit during the rollout of this kind of feature

Around 2008 a core step in search was basically a grep over all documents. The grep was distributed over roughly 1000 machines so that the documents could be held in memory rather than on disk.

Inverted indices were not used as they worked poorly for “an ordered list of words” (as opposed to a bag of words).

And this doesn’t even start to address the ranking part.

They just be doing something crazy because any time I query my local llm the lights in my office dim and the temperature rises a few degrees. Definitely far more energy than running the microwave for 1 second.

.3 Wh is 1080 joules. A liter of gasoline contains over 30 million joules. So this is like .034 milliliters of gasoline. But with grid power so even less than that since gasoline is very inefficient.

With google serving AI overviews, now an average search query should cost more? Compute is getting cheaper but also algorithms getting more and more complex, increasing compute?

>Even before the AI boom, shouldn't we have a bunch of extra capacity under construction, ready for EV driving, induction stoves and heat-pump heating?

When it comes to the EV, the answer is simple: the EV takeover "by 2030" was 100% wishful thinking - the capacity is nowhere near there, starting from scaling the battery production, never mind the charge capacity.

That's just a press release. Here's the actual PDF of the actual tech report: https://services.google.com/fh/files/misc/measuring_the_envi...

> When I hear reports that AI power demand is overloading electricity infrastructure ...

It feels like dog-whistle tactics. "Aren't the technology companies bad for the environment!" "What about the water usage?" "What about the electricity?"

For me the peak of this is complaining about water consumption at the Dalles datacentre [0]. The buildings are next to the Colombia river and a few miles away from the Dalles Dam [1] which generates an average of 700MW. The river water should be used for cooling, taking out some of the water, warming it up by a few degrees and returning it to the river; one might argue that this is simply returning the heat to the river that would have come from the water flowing downhill.

[0] https://www.oregonlive.com/silicon-forest/2022/12/googles-wa...

[1] https://en.wikipedia.org/wiki/The_Dalles_Dam

I'm not sure why they would report on the median prompt, and not the average, which would give a better sense of (well average) consumption in this case

> When I hear reports that AI power demand is overloading electricity infrastructure, it always makes me think: Even before the AI boom, shouldn't we have a bunch of extra capacity under construction, ready for EV driving, induction stoves and heat-pump heating?

One of the problems might be that a data center puts a lot of demand into a small area and it needs that power soon.

Those other things are being phased in over time, so we only need modest annual capacity growth to deal with them, and they are spread out.

>~If they're running on, say, two RTX 6000s for a total draw of ~600 watts, that would be a response time of 1.44 seconds. So obviously the median prompt doesn't go to some high-end thinking model users have to pay for.

You're not accounting for batches for the optimal gpu utilization, maybe it can takes 30 seconds but it completed 30 requests.

Yes we should, which is yet another example of ineffective vision from the prior administration and outright incompetence and sabotage by the current one, regarding renewables and transmission infrastructure.

Should we? Yes. Do we? Nope.

China however is continuously providing double the energy they currently require, only to notice that every two years or so it actually did end up getting used.

TPUs are more efficient at LLMs because they do more TOPS/KWh.

"OpenTPU: Open-Source Reimplementation of Google Tensor Processing Unit (TPU)" (2025) https://news.ycombinator.com/item?id=44111452

"A Comparison of the Cerebras Wafer-Scale Integration Technology with Nvidia GPU-based Systems for Artificial Intelligence" (2025) https://arxiv.org/html/2503.11698v1

From https://news.ycombinator.com/item?id=44648575 :

> "Next-generation datacenters consume zero water for cooling" (2024) https://news.ycombinator.com/item?id=42376406

>> this design will avoid the need for more than 125 million liters of water per year per datacenter

"Microsoft’s Datacenter Community Pledge: To build and operate digital infrastructure that addresses societal challenges and creates benefits for communities" (2024-06) https://blogs.microsoft.com/blog/2024/06/02/microsofts-datac... :

> We will design and operate our datacenters to support society’s climate goals and become carbon negative, water positive and zero waste before 2030. [...]

> By 2025, we will procure 100% renewable energy on a global scale, both significantly expanding and decarbonizing local electricity grids.

> Our datacenter designs are more water efficient than traditional enterprise datacenters, and our plan by 2030 is to replenish more water than we consume locally.

Here's this about CNT cooling:

"Cyberpower begins selling desktop PCs with carbon nanotube CPU cooling" (2025) https://news.ycombinator.com/item?id=44899495

"A carbon-nanotube-based tensor processing unit" (2024) https://news.ycombinator.com/item?id=41322070

Graphene semiconductors should be at least 10X more energy efficient; but how much less water would graphene-based chips waste?

Maybe, but I would suggest that larger factors arise from utilization and overhead. These are going to be the competitive advantages that AI operators need to win the arms race.

Gallons and teaspoons ... the units of measurements in the 21. century!

But what about the training cost?

Yes, you did.

The fundamental flaw in AI energy/water doomerism has always been that energy costs money, water costs money, real estate costs money, but AI is being given away for free. There is obviously something wrong with the suggestion that AI is using all the energy and water.

Sure, but now multiply that "new use-case" that we weren't consuming before by 7 billion humans on the planet... that's the issue

We can always keep adding new stuff and say each time "oh but it's small"... sure, but if we keep adding more, altogether it becomes huge

Using AI to calculate a trivial multiplication...

also an average would make much more sense here than a median, but we can assume it's much higher and that might be why they don't communicate it?

Yupp, plus the fact that I unwillingly send prompts out by visiting a website that now has a chatbot nag me; Among other annoyances in the economy of scale that say "if it is cheap, just use more of it".

To set appropriate goal posts, it shouldn't just the total water/energy usage from start to finish: it's that, relative to the local regions they're being extracted from. Are those local regions at risk of being overused?

Focusing on water, Google does reports on their total water withdrawal at all their data centers. It was around 10 billion gallons per year in 2024. That's around 0.03% of water usage in the US (that's just for rough context - not all of Google's data centers are in the US). I don't think that's an eye-popping amount.

But when you consider that more than 1 billion gallons came from Council Bluffs, IA, you have to make sure that's being done responsibly, and that the local ecology can support it. Google admits that 28% of their water use comes from sources with "medium to high risk of depletion or scarcity." [0]

[0]: https://www.gstatic.com/gumdrop/sustainability/google-2025-e...

> If the tech was lean at it's heart in terms of energy consumption, we wouldn't be seeing that or even the private nuclear reactor arms race by big tech.

Eh, I somewhat disagree with this. The US energy grid has had almost no extra capacity for a long time. A lot of this was due to efficiency (not a bad thing) and little industrial growth (not a great thing) in the country. Data centers themselves, I don't think are the biggest cause of the issues, but the distribution grid. We've had tons of problems around distribution with new energy sources coming online, but issues distributing the power to where we need to.

We see things like private power plants, not because we can't generate power, but because we absolutely suck at distribution of power.

How in the world do they claim to have 0.24Wh per query? MIT estimates the LLama 3.1 405B model uses 1.86Wh.

GPT-4 is over a trillion parameters. Is there any reason to think they have 2.5x more parameters but somehow use 6x less energy?

That's surprisingly energy efficient, for a technology that seemed like magic back in 2022.

The median prompt being a google search, not agentic coding for example.

I also think this misses a bit the forest from the trees, emissions need to go down across all industries and tech is actively increasing theirs.

Destroying the water originates as a NIMBY talking point (to stop data centers) that was co-opted by anti-capitalist groups (higher membership rates among GenZ) as a boogeyman to blame for why AI is bad.

It's probably best (for your mental health) to not ask these questions in earnest.

Water shortages are highly local problems. Sure, running a data center in Arizona might have some genuine water concerns. But even then, it can be mitigated by changes like using wastewater. The Palo Verde plant does that for its heat exchangers.

Data centers use evaporative cooling.

Data centers don't just heat up the water and return it - they evaporate the water into the atmosphere (yes, I know, the H2O still exists, but it's in a far less usable form when it's gaseous atmospheric H2O)

Water molecules are definitely not being destroyed, but they are often completely removed from a place where they were useful.

Aral Sea was destroyed by farm irrigation: https://www.youtube.com/watch?v=oBjDF4SFIlk

I was also surprised when someone asked me about AI's water consumption because I had never heard of it being an issue. But a cursory search shows that datacenters use quite a bit more water than I realized, on the order of 1 liter of water per kWh of electricity. I see a lot of talk about how the hyperscalers are doing better than this and are trying to get to net-positive, but everything I saw was about quantifying and optimizing this number rather than debunking it as some sort of myth.

I find "1 liter per kWh" to be a bit hard to visualize, but when they talk about building a gigawatt datacenter, that's 278L/s. A typical showerhead is 0.16L/s. The Californian almond industry apparently uses roughly 200kL/s averaged over the entire year -- 278L/s is enough for about 4 square miles of almond orchards.

So it seems like a real thing but maybe not that drastic, especially since I think the hyperscaler numbers are better than this.