AI Is Thirsty

Each chat with a large language-model is like dumping a bottle of water on the ground

Today I used ChatGPT to get some help making a browser plugin. I posted my queries, then watched as the code and text spilled down the screen. This is the part of large language-models that I dig! As a hobbyist developer, getting suggestions of customized lines of software can be a powerful way to learn.

But as it turns out, using ChatGPT consumes a lot of an unexpected resource:

Water.

The code wasn’t quite what I was looking for, so I chatted with ChatGPT for 15 minutes or so, slowly coaxing it to revise. By the time I was done, we’d gone back and forth about 20 times.

And during that exchange? Microsoft’s servers probably used about as much water as if I’d just bought a half-liter bottle … and spilled it on the ground.

AI, it turns out, is incredibly thirsty tech — ploughing through torrents of fresh water every day. Given that we’re likely to see large-language-model AI woven into ever more apps and appliances these days, it’s worth pondering just how much water our booming use of AI will consume.

Why precisely does large-language-model AI require water? Back in April, a group of researchers pondered this question as they created an estimate of AI’s water consumption. As they note in their paper (which is here free in full), the main use of water is when tech firms train their AI, and when the firms are running inferences (i.e. when you, I or anyone else interacts with the model).

Tech firms like Microsoft and Google and Meta do all that training (and inferring) on their huge computational farms. That computation requires a ton of energy, which generates heat. To remove that heat from server farms, the tech firms generally use cooling towers, where water is evaporated to send the heat out into the outside world. That evaporation? That’s how AI consumes water. It is, it’s worth noting, mostly all freshwater.

Tech firms do not publish specific stats on how much freshwater they use for different forms of computation. So the academics did some estimates. They calculated how much energy it would take to train one of the well-known large language-models (and to run inferences using it). (Here, there actually is some useful public info: As Google explained in this paper, when they created their large language-model LaMDA it required 57.7 consecutive days of training.) Then the academics used metrics of efficiency for the cooling mechanism to figure out precisely how much water would be needed.

The upshot, they figured, is that …

Training GPT-3 in Microsoft’s state-of-the-art U.S. data centers can directly consume 700,000 liters of clean freshwater, enough for producing 370 BMW cars or 320 Tesla electric vehicles … Moreover, training GPT-3 is also responsible for an additional off-site water footprint of 2.8 million liters due to electricity usage (assuming water usage efficiency at the U.S. national average level 1.8L/kWh [32] and power usage effectiveness 1.2). Thus, combined together, this would put GPT-3’s total water footprint for training at 3.5 million liters if trained in the U.S.

And when you’re chatting with a large language-model?

ChatGPT needs to “drink” a 500ml bottle of water for a simple conversation of roughly 20–50 questions and answers, depending on when and where ChatGPT is deployed.

Are these calculations reasonable? It’s hard to tell; because tech firms (and the jurisdictions that supply them with water) don’t publish granular data on water usage (more on this below), the authors in this paper rely on third-party estimates of how much water it takes to cool a data center. Those estimates themselves could certainly be off! For the rest of this post I’ll be using the figures the scholars came up with, but I’d much prefer data from inside the firms themselves.

Obviously, as this analysis shows, it takes a ton of water to train a model.

But what really struck me was that estimate of how much water it takes to use a model. Consuming a half-liter bottle of freshwater for a 20-to-50-turn conversation? That’s not much per conversation … but the sheer number of such conversations has exploded.

Consider that in June 2023 there were, as per an estimate by similarweb, about 1.6 billion user visits to ChatGPT. The folks at similarweb further estimate that the average ChatGPT session was probably just under 8 minutes long. That would mean that, in June, there were 12.8 billion minutes of conversation.

I don’t know how many questions-and-answer exchanges happen per minute, when someone is talking to ChatGPT. (I haven’t been able to find a good estimate, either.) But just for the heck of it, let’s posit that during the average AI chat session, there’s one question and one reply per minute. That’d be 12.8 billion questions and answers per month. If the model requires a 500ml bottle of water for 50 of these exchanges, that would mean that ChatGPT conversations consumed 128 million liters of freshwater in one single month.

Even if my (admittedly very crude) estimates are off by one or two orders of magnitude, one can see how using a model consumes far more water than merely training it.

And usage of these models is still in its early days! They’re probably going to be woven into evermore digital interactions. I imagine we’ll soon see firms using them to power banking interactions, robotics, web-site interfaces, ecommerce, and more. Hell, you’ll probably be chatting with your damn microwave soon, via large language-model AI. That means energy and water usage will erupt, too.

Freshwater is an increasingly scarce resource. So one obvious question out of all this is how we’ll reduce the deep, eldritch thirst of these models for fresh and clean H2O.

The academics have some thoughts on that. They suggest that AI firms could time their training to reduce water demand. If they trained the models at night — when temperatures are coolest — the efficiency of the cooling towers would be better than during the hot daytime. (Much better, in fact: The authors estimate the worst-possible training time uses 3X the water of the best-possible training time.)

The problem is, this can conflict with other environmental goals, like reducing CO2 use. If you wanted your AI to emit as little CO2 as possible, you’d do your training during the bright daylight, when solar power is most plentiful. As the authors point out, to reduce the CO2 you “follow the sun”; to reduce water usage you “unfollow the sun”. The other problem here is that people using the models are all over the world, so there’ll always be a customer base demanding inferences when the sun is up.

These aren’t easy circles to square.

Now, tech firms themselves have promised to improve their overall water usage by 2030, so much so that they become “water positive”. Microsoft announced in 2020 that it “will replenish more water than it consumes on a global basis”. Reading their list of interventions — which includes things like rainwater collection on premises and “adiabatic cooling” that uses air and not water — sound terrific. But the savings they predict are only on the order of hundreds of millions of liters of water per year. If the academics’ numbers are anywhere close to correct, today’s increasing use of models would eat up those savings several times over.

Of course, AI is only the tip of the iceberg when it comes to tech firms’ use of water. All cloud computation requires cooling, right? So tech firms are constantly shopping around for states and towns that will sell them freshwater at massive scale for the lowest possible price, as Gizmodo notes …

Water consumption issues aren’t limited to OpenAI or AI models. In 2019, Google requested more than 2.3 billion gallons of water for data centers in just three states. The company currently has 14 data centers spread out across North America which it uses to power Google Search, its suite of workplace products, and more recently, its LaMDa and Bard large language models. LaMDA alone, according to the recent research paper, could require millions of liters of water to train, larger than GPT-3 because several of Google’s thirsty data centers are housed in hot states like Texas; researchers issued a caveat with this estimation, though, calling it an “approximate reference point.”

Now! This offers a civic lever we can use to impel tech firms to make their computation less energy- and water-intensive. If we, the public (via our local and state governments) are the ones selling the community-owned and -managed water, then we could strike a harder bargain here, and charge much more. Putting a stiff price on externalities is basically the only reliable way to get companies to price them into their cost of doing business.

Alas, I won’t hold my breath on this one. Cities have been stumbling over themselves for two decades now to give sweetheart energy and water deals to tech giants, under the frequently-dodgy assumption that it’ll generate cornucopian local prosperity. City officials seem to get Stockholm Syndrome around tech firms. (Consider this object lesson: In the city of The Dalles in Oregon, Google’s use of water had nearly tripled in the last five years. But the city wouldn’t release these figures when asked; indeed, the newspaper The Oregonian/Oregon Live had to sue the city until a judge’s ruling forced city bureaucrats to make the data public.)

In their defense, tech giants frequently argue that they are, in the grand scheme of things, not particularly profligate consumers of water. As Google pointed out, its 2021 use of water was comparable to golf courses …

Overall, Google’s consumption of water across the US during 2021 accounted for 3.3 billion gallons (12.4 billion liters), with “additional global locations” (ie, the rest of the world) representing an extra 971 million gallons (4.4 billion liters).

This may sound like a lot (and it is), but to put it in perspective, Google claims that the total annual water consumption of its datacenter operations is comparable to the water footprint of 29 golf courses in the southwest US.

Fair enough, though I’d note that they picked a rather terrible comparison here, because a) golf is on an absolute collision course with the challenge of increasingly-scarce freshwater supplies, so it’s not exactly an example of a sustainable industry; and what’s more, b) the number of golf courses is not rapidly increasing, while the usage of AI models is.

That said, it’s entirely possible that with sufficient pressure — and again, cities and states, I’m looking at y’all and your pricing of water — tech firms could be induced to become radically more efficient in their water usage.

This has happened with energy! Because energy (even provided under sweetheart deals) costs money, tech firms have over the last few decades worked steadily to improve the energy efficiency of their data centers. In 2010, data centers used about 1% of the world’s energy; by 2020, data centers were processing six times as much data, but they were still only using barely above 1% of global energy, as calculations in Science found. That’s pretty impressive.

Granted, our supply of water is quite different from our supply of energy. We have, in theory, a totally massive amount of energy on tap; we’ve barely begun to tap how much the sun gives us every day. In contrast, there’s nothing currently supplying the planet with new freshwater, at scale.

So the challenges of our thirsty AI — and thirsty computation in general — are still with us.

(Enjoyed this one? Then pour yourself a nice cool glass of water and hunt down that “clap” button. You can whack it up to 50 times, per story!)

I publish on Medium two times a week; follow me here to get each post in your email — and if you’re not a Medium member, you can join here, and about half your monthly fee goes directly to supporting my writing on Medium, while also giving you access to everything else on the site.

I’m a contributing writer for the New York Times Magazine and Wired, and author of “Coders”. Follow me on Mastodon or Instagram.