avatarJano le Roux

Summary

The article discusses the impending limitations of AI growth due to challenges with data scarcity, energy consumption, scaling issues, and resource management.

Abstract

The author initially believed in the unstoppable progress of AI, inspired by achievements like solving protein folding. However, over the past six months, the focus has shifted to the constraints AI faces, particularly in data quality, energy demands, and the scalability of technology. The article outlines three core problems: the scarcity of fresh, high-quality data; the significant energy consumption required for AI operations; and the diminishing returns of scaling up AI models. It suggests that without addressing these issues, AI's progress could plateau. The author emphasizes the importance of nuclear energy as a stable power source for AI, the need for innovative solutions beyond brute-force scaling, and the potential for a shift towards more efficient, smaller-scale AI models. The piece concludes by highlighting the strategic advantage of owning private energy sources and the potential for a divide between tech giants and smaller labs.

Opinions

  • The author believes that AI's progress is not limitless and will encounter a plateau without addressing core issues.
  • There is a concern that the internet's supply of useful, human-created content for AI training is finite and depleting.
  • The article expresses skepticism about the effectiveness of recycling AI-generated content for further training.
  • It is suggested that the current growth in AI is unsustainable without significant advancements in energy production, particularly through nuclear power.
  • The author questions the continued effectiveness of simply increasing the size of AI models, pointing out the law of diminishing returns.
  • There is an opinion that the future of AI may rely more on qualitative improvements, such as new neural architectures and training strategies, rather than quantitative increases.
  • The piece highlights the fragility of the AI ecosystem, where shortages in resources like skilled engineers, hardware, or electricity could cause significant setbacks.
  • The author predicts a potential shift towards smaller, more optimized AI models as a response to the challenges of scaling up.
  • Owning private energy sources is seen as a strategic move by tech giants, potentially creating a competitive gap in the AI industry.

Why AI’s Growth Will Hit A Wall Very Very Soon

And how this shifted my view on tech progress in the last half-year.

CDD20

I used to think AI was unstoppable. (Just like Sam Altman.)

I remember reading about AI solving protein folding and thinking, ‘This is unstoppable.’ But six months later, the conversation shifted to energy costs and scaling problems.

It felt like this technology had no upper limit. But then I started looking deeper. I realized that even the biggest machines can grind to a halt without the right fuel.

Over the past half-year, I’ve become convinced that AI will reach a plateau if it doesn’t solve a few core problems.

  • Data.
  • Energy.
  • Scale.

These are the pillars of AI’s future, and each one has cracks. Let’s break down why I think these cracks matter and what might happen if we don’t patch them.

I’m not a pessimist by nature.

But I can’t help but wonder if Sam Altman ever has sleepless nights about this.

I believe that knowing the limits of a thing is the best way to make it better. Here’s where I see the breaking points and why they’ve changed how I view tech progress.

The data drought

Data is the lifeblood of AI.

Big models need massive amounts of fresh, high-quality text, images, and more to learn. Much of this data comes from the internet. But there’s a problem: that rich source is running out of “clean” information. It’s not that the internet will vanish. It’s that useful human-created content is finite.

I used to assume there would always be an endless stream of new text, research articles, blog posts, and social media chatter. But once you dig in, you see the limits:

  • AI can’t just recycle its own output: When AI pulls from AI-generated text, it’s like making a copy of a copy. Each iteration loses clarity. Distortion creeps in.
  • Fresh data takes time and effort to create: People need to write more, discover more, and share more. But we’re already saturating many topics.
  • Duplicates don’t help as much as unique, high-quality info: An AI model sees the same snippet over and over, and eventually, the gains shrink.

This hits home for me because I’ve started noticing the same facts repeated online. It feels like we’re swirling in an echo chamber. If AI is forced to feed on that echo, progress stalls.

The energy squeeze

AI doesn’t just need data. It devours power. Training the biggest AI models is like running a small factory around the clock. And that factory only grows as more developers and companies jump into machine learning.

Six months ago, I didn’t worry much about power. I figured we’d keep scaling up renewables like solar and wind. But when you look at the raw numbers, you see the strain:

  • Our electric grids are already stretched.
  • Renewables are helpful, but they can’t ramp fast enough to fuel exponential AI growth.
  • Fossil fuels are a fallback, but they damage the planet (and have their own limits).

Why Nuclear makes sense

Nuclear power is the clear path if we want stable, large-scale energy without pumping out more carbon.

But nuclear has its own set of hurdles — cost, public perception, regulatory hoops. If we don’t invest in it (or something equally robust), I see AI’s expansion getting capped by electricity before it’s capped by talent.

Lately, I find myself looking at stories about new nuclear startups or advanced reactor designs. It feels like we’re standing at a crossroads. If we pick the right path, AI thrives. If we pick the wrong one, the AI wave might slow to a ripple.

Bigger isn’t always better

For years, the formula seemed straightforward:

Throw more data and more computing power at a problem, get better AI.

That’s what Sam Altman did.

That’s what Google did.

That’s what Anthropic did.

That’s what Elon is doing.

But scaling up isn’t yielding the same breakthroughs anymore. It reminds me of building endless highway lanes to fix traffic — sooner or later, you still hit congestion.

I’ve worked with a few tech teams where the motto was “Let’s just add more nodes, more GPUs, more time.” But each leap in size offered smaller boosts in performance. That’s when I realized we need to be smarter, not just bigger.

The next leap in AI might come from things like new neural architectures, improved training strategies, or quantum computing. But a raw brute-force approach seems to be hitting diminishing returns. This reframed how I look at AI labs that brag about model size. I wonder how much of that is marketing versus real progress.

Resource juggling

Imagine a giant puzzle. Each piece is crucial, and any missing piece means the entire puzzle is incomplete. AI is that puzzle:

  1. Data centers — Need land, cooling, and top-tier infrastructure.
  2. Hardware — GPUs or specialized chips don’t build themselves. Production lines are maxed out.
  3. Cooling systems — Heat is a killer. You need serious solutions that consume water and power.
  4. Skilled engineers — People who can design, train, and deploy AI models. They’re in short supply.
  5. Electricity — Enough juice to keep everything running.

If one piece falters — say we can’t expand data centers fast enough — everything slows. The same goes for hardware supply. During the crypto mining boom, GPUs became scarce. Now imagine a worldwide AI boom that dwarfs that.

I once visited a data center run by a large tech company. It was massive: rows of servers humming, fans blasting, staff rushing to handle issues. It felt futuristic, but also delicate. A single failure in that chain — power, cooling, supply — could bring the entire operation down.

The Human Element

We forget how many people are needed to run these systems. Hiring top AI researchers is expensive. Recruiting talented ops teams is tough. If the workforce can’t keep up, it doesn’t matter how advanced your hardware is. You can’t scale.

I see it like spinning plates. If you spin too many, eventually one falls. That’s my biggest worry about AI’s future. We’re spinning a dozen plates — energy, data, hardware, water usage, engineers — and if just one hits a snag, the rest might crash.

Scaling vs. reality

Big models are impressive. We hear about language models with billions of parameters. But each leap in scale demands:

  • More storage space for parameters.
  • Higher computing power to run and train.
  • Specialized teams to keep everything from imploding.

At some point, the returns may not justify the costs. I wonder if we’ll hit that moment soon, where we say, “Does adding another billion parameters really make a difference?”

There’s a real possibility we’ll see a shift toward smaller, more optimized models. Why burn rivers of power and money for a marginal improvement?

Control your power, control your destiny

Some tech giants are buying solar farms or wind farms. Others are investing in nuclear or hydropower. Owning the power source directly means they’re not at the mercy of public grids. They can run big training jobs without hitting insane bills or blackouts.

In one sense, I get it. If you rely on external power, you risk shortages or price hikes. But when big players hoard clean energy, smaller labs might struggle to keep up. It sets the stage for a massive divide.

I’ve seen a few headlines about tech companies planning to build entire “AI training campuses.” These would have data centers, custom hardware, and direct lines to private energy. That’s a fortress strategy. It’s like building your own city with its own power plant. If you can do it, you’ll likely outpace those who can’t.

My five takeaways on AI’s upcoming limits:

  1. Fresh, accurate data is drying up.
  2. Energy costs can make or break AI growth.
  3. Chasing every AI domain at once is too expensive.
  4. Deals for exclusive data split the field into haves and have-nots.
  5. Owning your power source creates a huge advantage.

What part of this makes you rethink AI the most?

Artificial Intelligence
Business
Energy
Llm
ChatGPT
Recommended from ReadMedium