🧠 Is AI’s Computing Demand Sustainable?

Subscribe | AI Toolkit | Meet The Team

Welcome back AI prodigies!

In today’s Sunday Special:

• 📜The Prelude

• ⚙️How GenAI Uses Compute

• 📦Demand vs. Supply

• 🤔What’s the Verdict

• 🔑Key Takeaway

Read Time: 7 minutes

🎓Key Terms

• Generative AI (GenAI): When AI models create new content such as text, images, audio, video, or code.

• Parameters: Internal settings like weights, biases, and numbers that help AI models understand and predict words.

• Graphics Processing Units (GPUs): Specialized computer chips capable of performing mathematical calculations simultaneously.

• Floating Point Operations per Second (FLOPs): How many operations (i.e., addition, subtraction, multiplication, or division) a computer can solve within a second.

🩺 PULSE CHECK

📜THE PRELUDE

For centuries, Human Labor fueled economies. In recent decades, Knowledge Workers have driven progress. Now, AI is poised to do the same.

Historically, we’ve faced human constraints; Production Workers could only assemble so many goods in a factory, and Financial Analysts could only analyze so much data in a spreadsheet.

AI faces a different constraint: Compute, which is how quickly and efficiently computers can perform calculations and process data.

This year, Google will spend more money on Compute than employee salaries. As one of the world’s most valuable Big Tech companies, Google symbolizes a profound shift in how humans create economic value—from Labor and Intelligence to AI.

Today, GenAI consumes the most Compute. So, how exactly does GenAI rely on Compute? Why does a lack of Compute hinder GenAI progress?

⚙️HOW GENAI USES COMPUTE

The size of the largest AI models has grown 2.8x every year since 2018. In 2024, OpenAI’s GPT-4 leveraged 1.8 trillion Parameters. By 2028, the largest AI models will likely reach 15 trillion Parameters, which is more than 7x Compute of GPT-4.

But the size of AI models isn’t the only factor influencing the demand for Compute. Here are the top three use cases of Compute and roughly how much each one uses:

1. Training (i.e., 70%): This process involves feeding an AI model large amounts of high-quality datasets to help it perceive, interpret, and learn from the data. It requires tons of hardware, time, and mathematical calculations. For example, GPT-4 was trained on around 25,000 NVIDIA A100 Tensor Core GPUs over nearly 100 days, performing 2.15 × 1025 FLOPs, which is over 100x more mathematical calculations than the number of stars in the observable universe.

2. Fine-Tuning (i.e., 15%): This process involves taking a pre-trained AI model and adjusting it to fit your specific use case better. For example, Reinforcement Learning From Human Feedback (RLHF) relies on human feedback to teach AI models to align with human preferences. Every time ChatGPT responds to your question, you can click the Thumbs Up button below the response to support OpenAI’s RLHF efforts.

3. Inference (i.e., 15%): This process refers to everything that happens after you enter your prompt. For example, Reasoning Engines like OpenAI o3 use Chain-of-Thought (CoT) to break down complex problems into manageable steps.

📦DEMAND VS. SUPPLY

The demand for Compute is enormous and ever-increasing, with no end in sight. The BCG Henderson Institute (BHI) outlined an aggressive scenario for GenAI’s impact on Compute through 2028.

Compute Demand?

When ChatGPT first launched in November 2022, it reached 100 million active users in just two months. Now, ChatGPT has passed 400 million weekly active users, and GenAI adoption across Big Tech companies has skyrocketed.

Fortunately, most businesses using GenAI don’t engage in computationally intensive Training. Instead, they use existing GenAI models for Inference. Since Inference doesn’t require the same cutting-edge GPUs as Training, it can be distributed across less specialized hardware. Despite this, Inference workloads are expected to eclipse Training workloads by 2028 because of the sheer scale of user adoption.

Meta Ads reaches 3.3 billion users per day. This reach represents 20% of U.S. digital ad spending. By 2028, Meta may use GenAI to generate 30 personalized ads per user. Using GenAI on Meta Ads alone would represent a major increase in Inference workloads.

Compute Supply?

Faced with growing demand, will supply run out? The answer lies in the expansion of global Compute capacity. In 2023, NVIDIA shipped approximately 3.8 million cutting-edge GPUs to data centers. About 650,000 of them were NVIDIA H100 Tensor Core GPUs. Assuming an Average Utilization Rate (AUR) of 50%, BHI estimates the 2023 global AI Compute supply will reach 7 × 10^28 FLOPs by 2028, which is equivalent to counting every single grain of sand on every beach in the world.

SemiAnalysis projects that Compute will increase 60x by the end of 2025. If these projections hold, global Compute capacity should reach approximately 4 x 10^30 FLOPs by 2028, which is more than enough to meet even the most aggressive GenAI adoption scenarios.

🤔WHAT’S THE VERDICT

The world’s aging power grid is ill-equipped to handle the sudden surge in energy consumption from GenAI. For context, new data centers demanded 195 terawatt-hours (TWh) of electricity last year. That’s as much electricity as 18 million households per year. By 2027, new data centers could require 500 TWh, or 46 million households, worth of electricity. If a Compute shortage occurs, energy is more likely the culprit than hardware.

🔑KEY TAKEAWAY

AI’s largest obstacle to widespread adoption is arguably Compute capacity. While Training has historically required the most Compute, demand will increasingly come from Inference. Fortunately, Compute capacity will likely remain sufficient, so AI adopters can breathe a sigh of relief. But the world’s aging power grid could cause some issues.

📒FINAL NOTE

FEEDBACK

❤️TAIP Review of The Week

REFER & EARN

🎉Your Friends Learn, You Earn!

You currently have 0 referrals, only 1 away from receiving ⚙️Ultimate Prompt Engineering Guide.

Copy and paste this link to friends: https://theaipulse.beehiiv.com/subscribe?ref=PLACEHOLDER