• 2024 US Federal Energy Regulatory Commission (FERC) Rejection: In November 2024, FERC rejected Amazon's request to buy 180 MW of power directly from the Susquehanna nuclear power plant for a nearby data center, arguing it goes against other users' interests.
• US Power Demand Trend: Demand has been flat for nearly 20 years but is now shooting up. Part of the increase comes from data centers due to running sophisticated AI models.
• The AlexNet Moment: In 2012, Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton developed AlexNet for the ImageNet LSRVC. It had multiple layers and was too big to fit on a single GPU, so they split training between two. AlexNet won the competition and showed the decoupling of AI model size from single CPU/GPU capabilities.
• The Balancing Act: After AlexNet, using multiple GPUs to train AI became common. Between 2010 and 2020, data center power consumption was relatively flat due to GPU adoption and efficiency improvements. But with the rise of large language transformer models like ChatGPT in 2022, power consumption increased. Nvidia has continued to improve efficiency.
• The AI Lifecycle: LLMs use billions of neurons and a large number of parameters. Training is very computational and consumes a lot of power. Inference also consumes power over time.
• Trimming AI Models Down: To reduce power consumption, researchers use techniques like pruning (removing parameters) and quantization (changing parameter format). These optimize internal model workings.
• Finishing First: Optimizing data center operations can also save energy. Chung built Perseus to pace GPUs for more efficient training. Results were promising, but it takes time to deploy.
• Back of the Envelope: Research groups estimated data center energy consumption in 2028 at 325 - 580 TWh in the US. EPRI warns about concentration in certain locations. Nvidia says running ChatGPT is 12% of data center operations. FERC commissioner's numbers have some doubts.
• Closed AI Problem: Chowdhury and Chung don't trust the current power consumption estimates as companies like OpenAI and Google haven't released actual figures. They argue for a formal testing procedure.
• AI-Efficiency Leaderboard: ML Energy Initiative used ZeusMonitor to measure GPU power consumption. Results showed some LLMs are more energy-efficient than expected, but proprietary models' performance is unknown. Lack of transparency is a problem.
• Where Rubber Meets the Road: Energy efficiency in data centers follows a trend similar to Moore's law. There are more revolutionary technologies like photonic chips and 2D semiconductors on the horizon. But the Jevons paradox may apply as efficiency gains lead to more usage.