If Charles Dickens was alive in 2024, A Tale of Two Cities might be the divide between the “GPU poor” and the “GPU rich”.

We mentioned these terms in some of our previous episodes; they were originally coined by Dylan Patel of SemiAnalysis in his “Gemini Eats the World” post, put on blast by Sam Altman. SemiAnalysis are one of the most in depth research and consulting firms in the semis world, and have a unique insight into the design, production, and supply chain of GPUs based on their ground presence in Asia.

In this episode we break down the State of Silicon: when are more GPUs coming? Are there real GPU alternatives on the way? Should Microsoft buy AMD chips just to scare Jensen? Is there a “GPU poor is beautiful” manifesto?

Full show notes: https://www.latent.space/p/semianalysis

0:00 - Introductions
4:31 - Importance of infrastructure and hardware for AI progress
10:53 - GPU-rich vs GPU-poor companies and competing in AI
14:22 - Optimizing hardware and software for AI workloads
17:00 - Metrics for model training vs inference optimization
21:38 - Networking challenges for distributed AI training
23:19 - Google’s partnership with Broadcom for TPU networking
28:04 - What GPU-poor companies/researchers should focus on
34:47 - Innovation in AI beyond just model scale
38:03 - AI hardware startups and challenges they face
46:15 - Manufacturing constraints for cutting edge hardware
50:36 - Apple and AI
54:18 - AI safety considerations with scaling AI capabilities
57:37 - Complexity of rebuilding semiconductor supply chain
1:00:08 - Recommended readings to understand this space
1:04:31 - Dylan’s process for writing viral blog posts
1:07:27 - Dylan’s “magic genie” question