Source item
Open original source ↗We dove into the H100's performance improvement over time from software over 2 years.
Covered power usage + $ cost for training in a very detailed way for training runs on thousands of GPUs
Equating this to US household power consumption @JeffDean
+ GB200 reliability challenges
Quoted context
These are the quoted author’s words.
H100 vs GB200 NVL72 Training Benchmarks Power, TCO, and Reliability Analysis, Software Improvement Over Time Joules per Token, TCO Per Million Tokens, MFU Tokens Per US Annual Household Energy Usage, DeepSeek 670B GB200 Unreliability, Backplane Downtime https://t.co/DDDitLaT2N
SemiAnalysis_ ↗