Source item
Open original source ↗.@deepseek_ai highest impact innovations can be summarized as “communications efficiency.” AI is a distributed systems problem!
Quoted context
These are the quoted author’s words.
@karpathy Still reading, a lot here - is it right to say 1) dualpipe/comms optimizations and 2) fine grained MOE/load balancing strategy for expert utilization are the biggest drivers of compute efficiency? with FP8 as enabler (reducing mem footprint, efficient comms, bigger batches)
saranormous ↗