Source item
Open original source ↗Open models show 2.5x faster, 6x more expensive
Lower batch size, speculative decoding harder
Pareto optimal curve for Deepseek at https://t.co/d9dNCumX0I shows this
Claude Opus 4.6 is 100 Tok/s/user
Deepseek at 100 is 6k Tok/s/GPU
At 250 tok/s/user it's closer to 1k https://t.co/X294HzM3Zo
Quoted context
These are the quoted author’s words.
2.5x faster but 6x more expensive. This can’t be achieved by inference optimization, must be new chips. TPU? B200? AWS Inferentia? Cerebras?
Yuchenj_UW ↗