I’m very excited to see the Ant and OAI S-1s.
Here are some of the questions I am very interested in:
1. What is the best guess of the steady state number of tokens to accomplish a fixed benchmark task that actually represents real work in a company.
To understand what is worth knowing: fix the real world task and then measure the token consumption and cost across the important combinations of the following:
Model + Harness (native and 3rd party) : Task
What is it today and considering intelligence scaling, what is our best guess for its descent down the cost curve?
2. If 3rd party harnesses drive token efficiency and most tokens today exist without such efficiency how do you model this in the future?
3. If 3rd party harnesses can enable near-frontier performance using smaller/cheaper models, how do you model this cost competition in the future?
4. If there are a lot of COGS due to reselling or distribution through hyperscalers, how do you model this drag as electricity/power become THE constraint. Because the hyperscalers will have a near monopoly on scale capacity but it will come with strings attached (ie flow your model thru Bedrock and pay them a vig).
I’m sure there are other higher order questions but these are the ones at the top of my mind.
Quoted context
These are the quoted author’s words.
From yours truly on Friday’s @theallinpod
“On the margin, Anthropic is losing share to OpenAI, to open source, and to Grok, and they're still growing so fast.”
If the $65b is correct, then someone leaked it to help Anthropic on their forthcoming IPO roadshow. Basically, they are hoping will be able to say “we are now at a much higher ARR after bringing on capacity during July, August and September.”
And after the Anthropic commentary during their potential September/October IPO roadshow and given that OpenAI is almost certainly taking significant share given how aggressive they were securing compute capacity, we should then expect a leak from OpenAI showing they are still growing faster.
There is going to be so much gamesmanship into this IPO.
I read the $65b as impressive both for Anthropic given their limited capacity and for AI overall given that Anthropic is likely losing some share. Reasonable minds can disagree.
We shall see!
Codex user numbers for reference; users have almost tripled over the last 5 weeks:
GavinSBaker ↗