Source item
Open original source ↗Most AI benchmarks test retrieval or reasoning puzzles, tasks with fixed answers that the next model release just clears. Markets are different. The answer key keeps changing. That makes them the truest test of whether these models are actually getting more intelligent in a way that matters.
Worth following the Intelligent Earnings Benchmark from @iafunds!
Quoted context
These are the quoted author’s words.
Today we're launching the @iafunds Intelligent Earnings Benchmark. 8 frontier AI models. 715 large cap stocks. One question: Where are earnings estimates headed over the next 60 days? Not an AI puzzle. Not retrieval. A live test of how well frontier models can predict the direction of earnings. Follow along as we score the models and see this quarter's winner in June.
dougclinton ↗