Source item

New dominant question is not “is this the end of scaling” but how does the base model quality interact with test-time scaling / (o1/o3 style RL)? Is it multiplicative (a little bit moar in the base model will you get advanced new behaviors in the reasoning version still)? Otherwise, next OOM of $$$ shifts away from pretraining (with big implications for infrastructure)