Source item

1/ thinking: so many default behaviors of models are based on the failures of internet data. secondary framings like rankings, summaries, takes, etc appear far more often in training than the primary sources they compress. 2/ so they’re the low-effort, “feels fluent” default, and the model emits them from memory instead of reading the source, unless user/agent forces it to retrieve. they don’t want to think critically! in novel ways! it is hard! 3/ folks broadly complain about models being “nerfed” but interesting to think about specific bad behaviors in models that I expect might get worse as companies “compute/cost-optimize” their general products 4/ unclear if it’s just the upstream pretraining data or the actual reward today of faster time-to-answer/less external call