Source item

there are diminishing returns to (compute) scale for pretraining in the current paradigm