Source item

Users won’t be necessarily able to articulate it — but they are going to be increasingly frustrated when companies are holding back inference, restricting tokens naively at the cost of performance Once you have tasted the nectar of scaling tokens you just expect it