Source item

1/ rStar-Math is super cool, underrated. It demonstrates that small language models (7B!) can match larger models (like o1) at math reasoning through self-improvement: 2/ - self-generating high-quality math training data (for a policy SLM) using MCTS and code verification - a process preference model, based on predicted reward for each intermediate reasoning step - iteratively improving both w/multiple rounds of MCTS and retraining on validated data 3/ Self-improvement strategies work in much smaller systems!