Think Less, Locally: Local Preference Learning for Reasoning Compression
The loop: A reasoning model detects candidate overthinking states in its own traces and regenerates shorter verified continuations. These self-generated preference pairs are then used to train the model via DPO to compress its reasoning.
A local preference learning method that trains a reasoning model to compress its trajectories by learning when to conclude from its own generated traces.







