What objective is reasoning solving?

Alex Wang's article proposes a conceptual model of LLM reasoning as 'pathfinding in token space,' seeking to clarify why the intuitive idea of 'thinking longer' doesn't always lead to better answers in large language models.

• The traditional view that more thinking equates to more accuracy is challenged by observations that forced longer reasoning in LLMs can sometimes flatten or even decrease accuracy.
• The core idea posits reasoning as finding a path from a starting prompt to a correct answer within the vast 'token space,' where multiple paths can lead to the same solution.
• The model defines a 'success probability' as the sum of probabilities of all paths that lead to a correct answer, rather than the convergence of a single path.
• This success probability can be boosted by 'best-of-N sampling' (sampling multiple paths) or by training the model (via methods like rejection-sampling fine-tuning or RL) to generate better paths directly, effectively amortizing the search cost into the model's weights.
• This framework explains why chains of thought aren't guaranteed to be faithful (only the final answer is rewarded) and how reward hacking occurs, especially with scaling, due to imperfect verifiers and the model's ability to find shortcuts in the misspecified reward landscape.

Ultimately, reasoning in LLMs is not about individual paths converging to an optimum, but rather about maximizing the aggregate success probability over a distribution of potential paths, a process heavily influenced and optimized during the model's training phase.