LATIDIA · Investigación
El MCTS está contando lo que no debe: la exploración del aprendizaje cuenta para el razonamiento de los LLM
arXiv:2611.22301v1 Tipo de anuncio: nuevo Resumen: La búsqueda en árbol de Monte Carlo (MCTS) ha impulsado logros históricos en el aprendizaje por refuerzo, pero sus avances en el razonamiento de los LLM pueden seguir siendo modestos a pesar de un esfuerzo de búsqueda considerable.
WhatsApp ↗Telegram ↗
La noticia
arXiv:2611.22301v1 Announce Type: new Abstract: Monte Carlo tree search (MCTS) has powered landmark successes in reinforcement learning, yet its gains in LLM reasoning can remain modest despite substantial search effort. We identify an overlooked bottleneck: what MCTS counts. Standard UCT assigns independent visit counts to textual branches, allowing different formulations of the same reasoning strategy to repeatedly receive exploration bonuses. Our theoretical analysis shows that this count fragmentation can impair exploration efficiency, as quantified by increased cumulative regret. To address this problem, we introduce Learned Pseudo-Count MCTS (LPC-MCTS), which changes only the exploration statistic in UCT. A lightweight Coin-Flipping Network head on the policy's hidden representations estimates representation coverage, enabling