LATIDIA · Investigación

La densidad del pensamiento: predecir y dirigir el razonamiento en modelos de lenguaje grandes

arXiv: 2610.21286v1Tipo de anuncio: nuevo Resumen: Investigamos si la variación en la densidad de información a través de los pasos de razonamiento de la cadena de pensamiento (CoT) predice la corrección de la respuesta y puede servir como una inferencia controlable-

La noticia

arXiv:2610.21286v1 Announce Type: new Abstract: We investigate whether variation in information density across chain-of-thought (CoT) reasoning steps predicts answer correctness and can serve as a controllable inference-time target. We prompt frozen pretrained models to solve problems via CoT and measure the density of each reasoning step with two independent metrics: LP variance (how unevenly information is distributed across steps, computed as the variance in mean per-token negative log-likelihood) and the Caveman compression ratio. We then test whether density correlates with, and can be steered to improve, answer correctness. Experiments across six model scales, two training paradigms, and 60 GSM8K problems yield four main findings. First, lower LP variance predicts correctness