LATIDIA · Investigación

Sci$\Pi$Voto: Evaluación comparativa del juicio del investigador principal en momentos cruciales de los agentes científicos de investigación automática

arXiv:2610.21082v1 Tipo de anuncio: nuevo Resumen: los agentes del modelo de lenguaje grande (LLM) pueden mantener los experimentos científicos en funcionamiento durante meses con poca supervisión humana, pero la investigación autónoma confiable también requiere

La noticia

arXiv:2610.21082v1 Announce Type: new Abstract: Large language model (LLM) agents can keep scientific experiments running for months with little human supervision, but reliable autonomous research also requires scientific judgment. Evaluating scientific judgment is difficult because research decisions are open-ended and depend on project context. We construct Sci$\Pi$vot, a 248-case benchmark of scientific judgment based on human corrections. The benchmark draws on records from six months of research in which senior researchers ran LLM agents on 26 projects across six scientific fields and repeatedly intervened to correct the agents' research judgments. In each case, an agent chooses between the original LLM plan and the human-corrected alternative, using only the project record