LATIDIA · Investigación

Forecast-Dojo: Entornos reproducibles para la evaluación comparativa y la capacitación de agentes de pronóstico de LLM

arXiv: 2609.28876v1Tipo de anuncio: nuevo Resumen: Presentamos Forecast-Dojo, un entorno rejugable para la evaluación comparativa y la capacitación de agentes de pronóstico de LLM. Combina preguntas resueltas del mercado de predicción con noticias fechadas,

La noticia

arXiv:2609.28876v1 Announce Type: new Abstract: We introduce Forecast-Dojo, a replayable environment for benchmarking and training LLM forecasting agents. It combines resolved prediction-market questions with dated news, allowing agents to research an event and revisit their predictions at successive historical dates. The same tasks and tools support repeated evaluation, collection of training interactions, and feedback from recorded outcomes without waiting for new events to resolve. Forecast-Dojo contains 1,568 Polymarket events, split by time into training and evaluation periods, and 18.8M dated news articles. In an evaluation of 12 models, research tools lower Brier score for all 12. Forecasts also improve as events unfold, with the largest gains at steps where more