LATIDIA · Investigación

ALCANCE: Autodestilación selectiva en el paso crítico de la política para el aprendizaje de refuerzo agénico

arXiv:2610.38391v1 Anuncio Tipo: nuevo Resumen: El aprendizaje por refuerzo basado en resultados proporciona una supervisión gruesa para agentes LLM de largo horizonte: las trayectorias fallidas pueden contener pasos correctos, mientras que las trayectorias exitosas c

La noticia

arXiv:2610.38391v1 Announce Type: new Abstract: Outcome-based reinforcement learning provides coarse supervision for long-horizon LLM agents: failed trajectories may contain correct steps, while successful trajectories can contain errors. Trajectory-level skills can also shift teacher preferences at unrelated steps, introducing off-target distillation signals. We introduce SCOPE, a framework for Selective Critical-Step On-Policy Self-Distillation. After a supervised warm-up, the policy analyzes its own trajectories, generates step-specific hindsight skills, and distills their guidance only at selected error steps. A bounded signed teacher-student coefficient stabilizes auxiliary updates while preserving positive reinforcement and negative correction. A success-relative pairwise length reward further encourages efficient task completion. Experiments on ALFWorld and WebShop demonstrate consistent improvements across multiple model