LATIDIA · Investigación

EliteReplay: Preservar diversas soluciones en RL después de la capacitación a través de la búsqueda de calidad y diversidad

arXiv: 2610.22290v1Tipo de anuncio: nuevo Resumen: El aprendizaje por refuerzo se ha convertido en una técnica común posterior a la capacitación para mejorar la precisión de los LLM en tareas verificables. Sin embargo, dicho entrenamiento puede reducir el conjunto de soluciones

La noticia

arXiv:2610.22290v1 Announce Type: new Abstract: Reinforcement learning has become a common post-training technique to improve the accuracy of LLMs on verifiable tasks. However, such training can narrow the set of solutions the model can produce. Several approaches have attempted to address this behavior-sharpening effect, but they often optimize metrics that do not reflect the diversity of correct solutions. Ideally, we want a model that can produce multiple good solutions, not only a single reliable one. We propose EliteReplay, a post-training recipe that pairs GRPO with an "elite" archive built by quality-diversity search. Before training, we use the frozen base model to collect solutions that span distinct approaches; during training, GRPO