LATIDIA · Investigación

Aprendizaje de refuerzo eficiente en recursos después de la capacitación a través del paralelismo experto elástico

arXiv:2611.06984v1 Anuncio Tipo: nuevo Resumen: El aprendizaje por refuerzo posterior al entrenamiento mejora las capacidades de los grandes modelos de lenguaje, pero la generación de respuestas de entrenamiento es un importante cuello de botella computacional. En la mezcla de

La noticia

arXiv:2611.06984v1 Announce Type: new Abstract: Reinforcement learning post-training improves the capabilities of large language models, but generating training responses is a major computational bottleneck. In mixture-of-experts models, workers that finish their assigned responses may remain occupied because their experts are still needed by workers processing longer responses. Consequently, approximately one-third of occupied worker time in the baseline occurs after local response generation finishes. Here we present FlexFloor, a system that enables earlier resource release during synchronous response generation while preserving the expert computation required by ongoing responses. FlexFloor jointly adapts response scheduling and the active group of workers. It uses historical response lengths and memory constraints to guide batching and