LATIDIA · Investigación

SteerDistill: cuantificación de agentes de LLM a través de la dirección del maestro de circuito cerrado

arXiv: 2611.13410v1Announce Type: new Resumen: La cuantificación de bits bajos reduce el coste de inferencia de los agentes LLM, pero puede comprometer el éxito de la tarea. Al examinar las implementaciones fallidas, encontramos que los agentes cuantificados pasan por alto el requerimiento de tareas

La noticia

arXiv:2611.13410v1 Announce Type: new Abstract: Low-bit quantization reduces the inference cost of LLM agents but can compromise task success. Examining failed rollouts, we find that quantized agents overlook task requirements or environmental conditions, producing environment transitions that diverge from successful full-precision references. Correcting these behavioral divergences remains difficult: offline data may miss the student's failure contexts, while its own rollouts may provide few correct responses in those contexts. We introduce SteerDistill, a quantization-aware distillation method in which a full-precision teacher guides the student to revise failed responses and iteratively refines its guidance using feedback from student execution. This closed-loop teacher steering produces locally verified behavioral corrections. We internalize these corrections