LATIDIA · Investigación

Localización funcional de subredes para un aprendizaje continuo eficiente en grandes modelos lingüísticos

arXiv: 2610.21537v1Tipo de anuncio: nuevo Resumen: La inteligencia biológica se adapta continuamente a la nueva información al tiempo que conserva las habilidades previas a través de la plasticidad localizada. Por el contrario, el aprendizaje continuo en idiomas grandes m

La noticia

arXiv:2610.21537v1 Announce Type: new Abstract: Biological intelligence continually adapts to new information while retaining prior skills through localized plasticity. In contrast, continual learning in large language models (LLMs) remains vulnerable to catastrophic forgetting. A promising direction is to identify functionally specialized subnetworks within LLMs that enable targeted and resource-efficient adaptation. Using gradient-attribution profiling, we localize cognition-associated functional subnetworks exhibiting extremely low mutual overlap. Building on this finding, we propose Function-isolated Memory-Efficient Optimization (CiMEO), a continual learning framework that routes each training example to its corresponding functional subnetwork, restricting parameter updates and optimizer states to selected regions. Across 1.4 million sequential instructions, CiMEO achieves the highest average performance among the compared