LATIDIA · Investigación

FLoKD: Destilación adaptativa de conocimientos para LLM federado de bajo rango a través de redes inalámbricas

arXiv:2609.13580v1 Tipo de anuncio: nuevo Resumen: Los modelos de lenguaje grandes (LLM) han demostrado capacidades sólidas en una amplia gama de tareas de procesamiento de lenguaje natural. Sin embargo, el ajuste fino convencional típicamente rel

La noticia

arXiv:2609.13580v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated strong capabilities across a wide range of natural language processing tasks. However, conventional fine-tuning typically relies on centralized data collection, bringing in privacy concerns. Federated learning (FL) enables collaborative LLM fine-tuning without sharing raw client data, but its deployment over bandwidth-constrained wireless networks is hindered by the communication overhead of model-parameter transmission. Although Low-Rank Adaptation (LoRA) reduces the number of trainable parameters, its communication cost still increases with model scale. Knowledge distillation avoids parameter sharing via output logits, but token-level logits in LLMs incur high communication cost due to sequence length and vocabulary size. Reducing logits lowers the cost