LATIDIA · Ciberseguridad

Deduplicación durante la capacitación: un paradigma resiliente para la deduplicación entre clientes que preserva la privacidad en el aprendizaje federado

arXiv:2609.31262v1 Announce Type: new Abstract: Cross-client duplicate data in large language model training corpora degrada la eficiencia del aprendizaje federado (FL) al tiempo que exacerba la memorización del modelo y la privacidad ri

La noticia

arXiv:2609.31262v1 Announce Type: new Abstract: Cross-client duplicate data in large language model training corpora degrades the efficiency of federated learning (FL) while exacerbating model memorization and privacy risks. Privacy-preserving cross-client deduplication effectively mitigates this issue by eliminating duplicate training data. However, existing schemes all follow a "Deduplication-before-Training" paradigm. This serially coupled paradigm incurs high fault-tolerance costs and lacks support for dynamic client joining. To this end, we propose an unexplored paradigm called "Deduplication-while-Training (DwT)", which enables concurrent deduplication and training. DwT transforms cross-client deduplication from a one-time, globally synchronous preprocessing operation into a continuous online service with state management, concurrent claiming, and failure recovery. By enabling state synchronization and task