LATIDIA · Ciberseguridad

Falla de métricas en la reparación de vulnerabilidades de código basada en LLM: un estudio empírico y una pantalla consciente del cambio

arXiv: 2609.26749v1Tipo de anuncio: cross Resumen: Los modelos de lenguaje grandes (LLM) se aplican cada vez más a la reparación automatizada de vulnerabilidades de seguridad de C/C++, y la tasa de compilación es un proxy comúnmente reportado para progres

La noticia

arXiv:2609.26749v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly applied to the automated repair of C/C++ security vulnerabilities, and compile rate is a commonly reported proxy for progress: whether the generated patch compiles. We argue that compile rate is a scientifically unreliable metric for single-function vulnerability repair, and we support this with five controlled experiments over 203 vulnerable functions from Big-Vul, three open-source code LLMs (350M to 6.7B parameters), and three prompting strategies. Compile rate (i) barely responds to an intervention that substantially improves the generated code; (ii) is dominated by evaluation-harness and dataset artifacts rather than model quality, with about 64% of compile failures not attributable to