LATIDIA · Ciberseguridad
La tasa de éxito del ataque no es un número: sobre la validez de la medición en la evaluación de seguridad de la IA genética
arXiv:2609.25173v1 Tipo de anuncio: nuevo Resumen: La tasa de éxito de ataque (ASR) es la métrica principal en casi todas las evaluaciones publicadas de ataques y defensas para agentes de LLM. Argumentamos que ASR como se usa actualmente no es
WhatsApp ↗Telegram ↗
La noticia
arXiv:2609.25173v1 Announce Type: new Abstract: Attack success rate (ASR) is the headline metric in nearly every published evaluation of attacks on, and defenses for, LLM agents. We argue that ASR as currently used is not a single quantity but a family of metrics parameterized by six design choices that papers seldom specify and never hold constant across the literature. We support this with two studies that require no proprietary access. First, a full-text meta-analysis of 259 agentic-security papers posted to arXiv between February 2025 and September 2026 finds that most report neither a variance estimate nor repeated runs for their headline attack metric: 58% (95% CI 44-71) in a hand-coded