LATIDIA · Ciberseguridad
PIPETTE: Purple teaming de inyección de indicaciones para agentes de LLM robustos
arXiv:2611.24362v1 Tipo de anuncio: nuevo Resumen: Los agentes de modelos de lenguaje grande (LLM) interactúan habitualmente con datos externos no fiables, lo que los expone a ataques de inyección indirecta de «prompts» (IPI) que secuestran a los agentes para que ejecuten
WhatsApp ↗Telegram ↗
La noticia
arXiv:2611.24362v1 Announce Type: new Abstract: Large language model (LLM) agents routinely interact with untrusted external data, exposing them to indirect prompt injection (IPI) attacks that hijack agents into executing attacker-chosen tasks. To uncover these vulnerabilities at scale, automated red-teaming methods often leverage reinforcement learning (RL) to generate attacks that achieve high attack success rates (ASR). Model developers use the attacks discovered during internal red-teaming to harden agents through adversarial training before deployment, a collaborative offensive-defensive process known in cybersecurity as purple-teaming. However, we show that achieving high ASR during red-teaming does not guarantee robustness gains from subsequent adversarial training. Specifically, we find that two state-of-the-art RL-based red-teaming methods, RL-Hammer and