LATIDIA · Ciberseguridad

Evaluación de seguridad basada en especificaciones de llamadas de herramientas de agentes del mundo real bajo tareas de contexto de horizonte largo

arXiv: 2506.15253v2 Tipo de anuncio: reemplazar Resumen: A medida que los agentes del modelo de lenguaje grande (LLM) se implementan cada vez más en escenarios desatendidos de largo horizonte, la evaluación de seguridad de ellos antes de la implementación se vuelve cada vez más

La noticia

arXiv:2506.15253v2 Announce Type: replace Abstract: As large language model (LLM) agents are increasingly deployed in long-horizon unattended scenarios, security evaluation of them before deployment becomes increasingly essential and challenging. Existing evaluation methods mainly follow the paradigms of LLM-as-a-Judge or Agent-as-a-Judge. However, such judges are themselves often vulnerable to hallucination and can even be affected by perturbations. Furthermore, because capturing full environment states at every step is often prohibitively expensive in open-ended environments, existing evaluations typically rely on textual logs and the final environment state. As a result, they may miss transient violations that are overwritten before the trajectory ends or never reflected in textual logs. We present RAS-Eval, a specification-based