LATIDIA · Ciberseguridad

¿Respondió el LLM a la pregunta? Un juego de reconstrucción semántica para la evaluación de Jailbreak

arXiv: 2610.19306v1Announce Type: cross Resumen: Los ataques de jailbreak contra modelos de lenguaje grandes (LLM) se evalúan utilizando la tasa de éxito de ataque (ASR), que depende de identificar correctamente las respuestas que cumplen con prohibir

La noticia

arXiv:2610.19306v1 Announce Type: cross Abstract: Jailbreak attacks against large language models (LLMs) are evaluated using attack success rate (ASR), which depends on correctly identifying responses that fulfill prohibited requests. Existing evaluators can mistake refusals or topically related responses for successful attacks. We show that changing only the judge LLM can alter reported ASR by up to $82$ percentage points. We introduce the semantic reconstruction game, a training-free evaluator that decomposes compliance judgments into constrained semantic comparisons. Given only a response, an LLM reconstructs the request answered by its substantive content, excluding information merely repeated or refused. The game compares this reconstruction against similar decoy requests to determine whether it matches