LATIDIA · Ciberseguridad
IH-Benchmark: un punto de referencia centrado en los conflictos para evaluar la solidez de la jerarquía de instrucciones en los grandes modelos lingüísticos
arXiv:2607.25987v3 Tipo de anuncio: sustitución Resumen: Cuando un modelo de lenguaje recibe instrucciones contradictorias de diferentes niveles de prioridad, ¿cuál sigue realmente? Esta pregunta es fundamental para un modelo lingüístico fiable.
WhatsApp ↗Telegram ↗
La noticia
arXiv:2607.25987v3 Announce Type: replace Abstract: When a language model receives conflicting instructions from different priority levels, which one does it actually follow? This question lies at the heart of reliable LLM deployment. Existing benchmarks answer this only partially, often focusing on a single hierarchy component or adapting public datasets with limited tool-use coverage. We present IH-Benchmark, a conflict-centered benchmark for instruction-hierarchy robustness across direct System $\succ$ User (S $\succ$ U) conflicts and tool-mediated User $\succ$ Tool (U $\succ$ T) conflicts. We organize tests into seven constraint categories according to the higher-priority requirements under evaluation. Each test is evaluated with a uniform binary pass/fail protocol combining a predicate domain-specific language (DSL)