LATIDIA · Ciberseguridad

Los jueces más fuertes no ven más: ceguera de integridad de implementación en la evaluación de LLM

arXiv:2610.21135v1 Announce Type: new Abstract: LLM-agent evaluation reports behavioral scores but leaves the identity of the deployed runtime stack not reported. Una implementación manipulada que preserva las salidas evaluadas pasa

La noticia

arXiv:2610.21135v1 Announce Type: new Abstract: LLM-agent evaluation reports behavioral scores but leaves the identity of the deployed runtime stack unreported. A tampered deployment that preserves evaluated outputs passes every output-only evaluator. We introduce DIB-Bench, a benchmark that measures this deployment-integrity blindness across 7,560 scored examples, six tamper types, and an evaluator ladder from regex scorers through LLM judges to GPT-5-nano. Stronger evaluators accept more tampered outputs: mean acceptance rises from 88.6% (rule-based) to 94.1% (GPT-5-nano), and a trigger-conditional tamper invisible on the evaluation suite produces 48-100% harmful outputs, rising with model capability. A PAC-style bound shows that tampers with zero output shift are undetectable from outputs at any sample size;