LATIDIA · Ciberseguridad

Más allá de las etiquetas binarias: evaluación diagnóstica de la seguridad de LLM con rúbricas de nivel rápido

arXiv: 2610.29225v1Tipo de anuncio: Cross Resumen: Los puntos de referencia actuales de seguridad de LLM se basan principalmente en etiquetas binarias seguras o inseguras, que carecen de profundidad explicativa y sufren de baja consistencia en los jueces automatizados. Para

La noticia

arXiv:2610.29225v1 Announce Type: cross Abstract: Current LLM safety benchmarks predominantly rely on binary safe or unsafe labels, lacking explanatory depth and suffering from low consistency across automated judges. To address this, we introduce DiagSafe, a diagnostic evaluation framework that moves beyond binary classification by employing prompt-level rubrics. Grounded in globally applicable AI safety policies, each rubric provides explicit criteria along two dimensions: safety, which targets concrete harms with specific exemptions to suppress false positives, and usefulness, which independently evaluates instruction fulfillment. To ensure a robust and realistic evaluation, we construct a bilingual (English-Chinese) benchmark grounded in a comprehensive 84-label safety taxonomy, consisting of two subsets: DiagBase, which covers single- and