LATIDIA · Investigación
A los auditores les cuesta detectar sabotajes en las transcripciones integrales de las investigaciones sobre seguridad de la IA
arXiv:2611.21832v1 Tipo de anuncio: nuevo Resumen: A medida que los sistemas de IA se utilizan cada vez más para automatizar la I+D en IA, los sistemas desalineados podrían sabotear la investigación destinada a mejorar la seguridad y la alineación de los modelos. Para someter a pruebas de resistencia las defensas,
WhatsApp ↗Telegram ↗
La noticia
arXiv:2611.21832v1 Announce Type: new Abstract: As AI systems are increasingly used to automate AI R&D, misaligned systems could sabotage research intended to improve model safety and alignment. To stress-test defenses against such sabotage, we use a human expert as a proxy saboteur on AI safety research. We conduct a red-team/blue-team game where a human researcher runs AI safety experiments from scratch and auditors try to tell which were sabotaged and how. Given access to the full research transcript and the ability to run experiments, human auditors mostly fail to distinguish sabotaged from benign research: they achieve an AUROC of 0.59, not significantly better than chance, and fully identify the specific