LATIDIA · Investigación

¿Está seguro de que su LLM está seguro? Reparación y auditoría de la confianza

arXiv:2610.22139v1 Announce Type: new Abstract: Las puntuaciones de confianza determinan si una respuesta LLM es aceptada, revisada o regenerada, pero la recalibración escalar solo puede corregir errores visibles a través de la propia puntuación.

La noticia

arXiv:2610.22139v1 Announce Type: new Abstract: Confidence scores determine whether an LLM answer is accepted, reviewed, or regenerated, yet scalar recalibration can only correct errors visible through the score itself. We introduce RADAR (Repair, Adopt, and Diagnose After Recalibration), a reliability framework that separates confidence repair from deciding whether to deploy the repair and auditing the errors that remain. Under squared loss, we decompose prediction risk into outcome noise, information hidden by the score, and mapping error; restricting repair to order-preserving maps additionally introduces a pooling gap. For repair, we propose Probe-Anchored Isotonic Calibration (PAIC), which learns a monotone map from correctness-probe predictions on unlabeled target prompts. PAIC's excess risk depends