LATIDIA · Investigación

FlashAttention con estadísticas: características de atención de baja sobrecarga para detectores de alucinación compactos

arXiv:2610.21483v1 Announce Type: new Abstract: Los modelos de lenguaje grandes pueden producir respuestas fluidas que el contexto proporcionado no admite. Un modelo compacto de proxy congelado puede detectar tales respuestas leyendo el contexto y

La noticia

arXiv:2610.21483v1 Announce Type: new Abstract: Large language models can produce fluent answers that the supplied context does not support. A compact frozen proxy model can detect such answers by reading the context and another model's response, even from a proprietary generator. Attention-based detectors use summary statistics of the proxy's attention weights, but standard pipelines materialize the attention matrix, whose memory grows quadratically with sequence length. We compute these statistics without materializing attention, by accumulation in the FlashAttention pass, sparse lookups after it, or a second tiled pass over the scores, and classify them by this path into nearly free and costlier ones. On an H100, the statistics of five published