LATIDIA · Investigación

GraphACM: Medición de la consistencia de la acción basada en gráficos para la cuantificación de la incertidumbre en agentes de LLM

arXiv:2610.22031v1 Tipo de anuncio: nuevo Resumen: La consistencia del comportamiento en ejecuciones repetidas proporciona una señal informativa para evaluar la confiabilidad de los agentes del modelo de lenguaje grande (LLM). Sin embargo, el aprox. existente

La noticia

arXiv:2610.22031v1 Announce Type: new Abstract: Behavioral consistency across repeated executions provides an informative signal for assessing the reliability of large language model (LLM) agents. However, existing approaches typically summarize consistency with a single task-level score, without identifying where behavioral differences occur or whether actions are compared under equivalent decision contexts. We propose a graph-based framework for fine-grained behavioral consistency measurement and uncertainty quantification. Given multiple rollouts of the same task, we merge equivalent states into a directed, action-labeled graph, enabling action comparisons at shared decision states. We introduce node-level and global measures of action consistency, using the global consistency score as a proxy for uncertainty to predict task success or