LATIDIA · Investigación

Vibe Patenting: Evaluación de jueces de LLM para agentes profesionales de redacción de patentes

arXiv: 2609.13422v1Tipo de anuncio: nuevo Resumen: Los jueces de LLM se utilizan cada vez más para evaluar y mejorar los resultados generados por la IA, sin embargo, su confiabilidad para el trabajo profesional complejo sigue sin estar clara. Estudiamos este problema th

La noticia

arXiv:2609.13422v1 Announce Type: new Abstract: LLM judges are increasingly used to evaluate and improve AI-generated outputs, yet their reliability for complex professional work remains unclear. We study this problem through Vibe Patenting, an end-to-end patent-drafting testbed for AI-agent evaluation. A separately-invoked LLM judge evaluates generated patent drafts and provides structured feedback for iterative revision. Across multiple inventions and drafting-agent configurations, judge-guided revision consistently improves judge-assessed quality, while unguided revision tends to saturate. Notably, iterative judge feedback enables a low-reasoning agent to approach the performance of a substantially more expensive high-reasoning agent. Stronger models and increased reasoning generally improve judge-assessed drafting quality, while domain-specific agentic workflows provide further gains. We validate