LATIDIA · Investigación
SpecSearch: Hacia una decodificación especulativa eficiente que tenga en cuenta el «hardware»
arXiv:2611.22296v1 Tipo de anuncio: nuevo Resumen: La decodificación especulativa (SD) acelera la inferencia de los grandes modelos de lenguaje (LLM), pero su configuración óptima depende conjuntamente de las características de la carga de trabajo, del hardware de la GPU y de la disponibilidad
WhatsApp ↗Telegram ↗
La noticia
arXiv:2611.22296v1 Announce Type: new Abstract: Speculative decoding (SD) accelerates large language model (LLM) inference, but its optimal configuration depends jointly on workload characteristics, GPU hardware, and available memory. For example, on the RTX~5090, the state-of-the-art speculative drafter DFlash achieves $3.50\times$ the throughput of autoregressive (AR) decoding at low concurrency, but only $0.76\times$ at high concurrency with short outputs. We present SpecSearch, a hardware-aware search engine that jointly optimizes decoding strategy, draft length, and precision across workload--GPU pairs. Its analytical cost model captures memory and compute constraints to identify feasible configurations and narrow the search space for throughput optimization. Guided by this model, calibration measurements are collected from multiple GPUs. Using