LFM2.5-VL-DSpark: Accelerating Vision-Language Models with Minimal Overhead
LFM2.5-VL-DSpark：以超低開銷將多模態模型推論速度提升達 3 倍
Liquid AI released the DSpark speculative decoding draft model for LFM2.5-VL-3B, adding only 280M parameters (an 8.9% overhead). By projecting image patches and text into a shared-dimensional hidden state, the drafter shares the same architecture as text LFM2.5-DSpark models. It achieves decoding speedups of up to 3.13x on Apple M5 Max and 2.66x on NVIDIA H100, featuring day-one integration with llama.cpp, MLX-VLM, and SGLang.
Key points
Significant Decode Speedups
Delivers up to 3.13x decoding speedup (2.62x end-to-end) on M5 Max, and up to 2.66x decoding speedup (2.27x end-to-end) on H100.
Minimal Memory Cost
The draft model adds only 280M parameters, introducing a tiny 8.9% parameter overhead on top of the 3B target model.
Unified Multimodal Architecture
Projects image patches and text tokens into a shared representation, allowing the drafter to run identical inference algorithms to text models.
Day-One Open Source Support
Released on Hugging Face with native support for llama.cpp, MLX-VLM, and SGLang right out of the box.
How it works
Why it matters
This work demonstrates that speculative decoding is highly viable for multimodal models. By adding less than 9% memory overhead, it achieves major speedups on both edge devices and datacenter GPUs with zero loss in output quality. This lowers the barrier for deploying fast, responsive visual AI assistants on consumer-grade local hardware.
Who it affects
- AI Developer
- AI Researcher
- Enterprise Leader
How to use it
- 1On-device real-time Vision Question Answering (VQA) and chart analysis on Apple Silicon laptops.
- 2High-throughput, low-latency deployment of VLM services using SGLang or llama.cpp.
Limitations & caveats
- Speculative decoding only accelerates the decoding phase; prefill and vision encoding phases remain bottlenecked by hardware compute (Amdahl's law).
- End-to-end speedup gains are less pronounced in scenarios with very long text prompts or complex visual inputs where prefill dominates.
Related
Minimally Invasive Steering of LMs: Optimizing Rewards Without Quality Degradation
微創型語言模型導向技術：利用 MISVO 在不損害生成品質下優化輸出
This paper introduces MISVO, a minimally invasive steering method that uses local KL geometry to optimize LLM outputs for test-time rewards without parameter updates or quality degradation.
Accelerating MoE Training for Biological Foundation Models with NVIDIA Transformer Engine
NVIDIA Transformer Engine 加速生物基礎模型 MoE 訓練：吞吐量提升達 2.21 倍
This guide demonstrates how to use NVIDIA BioNeMo and Transformer Engine's optimized primitives to overcome MoE training bottlenecks in biological models, boosting throughput by up to 2.21x.
Reproducing OLMo 3 7B Pre-training in MaxText: A Case Study of Large-Scale Training on TPUs
以 MaxText 重現 OLMo 3 7B 預訓練：Google Cloud TPU 大規模訓練實戰指南
This case study details the successful reproduction of AI2's OLMo 3 7B pre-training and mid-training on Cloud TPUs using MaxText, detailing key performance tuning and debugging insights.