New Draft Model Released

Liquid AI has announced the release of an experimental DSpark draft model designed for its LFM2.5-VL-3B vision-language model. This release introduces a speculative decoding path that aims to trade a minimal increase in memory footprint for a substantial speedup in inference without altering the quality of the output. The model is part of Liquid AI's broader effort to optimize its LFM2.5 family for efficient deployment across various hardware environments.

Performance Gains on Edge and GPU

The company reports significant performance improvements across different hardware configurations. On-device inference using MLX on an Apple M5 Max chip sees decoding speeds increase by 2.30x to 3.13x depending on the specific task. End-to-end latency improvements range from 1.56x to 2.62x. For users relying on llama.cpp on an M3 Ultra, decoding speeds improve by 1.57x to 2.14x, with end-to-end gains between 1.30x and 1.77x. On high-end GPU hardware, specifically the NVIDIA H100, the drafter delivers decoding speedups of 20.4x to 2.66x, with end-to-end improvements of 1.64x to 2.27x.

Architectural Design and Training

The vision drafter utilizes the same architecture as the text-based LFM2.5-DSpark drafters. It captures hidden states from the target model at a fixed set of tapped layers and uses them to draft a block of candidate tokens. Image patches and text tokens are projected into a shared representation before these layers, ensuring the drafter operates on hidden-state vectors of identical dimensionality regardless of the input modality. Following the DSpark recipe, the model was trained on a mixture of vision-language supervised fine-tuning data, weighted toward expected workloads. The final draft model is a simplified attention-only drafter with 4 layers and a block size of 9, trained for 10 epochs.

Memory Footprint and Efficiency

A key feature of this release is its low memory overhead. The resulting drafter contains approximately 280 million parameters, which increases the deployed model’s parameter count by just 8.9% on top of the 3B target model. This small memory cost allows for significant speedups without requiring substantial additional hardware resources. The inference algorithm remains unchanged from the text models, ensuring that the speculative decoding process is exact; the target model verifies every proposed token, meaning greedy output is identical to the target model running alone.

Framework Integration and Availability

The DSpark draft model for LFM2.5-VL-3B ships with day-one support for major inference frameworks, including llama.cpp, MLX-VLM, and SGLang. Users can run the model with SGLang by using a build with DSpark support for LFM2 targets, launching the target with the draft attached. The block size is read from the draft's config.json file. For llama.cpp and MLX-VLM, specific builds are required to enable the functionality. The model is available on Hugging Face in both Safetensors and GGUF formats, facilitating easy integration into existing pipelines.

Limitations in Vision Workloads

While speculative decoding significantly accelerates the decode phase, it does not speed up vision encoding or prefill. In vision-language models, the image first passes through a vision encoder, and the language backbone processes hundreds of visual tokens along with the text prompt. On edge devices with less compute power than datacenter GPUs, prefill takes up a larger portion of the end-to-end latency. Consequently, even large decode speedups may result in only modest end-to-end gains due to Amdahl's law, where the overall speedup is capped by the unaccelerated parts of the workload.

Sources

• Hugging Face BlogAccelerating vision-language models with LFM2.5-VL-DSpark