Liquid AI Releases LFM2.5-VL-3B-DSpark For Faster VLM Decoding

Liquid AI has announced the release of LFM2.5-VL-3B-DSpark, an experimental speculative decoding draft model designed to accelerate the inference process for its LFM2.5-VL-3B vision-language model. This new component adds approximately 280 million parameters to the existing model and aims to enhance decoding speed without altering the final output of the vision-language model. According to Liquid AI's team, the DSpark model achieves decoding speeds up to 3.13 times faster on Apple silicon and up to 2.66 times faster on NVIDIA H100 hardware. The model's weights are now available on Hugging Face in Safetensors and GGUF formats, with immediate support integrated into SGLang, MLX-VLM, and llama.cpp. Liquid AI has labeled this release as experimental and it is distributed under the LFM Open License v1.0, which permits free commercial use for companies with annual revenues below $10 million. Speculative decoding operates by employing a smaller "drafter" model that proposes multiple tokens in advance. A larger "target" model then evaluates this entire block of proposed tokens in a single forward pass, retaining only those it agrees with. This approach contrasts with standard models that generate one token per forward pass. The DSpark model for vision-language tasks follows the methodology previously established by Liquid AI's text-model DSpark drafters, as detailed in the DSpark paper. The drafter model is designed to read hidden states from multiple layers of the target model and predict the subsequent 'k' tokens. A key innovation highlighted by Liquid AI is that the modality of the data (text or image) is irrelevant to the drafter, as both text and image patches are represented as tensors by the time they reach the hidden layers. This allows Liquid AI to utilize the same inference algorithm for both text-only and vision-language models. The drafter architecture is characterized as a simplified attention-only model. Ablation studies conducted by the Liquid AI team indicated that a configuration with 4 layers and a block size of 9 performed optimally. For inference, Liquid AI recommends a block size of 8 or 9, with the choice dependent on the specific hardware being used; Apple silicon implementations are suggested to use a block size of 8. The drafter component comprises several parts: a decoder stack with 4 layers contributing 193.0 million parameters, a hidden-state projection layer with 21.0 million parameters, a Markov head with 65.5 million parameters, and norms and a confidence head totaling 6.4 thousand parameters, bringing the total for the drafter to approximately 279.5 million parameters. The embedding and language model head are shared with the target model.