Liquid AI Released LFM2-Audio-1.5B: An End-to-End Audio Foundation Model with Sub-100 ms Response Latency

Liquid AI Released LFM2-Audio-1.5B: An End-to-End Audio Foundation Model with Sub-100 ms Response Latency

Estimated Reading Time

7 minutes

• LFM2-Audio-1.5B is Liquid AI’s compact, end-to-end audio-language foundation model.
• It achieves sub-100 ms latency for real-time conversational AI, a significant leap in responsiveness.
• The model features a unified architecture that seamlessly integrates speech understanding and generation within a single stack.
• Designed for resource-constrained devices, it offers powerful capabilities with a small footprint.
• Supports versatile applications with interleaved and sequential generation modes, validated by strong performance benchmarks.

• Estimated Reading Time
• Key Takeaways
• A New Paradigm for Audio-Language Interaction
• Unlocking Real-Time Conversational AI: Speed and Versatility
• Performance Benchmarks and Accessibility
• Actionable Steps for Developers and Innovators
• Conclusion
• Frequently Asked Questions (FAQs)

The landscape of artificial intelligence is constantly evolving, with new breakthroughs pushing the boundaries of what’s possible. One of the most critical frontiers remains the quest for truly seamless, real-time human-computer interaction, especially in voice AI. Traditional approaches often grapple with latency, complex pipelines, and resource demands that hinder widespread adoption on everyday devices. This challenge demands innovative solutions that integrate intelligence, speed, and efficiency.

Into this dynamic environment steps Liquid AI, making a significant stride with its latest release. They’re aiming to redefine how we interact with AI by tackling the core issues of responsiveness and architectural complexity head-on. Their new model promises to streamline the entire audio-language process, from understanding spoken words to generating articulate responses, all within milliseconds.

A New Paradigm for Audio-Language Interaction

In a move set to reshape real-time AI, Liquid AI has released LFM2-Audio-1.5B, a compact audio–language foundation model that both understands and generates speech and text through a single end-to-end stack. It positions itself for low-latency, real-time assistants on resource-constrained devices, extending the LFM2 family into audio while retaining a small footprint. This single-stack approach is precisely what differentiates LFM2-Audio-1.5B from many contemporary models, which often rely on a series of disconnected components.

But what’s actually new and innovative in its design? The core lies in its unified backbone with a brilliantly disentangled audio I/O system. LFM2-Audio extends the existing 1.2B-parameter LFM2 language backbone, allowing it to treat both audio and text as first-class sequence tokens. A crucial architectural decision was made to disentangle audio representations: continuous embeddings are projected directly from raw waveform chunks, approximately 80 milliseconds in duration, for inputs. In contrast, outputs are discrete audio codes. This clever design effectively bypasses discretization artifacts on the input path, ensuring high fidelity, while simultaneously maintaining autoregressive training and generation for both modalities on the output path.

Under the hood, this compact yet powerful model leverages a sophisticated combination of technologies. The backbone is the proven LFM2 architecture, incorporating a hybrid of convolutional and attention mechanisms, comprising 1.2 billion parameters focused on language. For audio encoding, it uses a FastConformer, boasting around 115 million parameters (specifically, the canary-180m-flash variant). The audio decoding is handled by an RQ-Transformer, which predicts discrete Mimi codec tokens across eight distinct codebooks. This robust setup is further supported by a substantial context window of 32,768 tokens, a text vocabulary of 65,536 tokens, and an audio vocabulary of 2049×8 tokens. Operating at bfloat16 precision under the LFM Open License v1.0, LFM2-Audio-1.5B currently supports English, making it ready for a wide array of applications.

Unlocking Real-Time Conversational AI: Speed and Versatility

The true power of LFM2-Audio-1.5B emerges in its ability to deliver astonishing speed, which is paramount for natural, interactive experiences. Latency has historically been a major bottleneck for voice-based AI agents, leading to frustrating pauses and unnatural interactions. Liquid AI’s innovation directly addresses this with remarkable results:

Liquid AI team reports end-to-end latency below 100 ms from a 4-second audio query to the first audible response—a proxy for perceived responsiveness in interactive use—stating it is faster than models smaller than 1.5B parameters under their setup. This sub-100 millisecond response time is a game-changer, making conversational AI feel less like talking to a machine and more like interacting with a human. Such speed is critical for maintaining engagement and providing a truly fluid user experience.

Beyond its raw speed, LFM2-Audio-1.5B offers two distinct generation modes, each optimized for different real-time agent scenarios:

• Interleaved generation: This mode is designed for live, speech-to-speech chat. The model dynamically alternates between generating text and audio tokens, a process that minimizes perceived latency by allowing the system to start vocalizing its response even before the full output is composed. This creates a much more responsive and natural conversational flow.
• Sequential generation: Ideal for more traditional Automatic Speech Recognition (ASR) to Text-to-Speech (TTS) applications, where the model switches modalities turn-by-turn. This mode supports scenarios where a complete transcription is needed before a full audio response, or vice-versa.

A Real-World Example: Imagine you’re driving and need to ask your in-car assistant for directions. With LFM2-Audio-1.5B, you speak your destination, and almost instantaneously, before you’ve even finished your sentence, the assistant begins to respond with the first instruction, “Turn left at the next intersection.” There’s no awkward silence, no noticeable delay, just a smooth, natural conversation that keeps your focus on the road. This technology transforms passive assistants into active, intuitive co-pilots.

This approach fundamentally matters in the voice AI trends because most “omni” stacks currently couple ASR, then an LLM, and then TTS. This multi-step process introduces inherent latency and creates brittle interfaces between components. LFM2-Audio’s single-backbone design, with its continuous input embeddings and discrete output codes, drastically reduces “glue logic.” This architectural elegance directly enables interleaved decoding for early audio emission, fundamentally simplifying the pipeline. For developers, this translates to less complex systems and dramatically faster perceived response times, all while retaining comprehensive support for ASR, TTS, classification, and sophisticated conversational agents from a single, unified model.

Performance Benchmarks and Accessibility

The capabilities of LFM2-Audio-1.5B are not just theoretical; they are rigorously validated through performance benchmarks. On VoiceBench—a comprehensive suite of nine audio-assistant evaluations introduced in late 2024 for LLM-based voice assistants—Liquid AI reports an impressive overall score of 56.78 for LFM2-Audio-1.5B. Specific per-task numbers, such as AlpacaEval at 3.71, CommonEval at 3.49, and WildVoice at 3.17, further illustrate its robust performance. The Liquid AI team proudly contrasts these results with those of larger models like Qwen2.5-Omni-3B and Moshi-7B, demonstrating LFM2-Audio’s competitive edge despite its compact size.

Further performance validation comes from the model card available on Hugging Face, which provides additional VoiceBench tables and includes classic Automatic Speech Recognition (ASR) Word Error Rates (WERs). Here, LFM2-Audio shows remarkable proficiency, matching or even improving upon the performance of Whisper-large-v3-turbo for several datasets, despite LFM2-Audio-1.5B being a generalist speech-text model. For instance, on the AMI dataset, it achieves a WER of 15.36 compared to Whisper-large-v3-turbo’s 16.13. Similarly, for LibriSpeech-clean, LFM2-Audio-1.5B records an impressive 2.03 WER against Whisper-large-v3-turbo’s 2.10. These figures underscore its accuracy and efficiency across diverse speech scenarios.

Liquid AI also ensures high accessibility for developers keen to experiment and integrate this technology. They provide a dedicated Python package, liquid-audio, alongside a Gradio demo, allowing users to easily reproduce and explore the model’s behaviors. This commitment to open access and developer tools fosters innovation and quick adoption, enabling a broader community to leverage its advanced capabilities.

Conclusion

Liquid AI’s release of LFM2-Audio-1.5B marks a significant milestone in the journey towards truly intuitive and instantaneous human-computer interaction. By pioneering an end-to-end, unified architecture with sub-100 ms response latency, Liquid AI has effectively dismantled many of the traditional barriers to real-time conversational AI. This compact, yet powerful, audio-language foundation model is not just an incremental improvement; it’s a paradigm shift, enabling developers to build more natural, responsive, and efficient AI assistants, especially for resource-constrained environments.

LFM2-Audio-1.5B stands out by merging sophisticated language understanding and generation with high-fidelity audio processing into a single stack. Its verified performance on benchmarks like VoiceBench and its competitive ASR results against larger, specialized models underscore its versatility and robustness. As AI continues to integrate more deeply into our daily lives, models like LFM2-Audio-1.5B will be crucial in making those interactions not just smart, but genuinely seamless and engaging. The future of voice AI is undoubtedly faster, smarter, and more unified.

Frequently Asked Questions (FAQs)

What is LFM2-Audio-1.5B?

LFM2-Audio-1.5B is a compact, end-to-end audio-language foundation model developed by Liquid AI. It is designed to understand and generate both speech and text through a single architectural stack, optimized for low-latency, real-time AI assistants on resource-constrained devices.

What is the key innovation of LFM2-Audio-1.5B?

Its key innovation lies in its unified architecture with a disentangled audio I/O system, allowing it to achieve sub-100 ms end-to-end latency from audio query to first audible response. This bypasses the typical multi-component pipeline (ASR-LLM-TTS) which introduces significant delays.

How fast is LFM2-Audio-1.5B?

The Liquid AI team reports an impressive end-to-end latency below 100 ms for a 4-second audio query to the first audible response, making it significantly faster than many comparable models and enabling truly natural, real-time conversational experiences.

What are the main applications of LFM2-Audio-1.5B?

LFM2-Audio-1.5B is ideal for real-time conversational AI, including AI assistants, smart home devices, in-car systems, and other applications requiring instantaneous speech-to-speech or speech-to-text interactions. It supports both interleaved and sequential generation modes for various use cases.

Is LFM2-Audio-1.5B open source?

The model operates under the LFM Open License v1.0, and Liquid AI provides a dedicated Python package (liquid-audio) and a Gradio demo for developers to experiment and integrate the technology.