LFM And Hybrid Operator Specification, Reference Oracles, And Portable Conformance

Full report

On this page

1. Exact Case-Study Identity and Source Hierarchy

The primary case study selected for this architectural specification and runtime conformance evaluation is the LiquidAI/LFM2.5-1.2B-Instruct model1. This model represents a modern hybrid architecture combining structured Linear-Input-Varying (LIV) convolutions with Grouped-Query Attention (GQA), optimized explicitly for resource-constrained edge environments, mobile platforms, and embedded execution1. The selection of this specific revision is predicated on its complex dual-state lifecycle—requiring both fixed-size recurrent buffers and dynamically growing attention caches—which rigorously stresses the safety and memory management of any portable Rust or WebAssembly (WASM) runtime7. The authoritative source hierarchy relies on the public Hugging Face repository, pinning the main branch to the immutable commit revision 868df744. Evidence derived for operator semantics, tensor constraints, and tokenization behavior originates entirely from the public source code within the transformers library, specifically the modeling\_lfm2.py file, which dictates the exact tensor operations for the forward pass8. No private TinyRustLM code, local runtime logs, or undocumented proprietary formats were accessed or validated; all specifications herein are derived strictly from verifiable public configuration files, network definitions, and the official transformers reference implementations8.

2. Artifact and License Inventory

The deterministic reconstruction of the model's inference graph demands an exact inventory of immutable repository artifacts. A structurally valid translation to a portable format relies on these cryptographically verifiable files to guarantee runtime numerical parity and behavioral compliance. Public code licenses governing the execution reference and model-weight licenses governing the parameters are separate legal frameworks and must be audited independently.

Authoritative Artifact Inventory

License Audit

The legal framework governing the weights of LiquidAI/LFM2.5-1.2B-Instruct is the "LFM Open License v1.0"12. An audit of this license reveals that it is structurally based on the Apache 2.0 license but introduces a strict commercial-use limitation. The license permits perpetual, royalty-free reproduction, modification, and distribution of the model and its derivatives13. However, rights to use the model or its derivatives for commercial purposes automatically terminate if the licensee's corporate entity exceeds $10 million USD in annual revenue12. The LFM Open License v1.0 contains no "copyleft" requirement; developers are not compelled to distribute fine-tuned weights or architectural derivatives12. Modifications must be marked, and original attribution notices must be retained15. Conversely, the reference implementation code found within the transformers library (modeling\_lfm2.py) is governed by the standard Apache 2.0 license without revenue thresholds8. Because the weight license terminates immediately upon a breach of the commercial threshold, automated continuous deployment pipelines within large enterprises must implement strict compliance gating to prevent unauthorized incorporation of these weights into production systems14.

3. Formal Forward-Pass Specification

The forward pass of the LFM2.5 architecture operates as a highly specific hybrid graph. It interleaves standard transformer attention elements with Linear-Input-Varying (LIV) depthwise convolutions. The derivation below translates the verified modeling\_lfm2.py source into strict mathematical notation, marking the provenance of each formulation. Unsupported or ambiguous semantics must fail closed in the target Rust runtime.

3.1. Embedding and Pre-Processing

For a sequence of input tokens [Figure omitted from source export], the token embedding lookup retrieves the initial dense representation. The vocabulary size is 65,536. [Figure omitted from source export] Where [Figure omitted from source export] \[Source: derived from public configuration files\]3.

3.2. Decoder Layer Iteration

The network consists of [Figure omitted from source export] sequential layers. A configuration array, layer\_types, dictates the specific mathematical operation for each layer index [Figure omitted from source export]. Specifically, the 16 layers adhere to the pattern: \["conv", "conv", "full\_attention", "conv", "conv", "full\_attention", "conv", "conv", "full\_attention", "conv", "full\_attention", "conv", "full\_attention", "conv", "full\_attention", "conv"\] \[Source: derived from public configuration files\]11. For every layer [Figure omitted from source export], the input [Figure omitted from source export] is first normalized using a Root Mean Square Normalization (RMSNorm) with an epsilon value [Figure omitted from source export]: [Figure omitted from source export] \[Source: derived from Lfm2RMSNorm public implementation\]8.

Branch A: Short Convolution Block (Lfm2ShortConv)

If the configuration dictates layer\_types\[i\] \== "conv", the layer applies a depthwise causal convolution8. The normalized state [Figure omitted from source export] is linearly projected to three times the hidden dimension: [Figure omitted from source export] The projected matrix is chunked equally into the three constituent matrices [Figure omitted from source export] and [Figure omitted from source export]. A gating mechanism scales the input: [Figure omitted from source export] A 1D depthwise convolution is subsequently applied to [Figure omitted from source export] along the sequence dimension. The convolution uses a kernel size of [Figure omitted from source export] and requires a left-side padding of [Figure omitted from source export] elements to maintain causality and prevent future token leakage:$$V\_s^{(c)} \= \\sum\_{j=0}^{2} W\_{conv}^{(c, j)} \\cdot Bx\_{s-j}^{(c)}$$Where [Figure omitted from source export] represents the channel index, and [Figure omitted from source export] represents the current sequence step. The output of the convolution is then multiplicatively gated by the [Figure omitted from source export] projection: [Figure omitted from source export] The final activation of the convolution block is projected back into the base hidden dimension: [Figure omitted from source export] \[Source: derived from Lfm2ShortConv public source code\]8.

Branch B: Grouped Query Attention (Lfm2Attention)

If the configuration dictates layer\_types\[i\] \== "full\_attention", a Grouped Query Attention (GQA) mechanism processes the normalized state [Figure omitted from source export]8. [Figure omitted from source export] Rotary Positional Embeddings (RoPE) are applied to the Query and Key tensors using a defined base frequency [Figure omitted from source export]11. $$Q', K' \= \\text{RoPE}(Q), \\text{RoPE}(K)$$The attention weights are calculated utilizing [Figure omitted from source export] query heads and [Figure omitted from source export] key-value heads, necessitating a broadcasting operation where every 4 query heads share 1 KV head during the scaled dot-product11. [Figure omitted from source export] [Figure omitted from source export] \[Source: derived from standard transformers RoPE algorithms and Lfm2Attention abstractions\]8.

Residual and Feed-Forward Network (Lfm2MLP)

Regardless of the hybrid branch taken, the layer concludes by applying a residual connection and a SwiGLU-based feed-forward network8. [Figure omitted from source export] [Figure omitted from source export] The Multilayer Perceptron (MLP) applies a custom SwiGLU projection utilizing an intermediate expanded dimension [Figure omitted from source export]11: [Figure omitted from source export] [Figure omitted from source export] \[Source: derived from Lfm2MLP public source code\]8.

3.3. Output Head

Following the sequence of 16 alternating blocks, a final RMSNorm is applied before a linear projection transforms the hidden state into the vocabulary space11. [Figure omitted from source export] [Figure omitted from source export] \[Source: derived from generalized Lfm2Model configuration parameters\]11.

4. Tensor and State Schema

Implementing the model safely in a portable Rust runtime demands strict adherence to the tensor schema. Guessed tensor names or mismatched storage layouts invariably lead to catastrophic memory corruption in WASM's linear memory space. The following matrix outlines the explicitly required tensors, derived directly from the configuration metadata and Python module definitions8.

Native PyTorch evaluates linear layers via [Figure omitted from source export]. Consequently, portable implementations parsing the .safetensors payload must mathematically transpose the loaded weight matrices if the underlying SIMD instructions perform [Figure omitted from source export], or they must utilize BLAS libraries explicitly configured for row-major, right-side accumulation to avoid continuous memory transposition overheads.

5. Prefill, Decode, Reset, and Cancellation State Machines

The state machine orchestrating LiquidAI/LFM2.5-1.2B-Instruct diverges structurally from pure Transformers due to its hybrid recurrent and attention-based nature7. Native Rust and browser WASM deployments require the rigorous management of linear memory across two distinct, unsynchronized types of state vectors.

5.1. State Creation and Initialization

A single generation session dictates the allocation of two completely distinct state buffers. For the 10 LIV convolution layers, the model requires a fixed, rolling buffer of shape \[hidden\_size, L\_cache \- 1\]. Given an [Figure omitted from source export] of 3, this buffer resolves to \[2048, 2\] per layer8. Upon sequence initialization, this buffer is strictly zeroed to simulate the implicit padding of the first token. Simultaneously, for the 6 GQA layers, the model allocates a dynamically growing or pre-allocated PagedAttention block cache7. The maximum logical shape per layer is \[num\_kv\_heads, max\_seq\_len, head\_dim\], equating to \[8, 32768, 64\].

5.2. Prefill Phase

During the prefill phase, the execution engine evaluates a contiguous chunk of prompt tokens [Figure omitted from source export] concurrently. Within the convolution blocks, the engine pads the left boundary of the token block with either the existing Conv State (retained from a prior prompt chunk) or zeros (if this is the absolute beginning of the sequence). The 1D convolution executes over the sequence dimension in parallel. Upon concluding the block evaluation, the final 2 tokens of the intermediate activation [Figure omitted from source export] are extracted, isolated, and permanently written into the Conv State Buffer to serve as the historical padding for the upcoming decode phase8. Concurrently, within the attention blocks, the system computes [Figure omitted from source export] and [Figure omitted from source export] representations for all tokens in [Figure omitted from source export], applies the RoPE rotations, and sequentially appends them to the Attention KV Cache8.

5.3. One-Token Decode Phase

During autoregressive generation, the sequence length processed per iteration is exactly 1\. The convolution update requires the system to shift the existing 2-token Conv State Buffer to the left, discarding the oldest token representation. The newly generated activation [Figure omitted from source export] is appended to the right edge of this buffer. The convolution kernel then executes a highly constrained dot-product, multiplying the 3-element buffer against the conv.weight parameters8. Meanwhile, the attention update generates a single [Figure omitted from source export] pair, mutates the KV cache at the specific memory address corresponding to the current seq\_len index, and executes a vector-matrix multiplication across the entire historical cache7.

5.4. Lifecycle: Sequence Reset, Prompt Replacement, and Cancellation

Memory boundaries must never leak across discrete sequence generations. A sequence reset must forcefully invoke a deterministic zeroing out of the Conv State Buffer and mathematically reset all KV Cache internal index pointers to 0\. Prompt replacement—such as re-evaluating a modified prompt with a shared prefix—requires isolating the mutation index, discarding the KV cache beyond the shared prefix length, and critically, restoring the Conv State Buffer to the exact, immutable historical snapshot that was captured at that specific prefix index. Cancellation and model switching present severe risks in a WASM environment because linear memory cannot rely on OS-level page reclamation routines. If an inference task is abruptly cancelled mid-generation by the host, the browser environment must invoke a specific Rust Drop trait routine that cleanly unbinds the heap allocations for the dynamic KV cache. Model switching requires synchronously dropping all graph parameters and forcing garbage collection before allocating the new model structure; failure to do so will trivially exceed the browser's strict 4GB WASM32 contiguous memory limit.

6. Memory and Compute Formulas

Precise, mathematically bounded memory formulas dictate whether the architecture successfully executes within WASM limits, mobile Neural Processing Units (NPUs), or constrained cloud instances5.

6.1. Weight Memory

The total active parameter count for this dense model is officially documented as [Figure omitted from source export] parameters17.

• BF16/FP16 Base: [Figure omitted from source export]4.
• Q4\_K\_M Quantization: A widely deployed format that averages approximately [Figure omitted from source export], yielding [Figure omitted from source export] of physical weight storage17.

6.2. State Memory

The primary architectural efficiency of the LFM2.5 design relies on its radically minimized state footprint7. The total state memory is the summation of the static convolution state and the dynamic KV cache.

• Conv State: Calculated as [Figure omitted from source export]. Assuming an FP32 inference target for highest precision, this requires [Figure omitted from source export] ([Figure omitted from source export]). This allocation remains completely static and is negligible compared to standard KV caches7.
• KV Cache (GQA): Because attention is restricted to only 6 layers, the cache size is [Figure omitted from source export]. Per generated token evaluated in BF16, the cost is [Figure omitted from source export]. At the maximum officially supported context boundary of 32,768 tokens, the absolute peak KV cache memory is [Figure omitted from source export]. By contrast, a 16-layer pure dense transformer of identical dimensions would consume over [Figure omitted from source export] of KV cache7.

6.3. Scratch, Activation, and Host Memory

During operator processing, intermediate activations demand transient scratch buffers. The largest intermediate tensor instantiated per token exists within the Lfm2MLP block, sized at [Figure omitted from source export] per sequence element11. In prefill mode processing a maximum batch size of 2048 tokens simultaneously, scratch memory bounds peak at [Figure omitted from source export]. Furthermore, the browser-host memory must safely allocate an additional systemic overhead of [Figure omitted from source export] to manage WASM-to-JS bridge buffers, string encoding arrays, and underlying allocator fragmentation.

7. Numerical-Risk Register

Implementing specific numerical kernels without relying on guesswork requires an exhaustive mapping of algorithmic risks. A failure to address these corner cases invariably causes divergent token generation and hallucination loops19. The configuration establishes the Lfm2RMSNorm epsilon value at [Figure omitted from source export]11. Squaring raw input tensors inside a bfloat16 or fp16 domain can trigger premature truncation to zero or overflow to infinity. Consequently, portable Rust implementations are strictly required to cast the hidden states [Figure omitted from source export] to FP32 before executing the squaring, accumulation, and division operations, later casting back to the native weight precision8. Long-sequence drift presents a secondary, cascading risk. The configuration scales the base frequency of the RoPE parameters to [Figure omitted from source export]11. At context lengths approaching the 32,768 boundary, the angular indices cause the trigonometric inputs to reach exceptionally large scalar values. The f32 implementations of sin and cos found within WebAssembly Math runtimes frequently diverge at high magnitudes from the native libm implementations compiled via GCC or Clang. This micro-divergence leads to an accumulating phase drift in attention scores, silently degrading semantic retrieval. Activation approximation introduces instability. The SwiGLU block computes [Figure omitted from source export], where [Figure omitted from source export], fundamentally requiring the calculation of [Figure omitted from source export]. WASM SIMD128 instruction sets do not possess a native, hardware-accelerated exponential operation. Any Taylor series approximations or bounded lookup tables engineered to bypass this limitation must be mathematically constrained to a [Figure omitted from source export] maximum absolute error threshold. Failing to enforce this boundary causes cascading, multiplicative divergence in the deep MLP outputs8. Finally, convolution padding and quantization accumulation represent edge-case vulnerabilities. During the evaluation of the initial token, the [Figure omitted from source export] history vector is artificially populated with literal zeros8. If the subsequent convolution kernel utilizes an unoptimized accumulation buffer, mathematically insignificant denormal numbers can propagate through the network. The runtime must explicitly instruct the CPU to flush denormals to zero (FTZ) to prevent massive computational stalling. When evaluating using integer-quantized Q8 or Q4 blocks, the kernel must execute the dot product accumulation entirely within a 32-bit integer boundary before applying the scaling factor to float, enforcing strict order-of-operations parity with the official PyTorch execution graph.

8. Reference-Oracle and Test-Vector Format

Before undertaking optimizations for portable SIMD or WebGPU acceleration, a deterministic scalar reference oracle must be constructed. This oracle ingests a verified, bounded test vector and validates mathematical parity across the execution stack. The proposed JSON schema for bounding and transmitting non-secret test vectors guarantees cross-runtime verification:

JSON { "schema\_version": "1.0", "model\_identity": { "repo": "LiquidAI/LFM2.5-1.2B-Instruct", "revision": "868df74", "dtype": "fp32" }, "test\_cases": \[ { "test\_id": "prefill\_short\_sequence", "input\_ids": \[1, 452, 1123, 7\], "layer\_outputs": \[ { "layer\_index": 0, "type": "conv", "output\_hash": "sha256:...", "state\_snapshot\_hash": "sha256:..." }, { "layer\_index": 2, "type": "full\_attention", "output\_hash": "sha256:..." } \], "final\_logits\_top\_k": { "indices": \[345, 98, 12\], "values": \[12.45, 11.02, 9.88\] } } \], "tolerances": { "absolute\_error\_max": 1e-4, "relative\_error\_max": 1e-3 } }

The oracle evaluates embeddings, isolates a single LIV convolution layer, isolates a single GQA attention layer, and evaluates an aggregate prefill sequence followed by an incremental autoregressive decode. Cryptographic state hashes ensure the Conv State Buffer and Paged KV Cache internally align byte-for-byte with the PyTorch reference implementation exported during test vector creation.

9. Native/WASM Differential-Conformance Matrix

Rigorous conformance testing mandates a differential analysis across four independent environments: (1) The official transformers PyTorch reference, (2) the llama.cpp independent C++ backend, (3) a future Native Rust scalar runtime, and (4) the final WebAssembly (WASM) execution target. When evaluating Floating Point 32 (FP32) verification, the absolute error (AE) must remain strictly below [Figure omitted from source export]. For BF16 or FP16 verification, relative error (RE) becomes the governing standard due to the architecturally reduced mantissa precision, with a tolerance set at [Figure omitted from source export]. When evaluating aggressively quantized models (Q8/Q4), direct token agreement becomes a flawed metric; token agreement can easily conceal dangerous underlying logit drift that only manifests during prolonged, high-temperature generations. Therefore, a differential test comparing the top 100 logit distributions utilizing Kullback-Leibler (KL) divergence is mandatory, and this KL divergence must remain tightly bounded below [Figure omitted from source export].

Historical operational data reveals that llama.cpp implementations encountered severe server 500 errors specifically when attempting to parse the Pythonic tool-calling syntax generated by LFM models20. The Rust runtime must test sequence cancellation, prompt replacement, and tool-parsing resilience extensively to avoid mirroring these systemic lifecycle failures.

10. Proposed Portable Format Extension

Current runtime ecosystems frequently rely on unstructured GGUF format heuristics to inject custom or hybrid architectures, relying on hardcoded name-matching17. To safely deploy the LFM2.5 architecture without risking silent corruption or erroneous loading into standard dense attention frameworks, a versioned, explicitly typed extension to the .safetensors manifest or a dedicated .slm (Small Language Model) format header is proposed. The portable format must define a highly explicit, non-optional manifest header embedded in the binary blob:

JSON { "format\_version": "1.1.0", "architecture\_flag": "lfm2\_hybrid", "required\_operators": \["causal\_conv\_1d", "gqa\_rope", "swiglu\_mlp", "rms\_norm"\], "endianness": "little", "alignment": 64, "quantization\_metadata": { "type": "q4\_k\_m", "group\_size": 32 }, "tensor\_schema": { "layer\_types": \["conv", "conv", "full\_attention", "conv"\], "conv\_L\_cache": 3 }, "tokenizer\_identity": "sha256:..." }

By explicitly declaring required\_operators and defining the precise layer\_types array, unsupported legacy runtimes will safely fail closed during the header parsing phase, rather than attempting the catastrophic operation of loading 1D convolution weights into expected attention-projection operators. The alignment constraint is strictly set to 64 bytes to guarantee cache-line efficiency when memory-mapping the payload files natively or when loading array buffers directly into WASM linear memory.

11. Kernel Implementation and Optimization Sequence

Optimization routines must never mathematically precede deterministic conformance. Premature optimization of SIMD vectors or WebGPU shaders actively conceals logical errors originating in the complex causal convolution padding mechanisms. The implementation backlog proceeds strictly via dependent, chronological phases characterized by effort variables ([Figure omitted from source export]). The initial phase ([Figure omitted from source export]) demands a strict Source Audit and the construction of a Scalar Oracle. Engineers must implement Lfm2ShortConv, Lfm2Attention, and Lfm2MLP utilizing purely scalar, unoptimized Rust. The scalar oracle must ingest the FP32 JSON test vectors and produce outputs mathematically identical to the PyTorch reference, establishing the ground-truth control graph. The stop rule for this phase is absolute error equivalence on single-layer tests. The subsequent phase ([Figure omitted from source export]) involves the Native Differential Test. Developers must implement the multi-layer looping structures and the dual-state management system. They will execute multi-token prefill and single-token decode sequences against the scalar oracle. This phase incorporates the FTZ (Flush-To-Zero) instructions and enforces strict RMSNorm epsilon constraints. The stop rule is identical token output for a 1,000-token sequence. Following baseline conformance, the Quantization Verification phase ([Figure omitted from source export]) introduces the Q8 and Q4 block dequantization routines. Engineers verify the KL divergence of the resulting logits against the scalar baseline. Once quantization is stable, the project advances to Portable SIMD and WASM Parity ([Figure omitted from source export]). This phase systematically replaces the scalar matrix multiplications and 1D convolutions with std::simd intrinsic operations. This kernel requires meticulous end-to-end testing of WASM SIMD128 implementations to ensure that identical memory boundary handling and NaN propagation behaviors exist in the browser as they do on native AVX2/NEON hardware. Only after the single-threaded WASM module achieves 100% token agreement does the architecture advance to the WebGPU phase ([Figure omitted from source export]). WebGPU requires entirely rewriting the causal\_conv\_1d update logic in WGSL shaders, relying heavily on the pre-existing scalar oracle to debug workgroup boundaries and GPU buffer dispatch protocols.

12. Fail-Closed Admission Criteria

A model family must not be admitted into the portable runtime simply because another established backend, such as llama.cpp or vLLM, manages to execute it via conditional heuristics21. The portable runtime must enforce strict "fail-closed" rejection protocols to guarantee safety. If the incoming format manifest lacks an exact, unambiguous array mapping of conv versus full\_attention layers, the loader must immediately abort execution. Guessing layer arrangements based on parameter shape inference results in unpredictable memory violations during sequence decoding. Similarly, if the parsed L\_cache differs from the hardcoded value of 3, and the runtime lacks a registered kernel dynamically capable of supporting variable cache windows, the runtime must reject the model instantiation. Tokenization mismatch presents another critical failure vector. If the embedded tokenizer hash identity does not precisely match the prompt generator's expected ChatML schema, the model must not be initialized, as it will hallucinate structural tokens3. Finally, during active execution, if a prefill request attempts to evaluate without the user application explicitly passing properly initialized, sized state buffers for both the Convolution and GQA subsystems, the execution engine must panic to prevent evaluating against garbage memory.

13. Unknown or Contradictory Public Facts

An exhaustive architectural review of the public ecosystem surrounding LiquidAI/LFM2.5 highlights several technical contradictions that require extreme caution during implementation. Early marketing narratives and broad community commentary often characterize Liquid models as entirely replacing the concept of a KV cache with fixed-size recurrent states23. However, deep architectural audits of the provided PyTorch code and production deployment guides explicitly reveal that 25% of the network (6 out of 16 layers) consists of standard GQA layers that do utilize a linearly growing, standard KV cache7. The portable runtime must categorically disregard "cache-less" marketing claims and securely allocate dynamic KV cache structures. Furthermore, discrepancies exist surrounding tool calling syntax. Official documentation states the model produces Pythonic tool calls natively enclosed in \<|tool\_call\_start|\>\[func(arg="val")\]\<|tool\_call\_end|\>24. However, developers relying on OpenAI-compatible JSON tool schemas frequently encounter catastrophic parsing failures (such as server 500 errors) in backends attempting to parse these outputs20. The portable runtime must proactively implement robust, fail-safe PEG parsers specifically tuned for this proprietary Pythonic syntax rather than defaulting to generic JSON validation hooks.

14. Annotated Primary-Source Bibliography with Revisions and Dates

The architectural specifications, tensor mathematics, and numerical constraints incorporated into this specification derive exclusively from the following verified public repositories and artifacts:

• Hugging Face transformers Repository
• File Focus: src/transformers/models/lfm2/modeling\_lfm2.py
• Revision/Branch: main (Code structure verified reflecting updates through January 2026\)8.
• Retrieval Date: July 15, 2026\.
• Significance: Provides the authoritative PyTorch source code for the Lfm2ShortConv, Lfm2Attention, Lfm2MLP, and Lfm2RMSNorm forward pass equations, directly substantiating the hybrid computational layout.
• Liquid AI Model Repository
• File Focus: LiquidAI/LFM2.5-1.2B-Instruct (config.json, tokenizer.json, LICENSE)
• Immutable Revision: 868df744.
• Update Date: January 202626.
• Retrieval Date: July 15, 2026\.
• Significance: Validates the exact layer architecture (layer\_types array), the strict LFM Open License v1.0 parameters, and core structural dimensions (e.g., hidden size 2048, vocab size 65536\)11.
• Independent Backend Trackers (vLLM & llama.cpp)
• File Focus: Multiple GitHub issues and pull requests (e.g., ggml-org/llama.cpp\#23838, \#23852)20.
• Update Date: Mid-2025 to Early 2026\.
• Retrieval Date: July 15, 2026\.
• Significance: Documents the exact parsing crashes associated with the Pythonic tool-call notation and highlights numerical edge cases encountered during backend porting.
• Hardware and Deployment Architectural Guides
• File Focus: Spheron Network Deployment Guides and independent benchmarks.
• Update Date: 20267.
• Retrieval Date: July 15, 2026\.
• Significance: Provides authoritative operational proof detailing that proportional layers in LFM models inherently utilize GQA mechanisms, factually refuting the prevalent "no KV cache" assumption and dictating memory allocation strategies.

Works cited

1. Liquid Foundation Models, https://www.liquid.ai/models
2. LFM2.5-1.2B-Instruct \- Liquid Docs \- Liquid Foundation Models, https://docs.liquid.ai/lfm/models/lfm25-1.2b-instruct
3. LiquidAI/LFM2.5-1.2B-Instruct \- Hugging Face, https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct
4. LiquidAI/LFM2.5-1.2B-Instruct at main \- Hugging Face, https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/tree/main
5. LFM2.5-230M: Built to Run Anywhere — Blog \- Liquid AI, https://www.liquid.ai/blog/lfm2-5-230m
6. Liquid AI's LFM2.5-230M Beats Models Twice Its Size Running on a Raspberry Pi, https://alphasignal.ai/news/liquid-ai-s-lfm2-5-230m-beats-models-twice-its-size-running-on-a-raspberry-pi
7. Deploy Liquid AI LFM2 Models (LFM2-8B-A1B, LFM2-2.6B) on GPU Cloud: Hybrid Architecture Guide (2026) | Spheron Blog, https://www.spheron.network/blog/liquid-foundation-models-lfm-deployment-gpu-cloud-2026/
8. transformers/src/transformers/models/lfm2/modeling\_lfm2.py at main \- GitHub, https://github.com/huggingface/transformers/blob/main/src/transformers/models/lfm2/modeling\_lfm2.py
9. lfm2 \- vLLM Documentation, https://docs.vllm.ai/en/v0.11.0/api/vllm/model\_executor/models/lfm2.html
10. config.json · BenjaminHelle/LFM2.5-1.2B-Instruct-Code-V1.1 at main \- Hugging Face, https://huggingface.co/BenjaminHelle/LFM2.5-1.2B-Instruct-Code-V1.1/blame/main/config.json
11. config.json · LiquidAI/LFM2.5-1.2B-Thinking-ONNX at 58de6ab4b249ec83e4393d621a5b3f4056fddaca \- Hugging Face, https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking-ONNX/blob/58de6ab4b249ec83e4393d621a5b3f4056fddaca/config.json
12. LFM License | Liquid AI, https://www.liquid.ai/lfm-license
13. Pricing \- Liquid AI, https://www.liquid.ai/pricing
14. Liquid AI's smallest model yet LFM2.5-230M beats models 4X its size at data extraction, can run 'anywhere' | VentureBeat, https://venturebeat.com/technology/liquid-ais-smallest-model-yet-lfm2-5-230m-beats-models-4x-its-size-at-data-extraction-can-run-anywhere
15. Model License \- Liquid Docs, https://docs.liquid.ai/lfm/help/model-license
16. Liquid AI LFM2.5-1.2B-Thinking: Compact Power \- HowAIWorks.ai, https://howaiworks.ai/blog/liquidai-lfm2-5-1-2b-thinking-release
17. Liquid LFM2.5: How To Run & Fine-tune | Unsloth Documentation, https://unsloth.ai/docs/models/tutorials/lfm2.5
18. LiquidAI/LFM2.5-1.2B-Instruct-GGUF \- Hugging Face, https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF
19. LFM2.5-8B-A1B: An Even Better On-Device Mixture of Experts — Blog \- Liquid AI, https://www.liquid.ai/blog/lfm2-5-8b-a1b
20. Eval bug: LFM2.5-8B-A1B tool parser rejects documented \<|tool\_call\_start|\>\[...\]\<|tool\_call\_end|\> format · Issue \#23838 · ggml-org/llama.cpp \- GitHub, https://github.com/ggml-org/llama.cpp/issues/23838
21. llama.cpp \- Liquid Docs, https://docs.liquid.ai/deployment/on-device/llama-cpp
22. vLLM \- Liquid Docs, https://docs.liquid.ai/deployment/gpu-inference/vllm
23. \[R\] Announcing the first series of Liquid Foundation Models (LFMs) – a new generation of generative AI models that achieve state-of-the-art performance at every scale, while maintaining a smaller memory footprint and more efficient inference. : r/MachineLearning \- Reddit, https://www.reddit.com/r/MachineLearning/comments/1fvgo7o/r\_announcing\_the\_first\_series\_of\_liquid/
24. oamazonasgabriel/lfm2-1.2b-tool \- Ollama, https://ollama.com/oamazonasgabriel/lfm2-1.2b-tool
25. hadad/LFM2.5-1.2B:F16 \- Ollama, https://ollama.com/hadad/LFM2.5-1.2B:F16
26. LFM2.5-1.2B-Instruct (free) \- API Pricing & Benchmarks | OpenRouter, https://openrouter.ai/liquid/lfm-2.5-1.2b-instruct:free
27. Eval bug: LFM2.5-8B-A1B reasoning is not captured in reasoning\_content · Issue \#23852 · ggml-org/llama.cpp \- GitHub, https://github.com/ggml-org/llama.cpp/issues/23852