Behavioral Distillation and Architecture Transfer for TinyRustLM: A Contamination-Resistant Factorial Experiment

Full report

On this page

Executive Experimental Recommendation

The deployment of sub-billion parameter language models into edge environments, specifically browser-based runtimes such as WebAssembly or WebGPU for TinyRustLM, imposes severe, non-negotiable constraints on active parameter count, execution latency, and context memory footprint. The core objective of this experimental design is to determine whether the Liquid Foundation Model (LFM) architecture, characterized by its hybrid Linear Input Variant (LIV) convolution and Grouped Query Attention (GQA) blocks, provides a tangible and measurable advantage over state-of-the-art dense Transformers when behaviorally distilled for conversational assistance. Because historical evaluations of compact models demonstrate that raw, pre-trained base models rarely justify immediate production promotion based on baseline quality alone, the application of post-training behavioral distillation remains a critical hypothesis to rigorously test rather than a foregone conclusion. The recommended approach establishes a strictly controlled, multi-axis factorial experiment comparing the Liquid LFM2.5 family models, ranging from 230 million to 1.17 billion parameters, against highly optimized dense and alternative-hybrid baselines, including Qwen3.5-0.8B, SmolLM2-360M, and Granite-4.0-350M1. This report details a comprehensive pipeline for sequence-level knowledge distillation utilizing a synthetically generated, mathematically decontaminated curriculum of one hundred thousand records. By strictly adhering to sequence-level supervised fine-tuning (SFT) and Direct Preference Optimization (DPO) while explicitly rejecting token-level Kullback-Leibler (KL) divergence distillation, the protocol seamlessly bypasses the severe architectural incompatibilities introduced by the highly divergent vocabulary sizes and embedding dimensions across the student models. The resulting experiment establishes exact promotion criteria, isolated ablation matrices, and catastrophic forgetting regression budgets to ensure that the adaptation of narrow, harmless conversational behavior does not overwrite the fundamental pre-trained capabilities of the base students.

Student and Teacher Candidate Matrix

The selection of optimal student architectures requires balancing theoretical execution efficiency within a browser environment against practical convergence properties during parameter-efficient fine-tuning. The evaluation matrix pits the Liquid LFM2.5 architecture against contemporary dense Transformers and alternative hybrid networks to isolate the impact of the underlying mathematical structure from the post-training curriculum. The LFM2.5 family utilizes a device-optimized structure that replaces the vast majority of traditional self-attention mechanisms with zero-cache LIV convolution blocks. These blocks are punctuated by periodic GQA layers responsible for global sequence coordination5. For example, the LFM2.5-350M variant consists of sixteen total layers, comprising ten double-gated LIV convolution blocks and six GQA blocks5. This architectural decision yields a substantially reduced key-value (KV) cache footprint, as the LIV layers process tokens through a structured recurrent update without maintaining an attention history over past tokens, effectively optimizing activation memory for long-context edge inference7. Conversely, the dense and alternative-hybrid baselines provide established gradient flow dynamics, extensive ecosystem support, and universal compatibility with standard low-rank adaptation abstractions.

The LFM2.5 models utilize a tied embedding configuration, requiring specific attention during optimization to ensure that the language modeling head and the input token embeddings remain synchronized without destabilizing the loss gradients10. Furthermore, the LFM Open License v1.0 explicitly permits commercial deployment only up to a ten million dollar annual revenue threshold, requiring diligent tracking of corporate scaling metrics, which stands in contrast to the universally permissive Apache 2.0 licenses governing the SmolLM2 and Granite models12. The Qwen3.5-0.8B model introduces an alternative hybrid approach via Gated DeltaNet layers, functioning as linear attention combined with a recurrent delta rule. However, its massive vocabulary expansion of 248,320 tokens heavily impacts the initial prefill memory boundary due to the sheer size of the embedding table, creating potential deployment friction within severely constrained browser sandboxes2. For the teacher ensemble, the generation protocol mandates the utilization of frontier-class large language models operating in the 70-billion to 120-billion parameter regime. Models such as the open-source 120B reasoning engines are ideal candidates for this role, as they possess the immense parameter capacity necessary to internalize complex generation rubrics and produce high-fidelity synthetic conversational data16. The teacher models serve exclusively as synthetic data generators and adjudicators. By restricting the teacher's role to data synthesis, the protocol effectively transfers behavioral patterns to the student models without requiring the compact students to emulate the hidden, internal reasoning steps or latent dimensionalities of the teacher, ensuring the resulting artifact remains a rapid, direct assistant.

LFM Training-Stack Capability Audit

The introduction of non-standard computational blocks, specifically the LIV convolutions present in the LFM2.5 architecture, fundamentally disrupts naive assumptions deeply embedded within standard training libraries. Standard implementations rely heavily on predictable weight distributions, uniform gradient checkpointing paths, and automated Low-Rank Adaptation (LoRA) module targeting designed explicitly for multi-head self-attention. An exhaustive audit of the current machine learning operations ecosystem reveals distinct gradients of support and critical configuration requirements for the LFM architecture. The LEAP-Finetune repository serves as the official open-source orchestration stack for Liquid models, providing dataset formatting, distributed launch mechanisms, and explicit support for both text and multimodal variants18. This stack integrates deeply with the Transformer Reinforcement Learning (TRL) library, which automatically handles sequence packing, ChatML template application, and Supervised Fine-Tuning (SFT) mapping without requiring manual tensor reshaping for the underlying LIV blocks21. For distributed execution, LEAP-Finetune natively supports environments such as SLURM, Kubernetes via KubeRay, and serverless GPU clouds like Modal19. However, the most critical element of the training stack audit involves the Unsloth library, which provides mathematically optimized kernels capable of reducing Video RAM (VRAM) requirements by up to seventy percent while accelerating execution speeds23. While Unsloth natively supports LFM2.5 models, standard LoRA target module discovery scripts rely on identifying q\_proj and v\_proj matrices located inside standard self-attention mechanisms24. Because the LFM architecture replaces up to seventy-five percent of these attention layers with convolutions, generic parameter-efficient fine-tuning configurations will fail to adapt the network effectively7. To bypass this architectural blind spot and ensure mathematical credibility during the backward pass, the training script must explicitly declare the LFM-specific linear projections—specifically targeting in\_proj and out\_proj alongside the standard feed-forward networks w1, w2, and w3, in addition to the standard attention projections within the minority GQA blocks26. Gradient checkpointing within this optimized stack is completely validated, specifically utilizing the "unsloth" checkpointing parameter which selectively recomputes activations at optimal block boundaries rather than requiring full computational graph traversal, safely and dramatically reducing activation memory when approaching the 32,768-token sequence limit23. Mixed precision via bfloat16 is strictly mandated for all gradient updates, avoiding the catastrophic float16 numerical overflow issues inherent in the multiplicative gating mechanisms of the LIV convolutional blocks4. While Quantization-Aware Training (QAT) via dynamic NVFP4 or INT4 is supported within the broader ecosystem, deploying QAT during the primary behavioral distillation phase introduces severe confounding variables for experimental validation9. Therefore, all primary experimental training must remain in raw bfloat16 precision before post-training export into execution formats such as GGUF, ONNX, or MLX32.

Method Comparison by Compatibility and Compute

Behavioral distillation into compact models requires selecting optimization algorithms that mathematically survive the architectural transition from a highly parameterized, dense teacher to a convolution-heavy, sub-billion parameter student. Methodologies that assume structural parity between the models will inevitably fail when applied to the diverse candidate matrix. Token-level Kullback-Leibler (KL) divergence, commonly utilized in direct knowledge distillation, attempts to minimize the statistical distance between the full continuous probability distributions of the teacher and the student at every single generation step. However, this method assumes a strict one-to-one mapping of token vocabularies. As identified in the candidate matrix, the LFM2.5 utilizes a 65,536-token vocabulary, while the Qwen3.5 model utilizes 248,320 tokens, and SmolLM2 operates on 49,152 tokens2. Consequently, continuous logit-matching is mathematically undefined without the introduction of destructive, computationally expensive projection matrices that introduce severe noise into the gradients. Similarly, hidden-state matching requires identical internal dimensionalities and layer depths, which the LIV convolutions and Gated DeltaNets explicitly reject. Therefore, the experiment must rely entirely on sequence-level optimization methods.

The methodology relies fundamentally on Assistant-Only Supervised Fine-Tuning combined with Low-Rank Adaptation (LoRA). By masking the user prompt in the loss calculation and applying cross-entropy exclusively to the teacher's final output sequence, the student maps its internal representations to the target text regardless of vocabulary structure. Full fine-tuning updates all model parameters and is overly destructive to the pre-trained weights of sub-billion parameter models, drastically increasing the risk of catastrophic forgetting21. Weight-Decomposed Low-Rank Adaptation (DoRA) offers theoretical advantages but introduces unnecessary implementation complexity for the LFM's custom convolution blocks. Upper-layer or attention-only adapters are entirely inappropriate for LFMs, as seventy-five percent of the network relies on non-attention convolution blocks that must be updated to alter conversational behavior7. To achieve optimal parameter efficiency and stability, the experiment incorporates Rank-Stabilized LoRA (rsLoRA). By setting the scaling factor mathematically to the alpha value divided by the square root of the rank, rather than dividing by the rank itself, training stability is dramatically improved when exploring the higher rank dimensions required to encapsulate broad conversational rules within small neural networks24. Direct Preference Optimization (DPO) serves as an optional secondary stage to align the model against specific negative behaviors without requiring the complex reward model infrastructure of traditional reinforcement learning36.

Teacher Ensemble and Generation Protocol

The synthesis of the curriculum requires an automated, robust pipeline to generate diverse, uncontaminated data that genuinely teaches decision-making rules rather than surface-level memorization. The generation protocol relies on a screened teacher ensemble, explicitly utilizing open-weight 120B reasoning engines to ensure the resulting synthetic data is free from restrictive commercial API terms of service that prohibit training competing models16. The teacher ensemble operates under a dual-phase decoding strategy. During the generation of raw user prompts and scenario contexts, the temperature is elevated (e.g., Temperature 1.0, Top-P 0.95) to maximize syntactic variance, vocabulary diversity, and structural entropy2. However, when the teacher is prompted to generate the target assistant response, the decoding parameters are strictly constrained (e.g., Temperature 0.1 to 0.2) to ensure deterministic, highly logical, and functionally sound outputs that serve as clean gradients for the student5. The protocol utilizes a production trace extraction methodology. By extracting sanitized, unstructured traces from real-world conversational logs and feeding them into the teacher model as domain context, the synthetic training examples accurately reflect the vocabulary, phrasing, and edge cases of actual user traffic rather than the teacher model's generic priors38. The generation framework explicitly demands final-answer extraction without rewriting; the teacher must not emit hidden reasoning tokens or chain-of-thought traces, as the sub-billion parameter students lack the context window and parameter depth to effectively emulate extended internal monologues40. An intermediate rubric review validation step utilizes a separate instance of the teacher model acting as an LLM-as-a-judge. This adjudicator scores the generated responses on a rigorous one-to-five scale for inference clarity, factual adherence, and utterance coherence38. Any record failing to achieve a perfect score is discarded. Disagreement adjudication is handled deterministically: if the primary generator and the adjudicator conflict, the record is purged to prevent the student from internalizing ambiguous loss signals. All generated records pass through a deterministic regular expression pipeline to strip malicious teacher artifacts, such as the ubiquitous "As an AI language model..." preamble, and to enforce strict Personally Identifiable Information (PII) removal. To ensure absolute scientific reproducibility, the generation pipeline records the exact deterministic replay parameters, specific teacher checkpoint hashes, and the origin seed of every accepted record within an immutable provenance ledger.

Curriculum Taxonomy with Target Counts

The curriculum must instruct the student in generalized decision rules, preventing the internalization of brittle, memorized responses. The distribution of data must enforce direct, positive assistance while establishing a firm but narrow boundary against concrete harms. Broad moralizing and excessive refusal protocols rapidly degrade the utility of sub-billion parameter models, as they lack the parameter capacity to distinguish nuanced edge cases and will frequently default to systemic refusal if overly constrained. The total curriculum size is scaled to exactly 100,000 records. Empirical evidence in sub-billion parameter distillation indicates that exceeding this threshold frequently induces catastrophic forgetting of the pre-training distribution without adding substantial behavioral quality, while dropping below 25,000 records fails to generalize across unseen paraphrases18.

Tool use training must strictly adhere to the target model's native formatting expectations. For instance, LFM2.5 defaults to Pythonic function calls encapsulated between \<|tool\_call\_start|\> and \<|tool\_call\_end|\> tokens5. Forcing an alternative, non-native JSON structure through post-training without deeply modifying the tokenizer's pre-trained expectations severely degrades generation reliability and increases syntax errors18.

Scenario-Factor Generation

To guarantee that the student model learns generalized rules rather than surface-level heuristics, scenario factors must be permuted extensively during generation. The generation protocol systematically cycles through diverse counterfactual resources, intentionally providing the teacher with fictional rulesets, alternate physical laws, or hypothetical company policies to ensure the resulting text relies on the provided context rather than pre-trained memorization. Authority permutations are injected by varying the perceived persona of the user—ranging from anonymous guests to system administrators—teaching the model to adjust its tone and compliance based on declared authorization states within the system prompt. Entity substitution algorithms automatically replace names, dates, units of measurement, and geographical locations within the generated text, multiplying the dataset diversity without requiring additional teacher compute. The pipeline introduces implicit negations and hard negatives, presenting the model with requests that subtly violate constraints or request impossible state changes, forcing the student to explicitly recognize the negation rather than blindly complying. Adversarial quoted instructions are embedded within the user prompts, testing the model's ability to distinguish between text that should be analyzed versus text that should be executed. Furthermore, answer-length variations are explicitly requested during teacher generation, ensuring the student learns to modulate verbosity based on the user's implicit or explicit requirements. Teacher disagreement is simulated by providing conflicting facts in the prompt history, forcing the model to articulate the discrepancy. Crucially, the entire curriculum is subjected to strict family-level splits, ensuring that a specific surface template or logical framing utilized in the training set cannot appear in the evaluation set, thereby preventing synthetic data memorization.

Contamination and Preservation Controls

The epistemological validity of this entire experiment hinges unconditionally on preventing the training curriculum from leaking the evaluation holds. If a model encounters the structural templates or semantic framing of the evaluation benchmark during training, the resulting metric improvements represent data contamination and benchmark saturation rather than genuine architectural capability transfer44. An independent auditor must be capable of verifying this strict separation without ever gaining access to the private evaluation prompts. To construct a mathematically contamination-resistant environment, the protocol deploys a multi-tiered filtration system. First, exact normalization is applied to all training and evaluation text, stripping punctuation, converting characters to lower case, and normalizing whitespace. Following this, a rigid 13-n-gram character and token overlap detection algorithm sweeps the entire curriculum45. Any training record sharing a contiguous 13-n-gram sequence with the evaluation set is immediately flagged and expunged. However, n-gram filtering is notoriously insufficient against paraphrased contamination. The protocol subsequently projects all records into a high-dimensional embedding space using a frozen, third-party semantic encoder. A semantic-neighbor search utilizing exact inner product calculations determines the cosine similarity between the training curriculum and the sealed holdouts. Records exceeding a conservative similarity threshold (e.g., \> 0.85) are flagged as potential semantic reconstructions and purged44. Furthermore, template-family detection ensures that if a specific structural permutation is used in the test set, all instances of that structural family are banned from the training set. Crucially, the experiment explicitly prohibits the practice of training directly on failed holdouts. Using evaluation failures as a direct feedback mechanism for training data generation guarantees metric inflation, triggers Goodhart's Law, and destroys construct validity45. Auditors verify these exclusions by executing open-source embedding and n-gram overlap scripts on the hashed cryptographic signatures of the training data against the public benchmark hashes. This mechanism proves absolute mathematical separation without revealing private plain-text answers to the auditing party.

Preservation Curriculum and Regression Budget

Fine-tuning neural networks, particularly highly compressed sub-billion parameter models, introduces the distinct and critical hazard of catastrophic forgetting. In this phenomenon, the aggressive optimization required to instill new conversational behavior inadvertently overwrites the model's fundamental pre-trained capabilities, resulting in the permanent loss of multilingual fluency, factual recall, and generalized reasoning48. Because the experiment utilizes highly aggressive learning rates to force behavioral compliance, the weights responsible for generalized world knowledge are at extreme risk49. To mitigate catastrophic interference, the protocol integrates a strict preservation curriculum comprising exactly five percent of the total training volume. This replay data consists of generic, high-quality pre-training text, multilingual web crawls, and established code instruction blocks51. Interleaving this preservation data forces the optimizer to preserve the parameter configurations essential to the original tasks while simultaneously accommodating the new behavioral adaptations via pseudo-rehearsal techniques51. A regression budget establishes the maximum permissible degradation of pre-trained capabilities before the fine-tuned checkpoint is deemed an irreversible failure. Evaluating training loss alone is profoundly insufficient; Evolution Strategies and gradient descent paths frequently demonstrate transient accuracy drops on prior tasks that occasionally recover over time, but often solidify into permanent capability loss if unchecked by rigorous evaluation53. The regression budget dictates a maximum allowable drop of 2.0% on zero-shot MMLU, 3.0% on multilingual translation benchmarks, and zero tolerance for degradation in established safety and harmlessness thresholds3. Continuous checkpoint evaluation against this budget serves as the primary early-stopping mechanism, aborting training the moment catastrophic interference breaches the mathematical threshold.

CPU Pilot and GPU Scale-Up Formulas

A rigorous mathematical derivation of Random Access Memory (RAM) and Video RAM (VRAM) constraints is mandatory to ensure that the experimental scaling transitions smoothly from local CPU prototyping to distributed GPU clusters. Training footprints consist of base model weights, adapter parameters, gradients, optimizer states, and the highly volatile activation memory7. For a model with [Figure omitted from source export] billion parameters, utilizing bfloat16 precision, the base model memory is calculated as [Figure omitted from source export] gigabytes. When implementing LoRA with a targeted rank of 16 across all linear modules, the adapter parameters represent roughly 1.5% of the total model size, requiring negligible static memory. However, during the backward pass, gradients are accumulated for these adapters in 32-bit floating-point precision, consuming an additional [Figure omitted from source export] gigabytes. The standard AdamW optimizer tracks master weights, momentum, and variance in 32-bit format, demanding exactly 12 bytes per trainable parameter54. The most volatile component is activation memory, which traditionally scales linearly with the sequence length ([Figure omitted from source export]), batch size ([Figure omitted from source export]), hidden dimension ([Figure omitted from source export]), and the number of layers ([Figure omitted from source export]), roughly equated to [Figure omitted from source export] bytes. However, by strictly enabling gradient checkpointing via the "unsloth" parameter, the computational graph selectively drops intermediate activations and recomputes them during the backward pass. This intervention mathematically crushes the scaling factor to approximately [Figure omitted from source export] bytes plus a fixed structural overhead, enabling massive context lengths on edge hardware23.

The derived data mathematically demonstrates that a CPU pilot or edge-device experiment is entirely feasible for the 230M and 350M models, requiring strictly under two gigabytes of unified memory for full fine-tuning. Scaling up to the 1.2B class for production requires a standard enterprise GPU, but easily fits within a single consumer-grade 8GB VRAM accelerator, bypassing the need for complex, multi-node gradient accumulation matrices. Cost and time formulas extrapolate linearly from these bounds: expected runtime on an NVIDIA L40S equates to [Figure omitted from source export], resolving to single-digit hours for the entire distillation protocol7. Optimization strategies mandate sequence packing to minimize padding waste, deterministic seeds for reproducibility, and durable restart checkpoints saved precisely every 250 gradient steps.

Factorial Experiment and Ablation Matrix

To isolate the precise origin of any capability gains and avoid the systemic trap of a misleading, single-checkpoint success story, the methodology relies on a strict multi-axis factorial pilot. This setup dissects the independent variables: base architecture, student size, distillation scale, and the presence of the preservation curriculum, ensuring that the source of behavioral improvement is mathematically proven.

1. Architecture Isolation: Compare the hybrid LFM2.5-350M against the dense Granite-4.0-350M and SmolLM2-360M using identical SFT hyperparameters, target modules (adjusted for topological variations), and deterministic random seeds. This confirms whether the LIV/GQA hybrid yields superior instruction following per parameter compared to standard self-attention mechanics.
2. Size Scaling Constraints: Compare LFM2.5-230M against LFM2.5-1.2B. This establishes the strict parameter elasticity curve, identifying if the advanced conversational logic of TinyRustLM fundamentally requires a billion-parameter state-space or if extreme distillation can compress it into 230 million weights.
3. Curriculum Depth Saturation: Train identically configured models at 10%, 50%, and 100% of the 100,000-record dataset. This determines the minimum data saturation point before overfitting or catastrophic forgetting initiates, optimizing future compute budgets.
4. Preservation Ablation: Train models with and without the 5% replay data. This tests the hypothesis that the preservation curriculum is strictly necessary to defend zero-shot world knowledge against catastrophic interference.

By minimizing overall compute overhead to rank-16 LoRA adaptations spanning a maximum of two epochs, the entire factorial grid can be executed efficiently, yielding a highly credible matrix of architectural performance.

Promotion Suite and Regression Budgets

A model checkpoint is never promoted to production based on an attractive qualitative chat response or a single peaked metric. The promotion suite mandates a rigorous, automated gauntlet testing the raw source, the base student, the adapter-merged model, and the final quantized browser runtime artifact (e.g., WebAssembly, ONNX, or MLX)11. The promotion criteria require evaluating untouched outputs across unseen paraphrases using repeated, declared generation seeds to mathematically map statistical variance. The fine-tuned student must exceed the raw base model by a minimum of 200% on specific conversational alignment scores and exact-match tool invocation metrics, while strictly adhering to the aforementioned regression budget (e.g., \<2.0% drop in generalized MMLU). Furthermore, the promotion suite tests extreme long-context behavior, demanding accurate retrieval of injected facts located precisely at the 32,768 token boundary7. The final quantized artifact is heavily scrutinized for latency and peak memory utilization; if the INT4 or NVFP4 precision reduction collapses the behavioral logic or spikes inference overhead past the strict product boundary, the checkpoint is disqualified from deployment9.

Stop Rules, Rollback, and Deletion Lineage

Strict, uncompromising guardrails are placed on the experimental pipeline to prevent resource waste and ensure compliance. The experiment will trigger an immediate halt, rollback, and deletion of all artifacts under the following conditions:

To enforce ultimate compliance, a strict deletion lineage protocol ensures that if a seed generation is found to violate safety boundaries or PII constraints post-generation, all subsequent student checkpoints, adapter weights, and deployment artifacts derived from that specific data point are cryptographically traced via their provenance ledger and eradicated entirely from the server infrastructure.

Unknowns Requiring Local Training or Evaluation

Theoretical analysis and literature review cannot definitively resolve the specific interaction between the Liquid architecture's LIV convolutional memory state and the generation of extremely constrained structural data (e.g., deeply nested JSON matrices) located at the far end of the 32,768 context window. Because the non-causal convolution layers lack a traditional KV cache over historical tokens, their ability to recall exact synthetic keys across extended multi-turn conversations without attention dilution requires empirical observation during local testing7. Furthermore, the actual latency delta between WebGPU-compiled LFM operators and highly optimized Transformer operations in varying browser environments remains an unknown variable requiring localized profiling11. Validation of mixed-precision stability across consumer hardware running quantized models requires deployment to a physical device lab, as emulation environments frequently fail to surface hardware-specific numerical precision faults. Finally, the ability of un-merged convolution adapters to perform consistently during high-throughput inference scenarios must be validated to ensure production reliability for TinyRustLM.

Works cited

1. Liquid Foundation Models, https://www.liquid.ai/models
2. onnx-community/Qwen3.5-0.8B-ONNX \- Hugging Face, https://huggingface.co/onnx-community/Qwen3.5-0.8B-ONNX
3. ibm-granite/granite-4.0-h-350m-base \- Hugging Face, https://huggingface.co/ibm-granite/granite-4.0-h-350m-base
4. HuggingFaceTB/SmolLM2-135M-Instruct · Hugging Face, https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct
5. Liquid LFM2.5: How To Run & Fine-tune | Unsloth Documentation, https://unsloth.ai/docs/models/tutorials/lfm2.5
6. LFM2.5-8B-A1B: Liquid AI's Edge Model Bet Is About Active Parameters \- LLM Rumors, https://www.llmrumors.com/news/liquid-ai-lfm25-edge-models-device-race
7. Deploy Liquid AI LFM2 Models (LFM2-8B-A1B, LFM2-2.6B) on GPU Cloud: Hybrid Architecture Guide (2026) | Spheron Blog, https://www.spheron.network/blog/liquid-foundation-models-lfm-deployment-gpu-cloud-2026/
8. LiquidAI/LFM2.5-350M \- Hugging Face, https://huggingface.co/LiquidAI/LFM2.5-350M
9. LiquidAI/lfm2.5-350m \- Ollama, https://ollama.com/LiquidAI/lfm2.5-350m
10. config.json · LiquidAI/LFM2.5-230M at main \- Hugging Face, https://huggingface.co/LiquidAI/LFM2.5-230M/blob/main/config.json
11. config.json · LiquidAI/LFM2.5-1.2B-Thinking-ONNX at 58de6ab4b249ec83e4393d621a5b3f4056fddaca \- Hugging Face, https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking-ONNX/blob/58de6ab4b249ec83e4393d621a5b3f4056fddaca/config.json
12. LFM License | Liquid AI, https://www.liquid.ai/lfm-license
13. Model License \- Liquid Docs, https://docs.liquid.ai/lfm/help/model-license
14. smollm2:135m/model \- Ollama, https://ollama.com/library/smollm2:135m/blobs/f535f83ec568
15. GabrieleConte/Qwen3.5-0.8B-LiteRT \- Hugging Face, https://huggingface.co/GabrieleConte/Qwen3.5-0.8B-LiteRT
16. Vibe-Tuning: The Art of Fine-Tuning Small Language Models with a Prompt \- distil labs, https://www.distillabs.ai/blog/vibe-tuning-the-art-of-fine-tuning-small-language-models-with-a-prompt/
17. A 0.6B model outperformed a 120B LLM by 29 points \- using dlt, distil labs, and Hugging Face, https://www.distillabs.ai/blog/a-0-6b-model-outperformed-a-120b-llm-by-29-points---using-dlt-distil-labs-and-hugging-face/
18. Overview \- Liquid Docs, https://docs.liquid.ai/lfm/fine-tuning/overview
19. Liquid4All/leap-finetune: A full model customization repo for Liquid Foundation Models, https://github.com/Liquid4All/leap-finetune
20. LEAP Finetune \- Liquid Docs, https://docs.liquid.ai/lfm/fine-tuning/leap-finetune
21. TRL \- Liquid Docs, https://docs.liquid.ai/lfm/fine-tuning/trl
22. Fine-Tune a Vision-Language Model on Satellite Imagery \- Liquid Docs, https://docs.liquid.ai/examples/customize-models/satellite-vlm
23. Unsloth \- Liquid Docs, https://docs.liquid.ai/lfm/fine-tuning/unsloth
24. LoRA \- Hugging Face, https://huggingface.co/docs/peft/package\_reference/lora
25. peft " target\_modules='all-linear' " have different behavior between x86 and aarch ? \#2878, https://github.com/huggingface/peft/issues/2878
26. I Gave an AI a Vocabulary of Only Emoji — And It Started to Communicate, https://kshreyas.dev/post/pantheon-ui/
27. GRPO Fine-tuning with Unsloth \- Colab \- Google, https://colab.research.google.com/github/Liquid4All/cookbook/blob/main/finetuning/notebooks/grpo\_with\_unsloth.ipynb
28. LFM2.5 \- CPT with Unsloth (Translation).ipynb \- Google Colab, https://colab.research.google.com/drive/1gaP8yTle2\_v35Um8Gpu9239fqbU7UgY8?usp=sharing
29. config.json · LiquidAI/LFM2.5-VL-450M at fc6221ca597f3315e4f82fc2df606783267b34ba \- Hugging Face, https://huggingface.co/LiquidAI/LFM2.5-VL-450M/blob/fc6221ca597f3315e4f82fc2df606783267b34ba/config.json
30. Introducing LFM2.5: The Next Generation of On-Device AI — Blog \- Liquid AI, https://www.liquid.ai/blog/introducing-lfm2-5-the-next-generation-of-on-device-ai
31. litert-community/Gemma3-1B-IT \- Hugging Face, https://huggingface.co/litert-community/Gemma3-1B-IT
32. unsloth/LFM2.5-230M-GGUF \- Hugging Face, https://huggingface.co/unsloth/LFM2.5-230M-GGUF
33. mlx-community/Qwen3.5-0.8B-MLX-4bit \- Hugging Face, https://huggingface.co/mlx-community/Qwen3.5-0.8B-MLX-4bit
34. ibm-granite/granite-4.0-h-350m-GGUF \- Hugging Face, https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF
35. LoRA fine-tuning Hyperparameters Guide | Unsloth Documentation, https://unsloth.ai/docs/get-started/fine-tuning-llms-guide/lora-hyperparameters-guide
36. Preference Fine-Tuning LFM 2 Using DPO \- Analytics Vidhya, https://www.analyticsvidhya.com/blog/2026/01/lfm-2-preference-fine-tuning-using-dpo/
37. maternion/lfm2.5:350m \- Ollama, https://ollama.com/maternion/lfm2.5:350m
38. GitHub \- distil-labs/distil-dlthub-models-from-traces: Demo repository showing how to create a model from production traces, https://github.com/distil-labs/distil-dlthub-models-from-traces
39. Your Traces Aren't Training Data Yet. Here's the Pipeline That Makes Them. \- dltHub, https://dlthub.com/blog/your-traces-aren-t-training-data-yet-here-s-the-pipeline-that-makes-them
40. oamazonasgabriel/lfm2-1.2b-tool \- Ollama, https://ollama.com/oamazonasgabriel/lfm2-1.2b-tool
41. Custom Distilled Models \- Lyceum Technology, https://lyceum.technology/solutions/custom-models/
42. Liquid AI Ships LFM2.5-230M with llama.cpp, MLX, vLLM, SGLang, and ONNX Support for On-Device Inference \- MarkTechPost, https://www.marktechpost.com/2026/06/27/liquid-ai-ships-lfm2-5-230m-with-llama-cpp-mlx-vllm-sglang-and-onnx-support-for-on-device-inference/?noamp=mobile&
43. Liquid AI Ships LFM2.5-230M with llama.cpp, MLX, vLLM, SGLang, and ONNX Support for On-Device Inference \- MarkTechPost, https://www.marktechpost.com/2026/06/27/liquid-ai-ships-lfm2-5-230m-with-llama-cpp-mlx-vllm-sglang-and-onnx-support-for-on-device-inference/
44. How can i build a High Quality dataset? \- Beginners \- Hugging Face Forums, https://discuss.huggingface.co/t/how-can-i-build-a-high-quality-dataset/176571
45. EvalSafetyGap: A Hybrid Survey and Conceptual Framework for LLM Evaluation-Safety Failures \- arXiv, https://arxiv.org/html/2606.30219v1
46. LiveClin: A Live Clinical Benchmark without Leakage \- OpenReview, https://openreview.net/forum?id=E0WSAugJ0j¬eId=CzYs093O5u
47. Code-driven Number Sequence Calculation: Enhancing the Inductive Reasoning Abilities of Large Language Models | OpenReview, https://openreview.net/forum?id=c8NK4icrZD
48. What is Catastrophic Forgetting? \- IBM, https://www.ibm.com/think/topics/catastrophic-forgetting
49. What is Catastrophic forgetting in the context of LLMs? Why its happening? How to mitigate it? | by Moe Moazzami | Medium, https://medium.com/@moe.moazzami/what-is-catastrophic-forgetting-in-the-context-of-llms-why-its-happening-how-to-mitigate-it-091dc4d388fd
50. How to Alleviate Catastrophic Forgetting in LLMs Finetuning? Hierarchical Layer-Wise and Element-Wise Regularization \- arXiv, https://arxiv.org/html/2501.13669v2
51. How are you handling catastrophic forgetting in multi-domain LLM fine-tuning pipelines?, https://www.reddit.com/r/mlops/comments/1rnhoa6/how\_are\_you\_handling\_catastrophic\_forgetting\_in/
52. Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning \- ACL Anthology, https://aclanthology.org/2025.emnlp-main.1108.pdf
53. How to Prevent Catastrophic Forgetting in LLM Fine-Tuning \- Cognizant, https://www.cognizant.com/us/en/ai-lab/blog/overcoming-forgetting-in-llm-fine-tuning
54. unknown\_url
55. Liquid AI's LFM2 Just Dropped — Here's How to Run It on Vast.ai, https://vast.ai/article/liquid-ai-lfm2-run-on-vast-ai
56. onnx-community/granite-4.0-350m-ONNX-web \- Hugging Face, https://huggingface.co/onnx-community/granite-4.0-350m-ONNX-web
57. mlx-community/Qwen3.5-0.8B-OptiQ-4bit \- Hugging Face, https://huggingface.co/mlx-community/Qwen3.5-0.8B-OptiQ-4bit
58. Wandler: Local OpenAI-Compatible Inference With Transformers.js and WebGPU, https://blog.kubesimplify.com/wandler-local-openai-compatible-inference-transformersjs-webgpu