Engineering a Compact Production Assistant: End-to-End Distillation and Quantization-Aware Training for the TinyRustLM Ecosystem

Full report

On this page

The deployment of localized, browser-based artificial intelligence requires navigating severe computational boundaries. Within the TinyRustLM and .slm ecosystem, execution occurs via a scalar WebAssembly (WASM) runtime on the main browser thread, restricted by a hard 128 MiB transfer ceiling and a transient 1,024-candidate sampling cap1. Constructing a highly capable 100M–400M parameter production assistant for this environment demands a fundamental departure from conventional scaling laws. It requires aggressive knowledge distillation, structural pruning, implicit reasoning representations, and quantization-aware training (QAT) to compress frontier-level instruction following and reasoning into a constrained memory envelope. The analysis herein outlines an exhaustive, staged engineering plan designed to yield a production-ready, q4-class model operating strictly within the physical limits of the custom SLM1 model container2. The resulting student model is designed to execute complex multi-turn conversations, code explanation, JSON formatting, and local-memory grounding without triggering catastrophic memory allocations or violating the execution transaction boundaries of the TinyRustLM framework1.

Foundation Architectures: Teacher and Student Selection

The selection of the student architecture must be precisely tuned to the memory allocation formulas inherent to the TinyRustLM C-style ABI. The core runtime allocates memory predictably: forward scratch space is defined as [Figure omitted from source export], the Key-Value (KV) cache consumes [Figure omitted from source export] bytes, and logits require [Figure omitted from source export] bytes1. To maximize parameter utility while remaining under the 128 MiB boundary, the baseline student model is designated as a 200M parameter causal transformer, conceptually aligned with the MiniModel-200M-Base paradigm3. The target student architecture comprises 16 layers ([Figure omitted from source export]), a hidden dimension of 768 ([Figure omitted from source export] heads), and a vocabulary size ([Figure omitted from source export]) of 32,000. To restrict KV-cache memory inflation—a critical necessity since the KV-cache resides in pre-allocated browser memory—the student relies on Grouped Query Attention (GQA)4. By utilizing 12 query heads but only 4 KV heads ([Figure omitted from source export]), the continuous memory footprint of the KV cache at a 2,048-token context length is strictly confined to approximately 8.3 MiB1. The teacher model must possess robust reasoning, structured output capabilities, and instruction compliance. An 8B parameter model, such as an advanced iteration of the Llama 3 or Qwen2.5 architectures, serves as the optimal teacher6. These models provide high-quality probability distributions and latent reasoning traces necessary for high-fidelity capability transfer, while remaining computationally tractable for extraction runs on multi-GPU nodes. Furthermore, Query-Key Normalization (QK-Norm) is mandatory for the student. In standard dot-product attention, attention logits can scale unboundedly, causing softmax saturation and vanishing gradients7. By inserting an RMSNorm operation directly on the query and key projections prior to the dot product, the attention score magnitude is constrained, decoupling vector direction from magnitude and stabilizing the distillation process at high learning rates without disrupting the residual stream8.

Tokenizer Inheritance Versus Vocabulary Reduction

The embedding matrix and final output projection often consume a disproportionate percentage of total parameters in small language models, frequently exceeding 15% of the parameter budget11. Furthermore, the cross-entropy loss computation over a massive vocabulary dominates the memory footprint during training12. Inheriting a modern 128,000-token vocabulary is mathematically incompatible with the 128 MiB transfer ceiling of TinyRustLM, as the tied embeddings alone would consume over 98 MiB in standard 8-bit quantization. To mitigate this bottleneck, the student does not inherit the complete vocabulary of the teacher model. Instead, the vocabulary undergoes a strict reduction process targeting a maximum of 32,000 tokens4. By adjusting the Byte Pair Encoding (BPE) merge rules based on the frequency statistics of the calibration corpus, the tokenizer size is physically truncated to form a compact subset11. Because reducing the vocabulary disrupts the mapping between the teacher's output space and the student's learning target, embedding factorization and probability mass redistribution are employed. For tokens present in both the teacher and the reduced student vocabulary, the teacher's pre-trained embeddings are directly inherited. For tokens that have been pruned, their probability mass in the teacher's output distribution is marginalized and pooled into a designated \<UNK\> token or distributed among their constituent surviving subwords11. The corresponding initialized embeddings for these fragmented tokens in the student are computed via the arithmetic mean of the existing subword tokens11. This factorization drastically reduces the logits memory allocation footprint in the SLM1 container while preserving the cross-lingual and syntactical structures required for downstream tasks.

Structural Pruning: Layer, Head, Channel, and FFN Reduction

Prior to intensive knowledge distillation, the architecture undergoes structured pruning to eliminate non-critical channels and attention heads, capitalizing on the MINI-LLM framework's memory-efficient gradient estimation14. Distilling an 8B teacher directly into a randomly initialized 200M student creates an optimization gap that is difficult to traverse. Instead, the teacher is progressively pruned to yield a structurally identical, albeit weaker, initialization point for the student. Classical pruning relies heavily on backpropagation, which is highly memory-intensive for an 8B teacher. The MINI-LLM approach circumvents this by calculating a hybrid sensitivity score using only forward passes to estimate gradients14. The sensitivity metric relies on the magnitude of the weights, the magnitude of the activations, and the approximated gradient to identify the least salient attention heads and feed-forward (FFN) channels17. Through this process, the teacher's 32 layers are depth-pruned down to 16 by measuring layer similarity and discarding redundant blocks. Within the surviving layers, the attention heads are pruned from 32 down to 12 (maintaining the 4 KV heads for GQA), and the SwiGLU FFN intermediate dimensions are truncated to eliminate outlier-prone or inactive channels. The resulting structurally reduced model serves as the weight initialization for the 200M student, providing a robust mathematical prior for the subsequent distillation phases.

Multi-Stage Knowledge Distillation Dynamics

The transfer of instruction-following and analytical capabilities relies on a rigorous distillation pipeline combining sequence-level and token-level optimization. The distillation objective is not merely to mimic the top-1 prediction, but to internalize the teacher's entire probability landscape.

Sequence-Level and Token-Level Knowledge Distillation

The training loss minimizes the divergence between the student's output distribution and the teacher's output distribution across all vocabulary tokens. The core optimization leverages both Forward Kullback-Leibler (KL) divergence and Reverse KL divergence, balanced dynamically by Jensen-Shannon (JS) divergence in the later epochs to prevent mode collapse. Forward KL divergence forces the student to cover all modes of the teacher's distribution, ensuring diversity in generation and preventing the student from over-committing to a narrow subset of phrases. Conversely, Reverse KL divergence penalizes the student for generating samples outside the teacher's high-probability regions, enforcing precision and factual accuracy. Because Forward KL is mode-covering and Reverse KL is mode-seeking, an unweighted combination can lead to training instability. Thus, JS Divergence is introduced as a symmetric metric to smooth the optimization landscape. Temperature scaling is applied to soften the teacher's hard logits, amplifying the signal from the "dark knowledge" contained in the near-zero probabilities of incorrect tokens. The temperature schedule begins at a highly softened state and anneals down to a sharper distribution as the student's confidence solidifies.

Hidden-State and Attention-Map Matching

Beyond matching the output logits, the student must replicate the internal geometric representations of the teacher to effectively grasp complex syntax and logic. Due to the dimensional mismatch between the 8B teacher ([Figure omitted from source export]) and the 200M student ([Figure omitted from source export]), a learnable linear projection matrix is introduced during training. This matrix up-projects the student's normalized hidden states to match the teacher's dimension. A Mean Squared Error (MSE) loss is then applied between the student's projected hidden states and the teacher's hidden states. Furthermore, attention-map matching is utilized to transfer the teacher's contextual weighting directly into the student's pre-normalized query-key space. This is achieved by computing a cross-entropy objective on the softmax-normalized attention matrices of the teacher and the student. By forcing the student's attention heads to distribute their weight identically to the teacher's most critical heads, the student learns to isolate relevant context (such as variable declarations in code or specific constraints in a prompt) with high efficiency.

Reasoning-Trace Distillation Without Exposing Hidden Chain-of-Thought

A primary challenge in engineering small models for the browser is instilling logical reasoning without relying on verbose explicit Chain-of-Thought (CoT) outputs. Standard CoT consumes excessive tokens, rapidly depletes the 1,024-token sampling cap, bloats the user interface, and exposes internal reasoning traces to the end user1. The architecture circumvents this by implementing Supervised Implicit Chain-of-Thought (SIM-CoT) and Continuous Chain-of-Thought via Self-Distillation (CODI)20. Under this paradigm, reasoning steps generated by the teacher are compressed into a sequence of latent, continuous representation tokens within the student's hidden states rather than being decoded into natural language23. During the training phase, a lightweight auxiliary decoder is attached to designated latent tokens generated by the student. This auxiliary decoder is tasked with reconstructing the explicit natural language reasoning trace of the teacher22. By penalizing the auxiliary decoder's outputs against the teacher's explicit CoT, the student's internal hidden states are forced to capture the distinct, step-by-step semantic computations required to arrive at a logical answer21. This prevents the latent instability and semantic homogenization that typically plague unguided implicit CoT attempts21. At inference time within the browser, the auxiliary decoder is completely discarded22. The student processes the prompt, transitions through the prescribed number of latent reasoning states purely in continuous space, and directly outputs the final formatted answer. This guarantees that internal thought processes remain hidden from the presentation layer, preserving token bandwidth and adhering to strict capability constraints.

Instruction Tuning, UAIX Memory Grounding, and Formatting

The model must seamlessly handle greetings, clarification, rewriting, summarization, code explanation, and JSON formatting. The instruction tuning corpus leverages the ChatML format, utilizing distinct system, user, and assistant roles to frame every interaction. A critical requirement of the .slm ecosystem is local-memory grounding, specifically adhering to UAIX memory and persona requirements. Personas in this architecture rely on a context injection (memory) limited to approximately 350 tokens, rarely exceeding 500 tokens, to prevent the student model from suffering trait theft—a phenomenon where the model over-identifies with the user's injected persona traits25. The memory structures are typically stored in memories.json format or passed as markdown layers parsed by a local vector database26. To instill robust UAIX memory grounding, the calibration corpus injects highly specific JSON-formatted persona blocks into the system prompt. The student is heavily penalized during training if its generations contradict the injected JSON state or if it hallucinates traits not present in the 350-token memory envelope. This ensures that the assistant strictly grounds its multi-turn conversational responses in the user's declared context rather than relying on its pre-trained parametric memory.

Preference Optimization, Safety, and Refusal Preservation

Following knowledge distillation, the student model undergoes Direct Preference Optimization (DPO) to align with conversational formatting constraints and safety guardrails. DPO operates directly on the model's policy without requiring a separate reward model, making it highly efficient for small-scale engineering. Preference pairs are generated to penalize verbosity, hallucinations, and harmful outputs. To preserve safety and refusal capabilities—which are often lost when compressing models—the DPO dataset includes targeted adversarial prompts. In these pairs, the teacher's correct refusal behavior is assigned as the chosen response, while compliant but harmful generations (often synthesized or drawn from red-teaming datasets) are rejected. To exert catastrophic-forgetting controls, the DPO loss function incorporates a stringent KL-divergence penalty parameter relative to the distilled base policy. This prevents the student from deviating too far from the linguistic priors established during distillation, ensuring that optimizing for safety does not degrade its ability to execute complex code explanations or JSON structuring tasks.

Quantization-Aware Training (QAT) and Outlier Handling

To fit within the 128 MiB runtime threshold, the model must be exported using 4-bit precision. Applying Post-Training Quantization (PTQ) natively to a 200M parameter model frequently results in a catastrophic degradation of perplexity. Therefore, Quantization-Aware Training (QAT) is strictly enforced.

Progressive Bit-by-Bit Strategy

The QAT process follows the progressive Bit-by-Bit block-wise strategy28. Training transitions from FP16 to W8A8 (8-bit weights, 8-bit activations), and eventually to W4A16 (4-bit weights, 16-bit activations)28. This stage-wise curriculum ensures stable initialization for low-bit optimization, smoothing the highly non-convex loss landscape that typically traps ultra-low precision networks in suboptimal local minima30.

Outlier Channel Splitting (OCS)

Activation outliers, frequently termed "Massive Outliers," emerge in specific feature dimensions and dramatically distort quantization scaling factors, leading to widespread accuracy loss32. To counteract this, Outlier Channel Splitting (OCS) is integrated into the QAT loop28. OCS identifies channels exhibiting massive outliers utilizing the metric [Figure omitted from source export]31. The identified channels are duplicated, and their corresponding weights are halved. This rounding-aware identity transformation redistributes the outlier magnitude across adjacent channels, drastically narrowing the dynamic range required by the integer quantization grid without altering the mathematical output of the linear projection31.

Activation and KV-Cache Quantization Constraints

While weight quantization reduces the storage footprint, activation and KV-cache memory must also be strictly managed. The TinyRustLM environment supports fixed buffer scaling. While standard QAT targets W4A16, the KV-cache must be maintained at 8-bit precision (K8V8) to prevent degradation during long multi-turn conversations. The QAT pipeline simulates 8-bit quantization noise specifically on the key and value projections during the forward pass, forcing the attention mechanism to adapt to lower-fidelity historical states.

Precision Allocation in SLM1

Theoretical model designs often call for arbitrary q5 or q6 mixtures to balance performance and size. However, the deployment artifact must map directly to the supported SLM1 types defined by the 108-byte header: f32 (dtype 1), q8\_0 (dtype 2), and q4\_0 (dtype 3\)2. Consequently, q5 and q6 precisions are not physically supported by the runtime matrix-vector kernels. To emulate the benefits of mixed precision, the model utilizes heterogeneous block allocation based on Activation-Aware Weight Quantization (AWQ) sensitivity analysis35. The embedding layer and the final LM head are strictly preserved in q8\_0 (dtype 2\) to maintain token discriminability. The bulk feed-forward networks (SwiGLU) and the query/key/value projection matrices are compressed into q4\_0 (dtype 3\)2. By simulating these exact bit-widths via nested integer grids during QAT, the simulated quantization noise perfectly matches the browser's eventual decoding arithmetic.

Data Engineering: Corpus Design, Deduplication, and Optimization

The dataset composition and optimizer configuration are crucial for efficiently training the compact assistant without exceeding compute budgets.

Dataset Categories and Contamination Controls

The calibration corpus is synthesized from highly curated, commercially viable (Apache 2.0 or MIT licensed) data streams.

• Pre-training/Distillation Base: A 20-billion token subset comprising high-quality web scrapes (e.g., Ultra-FineWeb) and specialized coding datasets (e.g., Python-edu-cleaned) to establish base syntactical structures3.
• Instruction Tuning (SFT): A 1-billion token corpus heavily weighted toward UAIX JSON memory interactions, multi-turn ChatML dialogues, JSON formatting tasks, and implicit CoT rationales generated by the 8B teacher.
• DPO Refusals: A 500-million token dataset containing adversarial injection attempts and corresponding safety refusals.

To enforce strict contamination controls against benchmark leakage, the dataset undergoes aggressive n-gram overlap removal against common evaluation suites (e.g., GSM8K, MMLU). Furthermore, MinHash Locality-Sensitive Hashing (LSH) is applied across the entire corpus to strip redundant semantic structures, ensuring high data efficiency.

Optimizer Schedules and Compute Estimates

Optimization of the 2D hidden layers relies on the Muon optimizer. Muon treats weight matrices as elements on a Stiefel manifold and leverages a Newton-Schulz iteration to approximate the polar decomposition, orthogonalizing the momentum updates37. This reduces the step requirement for generalization significantly37. The 1D parameters (biases, layer norms) utilize standard AdamW40. The learning rate follows a cosine decay schedule, peaking at [Figure omitted from source export] for Muon and [Figure omitted from source export] for AdamW41. The batch size is globally maintained at 64 sequences, with a maximum sequence length of 2,048 tokens to mirror the TinyLM KV-cache capacity limit1. Total compute required to complete the end-to-end pipeline is estimated at approximately 240 NVIDIA A100 GPU hours.

Deliverable 1: Staged Training Recipe

The overall training protocol is divided into five strictly gated sequential phases.

Deliverable 2: Exact Loss Composition and Schedules

The training regimen relies on a multi-component objective function designed to balance autoregressive modeling, structural imitation, and latent alignment.

Deliverable 3: Compression Targets and Predicted Byte Sizes

The explicit goal is to navigate the 128 MiB global transfer ceiling enforced by the TinyRustLM WASM ABI1. By strictly mapping the structural elements to the SLM1-supported dtypes, the memory profile is deterministically bounded.

This 111.00 MiB footprint safely clears the 128 MiB maximum transaction limit, ensuring successful loading behavior. The runtime allocations (KV cache at \~8.3 MiB and Forward Scratch at 47 KiB) remain comfortably within the limits of the main browser thread1.

Deliverable 4: Quantization-Aware Export Design for .slm

Exporting the model into the .slm ecosystem requires strict adherence to the SLM1 binary contract. The container begins with a mandatory 108-byte header2.

Following the header, each tensor is defined in a 64-byte directory entry mapping the FNV-1a name hash, dimensionality, payload offset, and dtype (1 for f32, 2 for q8\_0, 3 for q4\_0)2. The QAT export script must interleave the tensor blobs, aligning all data payloads to 64-byte boundaries to prevent WebAssembly memory misalignment panics during load2.

Deliverable 5: Required Provenance and Lineage Records

Because the TinyRustLM ecosystem utilizes isolated evidence boundaries, the provenance of the distilled model must be permanently coupled to the artifact via a sidecar JSON manifest1. The lineage record mandates the logging of:

1. Teacher Identity: Cryptographic SHA-256 hashes of the foundational 8B teacher weights utilized for inference generation.
2. Dataset Composition: Statistical distributions of the ChatML corpus, including exact proportions of implicit CoT latent traces, UAIX memory JSON structures, and standard instructional data.
3. Optimization Trace: Final loss values across all domains (JS Divergence, MSE, QAT penalty, Auxiliary Decoder Loss) to certify convergence limits.
4. Hardware Signature: Immutable record of the GPU environment, driver versions, and PyTorch runtime hash used during the final QAT compilation1.

Deliverable 6: Evaluation Plans (Public, Shadow, Sealed)

The verification of a heavily compressed, implicit-reasoning model requires a tri-layered evaluation architecture, designed to capture both linguistic degradation and mechanical failures in the WebAssembly environment.

Deliverable 7: Stop/Go Criteria for Every Training Stage

The transition between training phases is governed by strict empirical thresholds to prevent the amplification of quantization noise or the collapse of latent reasoning.

Deliverable 8: Failure-Recovery Plan

Should the compressed student suffer a catastrophic loss of instruction quality—often manifesting as endless repetition, formatting degradation, or failure to utilize the implicit reasoning space—a deterministic recovery protocol must be executed:

1. Halt and Isolate: Immediately terminate the optimization step and preserve the exact random seed, gradients, and optimizer states (specifically preserving the Gram matrices in the Muon optimizer to maintain historical velocity).
2. Identify the Degradation Vector: Analyze the auxiliary decoder's output log. If the decoder outputs nonsensical text from the latent tokens, the semantic alignment of the implicit CoT has collapsed21.
3. Rollback and Re-anchor: Revert to the last stable checkpoint from Phase 2\. Introduce an "Anchor Loss"—a heavy mean squared error penalty tethering the student's hidden states to the pre-QAT f32 master copy—to physically restrain the divergence of the feature maps during implicit tuning.
4. Recalibrate Outliers: A collapse during Phase 5 indicates that Outlier Channel Splitting failed to adequately detect heavy-tailed anomalies28. The OCS threshold parameter must be lowered to force a more aggressive duplication of outlier channels before restarting the quantization curriculum31.
5. Restart with Warmed Precision: Resume training at the q8\_0 boundary for an extended epoch before progressively stepping down to q4\_0, allowing the Newton-Schulz momentum iterations to restabilize the new low-precision topology before aggressive 4-bit clamping is enforced.

Works cited

1. Implementation operations \- MiRust, https://mirust.com/implementation-operations/
2. Implementation \- MiRust, https://mirust.com/implementation/
3. MiniModel-200M-Base : r/LocalLLaMA \- Reddit, https://www.reddit.com/r/LocalLLaMA/comments/1np5ey8/minimodel200mbase/
4. GitHub \- Ashx098/Mini-LLM: A ground-up LLM engineering project: tokenizer → architecture → training → scaling laws → inference. Starts at 80M, engineered to scale into 1B+ models with minimal changes. Clean, research-ready code for anyone serious about understanding and building LLMs from first principles., https://github.com/Ashx098/Mini-LLM
5. Tiny Model Composer \- MiRust, https://mirust.com/composer/
6. Awesome tiny LLMs \- a eipi1-0 Collection \- Hugging Face, https://huggingface.co/collections/eipi1-0/awesome-tiny-llms
7. Modern Transformer Architecture: RoPE, QK Norm, and Design Choices \- José David Baena, https://josedavidbaena.com/blog/nanochat/modern-transformer-architecture-rope-qk-norm
8. Enhanced QKNorm normalization for neural transformers with the Lp norm \- arXiv, https://arxiv.org/html/2602.05006v1
9. QK-Norm | Sebastian Raschka, PhD, https://sebastianraschka.com/llm-architecture-gallery/qk-norm/
10. Query-Key Normalization in Transformers \- Emergent Mind, https://www.emergentmind.com/topics/query-key-normalization-qk-norm
11. Efficient Vocabulary Reduction for Small Language Models \- ACL Anthology, https://aclanthology.org/2025.coling-industry.64.pdf
12. Cut Your Losses in Large-Vocabulary Language Models \- arXiv, https://arxiv.org/html/2411.09009v1
13. Small language model \- Grokipedia, https://grokipedia.com/page/small-language-model
14. \[2407.11681\] MINI-LLM: Memory-Efficient Structured Pruning for Large Language Models, https://arxiv.org/abs/2407.11681
15. MINI-LLM: Memory-Efficient Structured Pruning for Large Language Models \- ResearchGate, https://www.researchgate.net/publication/382302432\_MINI-LLM\_Memory-Efficient\_Structured\_Pruning\_for\_Large\_Language\_Models
16. arXiv:2407.11681v1 \[cs.CL\] 16 Jul 2024, https://arxiv.org/pdf/2407.11681
17. Mini-LLM: Efficient, Domain-Specific Language Models \- Emergent Mind, https://www.emergentmind.com/topics/mini-llm
18. Hidden Thoughts Are Not Secret: Reasoning Trace Exposure in LLMs \- arXiv, https://arxiv.org/html/2606.00642v1
19. Reasoning Trace Distillation \- Emergent Mind, https://www.emergentmind.com/topics/reasoning-trace-distillation
20. CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation \- arXiv, https://arxiv.org/abs/2502.21074
21. SIM-CoT: Supervised Implicit Chain-of-Thought \- Takara TLDR, https://tldr.takara.ai/p/2509.20317
22. SIM-CoT: Supervised Implicit Chain-of-Thought \[ICLR 2026 \] \- GitHub, https://github.com/InternLM/SIM-CoT
23. SemCoT: Accelerating Chain-of-Thought Reasoning through Semantically-Aligned Implicit Tokens, https://proceedings.neurips.cc/paper\_files/paper/2025/hash/3ddbd473456a57e3cafb1ee51ddf8ff6-Abstract-Conference.html
24. Implicit Chain-of-Thought Model \- Emergent Mind, https://www.emergentmind.com/topics/implicit-chain-of-thought-model
25. A simple guide to making “functional” personas : r/JanitorAI\_Official \- Reddit, https://www.reddit.com/r/JanitorAI\_Official/comments/1ij9m4l/a\_simple\_guide\_to\_making\_functional\_personas/
26. Persona Memory and Embedded Memories, a guide : r/claudexplorers \- Reddit, https://www.reddit.com/r/claudexplorers/comments/1s84tkt/persona\_memory\_and\_embedded\_memories\_a\_guide/
27. I spent 6 weeks building an "external memory" for my AI companion. Here's what I learned about identity. \- Reddit, https://www.reddit.com/r/ArtificialSentience/comments/1tb38nt/i\_spent\_6\_weeks\_building\_an\_external\_memory\_for/
28. Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs \- ACL Anthology, https://aclanthology.org/2026.acl-long.1020/
29. Bit-by-Bit: Progressive QAT with Outlier Channel Splitting for Stable Low-Bit LLMs | OpenReview, https://openreview.net/forum?id=wy5IaDDmun
30. Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs \- ACL Anthology, https://aclanthology.org/2026.acl-long.1020.pdf
31. PROGRESSIVE QAT WITH OUTLIER CHAN- NEL SPLITTING FOR STABLE LOW-BIT LLMS \- OpenReview, https://openreview.net/pdf?id=wy5IaDDmun
32. DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs \- NIPS, https://proceedings.neurips.cc/paper\_files/paper/2024/file/9febda1c8344cc5f2d51713964864e93-Paper-Conference.pdf
33. Top 12 Quantization Strategies That Keep Quality | by Thinking Loop \- Medium, https://medium.com/@ThinkingLoop/top-12-quantization-strategies-that-keep-quality-64dca6c1ff73
34. Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs \- arXiv, https://arxiv.org/html/2604.07888v1
35. A practical guide to INT4 quantization for SLMs: GPTQ vs AWQ, Olive, and real‑world results, https://medium.com/data-science-at-microsoft/a-practical-guide-to-int4-quantization-for-slms-gptq-vs-awq-olive-and-real-world-results-2f63d6963d1d
36. Optimizing LLMs for Performance and Accuracy with Post-Training Quantization, https://developer.nvidia.com/blog/optimizing-llms-for-performance-and-accuracy-with-post-training-quantization/
37. Muon optimizer \- Grokipedia, https://grokipedia.com/page/muon-optimizer
38. Muon and a Selective Survey on Steepest Descent in Riemannian and Non-Riemannian Manifolds | Franz Louis Cesista, https://leloykun.github.io/ponder/steepest-descent-non-riemannian/
39. Dao-AILab/gram-newton-schulz: Fast Polar Decomposition for Muon \- GitHub, https://github.com/Dao-AILab/gram-newton-schulz
40. AdaMuon: Adaptive Muon Optimizer \- arXiv, https://arxiv.org/html/2507.11005v1
41. KellerJordan/Muon: Muon is an optimizer for hidden layers in neural networks \- GitHub, https://github.com/KellerJordan/muon
42. Documentation \- MiRust, https://mirust.com/docs/