Architecting Subnetworks for Browser-Local Inference: Graph Closure, Elasticity, and Structured Pruning in Language Models

Full report

On this page

1. Executive Answer: Valid Submodels Versus Invalid Slicing

The hypothesis that a useful, narrow-domain language model can be instantiated by simply extracting a subset of parameters from a larger champion model fundamentally contradicts the geometric reality of pre-trained neural networks. In a standard dense transformer architecture, taking only the "useful parts"—whether by arbitrarily dropping layers, slicing attention heads, or deleting byte ranges—is mathematically invalid. Neural networks do not store discrete capabilities in modular, isolated parameter blocks; rather, they learn highly entangled, non-linear mappings where representations exist in a state of polysemantic superposition. When a model is subjected to naive slicing, the immediate result is catastrophic functional failure. This failure is driven by the destruction of graph closure. Removing a layer induces an abrupt magnitude gap in the residual stream, shifting the activation variance out of the domain expected by subsequent normalization layers and non-linearities1. Slicing a width dimension severs the orthogonal relationships within the attention projections and the delicate Hadamard symmetry of gated multi-layer perceptrons (MLPs). The resultant mathematical artifact is not a smaller model; it is a broken mathematical graph that outputs maximum-entropy noise. A subnetwork can only execute validly as a strict subset of a larger model's parameters under two explicit architectural regimes. The first is if the parent model was explicitly pre-trained for elastic inference utilizing nested or Matryoshka-style objective functions (e.g., MatFormer, M-MoE, ThinkingViT)4. In such architectures, the loss function enforces a coarse-to-fine representational hierarchy, guaranteeing that foremost parameter subsets independently map to a valid vocabulary distribution. The second regime requires a formally structured pruning pipeline that physically reduces the dimensionality of dense matrix multiplications (e.g., SliceGPT, ShortGPT), followed by either an offline magnitude compensation patch or supervised gradient-based recovery to heal the disrupted functional mappings7. For the TinyRustLM ecosystem utilizing .slm artifacts in a browser-local WebAssembly (WASM) environment, a parameter-count reduction is meaningless if the computational graph is not mathematically closed and the resulting kernels are not dense. Physical deduplication of weights on disk does not guarantee logical validity at runtime, and attempts to extract task specialists must adhere to strict structural constraints and rigorous recovery methodologies.

2. Explicit Assumptions and Terminology

To precisely evaluate subnetwork validity and compression paradigms for constrained browser execution, the following technical definitions and environmental assumptions are established:

• Subnetwork / Submodel: A strictly smaller subset of a parent neural network's parameter tensors that can successfully execute a forward pass to produce coherent, low-perplexity text.
• Graph Closure: The mathematical property wherein a modified neural network retains exact matching tensor dimensions for all matrix multiplications, residual additions, and normalization steps, ensuring the forward pass completes without dimensional exceptions or extreme variance shifts.
• Elastic Inference / Matryoshka Architecture: A paradigm where a model is natively designed and pre-trained such that varying continuous subsets of its parameters can be dynamically selected at runtime to execute with variable computational complexity, without post-hoc fine-tuning4.
• Structured Pruning: The removal of entire, coherent architectural components—such as contiguous transformer layers, full attention heads, or complete FFN channels—that physically shrinks the dimensionality of dense tensors7.
• Unstructured / N:M Sparsity: The zeroing of individual scalar weights within a tensor based on magnitude or gradient criteria, leaving the physical tensor shape unchanged but introducing algorithmic sparsity masks.
• Lottery Ticket Hypothesis: The theoretical claim that dense networks contain sparse, trainable subnetworks (winning tickets) that can reach the same accuracy as the parent when trained in isolation.
• Time-to-First-Token (TTFT): The latency from receiving the user prompt to emitting the first generated token. This is heavily bound by memory bandwidth for reading static weights during the parallel prefill phase.
• Decode Latency: The wall-clock time per token generated during the autoregressive phase, fundamentally bottlenecked by memory bandwidth scaling against the size of the static weights and the dynamic Key-Value (KV) cache.
• Peak Committed Memory: The absolute maximum memory footprint required by the runtime during execution. This includes static weights, dynamic KV cache, temporary scratch space, and the browser's JavaScript/WASM heap overhead.
• TinyRustLM Context: It is assumed that the runtime operates in a memory-constrained browser environment (governed by the strict 4GB limit of 32-bit WebAssembly linear memory) leveraging scalar CPU or SIMD128 instructions, with strict transfer size budgets for portable .slm files. TinyRustLM is a pre-release product; clean replacement is allowed, and no legacy artifact support is required.

3. Taxonomy of Subnetwork and Compression Methods

To extract a highly performant specialist from a larger champion model, compression techniques must be classified by their impact on the computation graph and their reliance on recovery training. Not all compression methods yield a loadable, coherent subnetwork suitable for a WASM environment.

Post-Hoc Deletion (Training-Free Pruning)

These methods modify the pre-trained weights without any gradient-based retraining, relying entirely on geometric or statistical adjustments.

• Depth Pruning (Layer Dropping): Removes entire transformer blocks based on redundancy metrics such as Block Influence (BI), perplexity degradation, or activation similarity9. Frameworks like ShortGPT identify consecutive or distributed layers that act as identity mappings and delete them. While structurally simple, layer dropping introduces severe activation magnitude gaps1.
• Transformative Width Pruning: Methods like SliceGPT project the weight matrices into a smaller orthogonal subspace, deleting columns and rows across the entire network while preserving the core embedding information. This requires calibration data to compute the orthogonal matrices but zero gradient updates8.
• Magnitude Compensation: Emerging techniques like Prune\&Comp or LinearPatch adjust the scales of remaining weights after layer removal to patch the activation variance, achieving near-original performance without backpropagation1.

Prune-and-Recover (Gradient-Based)

These techniques structurally alter the model and subsequently utilize supervised fine-tuning (SFT) or knowledge distillation (KD) to heal the network.

• Adaptive Structured Pruning: Frameworks like LLM-Pruner, Sheared LLaMA, or Adapt-Pruner evaluate the importance of coupled structures (heads, hidden channels) using Taylor approximations or gradients, remove them, and apply continued pre-training7. This guarantees a smaller, dense graph but requires substantial computational budgets to recover the lost functional mappings.
• Separately Distilled Students: A smaller architectural blueprint is initialized, and the champion model acts as a teacher, providing soft logits for the student to match. This allows complete architectural freedom but demands massive datasets and is highly susceptible to mode collapse.

Train-Time Elastic Subnetworks (Nested Checkpoints)

These models are explicitly pre-trained to house valid subnetworks within their dense parameters, enabling extraction without post-hoc recovery.

• Matryoshka Representation Learning (MRL): Models like MatFormer or ThinkingViT enforce a nested loss during pre-training. The model must predict accurately using the first fraction of its dimensions, and progressively larger fractions4.
• Slimmable Networks and Once-For-All (OFA) Training: The network is trained with "sandwich rules," where the maximum, minimum, and randomly sampled intermediate sub-architectures are optimized jointly in each training step. This prevents interference where subnetworks compete for representation capacity. The result is a single stored family with several independently executable identities.

Conditional Execution and MoE

• Mixture-of-Experts (MoE) Routing: Only a subset of parameters is active per token. However, standard MoE models require the entire expert pool to reside in memory, failing to reduce peak committed memory for browser deployment.
• Expert Pruning and Merging: Methods like ConMoE, C-PRUNE, and MoE-Pruner physically delete cold experts or merge similar experts into reusable prototypes, effectively converting a high-memory MoE into a smaller, statically loadable subnetwork17. This permanently alters the routing space but reduces the physical artifact size.

Sub-Optimal Topologies for WASM

• Low-Rank Factorization and Tensor Decomposition: Decomposing a large matrix into two smaller matrices ([Figure omitted from source export]). While reducing parameter count, it increases the depth of the computational graph, often increasing latency due to sequential memory reads.
• Sparse Subnetworks (Lottery Tickets): Finding a highly sparse unstructured mask that preserves performance. While theoretically elegant, unstructured sparsity requires index metadata that severely degrades SIMD efficiency on standard CPUs.

4. Transformer Graph-Closure Analysis by Tensor Family

A valid .slm subnetwork must strictly satisfy graph closure. If a tensor is arbitrarily sliced, the dimensional contracts of the transformer architecture fail. The mathematical requirements for subnetwork extraction across different architectural components are detailed below.

The Residual Stream and Normalization

The core of the transformer is the residual stream, [Figure omitted from source export], where [Figure omitted from source export] is the sequence length and [Figure omitted from source export] is the hidden dimension. Layer additions take the form [Figure omitted from source export]. If a pruning method reduces the width of a specific block's output to [Figure omitted from source export], it can no longer be added to [Figure omitted from source export] unless the entire residual stream is universally reduced to [Figure omitted from source export] across all layers, or a specific up-projection matrix is inserted. Furthermore, modern architectures rely on RMSNorm, which computes the variance across the [Figure omitted from source export] dimension: [Figure omitted from source export]. Slicing features fundamentally alters the variance scalar. Naive slicing breaks the mathematical closure of normalization, leading to extreme numerical instability.

Attention Projections and Grouped-Query Attention (GQA)

Self-attention relies on projection matrices [Figure omitted from source export]. Grouped-Query Attention (GQA) shares [Figure omitted from source export] and [Figure omitted from source export] heads across multiple [Figure omitted from source export] heads to reduce KV cache size20. If attention heads are pruned, they must be pruned in strict logical groups. Removing a single [Figure omitted from source export] or [Figure omitted from source export] head requires the simultaneous removal of all its associated [Figure omitted from source export] heads to maintain the mapping ratio. Furthermore, the output projection [Figure omitted from source export] must have its corresponding input columns removed. Graph closure requires that the internal intermediate dimension strictly matches the number of retained heads multiplied by the head dimension.

Rotary Position Embeddings (RoPE)

RoPE operates on pairs of adjacent features within the attention head dimension to inject relative positional information20. If the head dimension [Figure omitted from source export] is sliced arbitrarily (e.g., pruning [Figure omitted from source export] from 128 to 110), rotary adjacent pairs are severed, corrupting the complex number rotation. Subnetworks must preserve the exact [Figure omitted from source export] configured during the RoPE initialization, or the rotary basis frequencies must be completely re-calibrated.

MLP Gates (SwiGLU/GeLU)

Modern feed-forward networks (e.g., Llama, Qwen, Gemma) utilize gated MLPs: [Figure omitted from source export]. The intermediate size is expanded to a large dimension [Figure omitted from source export]. If an intermediate channel is pruned, the corresponding row in [Figure omitted from source export] and the corresponding columns in both [Figure omitted from source export] and [Figure omitted from source export] must be removed simultaneously16. The Hadamard product ([Figure omitted from source export]) enforces strict dimensional symmetry; a single misaligned index will cause a shape mismatch exception at runtime.

Vocabulary and Embedding Reduction

For narrow task specialists, the input embedding matrix [Figure omitted from source export] and the output language modeling head [Figure omitted from source export] represent a massive portion of the parameter budget. In Gemma-2-2B, the vocabulary size is 256,000, meaning the embeddings alone consume over 1.1GB in FP1621. By analyzing a narrow domain (e.g., local Rust code autocomplete), untriggered vocabulary rows (such as multilingual tokens) can be structurally sliced out of both [Figure omitted from source export] and [Figure omitted from source export]. However, this alters the tokenizer dependencies. A specialist cannot blindly share the champion tokenizer without carrying its full embedding table unless strict safety mechanisms are implemented. An unknown-domain fallback must trap out-of-vocabulary (\<UNK\>) generation attempts. If the user prompts the specialist with French text, the full tokenizer will emit French token IDs; if those rows are deleted, the subnetwork will encounter out-of-bounds memory access or segmentation faults. Graph closure here requires strict synchronization between the deployed tokenizer vocabulary and the dimensions of [Figure omitted from source export].

5. Current Primary-Source Architecture Matrix with Licenses and Revisions

To target the 0.3B to 2B dense-equivalent range for browser inference, an architectural matrix is established. This data reflects public foundational models amenable to structural extraction or acting as base champions.

Data sourced from architectural configurations and runtime implementations20. Architectural Note on Gemma 2: Gemma-2-2B exhibits a decoupled head dimension ([Figure omitted from source export]) which does not equal the standard hidden dimension divided by the number of query heads ([Figure omitted from source export])21. This anomaly necessitates careful parsing when calculating KV cache shapes or applying width pruning, as tools that inherently derive head dimension from the [Figure omitted from source export] ratio will miscalculate the graph closure requirements and generate invalid subnetworks.

6. Training/Recovery Requirements and Expected Failure Modes

When attempting to produce a smaller task specialist from a champion, the compression modality strictly dictates the required recovery phase. Arbitrary parameter reduction without corresponding recovery inevitably leads to mode collapse.

Expected Failure Modes of Unrecovered Slicing

If a subnetwork is created by deleting layers (e.g., shortening a 24-layer model to 18 layers) without further action, the immediate failure mode is perplexity explosion. Research demonstrates that layer removal creates a severe mismatch of activation magnitudes across layers and tokens at the pruning interface1. The activations from the layer preceding the pruning interface do not mathematically align with the expected inputs of the subsequent layer, destroying the model's zero-shot generative capacity.

Training-Free Offline Recovery

Methods like LinearPatch and Prune\&Comp recover performance by calculating a rescaling matrix or magnitude patch using a small calibration dataset (e.g., WikiText or C4). This process aligns the channel-wise magnitudes at the pruning interface1. This requires zero gradient updates and can be computed rapidly via forward passes, making it highly attractive for generating TinyRustLM .slm artifacts without relying on large-scale GPU clusters.

Supervised Fine-Tuning and Distillation

For heavy structural width pruning, the graph is permanently altered. To recover, the parent model acts as a teacher, and the pruned subnetwork acts as a student. The student is trained via knowledge distillation to minimize the Kullback-Leibler (KL) divergence between its output logits and the teacher's logits27.

• Requirements: High-memory GPU clusters, substantial high-quality token datasets, and rigorous hyperparameter tuning.
• Failure Modes: Catastrophic forgetting of generalized capabilities, over-fitting to the SFT dataset, and representation collapse where the model loses its capacity for logical reasoning. In-place distillation strategies attempt to mitigate this by jointly training the subnetwork inside the parent architecture, but this significantly increases memory pressure during the backward pass.

Elastic Architectures and Sandwich Rules

If a model is pre-trained as an elastic network (e.g., slimmable networks), the training phase incorporates "sandwich rules." During a single training step, the full model, the smallest predefined subnetwork, and a randomly sampled intermediate subnetwork are all updated based on the loss4. This explicit interference training forces the network to arrange its most critical feature detectors in the earliest layers and dimensions, rendering post-hoc extraction zero-shot. The recovery requirement for nested models is non-existent at extraction time; the heavy lifting was entirely front-loaded into the pre-training curriculum.

7. CPU, SIMD, and Browser-WASM Systems Implications

The TinyRustLM runtime targets WebAssembly environments running in web browsers. This hardware constraint completely invalidates certain classes of network compression that appear successful in academic literature targeting NVIDIA CUDA environments.

The Fallacy of Unstructured and N:M Sparsity

Unstructured pruning zeroes out weights based on magnitude, creating highly sparse matrices16. While this reduces the compressed binary size (via zlib/Brotli encoding during network transfer), it fundamentally fails to reduce wall-clock time or peak committed memory in a WASM execution environment. Standard sparse matrix representations (like Compressed Sparse Row \- CSR) require index indirection arrays. In CPU and WASM SIMD128 environments, computing dense Matrix-Vector multiplications (GEMV) is heavily optimized using contiguous memory access and predictable vector prefetching. Traversing a sparse index array destroys cache locality, forcing random memory access patterns that stall the CPU pipeline. Furthermore, the memory required to store the sparse metadata (indices and block pointers) often overshadows the memory saved by zeroing out the weights. Consequently, a 50% unstructured sparse model will frequently run slower and consume more linear heap memory than its dense counterpart on a standard CPU. Block sparsity introduces similar issues, as padding waste is required to align the blocks with SIMD registers.

The Superiority of Structured Pruning and Elasticity

Structured pruning (removing entire layers, attention heads, or channels) and elastic nested submodels physically shrink the tensor dimensions8.

• Contiguous Memory: The remaining parameters form completely dense blocks. Matrix multiplications map perfectly to standard BLAS routines or custom Rust std::arch::wasm32 SIMD128 intrinsics, maximizing throughput.
• Cache Utilization: A structurally smaller tensor fits more easily into L2/L3 CPU caches, dramatically improving the memory bandwidth bottleneck during the autoregressive decode phase.
• WASM Memory Limits: 32-bit WebAssembly environments are hard-capped at 4GB of linear memory. A structurally smaller .slm file directly reduces the contiguous memory allocation required for the WebAssembly.Memory object, allowing the model to load without triggering browser-level OutOfMemory aborts. A reduction in logical parameter count only matters if it translates to a reduction in dense array dimensions.

8. Honest Byte and Memory Accounting Formulas

For the TinyRustLM ecosystem, memory accounting cannot be obfuscated. Parameter count does not linearly equate to physical footprint, especially when considering the dynamic state required for autoregressive generation.

1. Logical Parameter Bytes ([Figure omitted from source export])

The total parameters in a standard dense transformer subnetwork are calculated as: [Figure omitted from source export] (Note: This approximation excludes minor bias terms and LayerNorm parameters which scale linearly with [Figure omitted from source export] or [Figure omitted from source export]. The term [Figure omitted from source export] assumes [Figure omitted from source export] for standard multi-head attention; for GQA, the projection sizes for [Figure omitted from source export] and [Figure omitted from source export] are proportionally smaller).

2. Runtime Resident Bytes (Static Weights)

Assuming execution in FP16 or BF16 (dtype\_bytes \= 2), the static footprint allocated in WASM linear memory is: [Figure omitted from source export] If block quantization (e.g., GGUF Q4\_0) is applied, dtype\_bytes averages approximately 0.55 bytes per parameter, significantly reducing [Figure omitted from source export].

3. Dynamic KV Cache Bytes

The KV cache is the most critical memory bottleneck for contextual conversations. It grows linearly with sequence length ([Figure omitted from source export]) and must generally be allocated in high-precision (FP16/BF16) to avoid degradation21. [Figure omitted from source export]

• For Llama-3.2-1B at a 4096 context length (FP16): [Figure omitted from source export].
• For Gemma-2-2B at a 4096 context length (FP16): [Figure omitted from source export]21. A subnetwork that prunes layers ([Figure omitted from source export]) or KV heads ([Figure omitted from source export]) yields a mathematically guaranteed, proportional reduction in peak KV memory.

4. Peak Committed Memory

The maximum allocation required to successfully process a request without triggering an abort: [Figure omitted from source export] [Figure omitted from source export] accounts for intermediate activations (e.g., the final logits tensor of size [Figure omitted from source export]). For Gemma-2-2B with a [Figure omitted from source export] vocabulary, the logit matrix for a batch of 1 token requires 1 MB, but during the parallel prefill phase for a 4096-token prompt, the attention matrix calculation causes significant memory spikes that must be pre-allocated.

5. Sparse Metadata and Physical Shared Bytes

If a subnetwork utilizes unstructured sparsity, [Figure omitted from source export] must be added to [Figure omitted from source export], typically consuming 1-2 bytes per non-zero weight for index pointers. Conversely, if the .slm artifact contains an elastic Matryoshka model, the parent champion ([Figure omitted from source export]) and the specialist subnetwork ([Figure omitted from source export]) physically share the same memory pointer. [Figure omitted from source export] The specialist uses a logical view (e.g., \&champion\_weights\[0..specialist\_size\]) requiring zero additional bytes in memory or storage.

9. Proposed Negative Controls and Local Experiment Matrix

To empirically validate that a generated .slm specialist is mathematically functional and not merely outputting memorized statistical noise that appears syntactically correct, strict negative controls must be established1. Evaluation answers cannot enter the training or extraction pipeline. Hiding a failed subnetwork behind repaired text or repetition penalties invalidates the benchmark.

The Negative Control

Hypothesis: A subnetwork produced by structurally dropping layers without magnitude compensation results in a severed computational graph and invalidates the model.Experiment:

1. Load the base champion model (e.g., Llama-3.2-1B).

2. Naively slice and delete the final 25% of layers (layers 12-15) without any activation scaling, fine-tuning, or magnitude patching14.

3. Run a forward pass on a deterministic test corpus (e.g., WikiText).Expected Observation: The negative control model will exhibit severe perplexity degradation (e.g., PPL spiraling \> 50\) and output incoherent repetition. The variance of the intermediate activations will display a sharp mathematical discontinuity at the interface of layer 11 and the final classification head. This ensures that the baseline quality failure remains explicitly visible.

The Local Experiment Matrix

To qualify a valid .slm subnetwork, execute the following evaluation matrix in a WASM testbed:

10. TinyRustLM Decision Tree and Stop Rules

To achieve the product hypothesis of one strong general champion plus several much smaller task specialists, the following decision logic is proposed for generating the .slm files. Node 1: Does the parent architecture natively support nested elasticity (MRL / MatFormer)?

• Yes: Extract the nested subnetwork by dynamically slicing the tensors at runtime (e.g., extracting the first [Figure omitted from source export] dimensions and [Figure omitted from source export] layers)4. The specialist requires zero additional transfer bytes because it executes as an alternative logical view over the resident champion memory. STOP.
• No: Proceed to Node 2\.

Node 2: Is the client bandwidth-constrained on .slm payload delivery?

• Yes: Utilize Depth Pruning (Layer Dropping) paired with Magnitude Compensation (LinearPatch / Prune\&Comp)1. Evaluate the Block Influence (BI) of the champion's layers. Remove the 20-30% least influential layers (often middle or late-middle layers, or strictly reverse-order14). Apply the training-free channel magnitude alignment patch offline. Deliver the distinct smaller .slm to the client. STOP.
• No (Storage/bandwidth is ample, but memory is tight): Proceed to Node 3\.

Node 3: Can the specialization be achieved via additive weights?

• Yes: Utilize a Trained Adapter (LoRA). The champion remains in memory, and the specialist is a tiny payload (typically \<50MB) containing low-rank additive matrices. This avoids subnetwork graph closure issues entirely but slightly increases peak committed memory during execution due to the extra adapter math. STOP.
• No (The base model itself must be shrunk): Proceed to Node 4\.

Node 4: Does the specialist require deep specialization in a narrow domain (e.g., local code autocomplete)?

• Action: Extract a structurally pruned descendant and apply heavy in-place Knowledge Distillation (KD) from the champion using domain-specific tokens. Implement Vocabulary Reduction to strip untriggered tokens from the embedding space, delivering a custom tokenizer mapping alongside the specialist. STOP.

11. Unknowns Requiring Private/Local Execution

Several critical parameters cannot be extrapolated from public literature and must be empirically measured within the TinyRustLM target environment.

WASM SIMD Compiler Quirks

While WebAssembly SIMD128 is standardized, the V8 (Chrome) and SpiderMonkey (Firefox) JIT compiler implementations vary wildly. The exact instruction overhead of masking for Block Sparsity versus pure dense GEMM loops must be benchmarked on actual client hardware, as browser sandboxing may introduce unexpected memory bandwidth bottlenecks.

MoE Router Recalibration and Rare-Domain Regressions

If utilizing an MoE model (e.g., Qwen-1.5B-MoE variants), identifying which experts represent "cold" knowledge to be pruned or merged17 is highly dependent on the user's specific conversational distribution. When experts are moved to disk or deleted, the router probabilities must be recalibrated. A rare-domain regression (e.g., the model suddenly failing at a niche coding language because its corresponding expert was pruned) cannot be predicted by general benchmarks and requires local logging.

License and Derivative-Work Implications

Structurally pruning models like Llama 3.2 or Gemma 2 creates a derivative weight artifact. Authorized legal review is required to determine if the resulting .slm specialist inherits the specific "Llama 3.2 License" or "Gemma License" restrictions (e.g., Monthly Active User limits, explicit attribution mandates) and whether a heavily distilled model escapes derivative classification. No legal conclusion is provided here; consult intellectual property counsel prior to commercial release.

12. Annotated Primary-Source Bibliography

• \[cite: 7\] Xia et al. (2024). "SliceGPT/Sheared LLaMA structured pruning LLM arXiv". Demonstrates structured pruning removes entire architectural components (layers, heads) and establishes that pruning outperforms training from scratch under limited token budgets. URL: https://arxiv.org/html/2606.14150v1
• \[cite: 11\] "Bonsai: Gradient-free structured pruning". Explores estimating module relevance via forward passes only, eliminating backpropagation memory requirements. URL: https://arxiv.org/html/2402.05406v4
• \[cite: 12\] "Self-Pruner". Proposes utilizing LLMs to automatically search for layer-wise pruning rates. URL: https://arxiv.org/html/2502.14413v1
• \[cite: 15\] "NIRVANA". Addresses structured pruning zero-shot degradation via adaptive sparsity allocation. URL: https://arxiv.org/html/2509.14230v1
• \[cite: 16\] "Adapt-Pruner". Confirms unstructured pruning faces speedup difficulties without specialized hardware, validating structured pruning. URL: https://arxiv.org/html/2502.03460v1
• \[cite: 8\] "IFPruning". Discusses input-dependent dynamic structured pruning. URL: https://arxiv.org/html/2501.02086v3
• \[cite: 9\] Men et al. (2024). "ShortGPT". Introduces Block Influence (BI) to delete redundant layers, outperforming complex pruning without retraining. URL: https://arxiv.org/html/2403.03853v1
• \[cite: 13\] Men et al. (2024). "ShortGPT v3". Re-affirms BI metrics for layer removal. URL: https://arxiv.org/html/2403.03853v3
• \[cite: 1\] "Prune\&Comp". Identifies that layer removal induces a significant magnitude gap in hidden states, solved by offline training-free weight rescaling. URL: https://arxiv.org/html/2507.18212v1
• \[cite: 2\] "LinearPatch". Details mathematical plugging of activation magnitude mismatches across layers at the pruning interface for layer-pruned LLMs. URL: https://arxiv.org/html/2505.24680v1
• \[cite: 14\] "LLM Layer Pruning Best Practices". Identifies reverse-order pruning (final 25% of layers) as a highly effective metric. URL: https://arxiv.org/html/2411.15558v1
• \[cite: 3\] "LinearPatch v2". Confirms activation scale misalignment as the root cause of layer pruning performance drops. URL: https://arxiv.org/html/2505.24680v2
• \[cite: 4\] "ThinkingViT". Introduces Matryoshka-style nested subnetworks for elastic inference without extra tuning. URL: https://arxiv.org/html/2507.10800v3
• \[cite: 10\] "ThinkingViT PDF". Confirms slicing components according to indices enables elastic inference. URL: https://arxiv.org/pdf/2507.10800
• \[cite: 5\] "MatFormer". Introduces a nested Feed Forward Network structure, extracting submodels natively. URL: https://arxiv.org/html/2310.07707v2
• \[cite: 6\] "Matryoshka MoE (M-MoE)". Instills coarse-to-fine structure in MoE models for elastic inference, bypassing degradation when altering activated experts. URL: https://arxiv.org/html/2509.26520v1
• \[cite: 29\] "Nemotron Elastic". Embeds nested submodels inside a parent model, sharing weights, extracted zero-shot. URL: https://arxiv.org/html/2511.16664v1
• \[cite: 30\] "ThinkingViT v1". Demonstrates token recycling for nested architectures. URL: https://arxiv.org/html/2507.10800v1
• \[cite: 23\] Qwen2.5-1.5B Architecture. Foundational integration data. URL: https://huggingface.co/unsloth/InternVL3-8B-Instruct-GGUF/commit/d1b12f182689dd2b2d4c75d46f5632f1ee1bc319
• \[cite: 31\] Qwen2.5-1.5B Reward Model. Config validation. URL: https://huggingface.co/seangogo/Qwen2.5-1.5B\_reward\_model\_v2\_normalized/commit/77281bf8cca27e663d38f9b7590802668a9b3d69
• \[cite: 32\] Qwen2.5-0.5B / 1.5B vLLM Support. URL: https://github.com/vllm-project/vllm/issues/16826
• \[cite: 33\] InternVL3-9B Pretrained. Identifies Qwen KV head setups. URL: https://huggingface.co/OpenGVLab/InternVL3-9B-Pretrained/commit/4274d3f8f730f41fff5ace48715512485e74ce73
• \[cite: 34\] InternVL3-1B Pretrained. Identifies Qwen 2.5 baseline usage. URL: https://huggingface.co/OpenGVLab/InternVL3-1B-Pretrained/commit/937bdd533bcfee0c1dffc1616a2f463c34ef2f77
• \[cite: 20\] TransformerLens Llama-3.2-1B/3B configs. Provides foundational tensor shapes (2048 hidden, 32 heads, 8 kv heads for 1B). URL: https://github.com/neelnanda-io/TransformerLens/blob/main/transformer\_lens/loading\_from\_pretrained.py
• \[cite: 24\] Llama-3.2-1B/3B Text2SQL. Validates 3072 hidden size for 3B. URL: https://huggingface.co/NarayanaGenai/Llama-3.2-1B-Instruct-txt2sql/blame/main/config.json
• \[cite: 28\] KVSizeCalc. Highlights KV size formulas and head\_dim discrepancies. URL: https://github.com/0-EricZhou-0/KVSizeCalc
• \[cite: 35\] Llama 3.2 Fine-Tuning. URL: https://colab.research.google.com/drive/17pKvi-R-wy\_R3QzCiZB4NPX\_DIueh36X?usp=sharing
• \[cite: 36\] Smol Training Playbook. Validates 1B and 3B capability classes. URL: https://huggingfacetb-smol-training-playbook.hf.space/
• \[cite: 25\] SmolLM2-360M / 1.7B Configs. URL: https://huggingface.co/datasets/louisbrulenaudet/mergekit-configs
• \[cite: 21\] Gemma-2-2B KV parameters. Validates [Figure omitted from source export], [Figure omitted from source export], [Figure omitted from source export], [Figure omitted from source export]. URL: https://www.reddit.com/r/LocalLLaMA/comments/1hr2noa/whats\_the\_deal\_with\_the\_bs\_anyways/
• \[cite: 22\] Gemma-2-2B config.json. Validates model properties. URL: https://huggingface.co/mlx-community/gemma-2-2b/blob/main/config.json
• \[cite: 26\] Gemma-2-2B dimensional math. Confirms total attention dimension anomaly ([Figure omitted from source export] hidden size). URL: https://blog.csdn.net/shizheng\_Li/article/details/144866526
• \[cite: 37\] Gemma-2-2B Pretraining. URL: https://huggingface.co/ChallengerSpaceShuttle/continued-trained-gemma2-2b
• \[cite: 38\] Gemma-2-2B vLLM issues. Highlights sliding window constraints. URL: https://github.com/vllm-project/vllm/issues/7464
• \[cite: 39\] GGUF Gemma support. URL: https://github.com/huggingface/transformers/pull/35887/files/5c0d61da78dbdf2961eb729e2ece8932f7345e86
• \[cite: 17\] C-PRUNE (Cluster-driven Expert Pruning). Expands on expert redundancy. URL: https://raw.githubusercontent.com/mlresearch/v317/main/assets/yao26a/yao26a.pdf
• \[cite: 40\] Generic TB-Coverage. Validates expert pruning using generic text corpora. URL: https://arxiv.org/html/2607.01710v1
• \[cite: 18\] SHAPE (Shapley-guided expert pruning). Evaluates intra-layer expert cooperation. URL: https://arxiv.org/html/2606.09886v1
• \[cite: 41\] REAM (Router-weighted Expert Activation Merging). Evaluates converting experts to prototypes. URL: https://arxiv.org/html/2604.04356v1
• \[cite: 27\] MoE-Pruner. Details expert-wise knowledge distillation recovery. URL: https://arxiv.org/html/2410.12013v1
• \[cite: 42\] MoE Expert Pruning. URL: https://arxiv.org/html/2402.14800v2
• \[cite: 19\] ConMoE. Formulates MoE compression as expert-pool consolidation without retraining. URL: https://arxiv.org/pdf/2605.29350
• \[cite: 43\] FUSE Taxonomy for Model Merging. Discusses parameter sharing and interpolation. URL: https://arxiv.org/html/2603.09938v2
• \[cite: 44\] SUPERMERGE. URL: https://huggingface.co/papers?q=chat%20vector%20merging
• \[cite: 45\] Nexus MoE Architecture. Evaluates upcycling dense models into MoE logic. URL: https://openreview.net/forum?id=WkHkwo8rpL\&noteId=ND7RiB7oPQ
• \[cite: 46\] Heterogeneous MoE Merging. URL: https://arxiv.org/html/2502.00997v2
• \[cite: 47\] Chemical Domain Model Merging. Outlines domain-specific knowledge dilution during parameter sharing. URL: https://papers.neurips.cc/paper\_files/paper/2025/file/379eb671b5b00ba37d6aa2a93e298d2a-Paper-Conference.pdf

Works cited

1. Prune\&Comp: Free Lunch for Layer-Pruned LLMs via Iterative Pruning with Magnitude Compensation \- arXiv, https://arxiv.org/html/2507.18212v1

2. A Simple Linear Patch Revives Layer-Pruned Large Language Models \- arXiv, https://arxiv.org/html/2505.24680v1

3. A Simple Linear Patch Revives Layer-Pruned Large Language Models \- arXiv, https://arxiv.org/html/2505.24680v2

4. ThinkingViT: Matryoshka Thinking Vision Transformer for Elastic Inference \- arXiv, https://arxiv.org/html/2507.10800v3

5. MatFormer: Nested Transformer for Elastic Inference \- arXiv, https://arxiv.org/html/2310.07707v2

6. Training Matryoshka Mixture-of-Experts for Elastic Inference-Time Expert Utilization \- arXiv, https://arxiv.org/html/2509.26520v1

7. Small LLMs: Pruning vs. Training from Scratch \- arXiv, https://arxiv.org/html/2606.14150v1

8. Instruction-Following Pruning for Large Language Models \- arXiv, https://arxiv.org/html/2501.02086v3

9. ShortGPT: Layers in Large Language Models are More Redundant Than You Expect \- arXiv, https://arxiv.org/html/2403.03853v1

10. ThinkingViT: Matryoshka Thinking Vision Transformer for Elastic Inference \- arXiv, https://arxiv.org/pdf/2507.10800

11. Everybody Prune Now: Structured Pruning of LLMs with Only Forward Passes \- arXiv, https://arxiv.org/html/2402.05406v4

12. Towards Efficient Automatic Self-Pruning of Large Language Models \- arXiv, https://arxiv.org/html/2502.14413v1

13. ShortGPT: Layers in Large Language Models are More Redundant Than You Expect \- arXiv, https://arxiv.org/html/2403.03853v3

14. Reassessing Layer Pruning in LLMs: New Insights and Methods \- arXiv, https://arxiv.org/html/2411.15558v1

15. NIRVANA: Structured Pruning Reimagined for Large Language Models Compression \- arXiv, https://arxiv.org/html/2509.14230v1

16. Adaptive Structural Pruning for Efficient Small Language Model Training \- arXiv, https://arxiv.org/html/2502.03460v1

17. Domain-Specific Expert Pruning for Mixture-of-Experts LLMs \- GitHub, https://raw.githubusercontent.com/mlresearch/v317/main/assets/yao26a/yao26a.pdf

18. SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs \- arXiv, https://arxiv.org/html/2606.09886v1

19. ConMoE: Expert-Pool Consolidation via Prototype Reassignment for MoE Compression \- arXiv, https://arxiv.org/pdf/2605.29350

20. TransformerLens/transformer\_lens/loading\_from\_pretrained.py at main \- GitHub, https://github.com/neelnanda-io/TransformerLens/blob/main/transformer\_lens/loading\_from\_pretrained.py

21. What's the deal with the B's anyways? : r/LocalLLaMA \- Reddit, https://www.reddit.com/r/LocalLLaMA/comments/1hr2noa/whats\_the\_deal\_with\_the\_bs\_anyways/

22. config.json · mlx-community/gemma-2-2b at main \- Hugging Face, https://huggingface.co/mlx-community/gemma-2-2b/blob/main/config.json

23. Add files using upload-large-folder tool · unsloth/InternVL3-8B-Instruct-GGUF at d1b12f1, https://huggingface.co/unsloth/InternVL3-8B-Instruct-GGUF/commit/d1b12f182689dd2b2d4c75d46f5632f1ee1bc319

24. config.json · NarayanaGenai/Llama-3.2-1B-Instruct-txt2sql at main, https://huggingface.co/NarayanaGenai/Llama-3.2-1B-Instruct-txt2sql/blame/main/config.json

25. louisbrulenaudet/mergekit-configs · Datasets at Hugging Face, https://huggingface.co/datasets/louisbrulenaudet/mergekit-configs

26. 解析大模型的配置文件（config.json）：以Gemma-2-2B为例原创 \- CSDN博客, https://blog.csdn.net/shizheng\_Li/article/details/144866526

27. MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router \- arXiv, https://arxiv.org/html/2410.12013v1

28. GitHub \- 0-EricZhou-0/KVSizeCalc: Per-token KV cache size calculator from HuggingFace config.json. Pluggable calculators: MHA/GQA/MQA, MLA (DeepSeek), hybrid Gated DeltaNet (Qwen3-Next), sliding/chunked windows., https://github.com/0-EricZhou-0/KVSizeCalc