Teleodynamic Evolution of Small Language Model Ecologies: A Rust-Native Paradigm for Multi-Model Breeding and Code Beading

Full report

On this page

The rapid expansion of artificial intelligence architectures has historically been driven by monolithic scaling laws, under which performance enhancements are sought through continuous increases in parameters, compute, and pretraining data.1 This trajectory faces immediate boundaries, including extreme energy and computational costs, physical hardware constraints, and behavioral homogenization.1 In response to these limitations, a profound paradigm shift is underway toward biology-mimicking, decentralized small-model ecologies.1 Under this decentralized approach, the unit of long-term optimization is not an individual, hyper-scaled network, but a dynamic population of small, mutable, and interchangeable models whose parameters, topologies, and execution flows evolve under bounded viability functions.1 This teleodynamic evolution operates across separate symbolic and parametric loops, coordinating localized competencies while leveraging high-performance, memory-safe system architectures built in Rust.1

The Teleodynamic Shift and the Digital 4Fs Framework

To achieve structural viability on resource-constrained edge hardware, decentralized AI systems must operate within the digital 4Fs framework, optimizing across four core operational axes: Fast, Flexible, Frugal, and Federated.1 This framework prioritizes execution velocity, topological plasticity, resource mindfulness, and decentralized collaborative learning.1

This digital architecture mirrors biological systems, where survival and reproduction are governed by primal drives.2 In neurobiology, the classical "Four Fs" proposed by Karl Pribram—fighting, fleeing, feeding, and mating—represent instinctual motivations regulated by the hypothalamus to maintain internal homeostasis and ensure the transmission of genetic material.2 Joseph LeDoux’s two-system framework demonstrates that survival relies on nonconscious, subcortical "defensive survival circuits" that process threats and execute rapid actions without the delay of conscious cognitive processing.1 In an artificial ecology, these biological imperatives translate into software-native operations designed to preserve functional organization under severe resource constraints.1

The interaction of these digital drives ensures that an AI ecosystem remains adaptive.1 Without the Flee operation, the system accumulates dead weight, leading to memory exhaustion and runtime failure.1 Without Fork, the system stagnates and loses the capacity to adapt to new environmental demands.1 Without Fight, variation remains arbitrary, degrading the system into chaotic noise.1 Without Feed, the thermodynamic engine lacks the resources to drive learning.1 In off-grid deployment zones, the speed of an edge-native AI ecology depends not only on fast forward-pass execution but also on its ability to rapidly adapt to security and privacy demands.1 For instance, on-device runtimes such as Google's LiteRT-LM achieve up to 2,000 tokens per second during the prefill phase on a mobile processor for a 270-million-parameter model, showcasing the capability of highly optimized, tiny models.1 However, to maintain security and respect privacy regulations, these networks must support rapid data deletion.1 Traditional model retraining to purge toxic, poisoned, or copyrighted data is computationally impossible at the edge.1 To bypass this, the ecosystem leverages accelerated unlearning algorithms, such as Fast-FedUL.1 This framework isolates and removes target gradient impacts up to 1,000 times faster than full retraining, ensuring the sanitized model preserves its primary task capabilities without introducing operational delays.1 Flexibility is achieved by allowing the underlying network architectures to dynamically adjust their depth and width during the active inference phase.1 Spiking Neural Networks (SNNs) play a pivotal role here, utilizing sparse activation paths to conserve energy during trivial tasks and scaling up to deeper topologies only when handling complex or noisy stimuli.1 This structural plasticity is mirrored in federated learning paradigms like FedFA, which allow heterogeneous edge clients to utilize varying sub-model widths and depths, tailored precisely to their localized hardware capacities.1

Code Beading and External Task-State Graph Architectures

A primary limitation of autonomous edge agents is their susceptibility to cognitive amnesia when forced to execute complex, multi-stage plans that exceed their immediate context windows.1 To resolve this operational constraint, the system leverages code beading.1 Code beading is a structural solution that establishes a persistent, queryable, and non-volatile external working memory for digital agents.1 It represents the digital tokenization of complex, continuous planning information into discrete, immutable, and interchangeable units called "beads".1 This methodology is directly inspired by Steve Yegge's Beads system, which operates as an issue-tracker for coding agents, exposing a structured memory of subtasks organized in a Git-based graph.1 Rather than forcing an LLM to generate long, linear, and volatile scripts that are prone to mid-generation failures, code beading allows an agent to dynamically append, modify, and link small task-specific beads over a persistent graph.1 This structural decomposition shifts agent operation from volatile runtime execution to transactional, stateful graphs.1

If a hardware node experiences power loss or an agent fails during execution, the state is preserved within the immutable Git-based bead registry.1 Upon reboot, the agent queries the registry, identifies the last successfully executed bead, and resumes operations, completely bypassing context-window amnesia.1 In practice, this compositional software logic is supported by modern programming tools, such as the Mojo language and modular engines, which utilize strict value semantics to optimize execution speeds up to 9 times faster than standard frameworks without memory safety side effects.1

Genomic Foundations of Model Breeding

While code beading manages symbolic task structures, model breeding focuses on the continuous evolutionary optimization of continuous parameter spaces.1 This paradigm draws a direct analogy to agricultural genomics and plant breeding.1 In plant biology, the development of new cultivars begins with selecting potential parental lines with desirable traits (e.g., disease resistance, increased yield, flavor, environmental adaptability) and making crosses to create a breeding population.4 This is followed by selection strategies to identify superior lines at the end of the breeding cycle.4 While traditional breeding methods rely on slow, physical crosses, Breeding 4.0 integrates big data, genome sequencing, and precise gene editing to make targeted modifications and accelerate the breeding process.6 Model breeding applies these exact principles to artificial neural networks.1 Rather than performing slow, computationally expensive joint training, developers treat existing fine-tuned models as parental lines.7 These models are crossed using mathematical merging operations to produce offspring that inherit specialized capabilities.7

Parent A (cultivar\_math) \\ \===\> Crossed (SLERP/TIES/DARE) \===\> Offspring (resilient\_generalist) Parent B (cultivar\_lang) /

A breakthrough in this domain is Sakana AI's Evolutionary Model Merge, which automates this breeding process by applying genetic algorithms in two distinct spaces 7:

• Parameter Space (PS): Operates on the continuous weight space of the neural networks, evolving layer-wise mixing weights and hyperparameter interpolation values.1 This is highly effective for blending features when models share a compatible base topology.1
• Data Flow Space (DFS): Operates on the discrete routing path of data through the layers of multiple models.1 This allows the evolutionary algorithm to discover novel routing recipes, such as passing tokens through layer [Figure omitted from source export] of Model A and then routing them to layer [Figure omitted from source export] of Model B.1

This dual-space optimization allows the evolutionary process to discover novel and unintuitive model combinations, facilitating cross-domain hybridization.7 For example, this method can automatically breed a high-performing Japanese language model with mathematical reasoning capabilities by crossing a specialized non-English language parent with a specialized mathematics parent, without requiring extensive training data or compute.7

The FUSE Taxonomy and Weight-Merging Mathematics

To systematically organize and execute these breeding strategies, the system adopts the FUSE taxonomy, which structures model merging along four dimensions: Foundations, Unification Strategies, Scenarios, and Ecosystem.9 Unification strategies represent the mathematical operations used to combine the weights of multiple customized networks into a single unified model.8

\+-----------------------+ | FUSE Taxonomy | \+-----------+-----------+ | \+---------+---------+---------+---------+ | | | | Foundations Unification Scenarios Ecosystem Strategies

Model Soup (Naive and Greedy)

The simplest unification strategy is Model Soup, which blends weights across multiple models fine-tuned from the same base network.8 The naive implementation performs a simple sequential linear average of the weights 8: [Figure omitted from source export] While computationally efficient, linear averaging can lead to degraded performance and loss of generalizability due to destructive interference in high-dimensional space.8 To address this, greedy model soup evaluates each candidate model sequentially and only retains its weights in the average if it improves performance on a validation set.8

Spherical Linear Interpolation (SLERP)

To preserve the geometric and scale properties of high-dimensional vector spaces, Spherical Linear Interpolation (SLERP) is preferred over linear interpolation.11 When interpolating between two weight vectors, linear averaging can decrease the magnitude of the interpolated vector, reducing the scale of the weights.11 Because the direction of weight vectors often represents learned features more than their magnitude, SLERP maintains a constant rate of change along a spherical path.11 First, the input weight vectors [Figure omitted from source export] and [Figure omitted from source export] are normalized to unit length to isolate their directions.11 The angle [Figure omitted from source export] between the vectors is computed using their dot product 11: [Figure omitted from source export] If the vectors are nearly collinear ([Figure omitted from source export]), the algorithm defaults to linear interpolation for efficiency.11 Otherwise, SLERP computes scale factors based on the interpolation factor $t \\in $ and the angular separation 11: [Figure omitted from source export] This ensures that the magnitude remains constant, preserving the representational scale of the model's weights across every layer of the network.11

Task Arithmetic

Task Arithmetic operates by representing the capabilities learned during fine-tuning as task vectors.8 A task vector [Figure omitted from source export] is computed by subtracting the weights of the pre-trained base model [Figure omitted from source export] from the weights of the fine-tuned model [Figure omitted from source export] 8: [Figure omitted from source export] Multiple task vectors can be linearly merged to combine capabilities (e.g., adding math reasoning and translation).8 The merged task vector is then added back to the base model to produce the final weights 8: [Figure omitted from source export] Where [Figure omitted from source export] represents a scaling hyperparameter for each task vector.8

TIES-Merging

When merging multiple task vectors, performance can degrade due to task interference, which is caused by redundant parameter changes and sign conflicts across models.8 TIES-Merging (TrIm Elect Sign and Merge) addresses this using a three-step pipeline 8:

• Trim: Retain only the most significant parameter changes by zeroing out the smallest [Figure omitted from source export] of values in each task vector based on absolute magnitude.8 This removes redundant parameter changes and isolates core updates.8
• Elect Sign: Resolve sign conflicts by computing the dominant sign of each parameter across all task vectors.8 The sign is selected based on the cumulative magnitude of updates in that direction 8:

[Figure omitted from source export]

• Merge: Average only those parameter updates that align with the elected dominant sign, while setting non-aligned updates to zero.8

Task Vectors \---\> \---\> \---\> \[ Merge (Average Aligned) \]

DARE (Drop and Rescale Elect)

DARE (Drop and Rescale) is an enhancement that can be used alongside other merging strategies, such as TIES.8 DARE operates on the observation that fine-tuned models contain many redundant weight changes.8 It randomly drops a large percentage [Figure omitted from source export] of the updates in a task vector by setting them to zero, and then rescales the remaining weights to preserve their expected value 8: [Figure omitted from source export] This process strips out small parameter updates, leaving only the most critical changes, which mitigates interference and improves the stability of multi-task model merging.8

High-Performance Rust Implementations with Candle and Burn

Deploying evolutionary small language model (SLM) ecologies to edge hardware requires a systems language that guarantees memory safety without garbage collection overhead, produces minimal binary footprints, and provides direct access to hardware accelerators.12 Rust has emerged as the standard for this architecture, supported by two primary deep learning frameworks: HuggingFace's Candle and Burn.12

Variable Management and safetensors Layout Control

In Candle, variable management and state tracking are handled by two core structs 17:

• VarMap: Keeps track of the active model variables and their gradients, making it necessary for fine-tuning, training, or executing online weight updates.17
• VarBuilder: Used to retrieve and load weights into the model's structural layers.17 During inference, loading weights via VarBuilder directly turns off gradient tracking, saving memory and accelerating execution.17

\+------------+ | VarMap | (Tracks gradients/variables) \+-----+------+ | v \+------------+ | VarBuilder | (Loads weights into layers) \+-----+------+ | \+--------------+--------------+ | | v v \[Finetuning Mode\] \[Inference Mode\] (Gradient tracking ON) (Gradient tracking OFF)

To load model parameters into RAM or GPU memory, Rust-native libraries use the safetensors format, which is designed to be faster and safer than PyTorch’s traditional pickle format.16 Safetensors provides zero-copy properties: while loading, the format maps the binary tensor buffers directly from disk into RAM or GPU VRAM using memory-mapped file access (MmapedSafetensors), bypassing intermediate allocation steps on the CPU.12 Safetensors also protects the runtime from denial-of-service (DoS) and execution exploits.18 It enforces structural constraints: addresses within the file cannot overlap, and the metadata header size is limited to 100MB to prevent parsing exploits.18 Tensors are stored in a packed, row-major layout without strides to ensure aligned memory reads.18

Rust use candle\_core::{Device, Result, Tensor}; use std::collections::HashMap;

/// Performs Spherical Linear Interpolation (SLERP) to merge model parameters in Rust. /// This method preserves geometric features in high-dimensional vector spaces. pub fn spherical\_linear\_interpolate(w0: \&Tensor, w1: \&Tensor, t: f32) \-\> Result\<Tensor\> { let shape \= w0.shape();

// Flatten both tensors to perform vector operations in 1D space let w0\_flat \= w0.flatten\_all()?; let w1\_flat \= w1.flatten\_all()?;

// Calculate dot product to find the angular separation let dot\_tensor \= w0\_flat.matmul(\&w1\_flat.unsqueeze(1)?)?; let dot \= dot\_tensor.to\_scalar::\<f32\>()?.clamp(-1.0, 1.0);

// If vectors are near collinear (dot product close to 1), fall back to standard LERP if dot.abs() \> 0.9995 { let term0 \= w0.affine((1.0 \- t) as f64, 0.0)?; let term1 \= w1.affine(t as f64, 0.0)?; return term0.add(\&term1); }

let theta\_0 \= dot.acos(); let theta\_t \= theta\_0 \* t;

let s0 \= (theta\_0 \- theta\_t).sin() / theta\_0.sin(); let s1 \= theta\_t.sin() / theta\_0.sin();

let term0 \= w0.affine(s0 as f64, 0.0)?; let term1 \= w1.affine(s1 as f64, 0.0)?;

// Sum and reshape back to the original matrix dimensions term0.add(\&term1)?.reshape(shape) }

/// Merges entire model parameter sets using a target interpolation factor. pub fn breed\_models( parent\_a: \&HashMap\<String, Tensor\>, parent\_b: \&HashMap\<String, Tensor\>, t: f32, device: \&Device, ) \-\> Result\<HashMap\<String, Tensor\>\> { let mut child \= HashMap::new();

for (name, tensor\_a) in parent\_a { if let Some(tensor\_b) \= parent\_b.get(name) { // Apply SLERP to multi-dimensional weights (e.g., projections, attention, MLP) // Fall back to LERP for 1D vectors like bias terms or normalization layers let merged\_tensor \= if tensor\_a.rank() \> 1 { spherical\_linear\_interpolate(tensor\_a, tensor\_b, t)? } else { let term0 \= tensor\_a.affine((1.0 \- t) as f64, 0.0)?; let term1 \= tensor\_b.affine(t as f64, 0.0)?; term0.add(\&term1)? }; child.insert(name.clone(), merged\_tensor.to\_device(device)?); } else { // If parameter exists only in Parent A, preserve it in the Child child.insert(name.clone(), tensor\_a.to\_device(device)?); } } Ok(child) }

This Rust implementation allows edge systems to perform live model crosses and breeding directly in parameter space.1 Additionally, crates such as candle-lora and ruvllm enable runtime adapter merging and hot-swapping.19 During training, LoRA updates are tracked separately.20 At inference time, candle-lora executes weight merging, adding the low-rank adapter matrices directly to the base weights to eliminate the computational overhead of running separate paths.20 These adapters can also be unmerged dynamically to allow zero-downtime, runtime task specialization on heterogeneous hardware (e.g., Apple Neural Engine via CoreML or NVIDIA GPUs via CUDA).19

Tiny LLM Architectures and Predictive Execution Signals

Deploying a model ecology requires selecting parent architectures that balance parameter density with raw execution efficiency.21 Small language models with fewer than 3 billion parameters have demonstrated competitive performance on complex reasoning tasks.21

Early Attention Signals as Error Predictors

While benchmark scores represent macro-level capabilities, a runtime teleodynamic controller must make micro-level, query-by-query routing decisions.1 To do this efficiently, the system utilizes early attention layer signals.23 Recent empirical studies have identified that the branching\_factor calculated on the attention distributions of early transformer layers (layers 0 through 4\) serves as a predictive indicator of output correctness.23 The branching\_factor is defined as the count of tokens that receive an attention weight greater than a critical threshold ([Figure omitted from source export]).23 This metric leverages the observation that when a model is confident and on a trajectory to generate a correct output, its early attention heads focus on a narrow, well-defined set of key tokens.23 Conversely, when the model is on a trajectory to hallucinate or generate an incorrect response, its attention distributions become highly diffused, attending to many irrelevant tokens.23

Layer 0-1 Attention Grid: Token A: \[██████░░░░\] \-\> Weight \> 0.01 (Counted) Token B: \[█░░░░░░░░░\] \-\> Weight \< 0.01 (Pruned) Token C: \[████████░░\] \-\> Weight \> 0.01 (Counted)

Branching Factor (BF) \= Count of High-Attention Tokens High BF in Early Layers \-\> High Output Correctness Probability.

This structural metric discriminates between correct and incorrect generation paths before the model completes decoding.23 For instance, on the Qwen-0.5B model, evaluating the branching\_factor in Layer 3 yields an Area Under the ROC Curve (AUC) of 0.708 for predicting output correctness, while the OPT-125M model exhibits an AUC of 0.913 in Layer 1\.23 The teleodynamic controller integrates these execution signals into its judging module.1 When a specialist SLM begins generating an answer, the judge monitors the early layers.1 If the branching\_factor drops below a critical threshold, indicating a high probability of structural failure or hallucination, the generation process is terminated.1 The controller then routes the task to a different model variant, avoiding computation waste.1

Orchestration, Viability Constraints, and Interoperability

To manage these specialized populations at runtime, the ecology uses a hierarchical orchestration architecture.1 This system relies on dynamic cooperation, routing, and a formalized selection regime.1 The runtime architecture consists of four core components 1:

• Router: Directs incoming inputs to specific specialist models or optimal pipeline coalitions based on capabilities and execution budgets.1
• Specialized Model Ensemble: A population of small, specialized models or model-chains (e.g., specialized in legal syntax, mathematical deduction, or API calling) that operate in parallel to propose solution candidates.1
• Judge: Evaluates and aggregates candidate solutions.1 Rather than relying on unstructured, open-ended multi-agent debates—which are prone to cascading hallucination and high token overhead—the judge uses selective aggregation methods such as majority voting or Bayesian consensus.1
• Teleodynamic Controller: Oversees the evolutionary optimization loop, tracking physical resource utilization and enforcing developmental constraints.1

Mathematical Foundations of Teleodynamic Viability

To prevent run-away architectural bloat, the controller evaluates candidate models using a formalized multi-objective utility score.1 A structural variation is only adopted if its performance gain outpaces its hardware footprint.1 The viability score [Figure omitted from source export] is defined as: [Figure omitted from source export] Where [Figure omitted from source export] represents primary task utility, [Figure omitted from source export] is model calibration and robustness, [Figure omitted from source export] is population-wide behavioral diversity, [Figure omitted from source export] is memory consumption (RAM/Flash), [Figure omitted from source export] is execution latency, and [Figure omitted from source export] is energy consumption in Joules.1 The positive scaling factors [Figure omitted from source export] and negative penalty weights [Figure omitted from source export] are dynamically tuned to reflect local environmental constraints.1 At the node level, local optimization updates are driven by a resource-aware loss function 1: [Figure omitted from source export] Where [Figure omitted from source export] represents standard cross-entropy loss on target tokens, [Figure omitted from source export] measures topological complexity (active parameter counts), and [Figure omitted from source export] tracks physical energy expenditure during training or inference.1

Components of System Interoperability

For a self-organizing ecology to function, its components must be interchangeable.1 Interoperability operates across four distinct levels:

• Contract Interchangeability (API level): Models must implement identical input/output interfaces and capability contracts so routers can swap modules without breaking pipelines.1
• Runtime Interchangeability (Format level): Models trained across different frameworks must deploy on a common engine using standardized exchange formats like ONNX or PyTorch's TorchScript.1
• Parameter Interchangeability (Weights level): Swapping or merging weights directly requires models to share compatible base topologies and embedding dimensions.1
• Semantic Interchangeability (Representation level): Translating the hidden states and embeddings between models that do not share the same tokenization or embedding space.1 This is highly complex, typically requiring the training of projection layers or gated adapters.1

Tactical Engineering Protocols for Rust-Native SLM Ecologies

Establishing a production-grade evolutionary AI system in Rust requires translating these theoretical models into structured systems engineering patterns.

Loop Separation

Ensure that code breeding and model breeding operate in separate, isolated loops.1 Code mutations (e.g., changes to attention kernels, routing strategies, compilation flags) must undergo sandboxed, automated testing to verify correctness and security before execution.1 Model mutations (e.g., weight merges, adapter updates) are managed in parameter space and evaluated against held-out task suites.1 The evaluation machinery must remain locked inside the code-breeding boundary to prevent models from accessing or mutating their own fitness metrics.1

Dynamic Resource Allocation

The teleodynamic controller must continuously track hardware resource usage, including available RAM, memory bandwidth, and GPU compute queues.1 Proposed model crosses or adapter activations must prove their viability under the current hardware budget using the viability score equation before being promoted to the active ensemble.1

Pipeline Orchestration and Traceability

Tasks should be decomposed into modular pipelines managed by stateful, immutable task registries using code-beading patterns.1 Every model variation, weight merge, or adapter swap must be logged with complete lineage metadata.1 If a performance regression or security failure is detected, the teleodynamic controller can audit the lineage logs and execute a rollback to a stable historical equilibrium.1

Works cited

1. AI Evolution\_ Small Models, Big Ecology.md
2. Four Fs (evolution) \- Wikipedia, accessed June 25, 2026, https://en.wikipedia.org/wiki/Four\_Fs\_(evolution)
3. Four Fs (evolution) \- Grokipedia, accessed June 25, 2026, https://grokipedia.com/page/Four\_Fs\_(evolution)
4. From text to traits: exploring the role of large language models in plant breeding \- PMC, accessed June 25, 2026, https://pmc.ncbi.nlm.nih.gov/articles/PMC12116590/
5. From text to traits: exploring the role of large language models in plant breeding \- Frontiers, accessed June 25, 2026, https://www.frontiersin.org/journals/plant-science/articles/10.3389/fpls.2025.1583344/full
6. Breeding 4.0: The Breeding Revolution of Genetic Information Integration and Editing | Fang, accessed June 25, 2026, https://genbreedpublisher.com/index.php/mpb/article/html/3992/
7. Evolutionary Optimization of Model Merging Recipes \- arXiv, accessed June 25, 2026, https://arxiv.org/html/2403.13187v2
8. An Introduction to Model Merging for LLMs | NVIDIA Technical Blog, accessed June 25, 2026, https://developer.nvidia.com/blog/an-introduction-to-model-merging-for-llms/
9. Model Merging in the Era of Large Language Models: Methods, Applications, and Future Directions \- arXiv, accessed June 25, 2026, https://arxiv.org/html/2603.09938v1
10. ArXiv Dives: Evolutionary Optimization of Model Merging Recipes \- Oxen.ai, accessed June 25, 2026, https://ghost.oxen.ai/arxiv-dives-evolutionary-model-merging/
11. Merge Large Language Models with mergekit \- Hugging Face, accessed June 25, 2026, https://huggingface.co/blog/mlabonne/merge-models
12. Building Sentence Transformers in Rust: A Practical Guide with Burn, ONNX Runtime, and Candle \- DEV Community, accessed June 25, 2026, https://dev.to/mayu2008/building-sentence-transformers-in-rust-a-practical-guide-with-burn-onnx-runtime-and-candle-281k
13. ML Library Comparison: Burn vs Candle : r/rust \- Reddit, accessed June 25, 2026, https://www.reddit.com/r/rust/comments/1op3ad1/ml\_library\_comparison\_burn\_vs\_candle/
14. burn-candle \- crates.io: Rust Package Registry, accessed June 25, 2026, https://crates.io/crates/burn-candle
15. huggingface/candle: Minimalist ML framework for Rust \- GitHub, accessed June 25, 2026, https://github.com/huggingface/candle
16. candle\_core::safetensors \- Rust \- Docs.rs, accessed June 25, 2026, https://docs.rs/candle-core/latest/candle\_core/safetensors/index.html
17. Let's Learn Candle 🕯️ ML framework for Rust. | by Cursor \- Medium, accessed June 25, 2026, https://medium.com/@cursor0p/lets-learn-candle-%EF%B8%8F-ml-framework-for-rust-9c3011ca3cd9
18. SafeTensors \- Lib.rs, accessed June 25, 2026, https://lib.rs/crates/safetensors
19. ruvllm \- crates.io: Rust Package Registry, accessed June 25, 2026, https://crates.io/crates/ruvllm
20. EricLBuehler/candle-lora: Low rank adaptation (LoRA) for Candle. \- GitHub, accessed June 25, 2026, https://github.com/EricLBuehler/candle-lora
21. Small Language Models (SLMs) Can Still Pack a Punch: A Survey \- arXiv, accessed June 25, 2026, https://arxiv.org/html/2501.05465v2
22. Introducing Qwen1.5 | Qwen, accessed June 25, 2026, https://qwenlm.github.io/blog/qwen1.5/
23. \[R\] Branching factor on early attention layers as an error-prediction signal — replicated on Qwen 0.5B, OPT 125M, TinyLlama 1.1B, Phi-1.5 \- Reddit, accessed June 25, 2026, https://www.reddit.com/r/deeplearning/comments/1tvfon8/r\_branching\_factor\_on\_early\_attention\_layers\_as/