Deep Technical Design: Exact, Atomic, Reversible Specialist Overlays on Quantized Transformer Bases

Full report

On this page

1. Executive Architecture Recommendation

The design of a local inference runtime for highly constrained environments—specifically Rust-based native CPU execution and browser-based WebAssembly (WASM)—demands a strict departure from traditional server-centric model mutation strategies. Server environments often utilize Segmented Gather Matrix-Vector Multiplication (SGMV) or unified paging mechanisms to serve thousands of concurrent adapters over a shared base model for high-throughput batching1. However, local environments optimize for a batch size of one (user-facing autoregressive decoding), strict memory boundaries, and minimal copy amplification. To satisfy the fixed constraints of peak memory minimization, zero cumulative drift, and atomic state transitions for the pre-release TinyRustLM runtime, the system must abandon destructive in-place parameter merging and full persistent floating-point materialization. The recommended architecture is a Dual-Path Ephemeral-Accumulation Engine combined with Content-Addressed Immutable Base Registries. Rather than merging an adapter into the base weights—which invariably induces quantization error drift or memory exhaustion—the runtime evaluates the quantized base and the specialist overlay as parallel mathematical streams during the forward pass. For environments where the arithmetic intensity of dual-path execution creates unacceptable latency, the system utilizes a Per-Layer Just-In-Time (JIT) Copy-on-Write (CoW) Materialization strategy. In this mode, only the single transformer layer currently undergoing execution is materialized into an f32 scratchpad, merged with the adapter, passed through the linear layer, and immediately discarded. This strictly bounds the memory copy amplification to the size of a single transformer block. This architecture treats the base .slm model as a cryptographically sealed, immutable memory-mapped (or linear-memory-loaded) oracle. Adapters are loaded as non-destructive sidecars in separate memory arenas. State transitions are governed by a robust finite state machine utilizing double-buffered atomic pointer swapping. This ensures that any fault during validation, allocation, or checksum verification results in a deterministic rollback to the exact prior composition, leaving the previous generation state entirely undisturbed.

2. Explicit Correctness and Atomicity Invariants

To guarantee absolute numerical stability, operational safety, and exact reversibility, the runtime architecture strictly enforces a core set of immutable invariants. Cumulative mutation is explicitly rejected as a mechanism for reversible switching. The foundational invariants are defined as follows:

1. The Oracle Invariant: The bytes representing the base model on disk, in host memory, and within the WASM linear memory arena must remain bit-for-bit identical from initialization to termination. Destructive mutations or in-place weight adjustments are strictly prohibited. The base model serves as an absolute mathematical oracle.

2. The Reversibility Invariant: A composition sequence defined as [Figure omitted from source export] must yield a numerical state and generation output for [Figure omitted from source export] that is bitwise equivalent to a sequence of [Figure omitted from source export]. No residual state, error term, or floating-point drift from [Figure omitted from source export] may persist in memory, caches, or logical tensor representations. Deactivation must restore physical bytes, logical tensor values, and final logits with absolute zero tolerance.

3. The Transactional Atomicity Invariant: Model composition is a singular, atomic transaction. The runtime must allocate all required memory, decode the adapter payloads, and verify all cryptographic checksums in an isolated staging arena. The active inference graph pointer is updated only upon successful completion of all staging checks. If any step fails, the staging arena is freed, and the existing inference pointer remains actively valid.

4. The Single-Truth Memory Bound: The peak memory commitment during a successful context switch must never exceed [Figure omitted from source export]. A full f32 or f16 unquantized model equivalent must never exist simultaneously in memory, preventing catastrophic Out-Of-Memory (OOM) exceptions in WASM limits.

5. The Isolation Invariant: Model or adapter switching inherently invalidates the key-value (KV) cache, prefix cache, speculative decoding drafts, and sampler state. State reuse is only permitted if an independently formalized and computationally proved compatibility transform exists for the specific adapter delta.

3. Composition Identity and Manifest Schema

Composition identity cannot be tracked merely by file names, user tags, or superficial metadata. It must be defined as a deterministic, cryptographically verifiable hash over the entire logical dependency tree. This ensures that a cache key or state snapshot uniquely maps to the exact mathematical function instantiated by the runtime. The identity [Figure omitted from source export] of a composed model is defined by the following cryptographic concatenation: [Figure omitted from source export] Every element in this sequence fundamentally alters the resulting probability distribution of the language model:

• Exact Base Bytes & Tokenizer ([Figure omitted from source export]): Provides the foundational numerical oracle. A modification to the tokenizer merges or splits subwords, fundamentally altering the embedding lookups and output logits.
• Tensor Layout & Quantization ([Figure omitted from source export]): Dictates block sizes, zero-points, and affine transformations. A q4\_K layout evaluates arithmetic differently than a q8\_0 layout, directly dictating the exact arithmetic of the dequantization kernels4.
• Ordered Adapters ([Figure omitted from source export]): The specific topological and temporal ordering of applied overlays.
• Runtime Implementation & Policy ([Figure omitted from source export]): Defines the mathematical reduction strategy (e.g., dual-path vs JIT layer materialization) and precision semantics (e.g., bf16 vs fp32 accumulation).

The Criticality of Adapter Ordering and Commutativity Failures

The architectural assumption that adapters—such as Low-Rank Adaptation (LoRA)—strictly commute is mathematically false and heavily rejected in this design. While basic linear low-rank additions applied to a purely linear system theoretically commute ([Figure omitted from source export]), this property breaks down rapidly under modern specialist frameworks and transformer topologies. First, activation functions in the forward pass of a transformer (such as SiLU or GELU) are non-linear. Modifying intermediate hidden states via successive adapters alters the input vector to all subsequent adapters in higher layers1. Second, advanced adaptation methodologies like Weight-Decomposed Low-Rank Adaptation (DoRA) decouple pre-trained weights into magnitude and direction components8. Applying a directional update and magnitude rescaling via DoRA Specialist [Figure omitted from source export] followed by DoRA Specialist [Figure omitted from source export] is fundamentally non-commutative, as the magnitude normalization applied by [Figure omitted from source export] depends on the directional output of [Figure omitted from source export]. Finally, task arithmetic and vector merging techniques rely heavily on orthogonalization and disentangled weight representations, where applying a suppression vector before an enhancement vector yields entirely different gradient trajectories and final logits than the reverse order7. Consequently, an arbitrary stack of adapters demands a strict, explicit application order defined in the composition manifest.

4. Runtime Strategy Comparison Matrix

The optimization space for applying specialist overlays over a quantized base is heavily constrained by memory bandwidth, compute latency, OS-level capabilities, and implementation complexity. The following analysis evaluates viable runtime strategies for the TinyRustLM ecosystem.

For TinyRustLM, the implementation must utilize either the Lazy Low-Rank Overlay (Dual-Path) or the Per-Layer JIT Materialization strategy. In WebAssembly environments where the wasm32 target enforces a strict 4GB linear memory maximum, a 7B to 8B parameter model encoded in a q4\_K or q5\_K format occupies approximately 3.8GB to 4.2GB4. Materializing a full unquantized copy requires roughly 14GB to 16GB, ensuring immediate runtime failure. Dual-path inference allows the base to remain in its heavily compressed 4-bit representation, allocating only minor megabyte segments for the active specialist adapter matrices ([Figure omitted from source export] and [Figure omitted from source export]) and keeping transient scratch memory strictly bounded.

5. Quantized-Base Numerical Analysis

Understanding the pathology of adapter merging requires dissecting the granular mathematical structure of block-quantized tensors. Modern quantized LLM formats, such as q8\_0 and the K-quant series (q4\_K, q5\_K), rely on block-wise affine transformations rather than global tensor quantization to preserve critical outlier information4. A standard q8\_0 block contains 32 weights and one f16 scale factor [Figure omitted from source export], storing weights at an effective 8.5 bits per parameter4. More complex structures, such as a q4\_K super-block, contain 256 weights divided into 8 sub-blocks of 32\. Each sub-block possesses a 6-bit scale and a 6-bit minimum (zero-point), which are subsequently governed by two overarching f16 scales for the entire super-block, yielding an effective 4.5 bits per parameter4.

The Pathology of Dequantize-Add-Requantize (DAR) Error

A naive merging operation attempts to produce a new quantized weight tensor [Figure omitted from source export] by extracting the base, adding the adapter, and quantizing the result: [Figure omitted from source export]. This operation introduces devastating numerical non-associativity and error accumulation:

1. Scale Recomputation Shift: The scale factor [Figure omitted from source export] for any given block is calculated based on the maximum absolute magnitude within that specific block. If the specialist adapter [Figure omitted from source export] introduces a localized, high-magnitude outlier to a single weight, the new block maximum increases drastically. Consequently, the newly computed scale factor [Figure omitted from source export] becomes much larger. When the remaining 31 weights in the block are divided by [Figure omitted from source export] during requantization, their quantized integer representations collapse toward zero, permanently destroying the granular precision of the surrounding weights.

2. Repeated Merge Degradation: Switching states iteratively ([Figure omitted from source export]) via DAR applies the highly lossy quantization function [Figure omitted from source export] repeatedly. Because [Figure omitted from source export], structural artifacts accumulate with every application. This leads to severe perplexity degradation and catastrophic forgetting of the pre-trained capabilities.

3. The Fresh-Base Oracle Definition: To achieve true zero drift, the mathematical oracle must be defined dynamically at inference time as [Figure omitted from source export]. The values of the physical representation [Figure omitted from source export] must never be recalculated, overwritten, or updated. If compute latency demands standard matrix multiplication over dual-path execution, the operation [Figure omitted from source export] must be materialized in an ephemeral f32 execution cache layer-by-layer, consumed solely for the instantaneous forward pass of that specific layer, and immediately destroyed before the next layer begins.

6. Proposed Activation/Deactivation State Machine

Robust lifecycle management for specialist sidecars dictates a rigid progression of states to ensure atomicity and exact reversibility. The state machine operates on atomic reference counters and isolated memory arenas.

Activation Flow

1. IDLE / ACTIVE\_BASE: The runtime is operating on the verified oracle base model. The inference pipeline pointer routes all requests to the base execution graph.

2. VALIDATION: An activation request for Specialist A is received. The system reads the adapter manifest, cryptographically hashes the incoming artifact, and checks the compatibility manifest (dimensions, tensor names, rank, target layers, quantization profile) against the immutable base.

3. ALLOCATION: The runtime computes the exact peak memory required for Specialist A. It checks the WASM linear memory arena (or native OS heap via std::alloc). If requested\_bytes \> available\_bytes, the operation is aborted instantly, and the state reverts to ACTIVE\_BASE without interrupting active requests.

4. STAGING: The adapter weights are loaded and decoded into the newly allocated contiguous block. For a WASM environment, this utilizes WebAssembly.Memory regions specifically partitioned away from the KV cache and base model space18.

5. CHECKSUM VERIFICATION: A final CRC32/SHA256 checksum is performed on the fully loaded adapter block to detect in-flight memory corruption, bit flips, or incomplete network/disk reads.

6. COMMIT / SWAP: The active model dependency graph is updated via a double-buffered atomic pointer swap (std::sync::atomic::AtomicPtr in Rust). The generation thread is synchronized via a memory barrier (or WASM Atomics.wait18), the composition pointer is updated, and the affected caches are invalidated.

7. ROLLBACK: If any of steps 2-5 fail, the allocated staging block is marked free in the arena allocator. The generation thread remains completely uninterrupted, satisfying the Transactional Atomicity Invariant.

Exact Deactivation Flow

Deactivation is the explicit process of reverting from an active specialist overlay back to the clean base model or a different specialist. Because the architecture mandates dual-path execution or ephemeral JIT materialization, the physical bytes of the base model were never mutated.

• Restoring Bytes: Deactivation requires zero disk I/O or memory rewriting. The runtime merely updates the atomic composition pointer to drop the reference to the adapter arena. The physical base bytes remain mathematically pristine.
• Restoring Logical Tensor Values: The logical view of the tensor instantly reverts to [Figure omitted from source export]. The addition of [Figure omitted from source export] is structurally bypassed.
• Restoring Logits within Tolerance: Because the base representation was immutable, removing the adapter guarantees that the output logits return to the exact bit-for-bit values of the original base model, achieving a tolerance of absolute zero ([Figure omitted from source export]). The claim perfectly matches the representation.

7. Memory, Byte, and Latency Formulas

Precise accounting of system resources is required for local constrained inference. A full f32 copy is strictly forbidden from memory accounting tricks; the actual peak switch memory must reflect true hardware allocations. The peak memory footprint [Figure omitted from source export] during an atomic swap from Adapter A to Adapter B over a Quantized Base is defined as: [Figure omitted from source export]

Persistent Base Bytes ([Figure omitted from source export])

For a model with [Figure omitted from source export] layers, each containing parameter matrices [Figure omitted from source export]: [Figure omitted from source export] where [Figure omitted from source export] is the parameter count of tensor [Figure omitted from source export], and [Figure omitted from source export] is the effective bits-per-weight of the quantization profile (e.g., 4.5 for q4\_K, 8.5 for q8\_04).

Low-Rank Adapter Bytes ([Figure omitted from source export])

For LoRA, applied to a subset of projections (e.g., [Figure omitted from source export]) with rank [Figure omitted from source export]: [Figure omitted from source export] where [Figure omitted from source export] bytes for f16 or bf16.

Sparse Overlay Bytes ([Figure omitted from source export])

For task vectors masked by DARE (Drop And REscale)14 or similar magnitude-pruning delta tuning, the parameters are highly sparse. The format utilizes Compressed Sparse Row (CSR) or direct index-value pairs: [Figure omitted from source export] where [Figure omitted from source export] is the number of non-zero delta parameters in layer [Figure omitted from source export], [Figure omitted from source export] is 4 bytes (uint32), and [Figure omitted from source export] is 2 bytes (f16).

Scratch Overhead ([Figure omitted from source export])

For Dual-Path execution, [Figure omitted from source export] is limited to the maximum activation tensor size: [Figure omitted from source export]. For Per-Layer JIT Materialization, [Figure omitted from source export] requires enough contiguous memory to hold the largest single f32 weight matrix, plus the f32 adapter expansion, and the result array before it is fed to the activation function: [Figure omitted from source export] Requantization scratch is non-existent ([Figure omitted from source export] bytes) because the architecture forbids in-place requantization. Allocator overhead ([Figure omitted from source export]) for standard dlmalloc or WASM bump allocators typically adds [Figure omitted from source export] bytes of header metadata per distinct tensor allocation.

8. Native and Browser-WASM Kernel Implications

The execution environments dictate vastly different computational boundaries. The implementation of exact reversible overlays must be heavily tailored to the instruction set architectures (ISAs) available.

Native Execution (AVX2/NEON/AVX-512)

On native CPU targets, the system relies on high-throughput SIMD intrinsics. Dual-path inference calculates the base dot product using highly optimized integer multiply-add instructions (e.g., VPMADDWD on AVX2 or SDOT on ARM NEON). Concurrently, the adapter product [Figure omitted from source export] is calculated using f32/f16 FMA (Fused Multiply-Add) pipelines.

• Prefill vs Decode: During prompt prefill, the batch dimension equals the sequence length. Dense matrix multiplication (GEMM) optimizations heavily favor Per-Layer JIT Materialization here, as the overhead of materializing a layer is amortized over thousands of tokens. During decode (batch size one), matrix-vector multiplication (GEMV) is memory-bandwidth bound; Dual-Path execution is vastly superior as it avoids writing a massive materialized matrix back to RAM.
• Cache Locality: Because native memory allocators handle virtual memory paging efficiently, caching the materialized layer in L2/L3 cache is highly performant. Kernel launch overhead is minimized by static dispatch in Rust.

Browser WASM Execution (wasm32, v128)

WebAssembly enforces a sandboxed linear memory array, initialized to zero, lacking a hardware translation lookaside buffer (TLB) accessible to the runtime, and lacking multi-process shared memory without specifically enabled SharedArrayBuffer headers13. WASM utilizes a single 128-bit SIMD proposal (v128)18.

• Kernel Constraints: The v128 specification allows for [Figure omitted from source export]\-bit float operations or [Figure omitted from source export]\-bit integer operations per cycle. Dequantization of q4\_K requires excessive bit-shifting and masking to unpack the 4-bit nibbles.
• Avoidance of Premerged Weights: Because WASM cannot utilize OS-level page faulting to swap memory to disk transparently without massive performance penalties or manual software MMUs13, materializing a full f32 premerged model will immediately crash a standard 4GB WASM linear memory limit. Therefore, dual-path direct low-rank matmul addition is strictly required for WebAssembly deployments.
• Batch Size One & Locality: In local browser inference, batch size is almost always [Figure omitted from source export] (single user autoregressive decoding). The arithmetic intensity (FLOPs per byte) is extremely low. Loading adapter matrices [Figure omitted from source export] and [Figure omitted from source export] from memory adds severe pressure to the WASM memory bus. Kernels must be fused: as the quantized base block is loaded into v128 registers for the base matrix-vector multiplication, the corresponding rows of [Figure omitted from source export] should be fetched and computed immediately to maximize data locality in the CPU's L1 cache, avoiding multiple passes over the activation vector.

9. Composition of Multiple Specialists

When multiple specialists are provided in the manifest (e.g., a champion routing module and a domain-specific coding specialist), the runtime applies them sequentially based on explicit composition policy rules.

• Conflicts and Ordering: As established in Section 3, adapters do not commute. If Specialist A is an attention adapter and Specialist B is an MLP adapter, there is minimal structural conflict. However, if both target [Figure omitted from source export], the runtime must define ownership. The policy explicitly defines whether combinations are additive ([Figure omitted from source export]) or replacing (Specialist B overwrites Specialist A's claim on [Figure omitted from source export]).
• Weighted Mixtures: Linear scalar weighting (e.g., [Figure omitted from source export]) is permitted at the final accumulation step but must be executed dynamically in the dual-path engine.
• Incompatible Training Bases: Adapters trained on structurally different bases (e.g., Llama-2 vs Llama-3) possess diverging latent spaces. The validation checksums will aggressively reject mismatched dependencies.
• Quality Receipts: Arbitrary stacking of task vectors often leads to catastrophic interference and degradation7. To support arbitrary stacks, the system requires a cryptographic "Quality Receipt"—a pre-calculated hash provided by the model author indicating that the specific mixture [Figure omitted from source export] has been explicitly trained, orthogonalized, or empirically tested for stability. If the receipt is absent, the runtime applies the mixture but raises a warning flag that the output is structurally unverified and prone to catastrophic interference.

10. KV, Prefix-Cache, and Speculative Invalidation Contract

The state of the Key-Value (KV) cache represents a sequence of hidden state projections that are inextricably tied to the exact numerical composition of the active model.

1. Strict Cache Invalidation: Any swap from Base [Figure omitted from source export] Specialist A mandates an immediate and total flush of the KV cache, the prefix cache tree, and any drafted tokens residing in speculative decoding pipelines. Reusing stale KV states generated by a different latent manifold results in hallucination and rapid perplexity divergence.

2. Cache Key Cryptography: To prevent accidental corruption and manage multi-turn conversations across context switches, the cache eviction policy relies on a deterministic key. The cache key for any cached token state is rigorously defined as [Figure omitted from source export]. If [Figure omitted from source export] changes, previously stored blocks automatically become orphaned and are rapidly reclaimed by the KV arena allocator.

3. Sampler Invalidation: Token sampling states, such as dynamic temperature or Min-P scaling histories, must be reset. Changes to the model composition directly alter the logit distribution variance, invalidating prior sampler heuristics.

4. Mathematical Compatibility Exception: KV reuse is permitted only if an independently proven compatibility transform exists. For example, if Specialist A modifies only the final MLP layers and leaves the Attention projections ([Figure omitted from source export]) completely untouched, the KV cache (which stores [Figure omitted from source export] and [Figure omitted from source export]) remains mathematically valid for the new composition. The runtime manifest must explicitly flag attention\_frozen: true to bypass the rigorous KV invalidation lock.

11. Clean Replacement Plan and Rejection Criteria

To prevent the technical debt associated with backward-compatibility shims, TinyRustLM must adopt a clean v0 specification for the .slm (Small Language Model) artifact and API boundary. This replaces all legacy mutation logic.

Rejection Criteria

• No Cumulative Mutation: Any API proposing model.apply\_adapter(A) without returning a strictly isolated new state object is rejected.
• No Hidden F32 Copies: Any design requiring a background thread to maintain an unquantized f32 replica of the model to bypass dual-path computation is rejected due to violating the Single-Truth Memory Bound.
• No Commutative Assumptions: Adapters cannot be arbitrarily mixed without a defined policy order (e.g., [Figure omitted from source export] applies before [Figure omitted from source export]).

v0 .slm Format API Boundary

The artifact format utilizes a flat-buffer or zero-copy C-struct layout compatible directly with WASM linear memory mapping (via WebAssembly.Memory buffer instantiation)19, eliminating deserialization allocation overhead.

Rust // Logical architecture of the API boundary (Agnostic to private TinyRustLM internals) \#\[repr(C)\] pub struct SlmManifestV0 { pub magic: \[u8; 4\], // "SLM0" pub base\_checksum: \[u8; 32\], // Validated against H\_base pub quant\_profile: QuantizationProfile, // e.g., Q4\_K, Q8\_0 pub endianness: u8, // 0 for Little-Endian (WASM native) pub layer\_count: u32, pub tied\_embeddings: u8, // Boolean flag for tied unembedding projection pub vocab\_size: u32, }

\#\[repr(C)\] pub struct AdapterManifestV0 { pub adapter\_checksum: \[u8; 32\], pub required\_base\_checksum: \[u8; 32\], // Fails validation if mismatched pub adapter\_type: AdapterType, // Dense, LowRank, Sparse (DARE) pub rank: u32, pub alpha\_scaling: f32, // Preserves LoRA alpha/rank scaling factor pub target\_tensors\_bitmask: u64, // Flags mapping to Wq, Wk, Wv, Wo, Gate, Up, Down }

// Runtime API Entry Points pub trait SlmRuntime { /// Applies an ordered sequence of adapters, returning a novel, isolated composition graph. /// Does not mutate the immutable base. fn compose( base: Arc\<ImmutableBase\>, adapters: &\[Arc\<SpecialistAdapter\>\] ) \-\> Result\<ComposedGraph, CompositionError\>;

/// Frees the composed graph, dropping references to adapters. Base remains loaded. fn deactivate(graph: ComposedGraph); }

Deep Compatibility Checks

During the compose execution, the runtime executes deep verification. It asserts that tensor dimensions match perfectly, that sparse index orders are strictly monotonic (to allow efficient CSR binary searching), that no NaN or Inf finite values exist in the f16 payloads, and that the quantization profile of the base explicitly matches the kernel implementations available in the runtime for the adapter matrix multiplication.

12. TDD and Fault-Injection Matrix

To ensure absolute adherence to the Correctness and Atomicity Invariants, the runtime requires an aggressive Test-Driven Development (TDD) harness executing differential testing across native and WASM builds.

13. Unknowns Requiring Local Source and Measurements

Several execution dynamics depend tightly on hardware and browser-specific JIT compiler implementations, which cannot be statically determined without local profiling and trace analytics on the actual TinyRustLM source tree:

1. WASM Engine JIT Tiering: The V8 (Chrome) and SpiderMonkey (Firefox) engines optimize WebAssembly v128 instructions differently. The exact computational penalty of LEB128 integer decoding overhead18 during adapter payload initialization requires cycle-accurate tracing to determine if custom binary formats outpace standard serialization.

2. L1/L2 Cache Eviction Rates: Dual-path inference loads the base quantized block, dequantizes it in registers, and simultaneously loads the adapter matrices [Figure omitted from source export] and [Figure omitted from source export]. If the footprint of [Figure omitted from source export] and [Figure omitted from source export] forces the base block out of the L1 cache prematurely, memory latency will spike. Cache line measurements on target CPUs (e.g., Apple Silicon M-series vs Intel Core) are required to tune the fused kernel loop unrolling and blocking factors.

3. Browser Garbage Collection (GC) Pauses: While the WASM linear memory is manually managed by the Rust allocator (e.g., wee\_alloc or dlmalloc), bridging input tokens and output string data to the host JavaScript runtime via typed arrays can trigger unpredictable GC pauses21. The exact boundary serialization cost for tokens must be empirically measured to prevent audio/visual stutter in front-end UI components.

4. Hardware FMA Precision Divergence: Subtle differences in Floating-Point Multiply-Accumulate (FMA) rounding logic between ARM NEON, AVX2, and WASM JIT execution may cause slight numerical divergence. Over thousands of autoregressive steps, this can lead to divergent token paths, requiring empirical tolerance bound mapping across the target device matrix.

14. Annotated Primary-Source Bibliography

Liu, S.-Y. et al. (2024). DoRA: Weight-Decomposed Low-Rank Adaptation. arXiv:2402.09353. Establishes the non-commutative nature of advanced adapters by decomposing pre-trained weights into magnitude and directional components, proving why strict adapter application order is mandated in the manifest.

Liu, S.-Y. et al. (2024). Weight-Decomposed Low-Rank Adaptation (PDF Source). github.com/mlresearch/v235. Reinforces the mathematical decoupling of directional learning, requiring complex adapter handling over simple matrix additions.

Liu, S.-Y. et al. (2024). DoRA: Weight-Decomposed Low-Rank Adaptation (Semantic Scholar Profile). Corroborates the broad industry adoption and necessity of handling non-linear adapter updates.

Sheng, Y. et al. (2023). S-LoRA: Serving Thousands of Concurrent LoRA Adapters. arXiv:2311.03285. Provides the foundational analysis for unified paging and heterogeneous batching, demonstrating how a single immutable base can support massive concurrent adapter querying efficiently.

Chen, L. et al. (2023). Punica: Multi-Tenant LoRA Serving. AAAI. Details the algorithmic complexity of batching multi-tenant adapters, highlighting the necessity of avoiding in-place merges for dynamic routing.

Chen, L. et al. (2023). Punica: Multi-Tenant LoRA Serving (Semantic Scholar Profile). Validates the performance bottlenecks associated with naive adapter switching.

Chen, L. et al. (2023). Punica Github Repository. github.com/punica-ai/punica. Details the Segmented Gather Matrix-Vector Multiplication (SGMV) CUDA kernel design, proving the mathematical viability of dual-path inference over shared backbones without materializing full model copies.

Yu, L. et al. (2023). Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch. arXiv:2311.03099. Introduces the Drop And REscale (DARE) sparsification method for task vectors, informing the runtime requirement for sparse overlay byte accounting, memory footprint calculations, and composition logic.

Yu, L. et al. (2023). Language Models are Super Mario (Semantic Scholar Profile). Corroborates the utility of masking and extreme sparsity in delta parameters.

Yu, L. et al. (2023). MergeLM Github Repository. github.com/yule-BUAA/MergeLM. Provides empirical ranges for SFT delta parameter limits, driving the sparse adapter memory formulas.

Ilharco, G. et al. (2022). Editing Models with Task Arithmetic. arXiv:2212.04089. Demonstrates how task vectors ([Figure omitted from source export]) can be manipulated arithmetically, establishing the critical need for "Quality Receipts" when composing multiple unverified vectors due to catastrophic task interference.

Ilharco, G. et al. (2022). Editing Models with Task Arithmetic (Semantic Scholar Profile). Links weight disentanglement challenges directly to the need for strict adapter isolation.

Ortiz-Jimenez, G. et al. (2023). Task Arithmetic in the Tangent Space. NeurIPS. Proves mathematically that task vectors in non-linear models do not behave strictly linearly, validating the rejection of commutative adapter assumptions.

Xiangpeng Systems (2024). How to LLM Inference. Details the exact bit-level structural memory layout of GGML quantization formats, specifically defining the block configurations of q8\_0 (32 weights/block) and q4\_K (256 weights/super-block), driving the numerical analysis of requantization drift and the pathology of the DAR error.

Anonymous (2026). Quantization Configurations in llama.cpp. arXiv:2601.14277. Analyzes 13 discrete quantization configurations and their affine transformations, reinforcing the necessity of dual-path execution over destructive v128 SIMD-level requantization logic.

llama.cpp Community (2023). GGML Quantization Discussions. github.com/ggml-org/llama.cpp/discussions. Outlines non-linear quantization polynomial mappings, dictating the complexity required for accurate dual-path evaluation.

JarvisLabs (n.d.). vLLM Quantization Complete Guide. Highlights AWQ/GPTQ mechanics, proving why sub-block scale recomputation is fatal to accuracy if an adapter is aggressively merged into the quantized domain.

OCamlStreet (2025). Hardcaml WebAssembly Wiki. Explores the internal structure of WASM execution, specifically the lack of built-in OS-level page faulting, the v128 SIMD module restrictions, LEB128 encoding overhead, and linear memory properties, dictating the WASM compatibility bounds for peak memory tracking.

Anonymous (2024). WASM Performance Overviews. arXiv:2410.08517. Provides baseline expectations for near-native performance degradation limits within browsers.

NDSS Symposium (2025). WAVEN: WebAssembly Memory Virtualization for Enclaves. Discusses the strict limitations of the linear memory model in WASM regarding page-level cross-module sharing, mathematically confirming that CoW OS-level paging is currently impossible in native browser implementations without injecting massive custom software MMUs.

MDPI (2023). WebAssembly Data Flow Architecture. Evaluates the stack-based VM nature of WASM and its single resizable array limitation, enforcing the contiguous memory allocation requirements.

Fenstermacher, D. (n.d.). WebAssembly, Emscripten, Quantization. Details the intricate interaction between C++/Rust manual memory allocators within WASM and the host JavaScript garbage collection boundaries, highlighting the zero-overhead manual struct layout required for performant LLM inference.

• 8
• 9
• 10
• 1
• 2
• 22
• 3
• 14
• 15
• 16
• 11
• 12
• 7
• 4
• 5
• 17
• 6
• 18
• 20
• 13
• 19
• 21

Works cited

1. Serving Thousands of Concurrent LoRA Adapters \- NSF PAR, https://par.nsf.gov/servlets/purl/10552444

2. Dynamic Operator Optimization for Efficient Multi-Tenant LoRA Model Serving, https://ojs.aaai.org/index.php/AAAI/article/view/34453/36608

3. Punica: Serving multiple LoRA finetuned LLM as one \- GitHub, https://github.com/punica-ai/punica

4. A system programmer's guide to LLM inference \- Xiangpeng's blog, https://blog.xiangpeng.systems/posts/how-to-llm-inference/

5. Which Quantization Should I Use? A Unified Evaluation of llama.cpp Quantization on Llama-3.1-8B-Instruct \- arXiv, https://arxiv.org/html/2601.14277

6. The Complete Guide to LLM Quantization with vLLM: Benchmarks & Best Practices, https://jarvislabs.ai/blog/vllm-quantization-complete-guide-benchmarks

7. Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models, https://papers.neurips.cc/paper\_files/paper/2023/file/d28077e5ff52034cd35b4aa15320caea-Paper-Conference.pdf

8. DoRA: Weight-Decomposed Low-Rank Adaptation \- arXiv, https://arxiv.org/html/2402.09353v6

9. DoRA: Weight-Decomposed Low-Rank Adaptation \- GitHub, https://raw.githubusercontent.com/mlresearch/v235/main/assets/liu24bn/liu24bn.pdf

10. \[PDF\] DoRA: Weight-Decomposed Low-Rank Adaptation | Semantic Scholar, https://www.semanticscholar.org/paper/DoRA%3A-Weight-Decomposed-Low-Rank-Adaptation-Liu-Wang/da053e2a4ba1b244940c8f2cad5dcdf0d730f85f

11. Editing models with task arithmetic \- OpenReview, https://openreview.net/forum?id=6t0Kwf8-jrj¬eId=isbjbtZou3k

12. Editing Models with Task Arithmetic \- Semantic Scholar, https://www.semanticscholar.org/paper/Editing-Models-with-Task-Arithmetic-Ilharco-Ribeiro/71ba5f845bd22d42003675b7cea970ca9e590bcc

13. WAVEN: WebAssembly Memory Virtualization for Enclaves \- NDSS Symposium, https://www.ndss-symposium.org/wp-content/uploads/2025-746-paper.pdf

14. Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch \- arXiv, https://arxiv.org/html/2311.03099v3

15. Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch \- Semantic Scholar, https://www.semanticscholar.org/paper/Language-Models-are-Super-Mario%3A-Absorbing-from-as-Yu-Bowen/c0230760f644f6b7538d93e4296a5e9aa7028e45

16. Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch \- GitHub, https://github.com/yule-BUAA/MergeLM/blob/main/README.md

17. Even more quantization types? \#5063 \- ggml-org llama.cpp \- GitHub, https://github.com/ggml-org/llama.cpp/discussions/5063

18. WASM | HardCaml Wiki \- GitBook, https://ocamlstreet.gitbook.io/hardcaml-wiki/wasm

19. An Overview of WebAssembly for IoT: Background, Tools, State-of-the-Art, Challenges, and Future Directions \- MDPI, https://www.mdpi.com/1999-5903/15/8/275

20. WasmWalker: Path-based Code Representations for Improved WebAssembly Program Analysis \- arXiv, https://arxiv.org/html/2410.08517v1

21. Boosting Web Performance: Implementing K-Means Clustering with WebAssembly and Emscripten | Doug Fenstermacher, https://dougfenstermacher.com/experiment/webassembly-emscripten-quantization

22. \[PDF\] Punica: Multi-Tenant LoRA Serving \- Semantic Scholar, https://www.semanticscholar.org/paper/Punica%3A-Multi-Tenant-LoRA-Serving-Chen-Ye/c2314751d367b34239a537fe27e2bd51a8b84528