Architecture for Minimal-Footprint Execution of .slm Models in WebAssembly

Full report

On this page

The Edge Inference Imperative and TinyRustLM Baseline

The transition of localized artificial intelligence from monolithic server deployments to constrained browser environments demands a fundamental rethinking of model container architectures, memory management, and execution paradigms. The current TinyRustLM implementation relies on a rigid SLM1 format, imposing a 128 MiB single-allocation transfer ceiling, main-thread blocking, and full-file memory materialization1. This approach fundamentally limits the system to small parameter counts, such as the 16-million parameter deterministic smoke test, and prevents the scaling of edge inference to highly capable 135M, 360M, and 500M models without triggering Out-Of-Memory (OOM) browser kills2. Furthermore, the existing scalar execution runs entirely on the main JavaScript thread, which forces synchronous blocking of the browser UI during inference and relies on a non-cryptographic checksum for artifact validation1. To execute useful models directly within the client, the architecture must evolve. This report details a comprehensive architectural overhaul via a highly optimized, streaming-capable, and backward-compatible SLM2 Application Binary Interface (ABI). By leveraging WebAssembly (WASM) Single Instruction, Multiple Data (SIMD)6, the Origin Private File System (OPFS) with synchronous access handles8, grouped mixed-precision K-quantization10, and layer-at-a-time weight paging13, the proposed architecture delivers maximum token throughput with a strictly bounded resident memory footprint.

Versioned .slm ABI Proposal

To support advanced quantization, piece-addressed loading, and metadata extensibility while maintaining backward compatibility with the existing 108-byte SLM1 header1, the .slm container must evolve into SLM2. The SLM1 format utilizes a fixed-header structure dictating model dimensions, RoPE theta, RMS epsilon, and a custom 64-bit checksum5. The SLM2 format expands this fixed contract to 128 bytes, absorbing previously zero-padded alignment spaces to encode advanced cryptographic and architectural features, while strictly maintaining the 64-byte alignment requirement for the subsequent tensor directory.

By inspecting the 4-byte Magic field and Version flag, the unified runtime establishes a seamless compatibility matrix. If SLM1 is detected, the parser routes to the legacy structural validator and enforces the 128 MiB full-file transfer ceiling1. If SLM2 is detected, the runtime unlocks hierarchical block quantization, piecewise streaming logic, and Merkle tree verification.

Exact Data Structures and Alignment Rules

Directly following the main header contract are the directory entries. The original SLM1 format defined 64-byte tensor entries comprising an FNV-1a name hash, dtype, rank, dimensions, data offset/length, and block size, supporting only basic q8\_0 and q4\_0 data types1. The SLM2 specification retains the 64-byte size but repurposes legacy padding to support grouped quantization, sparse outlier matrices, and layer-indexing parameters critical for weight paging.

Grouped Quantization Layouts and Mixed Precision

To shrink the memory footprint without degrading the emergent reasoning capabilities of models in the 135M to 500M parameter range, SLM2 adopts a hierarchical block quantization layout mirroring the GGUF K-quant family (Q4\_K, Q5\_K, Q6\_K)10. This mixed-precision architecture applies different bit-depths depending on the tensor's sensitivity, shifting away from the flat block structures that bake in precision loss. Traditional legacy quantization (like q4\_0) groups 32 weights per block with a single 16-bit float scale, which fails to accommodate the highly skewed weight distributions in modern transformer layers10. The SLM2 hierarchical block layout introduces super-blocks, offering finer granularity and minimizing scale overhead:

• Q4\_K (4.5 bits-per-weight): Designed for extreme memory constraint, this layout creates super-blocks of 256 weights, further subdivided into 8 sub-blocks of 32 weights. The 4-bit weights are scaled by a 6-bit sub-block scale and shifted by a 6-bit minimum, allowing asymmetric distributions without expanding metadata significantly11.
• Q5\_K (5.5 bits-per-weight): Mirroring the Q4\_K layout, the Q5\_K format utilizes super-blocks of 256 weights and 8 sub-blocks of 32 weights, but utilizes 5-bit weight representations for layers highly sensitive to precision degradation, such as the initial and final transformer blocks12.
• Q6\_K (6.56 bits-per-weight): Prioritizing fidelity, this layout utilizes super-blocks of 256 weights, subdivided into 16 blocks of 16 weights. Scales are encoded as 8-bit integers without individual block minimums, operating as a near-lossless stand-in for 16-bit floating point weights during critical projection phases11.

These per-channel, per-group, and block scale encodings utilize double-quantization. Rather than storing a full FP16 scale for every small block, the super-block maintains a base FP16 scale, and the sub-blocks store highly compressed 6-bit or 8-bit relative scales10. The tensor directory dictates exactly which scale encoding applies to which matrix.

Outlier Tensors and Sparse Residuals

Language model weights frequently feature extreme outliers in specific projection channels that drive internal activations10. Clipping these outliers during 4-bit uniform quantization destroys model coherence, while accommodating them artificially inflates the scale of the entire block, crushing the resolution of standard weights10. To counter this, the SLM2 format allows tensors to define a "Sparse Residual" payload. The primary block stores the densely quantized weights, while a secondary sparse tensor maps exact matrix indices to FP16 outlier values. During the SIMD dot product, the quantized kernel executes rapidly over the dense integer matrix, followed immediately by a sparse scalar addition for the outliers. This preserves high-precision mathematical accuracy without requiring higher overall bit precision across the entire matrix, effectively granting Q4\_K the emergent reasoning capacity of Q6\_K at a fraction of the footprint.

Tied Embeddings

To optimize spatial complexity, SLM2 explicitly supports tied embeddings via the header Flags register (Bit 0). By tying the input token embeddings (e.g., 32000 × 1024\) to the final output logits projection layer, the architecture eliminates the need to store duplicate parameter matrices2. For a 360M parameter model, this architectural decision saves tens of megabytes in both persistent storage and active memory footprints without a measurable loss of generative quality4.

Compression Dynamics: Tokenizer vs. Random-Access Weights

In highly quantized small models, the vocabulary embeddings and tokenizer dictionaries represent a disproportionate amount of the overall artifact size. The SLM2 format implements dictionary-based entropy compression (e.g., zstd or brotli) exclusively for the string-to-token mappings and metadata blocks. The tokenizer block is streamed and decompressed exactly once into a statically bounded 2 MiB memory arena during model initialization4. While applying entropy compression to the entire .slm artifact would reduce the initial network download size, it is explicitly rejected for the tensor payloads. Random-access weight loading—which is strictly essential for layer-at-a-time execution (weight paging)—requires mathematically predictable byte offsets13. Entropy compression introduces variable-length encoding, which completely defeats [Figure omitted from source export] offset addressing. If the entire file were compressed, the browser would be forced to decompress the entire artifact into a secondary memory buffer before execution could begin. For a 500M parameter model in Q8, this would require a 500 MiB decompression buffer, catastrophically violating the minimal-memory mandate and triggering browser tab eviction1.

Loading, Network, and Verification State Machines

Operating effectively within the browser sandbox requires mitigating network instability, preventing main-thread UI blocks, and navigating the profound limitations of JavaScript memory bindings. The architecture establishes a robust state machine for model acquisition, cryptographic verification, and persistence.

IndexedDB vs. OPFS Persistence

Legacy browser persistence relies heavily on IndexedDB. However, IndexedDB operates via a structured clone algorithm that is fundamentally unsuited for large binary model weights17. Retrieving a model from IndexedDB forces the browser to pull massive ArrayBuffer objects into the main Javascript V8 heap, causing severe latency spikes, garbage collection pauses, and frequent OOM crashes18. The proposed architecture wholly abandons IndexedDB for tensor storage in favor of the Origin Private File System (OPFS)8. OPFS provides a highly optimized, browser-internal private storage area that bypasses traditional quota prompts and offers near-native file I/O speeds18. Specifically, network acquisition and disk writing are moved to a Dedicated Web Worker, unlocking the exclusive FileSystemSyncAccessHandle8. This API allows the WASM runtime to execute synchronous, low-level POSIX-like file I/O operations (e.g., read(), write(), getSize()) directly against the local disk9. This mechanism ensures that weight data moves directly from the disk cache into WASM linear memory, completely bypassing the V8 Javascript garbage collector.

Streaming Admission and Direct Memory Writes

Browsers impose strict memory limits on individual network allocations1. To perform streaming admission without holding duplicate full artifacts in memory, the system dynamically orchestrates data acquisition through HTTP Range requests, P2P WebRTC data channels, and local file loading via showOpenFilePicker()20. A critical engineering hurdle in the browser involves writing network streams directly into WebAssembly. While the ReadableStreamBYOBReader (Bring Your Own Buffer) is theoretically designed to write directly into an existing buffer to prevent zero-copy overhead, the web specification mandates that any ArrayBuffer passed to a BYOB reader is detached and transferred to the stream23. Because WebAssembly linear memory (WebAssembly.Memory.buffer) cannot be detached without permanently destroying the WASM instance23, a naive BYOB implementation targeting WASM directly will fail with a memory rejection exception23. To solve this paradox, the architecture utilizes a bounded 4 MiB staging buffer system.

1. The network thread pulls a piece-addressed chunk into the static 4 MiB staging buffer.
2. The Dedicated Web Worker utilizes the OPFS FileSystemSyncAccessHandle.write() method to flush the chunk directly to the local disk cache8.
3. The WASM runtime subsequently uses the POSIX-style synchronous file read to load only the required layers into its linear memory. This guarantees that peak memory during admission never exceeds the small bounded staging arena, regardless of the overall model size4.

Merkle Verification and Crash-Safe Recovery

To prevent adversarial or corrupted model weights from generating toxic or nonsensical outputs, SLM2 enforces strict cryptographic verification. The header contains a 32-byte Merkle Root. As each piece-addressed chunk is downloaded, its SHA-256 hash is computed dynamically within WASM1. These chunk hashes form the leaves of a Merkle tree. Before a model state is marked as Ready, the tree is reduced, and the resulting root must exactly match the header payload. This allows for localized piece-addressed verification, ensuring that a P2P or multi-source download can instantly discard and re-request poisoned or corrupted pieces without discarding the entire artifact. Artifact downloads can be interrupted by network failure or mobile operating system backgrounding. The OPFS state machine records a "Chunk Manifest" independent of the artifact. If a download fails, the application reads the OPFS manifest, determines the last verified 4 MiB block, and initiates an HTTP Range request to resume exactly where it left off, providing crash-safe interrupted-download recovery without duplicate bandwidth consumption.

The Execution Engine: Paging, Threads, and Kernels

Large Language Models typically load all weights into VRAM or RAM simultaneously for maximum memory bandwidth28. In the browser, pushing a 500M parameter model into WASM linear memory risks immediate tab suspension by the operating system due to baseline memory pressure14. The architecture solves this using strict temporal boundaries and hardware-accelerated kernels.

Layer-at-a-Time Execution (Weight Paging)

Transformer inference is inherently sequential; processing layer [Figure omitted from source export] does not require the weights of layer [Figure omitted from source export] or [Figure omitted from source export]14. The architecture implements weight paging. Using the OPFS FileSystemSyncAccessHandle, the runtime maps only the active layer's parameters into the WASM linear memory13. To hide the latency of disk I/O, the architecture utilizes Web Workers and WASM threads backed by SharedArrayBuffer and atomics5. While the primary WASM thread executes the matrix multiplications for Layer [Figure omitted from source export], a background Web Worker prefetches Layer [Figure omitted from source export] from the OPFS cache into a secondary, fixed-size WASM buffer. Once Layer [Figure omitted from source export] completes, the pointers are swapped. This architectural shift reduces the peak memory requirement of the model weights from [Figure omitted from source export] to [Figure omitted from source export], permitting massive models to run on deeply constrained edge hardware4.

SIMD Kernels vs. WebGPU-Assisted Matmul

WebAssembly SIMD (128-bit) provides the primary vehicle for high-throughput quantized matrix multiplication on edge devices7. The architecture leverages the Rust core::arch::wasm32 module to generate optimized vectorized dot products6. The fixed-width 128-bit operations (v128) process multiple quantized weights concurrently, achieving up to 15x speedups over pure scalar JavaScript loops6. Where supported by the browser environment, the runtime aggressively offloads matrix multiplications to the GPU using WebGPU31. A major challenge with WebGPU is the CPU-to-GPU data transfer bottleneck, which frequently negates the compute advantages for smaller workloads28. The architecture implements a zero-copy buffer mapping approach. By utilizing queue.writeBuffer()33, the runtime avoids creating duplicate TypedArray objects in the JavaScript heap, writing the piece-addressed chunked weights directly to the GPU's memory space for decoding in WGSL (WebGPU Shading Language) compute shaders35.

Rust-Oriented Pseudocode for Quantized Kernels and Chunk Loading

The synergy between Rust's safe memory management, WASM SIMD intrinsics, and OPFS synchronous handles is best illustrated through the core runtime loops. The pseudocode below demonstrates the state machine for loading chunks via OPFS and dispatching them to a heavily optimized Q8\_0 SIMD kernel.

Rust // Core state machine and memory management structures pub struct Slm2Runtime { active\_layer\_idx: usize, opfs\_handle: OpfsSyncHandle, weight\_arena\_a: Vec\<u8\>, // Paged prefetch buffer 1 weight\_arena\_b: Vec\<u8\>, // Paged prefetch buffer 2 kv\_cache: Int8KVCache, }

impl Slm2Runtime { /// Synchronously pages the next layer from OPFS into WASM linear memory pub fn page\_next\_layer(&mut self, target\_buffer: &mut \[u8\], offset: u64) \-\> Result\<(), Error\> { // Read directly from the Origin Private File System into the WASM arena // bypassing the V8 JavaScript heap entirely. let bytes\_read \= self.opfs\_handle.read\_sync(target\_buffer, offset)?; if bytes\_read \!= target\_buffer.len() { return Err(Error::OpfsReadIncomplete); } Ok(()) } }

// SIMD Q8\_0 Dot Product Kernel for WebAssembly // Utilizes core::arch::wasm32 for 128-bit vectorization \#\[cfg(target\_arch \= "wasm32")\] \#\[target\_feature(enable \= "simd128")\] pub unsafe fn dot\_product\_q8\_simd( weights: \const u8, activations: \const u8, len: usize, block\_scale: f32 ) \-\> f32 { use core::arch::wasm32::\*;

let mut sum\_vec \= f32x4\_splat(0.0); let mut i \= 0;

// Process 16 bytes (128 bits) per iteration while i \< len { // Load quantized weights and 8-bit quantized activations directly from memory let w\_vec \= v128\_load(weights.add(i) as \const v128); let a\_vec \= v128\_load(activations.add(i) as \const v128);

// Extensional multiply of low and high bytes (i8 \* i8 \= i16) let mul\_low \= i16x8\_extmul\_low\_i8x16(w\_vec, a\_vec); let mul\_high \= i16x8\_extmul\_high\_i8x16(w\_vec, a\_vec);

// Pairwise addition into 32-bit integers let sum\_low \= i32x4\_extadd\_pairwise\_i16x8(mul\_low); let sum\_high \= i32x4\_extadd\_pairwise\_i16x8(mul\_high);

// Convert integer sums to floating point vectors let f\_low \= f32x4\_convert\_i32x4(sum\_low); let f\_high \= f32x4\_convert\_i32x4(sum\_high);

// Accumulate into the running total sum\_vec \= f32x4\_add(sum\_vec, f\_low); sum\_vec \= f32x4\_add(sum\_vec, f\_high);

i \+= 16; }

// Horizontal sum of the f32x4 vector (simplified for brevity) let final\_int\_sum \= horizontal\_sum(sum\_vec);

// Apply the block-level FP16 scale to the integer dot product final\_int\_sum \* block\_scale }

Context Management: Sliding Windows and KV Caches

Autoregressive decoding generates a Key-Value (KV) cache that grows linearly with the context length, threatening to overrun strict memory constraints38. Standard FP16 KV caches for a 500M parameter model at a 2048 context reach approximately 78 MiB, heavily impacting the available heap4. The SLM2 runtime enforces KV cache quantization, dropping the precision of cached keys and values to Int8 or Q8\_04. This mathematically predictable transformation halves the KV memory requirement to 39 MiB for the 500M model4. Furthermore, to prevent OOM errors during extended conversational agent loops, the runtime enforces a bounded context with a sliding window mechanism. When the prompt approaches the 2048-token limit, the earliest tokens (excluding the critical system prompt and first user interaction) are systematically evicted. The relative RoPE (Rotary Position Embedding) indices are subsequently shifted downward5, ensuring that the forward scratch arena and the KV cache never exceed their statically allocated bounds.

Desktop and Mobile Resource Budgets

To validate the viability of the architecture across deeply constrained edge devices, deterministic memory budgets for three representative model sizes (135M, 360M, and 500M) were calculated under both full-resident and weight-paged operational modes4. The calculations include the overhead of the WASM heap execution environment (roughly 16 MiB for static data and call stacks), the statically decompressed Tokenizer (2 MiB), the heavily optimized int8 KV cache at a context of 2048, the dynamic forward scratch arena (calculated as [Figure omitted from source export] bytes, where [Figure omitted from source export] is hidden dimension, [Figure omitted from source export] is feed-forward dimension, and [Figure omitted from source export] is context size), and a 16 MiB allocation for the bounded staging and streaming network verification buffers1.

TinyRustLM-135M Resource Budget

Architecture: 16 Layers, Hidden Dim: 768, FFN: 2304, Heads: 12, KV Heads: 4, Context: 2048, Tied Embeddings.

TinyRustLM-360M Resource Budget

Architecture: 24 Layers, Hidden Dim: 1024, FFN: 3584, Heads: 16, KV Heads: 4, Context: 2048, Tied Embeddings.

TinyRustLM-500M Resource Budget

Architecture: 26 Layers, Hidden Dim: 1152, FFN: 4096, Heads: 18, KV Heads: 6, Context: 2048, Tied Embeddings.

By employing the layer-at-a-time weight-paged execution mode over OPFS, the peak memory footprint for the 500M parameter model in q6 drops from an unmanageable 445.90 MiB down to a highly constrained 129.63 MiB4. This memory profile is well within the safety margins of constrained mobile device browsers, entirely dodging aggressive iOS Safari memory eviction thresholds, and maintaining operational capacity just over the previous absolute 128 MiB runtime ceiling1.

Compatibility and Migration Rules

The migration from the monolithic, main-thread SLM1 paradigm to the asynchronous, OPFS-backed SLM2 architecture demands strict compatibility enforcement to ensure seamless user experiences across fragmented edge environments. The Web Worker parses the initial 128 bytes upon network acquisition. If SLM1 is identified by the magic bytes, the system issues a deprecation warning to the console, disables the layer-at-a-time OPFS streaming logic, and falls back to full-memory materialization for backward compatibility5. Furthermore, if a legacy browser does not support the FileSystemSyncAccessHandle within Dedicated Web Workers9, the engine aborts the streaming load and gracefully degrades. It intercepts the HTTP stream, buffers it via standard asynchronous OPFS or IndexedDB, and informs the user of expected latency spikes and memory warnings. Upon instantiation, the runtime actively probes CPUID and browser features for v128 SIMD and WebGPU. It statically maps the fastest backend available to the execution graph, ensuring zero runtime branching overhead29.

Quality Assurance: TDD, Fuzzing, and Test Matrices

Local-first, client-side inference dictates that errors degrade gracefully rather than crash the browser. A rigorous testing matrix must be established across multiple axes to guarantee stability.

Prioritized Implementation Sequence

To seamlessly deliver these architectural improvements without breaking existing local pipelines, the development sequence is highly prioritized based on risk and footprint impact.

1. Phase 1: OPFS and Web Worker Orchestration. Refactor the synchronous main-thread runtime into a Dedicated Web Worker. Implement the HTTP Range streaming admission and OPFS FileSystemSyncAccessHandle persistence loop8. This immediately cures UI freezing and establishes a framework for bypassing the 128 MiB transfer ceiling1.
2. Phase 2: SLM2 Container and Merkle Verification. Expand the .slm parser to read the 128-byte v2 header. Implement the SHA-256 chunk hashing and Merkle root verification during the initial streaming download5.
3. Phase 3: Hierarchical K-Quants and SIMD. Introduce the Q4\_K, Q5\_K, and Q6\_K tensor directory structures. Port the inner loop dot products to utilize Rust core::arch::wasm32 SIMD intrinsics, drastically increasing scalar throughput6.
4. Phase 4: Weight Paging and Int8 KV Cache. Decouple the tensor memory allocator. Implement the layer-at-a-time active memory swap using OPFS synchronous reads and apply Int8 quantization to the KV Cache state plane4.
5. Phase 5: WebGPU Offloading. Introduce experimental WebGPU buffer mapping (queue.writeBuffer) and WGSL shaders for supported desktop environments, offering maximum token throughput33.

The TinyRustLM runtime currently suffers from self-imposed architectural limitations that restrict it to basic smoke tests. By adopting the SLM2 ABI, moving execution into Dedicated Web Workers via OPFS, utilizing hardware-accelerated SIMD intrinsics, and dynamically paging weights a single layer at a time, the resident memory footprint of a 500M parameter model is compressed by over 70%—from an untenable 445 MiB down to a highly efficient 129 MiB. This architecture proves that excessive downloads and peak browser memory are not absolute hardware limitations, but rather software boundaries that can be systematically engineered away, allowing local micro language models to run securely and swiftly inside any modern browser, entirely independent of cloud inference constraints.

Works cited

1. Implementation operations \- MiRust, https://mirust.com/implementation-operations/
2. Models \- MiRust, https://mirust.com/models/
3. Building ALLM: a Rust local LLM runtime that streams GGUF weights under a hard RAM budget \- Reddit, https://www.reddit.com/r/rust/comments/1ullby2/building\_allm\_a\_rust\_local\_llm\_runtime\_that/
4. unknown\_url
5. Implementation \- MiRust, https://mirust.com/implementation/
6. core::arch::wasm32 \- Rust, https://doc.rust-lang.org/core/arch/wasm32/index.html
7. Rust \+ WebAssembly 2025: Why WasmGC and SIMD Change Everything \- DEV Community, https://dev.to/dataformathub/rust-webassembly-2025-why-wasmgc-and-simd-change-everything-3ldh
8. How to Handle Files Synchronously in the Browser | R3gardless.dev, https://r3gardless.dev/en/blog/2026-03-20-sync-file-handling-in-browser/
9. FileSystemSyncAccessHandle \- Web APIs \- MDN Web Docs, https://developer.mozilla.org/en-US/docs/Web/API/FileSystemSyncAccessHandle
10. GGUF Optimization: A Technical Deep Dive (Part 1 of 2\) \- Medium, https://medium.com/@michael.hannecke/gguf-optimization-a-technical-deep-dive-for-practitioners-ce84c8987944
11. \[copilots\] Support non-uniform block quantization in ONNX (GGUF K-quant, super-block structures) · Issue \#7691 \- GitHub, https://github.com/onnx/onnx/issues/7691
12. Which Quantization Should I Use? A Unified Evaluation of llama.cpp Quantization on Llama-3.1-8B-Instruct \- arXiv, https://arxiv.org/html/2601.14277v1
13. MCAP: Deployment-Time Layer Profiling for Memory-Constrained LLM Inference \- arXiv, https://arxiv.org/html/2604.21026v1
14. Partial LLM Loading: Running Models Too Big for VRAM | TinyComputers.io, https://tinycomputers.io/posts/partial-llm-loading-running-models-too-big-for-vram.html
15. GGUF · Hugging Face, https://huggingface.co/docs/hub/gguf
16. GGUF's "Q4\_K\_M": What are you actually choosing?｜Komugi @ PowerPlatform \- note, https://note.com/quick\_pipit7468/n/n876606cf31e7?hl=en
17. The browser is your database: Local-first comes of age | InfoWorld, https://www.infoworld.com/article/4133648/the-browser-is-your-database-local-first-comes-of-age.html
18. Why Local-First Software Is the Future and its Limitations | RxDB \- JavaScript Database, https://rxdb.info/articles/local-first-future.html
19. Rewriting Reflect in SQLite, https://reflect.app/blog/sqlite-rewrite-techical-explanation
20. Origin private file system \- Web APIs | MDN, https://developer.mozilla.org/en-US/docs/Web/API/File\_System\_API/Origin\_private\_file\_system
21. Breaking change: sync methods for AccessHandles | Blog \- Chrome for Developers, https://developer.chrome.com/blog/sync-methods-for-accesshandles
22. Sync methods for SyncAccessHandle in File System Access API \- Chrome Platform Status, https://chromestatus.com/feature/5149644305203200
23. We deserve a better streams API for JavaScript \- The Cloudflare Blog, https://blog.cloudflare.com/a-better-web-streams-api/
24. ReadableStreamBYOBReader: read() method \- Web APIs | MDN, https://developer.mozilla.org/en-US/docs/Web/API/ReadableStreamBYOBReader/read
25. Why can I not reuse the buffer I brought to the ReadableStreamBYOBReader?, https://stackoverflow.com/questions/79449404/why-can-i-not-reuse-the-buffer-i-brought-to-the-readablestreambyobreader
26. Zero-copy pass ArrayBuffer from JS-land to WebAssembly-land · Issue \#1162 \- GitHub, https://github.com/WebAssembly/design/issues/1162
27. GGUF \- Wikipedia, https://en.wikipedia.org/wiki/GGUF
28. The Engine Behind Modern LLM Inference, Part 1: Continuous Batching, PagedAttention, and the End of Naive Serving | by Parth Joshi | Medium, https://medium.com/@parth010872/the-engine-behind-modern-llm-inference-part-1-continuous-batching-pagedattention-and-the-end-of-859215173d34
29. WASM \+ SIMD for On-Device AI: Private, Fast, Offline | by Thinking Loop | Medium, https://medium.com/@ThinkingLoop/wasm-simd-for-on-device-ai-private-fast-offline-3ef82c47172d
30. I built a WASM SIMD inference engine in Rust \- semantic embedding model in 7MB \- no torch, no ML framework (\~2ms embedding inference) \- Reddit, https://www.reddit.com/r/rust/comments/1urd2hp/i\_built\_a\_wasm\_simd\_inference\_engine\_in\_rust/
31. LiteRT LM on the Web: Running Generative AI Locally in the Browser | by Shivay Lamba, https://shivaylamba.medium.com/litert-lm-on-the-web-running-generative-ai-locally-in-the-browser-f6d5393c3756?source=rss-------1
32. Serverless AI in a Browser Tab: Java WebAssembly \+ Local WebGPU LLMs, https://dev.to/vishalmysore/serverless-ai-in-a-browser-tab-java-webassembly-local-webgpu-llms-f4g
33. GPUQueue: writeBuffer() method \- Web APIs | MDN, https://developer.mozilla.org/en-US/docs/Web/API/GPUQueue/writeBuffer
34. Unweight: how we compressed an LLM 22% without sacrificing quality \- The Cloudflare Blog, https://blog.cloudflare.com/unweight-tensor-compression/
35. 7 WebGPU \+ WASM Plays for Private On-Device ML | by Bhagya Rana | Medium, https://medium.com/@bhagyarana80/7-webgpu-wasm-plays-for-private-on-device-ml-a114bbba8c58
36. Buffer in wgpu \- Rust \- Docs.rs, https://docs.rs/wgpu/latest/wgpu/struct.Buffer.html
37. WebGPU Speed and Optimization, https://webgpufundamentals.org/webgpu/lessons/webgpu-optimization.html
38. Streaming LLM Responses: Make Your AI App Feel Fast \- Redis, https://redis.io/blog/streaming-llm-responses/
39. Guides \- MiRust, https://mirust.com/guides/
40. aprender-compute \- crates.io: Rust Package Registry, https://crates.io/crates/aprender-compute/
41. What's New in WebGPU (Chrome 145\) | Blog, https://developer.chrome.com/blog/new-in-webgpu-145