Browser-Native LLM Deployment: A Comprehensive Byte Budget Study of Tiny Llama Architectures (10M-42M)

Full report

On this page

The paradigm of large language model (LLM) deployment is undergoing a radical, structural shift. Historically, the immense computational and memory requirements of transformer-based architectures forced deployment strictly onto centralized, cloud-centric infrastructure equipped with high-bandwidth, discrete server GPUs. However, the maturation of machine learning compilation, coupled with the rapid evolution of client-side web standards—specifically WebAssembly (Wasm) and WebGPU—has catalyzed a migration toward local, on-device execution environments.1 Within this transition, the modern web browser represents the most universally accessible cross-platform deployment vector, offering zero-installation runtime environments that naturally abstract away underlying hardware heterogeneity across different vendors and operating systems.3 Despite the theoretical promise of browser-based AI, the practical execution of these workloads is hindered by draconian memory ceilings, execution limits, and security constraints imposed by browser engines.1 Browsers are fundamentally designed to sandbox applications, preventing any single webpage from monopolizing system resources or inducing operating system volatility. Consequently, running LLM inference inside a web application is not merely a matter of computational throughput, but a rigorous exercise in extreme memory choreography. This exhaustive research report investigates the precise "byte budget" required to securely and reliably execute sub-50-million parameter LLMs—specifically the 10M, 15M, and 42M "tiny" Llama-style architectures—within modern browser engines. To properly deploy these models without triggering silent out-of-memory (OOM) evictions or browser tab crashes, engineers must map the precise memory topology of the application lifecycle. This analysis dissects every byte allocated during execution, spanning the static weight artifacts in varying quantization precisions (F32, Q8, Q4), the tokenizer dictionary footprint, the dynamic scaling of the key-value (KV) cache during autoregressive decoding, and the statically allocated execution scratch buffers.5 By mathematically deriving the memory topology of these components and contrasting them against the strict memory ceilings of mobile and desktop browsers (such as iOS Safari and Chrome Android), this report establishes the definitive practical deployment targets for browser-native inference.

Topological Architecture and Parameter Distribution of Tiny Llama Models

To accurately model the memory footprint of a neural network, it is first necessary to deconstruct its architectural hyperparameters. The "Tiny Llama" models pioneered for edge experimentation (most notably popularized by Andrej Karpathy's llama2.c repository and the TinyStories dataset evaluation) are not novel architectures; rather, they retain the exact structural topology of Meta's foundational Llama 2 architecture.8 They merely scale down the internal dimensionality, layer count, and attention heads to fit within constrained parameter counts.8 The Llama architecture utilizes a standard autoregressive transformer decoder. It abandons absolute positional encodings in favor of Rotary Positional Embeddings (RoPE), replaces standard layer normalization with Root Mean Square Normalization (RMSNorm) for enhanced training stability, and utilizes SwiGLU feed-forward networks (FFNs) instead of standard ReLU-based multi-layer perceptrons.9 The defining hyperparameters for the 15M, 42M, and experimental extreme-micro models dictate the mathematical footprint of both the static weights and the runtime activations:

• 15M Architecture: This topology features a hidden dimension ([Figure omitted from source export]) of 288, 6 hidden transformer layers ([Figure omitted from source export]), 6 attention heads ([Figure omitted from source export]), 6 key-value heads ([Figure omitted from source export]), and an intermediate SwiGLU FFN dimension ([Figure omitted from source export]) of 768\.8 The typical context window ([Figure omitted from source export]) during training is restricted to 256 tokens.8
• 42M Architecture: This model scales upward, expanding to a hidden dimension ([Figure omitted from source export]) of 512, 8 hidden transformer layers ([Figure omitted from source export]), 8 attention heads ([Figure omitted from source export]), 8 key-value heads ([Figure omitted from source export]), and a typical context window ([Figure omitted from source export]) of 1024 tokens.8
• 10M / 260K Architectures: Extreme micro-models have also been evaluated to test the absolute lower bounds of transformer reasoning. For example, a 260K parameter variant utilizes an exceptionally narrow [Figure omitted from source export] of 64, 5 hidden layers, 8 attention heads, and crucially, only 4 key-value heads.8 This disparity between attention heads and KV heads indicates the use of Multi-Query Attention (MQA) or Grouped-Query Attention (GQA), a paradigm where multiple query heads share a single key/value projection, drastically reducing both parameter count and runtime KV cache memory.7

Mathematical Derivation of Transformer Layer Parameters

The exact parameter count of the transformer layers can be derived mathematically using the standard attention and feed-forward layer dimensions. For a single layer, the attention mechanism dictates four projection matrices ([Figure omitted from source export]). In a standard multi-head attention (MHA) setup where the number of attention heads equals the number of key-value heads ([Figure omitted from source export]), these matrices each possess a total shape of [Figure omitted from source export]. The SwiGLU feed-forward network utilizes three separate weight matrices to perform its non-linear activation: a gate projection ([Figure omitted from source export]), a down projection ([Figure omitted from source export]), and an up projection ([Figure omitted from source export]).9 The gate and up matrices project the hidden state outward, possessing shapes of [Figure omitted from source export], while the down matrix projects the intermediate state back to the hidden dimension, possessing a shape of [Figure omitted from source export].9 The total parameter count per individual transformer layer ([Figure omitted from source export]) is formalized as: [Figure omitted from source export] Applying this formula to the 15M architecture ([Figure omitted from source export]), a single layer requires [Figure omitted from source export] parameters. Across the 6 designated layers, the core transformer blocks account for exactly 5,971,968 parameters.11 For the 42M architecture ([Figure omitted from source export], and assuming an estimated [Figure omitted from source export] based on standard Llama 2 feed-forward scaling ratios), a single layer requires roughly 3,162,112 parameters. Across 8 layers, the reasoning blocks consume approximately 25.29 million parameters.8

The Vocabulary Paradox in Micro-Architectures

When analyzing the parameter distribution of sub-50M models, a critical secondary insight emerges regarding the disproportionate dominance of the embedding layer and the language modeling (LM) output head. Standard foundational models like Meta's 7B and 13B models utilize a vocabulary size ([Figure omitted from source export]) of 32,000 tokens.8 Because these tiny models copy the Llama architecture directly to maintain compatibility with existing inference frameworks and tokenizers, they frequently inherit this massive 32,000-token dictionary.8 The parameter count for the input embedding matrix ([Figure omitted from source export]) is a simple product of the vocabulary size and the hidden dimension: [Figure omitted from source export] For the 15M architecture, maintaining a standard 32,000 token vocabulary dictates an embedding matrix of [Figure omitted from source export] parameters.11 If the LM output head is not tied to the input embeddings (a configuration denoted by tie\_word\_embeddings \= false), the architecture requires a secondary output matrix of identical size, contributing a staggering 18,432,000 parameters solely to vocabulary mapping.11 This presents a foundational paradox in micro-architecture design. If the actual reasoning layers (the 6 transformer blocks) consume only 5.97M parameters, yet untied embeddings consume 18.4M parameters, the so-called "15M" model is effectively functioning as a massive vocabulary lookup table attached to a miniature reasoning engine. To adhere strictly to a \~15M total parameter count, practitioners generally tie the embeddings (tie\_word\_embeddings \= true), sharing the weights between the input embedding and the final pre-softmax linear projection, merging them into a single 9.2M parameter block.17 However, even when tied, the vocabulary matrix accounts for over 60% of the entire model's weight footprint. For ultra-lightweight deployment scenarios where a broad linguistic scope is unnecessary, training models with custom micro-tokenizers featuring aggressively reduced vocabularies (e.g., 4,096 or 512 tokens) is highly recommended.8 Applying a 512-token vocabulary to the 15M architecture shrinks the embedding layer from 9.2M parameters to a mere 147,456 parameters, radically freeing the byte budget to be reallocated toward deeper, more complex reasoning layers without increasing the total payload size.8

Weight Quantization Semantics and Memory Mapping

Executing unquantized 32-bit floating-point (F32) models in a browser environment is heavily discouraged, not only due to network transmission overhead but primarily due to severe memory-bandwidth bottlenecks.9 Neural network inference—specifically autoregressive text generation—is notoriously memory-bandwidth bound on modern hardware.9 Every time a new token is generated, the inference engine must stream the entirety of the model's weights from the system's global memory into the computational cores, performing only a sparse handful of floating-point operations per loaded weight.9 This FLOPs-to-byte ratio dictates that execution speed is directly proportional to how rapidly the weights can be read. To mitigate this, the GGUF (GPT-Generated Unified Format) standard provides a rigorous, highly optimized methodology for quantizing these weights into lower bit-depth representations.18 Quantization dramatically reduces artifact sizes and memory bandwidth requirements while incurring negligible degradation to the model's linguistic perplexity.18

Precision Formats and Hierarchical Block Structures

Modern browser inference engines (such as Wllama and LlamaWeb) rely heavily on block-wise quantization. Instead of applying a single scaling factor across an entire tensor, weights are grouped into small clusters, and a localized scaling factor is extracted to map the high-precision floating-point values down to restricted integer ranges.18 This allows the quantization to adapt to local weight distributions, preserving outlier activations. F32 (The 32-bit Baseline): In an unquantized state, every parameter occupies 4 bytes of memory. An F32 representation requires zero dequantization overhead during execution, providing maximum mathematical fidelity, but poses maximum stress on the WebGPU VRAM, system RAM, and network download times.18 Q8\_0 (8-bit Legacy Block Quantization): The Q8\_0 format is generally considered the baseline for optimized, mathematically lossless inference.20 It utilizes a Type-0 (scale-only) symmetric quantization scheme centered around zero.19 Weights are clustered into linear blocks of 32 values. Each block stores a single 16-bit floating-point (FP16) scaling factor (consuming 2 bytes) alongside 32 quantized 8-bit integers (consuming 32 bytes).19 [Figure omitted from source export] [Figure omitted from source export] The Q8\_0 format inflates model perplexity by a statistically insignificant 0.01 points compared to the FP16 reference, whilst compressing the active memory footprint by nearly 73% relative to F32.18 Q4\_0 (4-bit Legacy Block Quantization): The Q4\_0 format similarly clusters 32 weights per block but aggressively packs the quantized values into 4-bit integers.18 A single block contains a 2-byte FP16 scale and 16 bytes of data (which securely holds thirty-two 4-bit values).19 [Figure omitted from source export] [Figure omitted from source export] While Q4\_0 is the fastest 4-bit format to dequantize, it suffers from noticeable quality degradation because a single scaling factor struggles to accurately represent 32 highly variable weights compressed into just 16 discrete bins.18 Q4\_K\_M (4-bit K-Quantization): To resolve the quality loss of naive 4-bit quantization, the ecosystem migrated toward K-Quants.18 The Q4\_K\_M format utilizes a sophisticated hierarchical super-block structure. A super-block of 256 weights contains a master FP16 super-scale, alongside 8 sub-blocks of 32 weights that possess their own heavily quantized sub-scales.19 This double-quantization significantly reduces scaling overhead. Furthermore, Q4\_K\_M intelligently mixes precisions throughout the network; it stores highly sensitive attention weights at slightly higher bit-depths (e.g., 5-bit or 6-bit) while leaving less sensitive feed-forward weights at 4-bit.18 The resulting average footprint is approximately 0.6 bytes per parameter, offering vastly superior linguistic quality to Q4\_0 at a virtually identical byte cost.18

Total Artifact Size Projections

Applying these average byte-per-weight metrics allows for a precise calculation of the model artifact sizes that must be downloaded over HTTP, cached by the browser, and mapped into the local memory space.

Table 1: Theoretical artifact sizes for Tiny Llama LLMs based on strict quantization mathematical averages.8 Note: True file sizes on disk may vary slightly due to GGUF header metadata padding.22 These sizes represent the raw payload delivered over the network. In web environments, user experience dictates that interactive elements load in under 3 seconds. An unquantized 168 MB download for a 42M F32 model is entirely prohibitive on mobile network connections. Quantizing the 42M model down to a 25.2 MB Q4\_K\_M footprint makes rapid web application initialization a reality.18

The Tokenizer Payload and Dictionary Footprint

Orthogonal to the neural network parameters is the auxiliary data required for natural language processing: the tokenizer dictionary. Models based on the Llama 2 paradigm rely on Byte-Pair Encoding (BPE), typically implemented and managed via SentencePiece mechanisms.14 The tokenizer is the software interface responsible for translating raw user string characters into the corresponding integer sequences mapped to the model's input embedding matrix.14 The standard Llama tokenizer, encompassing a vocabulary size of 32,000, is usually serialized into a tokenizer.json or tokenizer.model file structure.14 The byte footprint of a 32,000-token BPE dictionary structure on disk is generally constant regardless of the model's reasoning size, routinely measuring between 1.80 MB and 1.84 MB of raw text.25 However, measuring the static file size is deceiving when calculating a browser byte budget. When the browser downloads this JSON payload, it must parse it into active JavaScript memory structures (such as Hash Maps, nested objects, Tries, or standard arrays) to perform rapid, optimized string-matching and recursive token splitting during the generation loop. A 1.8 MB flat JSON file typically expands by a factor of 3 to 4 when instantiated as a live JavaScript object due to the V8 (Chrome) or JavaScriptCore (Safari) engine's object overhead, hidden classes, and 64-bit pointer alignments. Consequently, the tokenizer mandates a persistently active memory budget of approximately 5 MB to 8 MB within the main JavaScript thread or Web Worker heap.14 While a 5 MB overhead is negligible when deploying multi-gigabyte 7B models, it is highly conspicuous when attempting to deploy a 15M model whose Q4 weights only consume 9.1 MB. If custom micro-tokenizers are utilized (e.g., training a model specifically on a 512-token or 4,096-token vocabulary for a narrow domain like TinyStories), the JSON artifact collapses to less than 100 KB, and the resulting in-memory JavaScript footprint drops below 1 MB, entirely neutralizing the tokenizer as a systemic memory concern.8

The KV Cache: Memory Scaling During Autoregressive Decoding

Large Language Models generate text autoregressively, processing and emitting exactly one token at a time.2 To prevent the computationally disastrous recalculation of all preceding tokens during each forward pass, execution engines maintain a Key-Value (KV) cache.9 The KV cache operates as an optimized memory buffer that stores the derived Key ([Figure omitted from source export]) and Value ([Figure omitted from source export]) attention vectors for every token generated thus far in the sequence.26 When generating the next token, the model only needs to calculate the Query vector for the newest token and compute its attention against the cached keys and values.26 However, this optimization trades compute for memory. As the sequence length grows, the KV cache grows linearly, continuously demanding additional RAM. In web browser limits, predicting the maximum bounds of this cache is paramount to preventing Out-of-Memory (OOM) evictions midway through a generation stream. The exact memory size of the KV cache is dictated by the layer count ([Figure omitted from source export]), the number of KV heads ([Figure omitted from source export]), the dimension of each head ([Figure omitted from source export]), the current sequence length ([Figure omitted from source export]), and the precision format of the cache.7 Because FP32 KV caches waste bandwidth, and INT8/INT4 caches require complex dynamic quantization overhead, most browser frameworks default to utilizing FP16 format for the KV cache, which requires exactly 2 bytes per element. The full footprint equation is expressed as: [Figure omitted from source export] Where [Figure omitted from source export] is the bytes per element (2 for FP16). The leading factor of 2 at the beginning of the equation accounts for the separate storage requirements of both the Keys tensor and the Values tensor.7

KV Cache Profiles for 15M and 42M Architectures

To establish the exact byte budget, we must apply the specific topological parameters of the models to the KV scaling equation. 15M Architecture ([Figure omitted from source export]): For the 15M model, the head dimension is derived by dividing the total hidden size (288) by the number of attention heads (6), yielding a [Figure omitted from source export] of 48\.10 At a typical training context window of 256 tokens 8: [Figure omitted from source export] 42M Architecture ([Figure omitted from source export]): For the 42M model, dividing the 512 hidden size by 8 heads yields a slightly wider [Figure omitted from source export] of 64\.8 At its standard evaluated context window of 1024 tokens 8: [Figure omitted from source export]

Table 2: Scaling characteristics of the FP16 KV Cache at various sequence lengths. The 110M model ([Figure omitted from source export]) is provided for broader context scaling comparison.7 While a 1.77 MB or 16.78 MB cache appears trivial in desktop computing paradigms, it represents a substantial continuous allocation block within heavily constrained browser environments. Furthermore, because the WebGPU specification generally prevents the runtime resizing of active memory buffers to avoid driver sync issues, prominent browser inference frameworks like LlamaWeb and WebLLM must statically pre-allocate the absolute maximum theoretical size of the KV cache at initial startup.6 Therefore, if an application developer configures the engine with a maximum context limit of 2048 for a 42M model, it will force the browser to immediately allocate and lock 33.55 MB of GPU VRAM, regardless of whether the user inputs a short 5-token prompt or a massive 2000-token prompt.6 Accurate prediction of required sequence lengths is critical for minimizing wasted allocations.

Activation Memory and WebGPU Scratch Buffers

Executing a neural network is not merely a static process of holding weights; it actively involves pushing massive data matrices through a long chain of mathematical transformations.9 During a forward pass, the model generates intermediate activation tensors—outputs from RMSNorm layers, pre-softmax attention logits, SwiGLU gating combinations, and query-key dot products.9 In native, desktop-class deployment environments utilizing direct CUDA or Metal APIs, activation memory can be dynamically allocated and freed in microseconds via custom allocators. However, in WebGPU, dynamic allocation during the active rendering or compute loop incurs catastrophic CPU-to-GPU synchronization latency, ruining inference speeds.6 To circumvent this limitation, WebGPU frameworks manage intermediate data through statically allocated memory pools colloquially known as "scratch buffers".1 A scratch buffer is a singular, pre-allocated pool of memory large enough to hold the largest single intermediate tensor generated at any point in the network architecture. As the forward pass moves sequentially from layer 1 to layer [Figure omitted from source export], the scratch buffer is endlessly overwritten by subsequent operations, preserving a perfectly flat memory footprint.6

Deriving the Scratch Buffer Budget

The size of the scratch buffer is entirely contingent on the sequence length during the prefill phase (when the entire input prompt is processed concurrently in a massive parallel matrix multiplication, as opposed to the decode phase which processes one token at a time), and the internal dimension parameters of the model. The peak intermediate memory pressure typically occurs in one of two places: during the Feed-Forward Network expansion or during the Attention mechanism's QK matrix multiplication.6

1. FFN Expansion State: The SwiGLU FFN temporarily projects the hidden state out to the much wider intermediate dimension ([Figure omitted from source export]). For an input sequence of length [Figure omitted from source export], this expanded tensor requires [Figure omitted from source export] bytes (assuming FP32 intermediate activation precision for numerical stability). For the 42M model ([Figure omitted from source export]) processing a maximum 1024-token prompt, this single intermediate tensor demands approximately 5.6 MB of VRAM.
2. Attention Logit Matrices: The multiplication of Queries and Keys generates an attention score matrix of shape [Figure omitted from source export] per attention head. For the 42M model ([Figure omitted from source export]) acting over 1024 tokens, this produces a multi-dimensional [Figure omitted from source export] matrix. At FP32 precision, this single tensor consumes an immense 33.55 MB.6

If naive, standard execution graphs are utilized, the scratch buffer for the 42M model would require approximately 40 MB of statically allocated WebGPU buffer space just to safely house these intermediates during the prefill phase.6

The FlashAttention Optimization

To mitigate this exorbitant scratch memory requirement, optimized inference engines universally deploy FlashAttention kernels.30 FlashAttention fundamentally restructures the attention computation through advanced register tiling algorithms. Instead of computing the massive [Figure omitted from source export] attention matrix and writing it to global GPU memory (VRAM), FlashAttention computes the attention scores in small blocks, applies the softmax reduction continuously, and multiplies it by the Values tensor while the data remains strictly within the GPU's ultra-fast, on-chip SRAM.30 By preventing the attention matrix from ever being fully materialized in global memory, FlashAttention drops the attention memory complexity from [Figure omitted from source export] down to [Figure omitted from source export]. WebLLM, LlamaWeb, and Wllama implementations rely heavily on WebGPU or Wasm adaptations of FlashAttention to eliminate the need for the massive 33.5 MB activation buffer, reducing the required baseline scratch buffer to strictly under 10 MB for all sub-50M models, regardless of prompt length.6

Browser Runtime Guardrails: WebGPU and WebAssembly Limits

The most perilous bottlenecks in browser-native machine learning are not found in theoretical compute metrics, but in the artificial limiters enforced by the browser's architecture. Surpassing the weight budget, the KV cache budget, or the scratch budget will inevitably trigger these OS-level traps, resulting in silent tab crashes, frozen UI threads, or explicit JavaScript exceptions.28

WebGPU Buffer Constraints

The WebGPU specification provides a direct, low-overhead interface to the device's graphics hardware.4 However, it enforces strict maximum buffer limits to ensure security, guarantee hardware portability, and prevent malicious webpages from exhausting shared rendering pipelines.1 The two primary limits dictating LLM inference bounds are maxBufferSize and maxStorageBufferBindingSize.28

• maxBufferSize: This parameter defines the absolute maximum size of a single GPUBuffer allocation request. By W3C standard default, this is capped at 268,435,456 bytes (256 MB).34
• maxStorageBufferBindingSize: This is the far more restrictive parameter. It dictates the maximum size of a memory buffer that can be mapped directly into a shader (the WGSL execution kernel) for active computation. The WebGPU spec dictates a default minimum support limit of 134,217,728 bytes (128 MB).35

While powerful desktop discrete GPUs (like an NVIDIA RTX series) or Apple's M-series chips with unified memory architectures may dynamically negotiate this limit upward to 2 GB or 4 GB 28, mobile devices enforce the limits strictly. Specifically, Chrome on Android (running on Qualcomm Adreno or ARM Mali chips) and standard iOS Safari WebKit strictly enforce the baseline 128 MB maxStorageBufferBindingSize.28 This yields a profound architectural revelation: An unquantized 42M model (168 MB artifact) natively violates the 128 MB maxStorageBufferBindingSize limit of all mobile browsers.28 If an inference framework attempts to map the F32 weights as a contiguous tensor for execution, the WebGPU API will unconditionally throw a validation error and reject the operation.28 To successfully execute the 42M model on a mobile browser, developers are forced to choose between two paths: either fragment the model weights into disjointed, cumbersome sub-128 MB chunks (which shatters the contiguous tensor abstraction and severely degrades execution speed), or quantize the model. Utilizing a Q4\_0 format (25.2 MB) allows the entire model to comfortably bypass the WebGPU binding limits on every single device class globally.28

WebAssembly (Wasm) Memory and iOS Safari Evictions

While WebGPU handles the massively parallel matrix multiplications, the overarching control flow, token sampling logic, and grammar constraint engines (often written in C++ and compiled via Emscripten) operate within WebAssembly.4 Wasm utilizes a contiguous linear memory structure known as the "Wasm Heap".40 Historically, Wasm is restricted by 32-bit addressing mechanisms, setting an absolute theoretical maximum heap limit of 4 GB.42 However, in reality, mobile operating systems enforce vastly more aggressive memory guardrails. iOS Safari, operating in an environment entirely devoid of disk swap memory capabilities, heavily penalizes background execution and runaway RAM usage.33 Safari will aggressively terminate tabs that exceed roughly 384 MB to 500 MB of total allocation (a sum combining the JavaScript engine footprint, the HTML DOM, the Wasm heap, and the active WebGL/WebGPU context VRAM) without ever throwing an identifiable error event.46 If a framework clumsily forces the entire model weight artifact into the Wasm heap simply to pass it down to the WebGPU execution engine, the heap must grow massively to accommodate both the weights and the intermediate logic.48 For reliable deployment on iOS architectures, the entire application ecosystem must aim for a strict ceiling of \~300 MB.47

Execution Engine Paradigms and Memory Strategies

Various open-source frameworks navigate these complex browser bottlenecks using distinct memory architectures. The choice of inference engine drastically alters the baseline byte budget overhead. WebLLM (Apache TVM Compilation): WebLLM utilizes the Apache TVM machine learning compiler to translate specific model graphs ahead-of-time into highly optimized WebGPU WGSL shaders.4 By utilizing sophisticated kernel fusion, WebLLM combines distinct sequential operations (e.g., Matrix Multiply followed by a Bias Add, followed by a SwiGLU Activation) into singular GPU execution dispatches. This prevents intermediate tensors from having to materialize in global memory, severely reducing scratch buffer requirements.6 However, WebLLM conducts its token biasing and sampling mechanisms on the CPU (within the Wasm heap), incurring some CPU-GPU synchronization overhead and additional Wasm memory footprint during autoregressive decoding.2 Transformers.js (ONNX Runtime Web): Operating on the industry-standard ONNX format, Transformers.js is robust and supports vast multimodal ecosystems.42 It interfaces with WebGPU via the underlying ONNX Runtime execution provider.49 Because ONNX runtime must support generalized graph execution rather than specific fused Llama topologies, it often relies on dynamic memory pooling for its scratch buffers. This generalization can induce unexpected peak memory spikes and "invisible" GPU buffer leaks during execution, sometimes ballooning the prefill memory profile dangerously beyond the limits of constrained mobile hardware.6 Furthermore, ONNX Runtime Web struggles with zero-copy weight loading in some browser contexts, occasionally double-allocating weights inside the Wasm heap before they reach the GPU.50 Wllama (Wasm SIMD \+ WebGPU \+ OPFS): Designed as a direct web binding for the acclaimed llama.cpp library, Wllama employs highly specific techniques to bypass Wasm memory limits.23 To completely prevent OOM errors, Wllama utilizes the Origin Private File System (OPFS) and memory mapping (mmap) to load GGUF files directly from the browser's persistent disk cache.22 The massive model weights completely bypass the Wasm linear heap, remaining locked in external file-backed memory until required dynamically by the GPU or SIMD cores.6 This architecture makes Wllama exceptionally memory-efficient, allowing larger models to run seamlessly without ever tripping the notorious Safari 384 MB eviction limit.47 LlamaWeb (Static Memory Planning): A highly optimized, custom WebGPU backend designed to run alongside llama.cpp, LlamaWeb takes an aggressive static approach to memory management.6 It completely eradicates dynamic allocation during the execution loop, pre-allocating a single, massive slotted memory region upon startup that exactly conforms to the model's theoretical limits.6 It also implements templated GPU kernels that read quantized formats directly inside the shader program. This eliminates the need to temporarily dequantize weights into bloated FP16 buffers before executing matrix multiplications. Benchmarks reveal that LlamaWeb's strict static memory planning reduces peak memory usage by 29% to 33% compared to WebLLM and Transformers.js, locking its maximum footprint securely and predictably.6

End-to-End Browser Fit Targets for Practical Deployment

To construct practical deployment profiles for real-world edge computing, all the underlying constants—artifact size, tokenizer payload, sequence scaling, and engine architecture—must be aggregated into a comprehensive "Total Byte Budget." This budget accounts for the exact memory locked by the browser during an active, ongoing LLM generation session. The Total Byte Budget formula is synthesized as: [Figure omitted from source export] For this baseline analysis, we assume the use of a highly optimized engine (like LlamaWeb or Wllama) that prevents weight duplication in the Wasm heap, utilizing a 10 MB statically allocated scratch buffer (enabled by FlashAttention), and enforcing a fixed engine overhead of approximately 15 MB to run the underlying JavaScript runtime and Wasm control logic.4

Profile 1: The 15M Architecture Budget

Context Length Target: 256 tokens.Tokenizer: Standard 32,000 vocab (approx. 5 MB RAM footprint).KV Cache Target: 1.77 MB.

Table 3: Comprehensive memory budgets for the 15M Tiny Llama architecture. Sizes are aggregated and rounded to the nearest tenth of a megabyte. Deployment Feasibility: The 15M model is phenomenally suited for absolute edge deployment across the widest variety of hardware. Even in unquantized F32 format, the 92.5 MB total footprint rests comfortably below the 128 MB Chrome Android WebGPU binding limit 28, and operates far below the 384 MB Safari iOS death threshold.46 When quantized to Q4\_K\_M, the entire neural reasoning engine, alongside its context window and execution buffers, consumes just 40.9 MB. This represents a remarkably trivial footprint that can be safely embedded almost invisibly into the background of standard websites, operating smoothly alongside heavy visual DOM elements.

Profile 2: The 42M Architecture Budget

Context Length Target: 1024 tokens.Tokenizer: Standard 32,000 vocab (approx. 5 MB RAM footprint).KV Cache Target: 16.78 MB.

Table 4: Comprehensive memory budgets for the 42M Tiny Llama architecture. Deployment Feasibility: As mathematically predicted during the architectural teardown, the 42M F32 configuration triggers immediate systemic failures on strict edge environments. The 168 MB static weight payload instantly shatters the 128 MB maxStorageBufferBindingSize default of mobile WebGPU implementations.28 Furthermore, a total systemic memory load of 214.8 MB pushes aggressively close to the 300-400 MB volatility limits of the iOS Safari engine 46, massively increasing the probability of a silent OS-level tab termination during standard spikes in JavaScript garbage collection. Conversely, the Q4\_K\_M footprint proves resilient and highly practical. At a combined total of exactly 72.0 MB, the Q4-quantized 42M model easily avoids all WebGPU hardware limitations and resides safely within iOS memory constraints. This solidifies 4-bit block quantization not merely as a convenient network optimization, but as an indispensable, non-negotiable requirement for the practical cross-platform deployment of 40M+ architectures.

Strategic Conclusions

The empirical and architectural deconstruction of sub-50M Llama architectures within the browser runtime yields several foundational directives for software deployment:

1. Vocabularies Dominate the Byte Budget in Micro-Architectures: The mathematical structure of embedding and language modeling head projection matrices forces a standard 32,000-token dictionary to consume the vast majority of a 15M model's parameter budget.8 To deeply optimize tiny LLMs (sub-20M scales), engineering teams must aggressively scale down the vocabulary footprint, ideally training models on narrowed domain spaces with custom 512-token or 4,096-token dictionaries.8
2. WebGPU Binding Ceilings Act as Hard Blockers for F32 Deployment: The W3C specification strictly bounds mobile WebGPU maxStorageBufferBindingSize limits to 128 MB to maintain compatibility with older graphics driver stacks.34 Because an unquantized 42M model exceeds this constraint in its raw state, framework developers must rely strictly on 8-bit or 4-bit K-quants (Q8\_0, Q4\_K\_M) which reliably condense weight artifacts to 1.06 bytes or 0.6 bytes per parameter, easily sidestepping graphics hardware limits without fracturing the tensor arrays.18
3. Static Memory Paradigms are Essential for Browser Stability: Execution engines like Transformers.js that rely on dynamic buffer scaling risk exposing the browser to catastrophic memory volatility.6 Modern WebGPU systems designed specifically for local execution—such as Wllama utilizing OPFS-driven mmap limits, and LlamaWeb enforcing strict static memory slotted regions—are mandatory for mitigating iOS Safari's harsh 384 MB silent-termination threshold.6
4. FlashAttention is a Non-Negotiable Requirement for Prefill Memory Management: At a sequence length of 1024, the prefill generation of a 42M model requires nearly 34 MB of dynamic scratch buffer space just to house the [Figure omitted from source export] attention matrix intermediate state.6 Utilizing WebGPU FlashAttention implementations drops this overhead strictly into SRAM, restricting total system activation memory to under 10 MB, preventing OOM crashes on heavily memory-constrained edge mobile devices.6

In closing, the successful deployment of Tiny Llama models inside the browser is not limited by theoretical compute constraints, but rather by sophisticated, highly orchestrated memory choreography. By intelligently limiting KV caches, strictly enforcing Q4\_K\_M block quantization formats, tying embeddings, and utilizing statically allocated execution engines, both 15M and 42M architectures can operate safely and performantly inside the strict \~100 MB limits necessary for invisible, localized artificial intelligence on the modern web.

Works cited

1. Characterizing WebGPU Dispatch Overhead for LLM Inference Across Four GPU Vendors, Three Backends, and Three Browsers \- arXiv, accessed June 29, 2026, https://arxiv.org/html/2604.02344v1
2. WeInfer: Unleashing the Power of WebGPU on LLM Inference in Web Browsers \- OpenReview, accessed June 29, 2026, https://openreview.net/pdf?id=Qu2itILaoZ
3. Daily Papers \- Hugging Face, accessed June 29, 2026, https://huggingface.co/papers?q=CPU%20inference
4. Democratizing On-Device LLM Inference with Machine Learning Compilers and Web Technologies \- Carnegie Mellon University, accessed June 29, 2026, http://reports-archive.adm.cs.cmu.edu/anon/2025/CMU-CS-25-112.pdf
5. Overview of GGUF quantization methods : r/LocalLLaMA \- Reddit, accessed June 29, 2026, https://www.reddit.com/r/LocalLLaMA/comments/1ba55rj/overview\_of\_gguf\_quantization\_methods/
6. Llamas on the Web: Memory-Efficient, Performance-Portable, and Multi-Precision LLM Inference with WebGPU \- arXiv, accessed June 29, 2026, https://arxiv.org/html/2605.20706v1
7. Why Your Local LLM Is Slow — llama.cpp Config Guide | OmniForge Blog, accessed June 29, 2026, https://omniforge.online/blog/your-local-llm-is-slow-because-of-five-config-flags
8. GitHub \- karpathy/llama2.c: Inference Llama 2 in one file of pure C, accessed June 29, 2026, https://github.com/karpathy/llama2.c
9. Fast LLM Inference From Scratch \- Andrew Chan, accessed June 29, 2026, https://andrewkchan.dev/posts/yalm.html
10. gpt-fast/model.py at main \- GitHub, accessed June 29, 2026, https://github.com/meta-pytorch/gpt-fast/blob/main/model.py
11. config.json · Xenova/llama2.c-stories15M at ... \- Hugging Face, accessed June 29, 2026, https://huggingface.co/Xenova/llama2.c-stories15M/blob/9397aa7180a7b88fa20a0197da2a0601973373e9/config.json
12. Vectorizing Pytorch for RISC-V RVV \- UPCommons, accessed June 29, 2026, https://upcommons.upc.edu/bitstreams/62044513-c060-49e8-bc4b-993a7e2fb730/download
13. stories260K/readme.md · karpathy/tinyllamas at 0bd21da7698eaf29a0d7de3992de8a46ef624add \- Hugging Face, accessed June 29, 2026, https://huggingface.co/karpathy/tinyllamas/blob/0bd21da7698eaf29a0d7de3992de8a46ef624add/stories260K/readme.md
14. LlamaTokenizer \- Keras, accessed June 29, 2026, https://keras.io/keras\_hub/api/models/llama/llama\_tokenizer/
15. llama \tie\_word\_embeddings\ ignored on cpu and with auto dtype only · Issue \#33689 · huggingface/transformers \- GitHub, accessed June 29, 2026, https://github.com/huggingface/transformers/issues/33689
16. Upload folder using huggingface\_hub · Xenova/llama2.c-stories42M, accessed June 29, 2026, https://huggingface.co/Xenova/llama2.c-stories42M/commit/95a62cf2b1d20bf52aaab1450023b90069b605ab
17. Upload 21 files · teragron/TinyStories at 233119d \- Hugging Face, accessed June 29, 2026, https://huggingface.co/spaces/teragron/TinyStories/commit/233119d4973907e01db21718b5fe6285e3b338f3
18. GGUF Quantization Explained: Q4\_K\_M vs Q8\_0 and When Each Matters | Pristren Blog, accessed June 29, 2026, https://pristren.com/blog/gguf-quantization-guide-2026/
19. GGUF Optimization: A Technical Deep Dive (Part 1 of 2\) \- Medium, accessed June 29, 2026, https://medium.com/@michael.hannecke/gguf-optimization-a-technical-deep-dive-for-practitioners-ce84c8987944
20. GGUF · Hugging Face, accessed June 29, 2026, https://huggingface.co/docs/hub/en/gguf
21. tmc/go-llama2: Llama 2 inference in one file of pure Go \- GitHub, accessed June 29, 2026, https://github.com/tmc/go-llama2
22. README-dev.md \- ngxson/wllama \- GitHub, accessed June 29, 2026, https://github.com/ngxson/wllama/blob/master/README-dev.md
23. ngxson/wllama: WebAssembly binding for llama.cpp \- Enabling on-browser LLM inference · GitHub, accessed June 29, 2026, https://github.com/ngxson/wllama
24. Understanding Llama2.c And ChatGPT Inferencing – A Visual Design Walkthrough, accessed June 29, 2026, https://www.signalpop.com/2024/02/10/understanding-llama2-c-and-chatgpt-a-visual-design-walkthrough/
25. Llama 2: Prompt, Tokenizer and Padding Guide \- Colab, accessed June 29, 2026, https://colab.research.google.com/github/TrelisResearch/llama-2-setup/blob/main/Llama\_2\_Prompt\_and\_Tokenizer\_Format.ipynb
26. An Architecture for Web-Based Distributed LLM Inference \- reposiTUm, accessed June 29, 2026, https://repositum.tuwien.at/bitstream/20.500.12708/227204/1/Kitzberger%20Gabriel%20-%202026%20-%20An%20Architecture%20for%20Web-Based%20Distributed%20LLM...pdf
27. Scaling On-Device GPU Inference for Large Generative Models \- CVF Open Access, accessed June 29, 2026, https://openaccess.thecvf.com/content/CVPR2025W/EDGE/papers/Tang\_Scaling\_On-Device\_GPU\_Inference\_for\_Large\_Generative\_Models\_CVPRW\_2025\_paper.pdf
28. WebGPU Memory Limits: maxStorageBufferBindingSize \- Ayoob AI, accessed June 29, 2026, https://ayoob.ai/blog/webgpu-maxstoragebufferbindingsize-limits-enterprise
29. llama.cpp/tools/completion/README.md at master · ggml-org/llama.cpp · GitHub, accessed June 29, 2026, https://github.com/ggml-org/llama.cpp/blob/master/tools/completion/README.md
30. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness | Request PDF, accessed June 29, 2026, https://www.researchgate.net/publication/360936499\_FlashAttention\_Fast\_and\_Memory-Efficient\_Exact\_Attention\_with\_IO-Awareness
31. JChunX/web-control-net: ControlNet on the browser using WebGPU backend. \- GitHub, accessed June 29, 2026, https://github.com/JChunX/web-control-net
32. Run AI Models in the Browser with WebGPU & WASM \- Mad Devs, accessed June 29, 2026, https://maddevs.io/writeups/running-ai-models-locally-in-the-browser/
33. How We Stopped iOS Safari From Crashing Our E-Commerce Site | by Shilpe Saxena, accessed June 29, 2026, https://medium.com/@shilpecsaxena9098/how-we-stopped-ios-safari-from-crashing-our-e-commerce-site-beeb948ded34
34. WebGPU \- W3C, accessed June 29, 2026, https://www.w3.org/TR/webgpu/
35. GPUSupportedLimits \- Web APIs | MDN, accessed June 29, 2026, https://developer.mozilla.org/en-US/docs/Web/API/GPUSupportedLimits
36. Chat demo does not work on Android because of maxStorageBufferBindingSize · Issue \#209 · mlc-ai/web-llm \- GitHub, accessed June 29, 2026, https://github.com/mlc-ai/web-llm/issues/209
37. What's New in WebGPU (Chrome 133\) | Blog, accessed June 29, 2026, https://developer.chrome.com/blog/new-in-webgpu-133
38. Why is webgpu on mac "max binding size" much smaller than reported "max buffer size"?, accessed June 29, 2026, https://stackoverflow.com/questions/78034628/why-is-webgpu-on-mac-max-binding-size-much-smaller-than-reported-max-buffer-s
39. WebLLM: A High-Performance In-Browser LLM Inference Engine \- arXiv, accessed June 29, 2026, https://arxiv.org/html/2412.15803v2
40. How is heap memory managed in WebAssembly? \- Stack Overflow, accessed June 29, 2026, https://stackoverflow.com/questions/67340197/how-is-heap-memory-managed-in-webassembly
41. Play the Chaos Game to Understand WebAssembly Memory Management | by Jeremy Likness | Developer for Life | Medium, accessed June 29, 2026, https://medium.com/developer-for-life/play-the-chaos-game-to-understand-webassembly-memory-management-5feaa7553a5
42. Working with Large Models | onnxruntime, accessed June 29, 2026, https://onnxruntime.ai/docs/tutorials/web/large-models.html
43. The State of WebAssembly – 2024 and 2025 \- Uno Platform, accessed June 29, 2026, https://platform.uno/blog/state-of-webassembly-2024-2025/
44. Up to 4GB of memory in WebAssembly \- V8 JavaScript engine, accessed June 29, 2026, https://v8.dev/blog/4gb-wasm-memory
45. Mobile Safari on iOS crashes on big pages \- Stack Overflow, accessed June 29, 2026, https://stackoverflow.com/questions/11831429/mobile-safari-on-ios-crashes-on-big-pages
46. Safari ram limitation : r/iosdev \- Reddit, accessed June 29, 2026, https://www.reddit.com/r/iosdev/comments/1ugywet/safari\_ram\_limitation/
47. This unsourced SO answer says: \> Unfortunately the memory limit on iOS Safari is... | Hacker News, accessed June 29, 2026, https://news.ycombinator.com/item?id=39039593
48. Wasm needs a better memory management story · Issue \#1397 · WebAssembly/design \- GitHub, accessed June 29, 2026, https://github.com/WebAssembly/design/issues/1397
49. Using WebGPU | onnxruntime, accessed June 29, 2026, https://onnxruntime.ai/docs/tutorials/web/ep-webgpu.html
50. Native WebGPU EP fails to run model with in-memory external data · Issue \#24768 \- GitHub, accessed June 29, 2026, https://github.com/microsoft/onnxruntime/issues/24768
51. Helper methods for WebGPU runtime · microsoft onnxruntime · Discussion \#15937 \- GitHub, accessed June 29, 2026, https://github.com/microsoft/onnxruntime/discussions/15937
52. Wllama, accessed June 29, 2026, https://mikeesto.com/posts/wllama/
53. wllama/wllama \- UNPKG, accessed June 29, 2026, https://app.unpkg.com/@wllama/wllama@2.3.6/files/esm/workers-code/generated.d.ts
54. Llamas on the Web \- Reese Levine, accessed June 29, 2026, https://reeselevine.github.io/llamas-on-the-web/
55. LlamaWeb: Efficient LLM Inference in the Browser \- YouTube, accessed June 29, 2026, https://www.youtube.com/watch?v=3611Rv7w9hw