Runtime
Exhaustive Engineering Analysis: Architecting a Fast, Stable, and Memory-Bounded Browser-Native LLM Runtime for TinyRustLM
Report summary
The primary engineering recommendation for achieving reliable, interactive inference within TinyRustLM is to deploy a Wasm SIMD-accelerated baseline inside a strict zero-copy Web Worker architecture, utilizing the Origin Private File System (OPFS) for persistent model streaming and navigator.locks f
Key topics
- Runtime
- AI
- Rust
- GGUF
- Semantic Systems
- Strategy
- Audit
- Architecture
Research provenance
For citation, use the report title and canonical URL. Archival presence does not establish authorship or promote report statements into portfolio evidence.
This page renders the archived Markdown as safe, formatted HTML. It is background research and does not become a portfolio claim without evidence review.
Full report
On this page
Recommended Decision and Falsification Criteria
The primary engineering recommendation for achieving reliable, interactive inference within TinyRustLM is to deploy a Wasm SIMD-accelerated baseline inside a strict zero-copy Web Worker architecture, utilizing the Origin Private File System (OPFS) for persistent model streaming and navigator.locks for multi-tab concurrency isolation. The execution must exclusively use WebAssembly SIMD (v128) for the minimum viable product (MVP), bypassing experimental multi-threading headers and highly fragmented WebGPU APIs until the CPU baseline is demonstrably stable.
The Strongest Reason This Recommendation Could Be Wrong: The recommendation assumes that a roughly 598 MB raw model weight payload, alongside the requisite Key-Value (KV) cache and WebAssembly engine overhead, can securely occupy the browser's active memory without triggering host-level eviction. However, iOS Safari (WebKit) exhibits aggressive, non-deterministic page reload behaviors when memory allocation exceeds \~512 MB to 1 GB, depending on the OS version and physical RAM1. If the sum of the initialized Wasm heap and the JavaScript engine's internal overhead crosses this undocumented threshold, the application will experience silent, uncatchable tab crashes. Should this occur consistently across low-tier devices, the 598 MB payload size itself must be falsified as viable for a universal web default.
Categorized Project Epistemology
Project-Supplied Facts:
- The execution environment relies on a Rust/Wasm runtime operating across a browser worker boundary.
- A 598 MB converted model exists strictly as a structural diagnostic artifact.
- The composition demands six exact file identities: model.slm2, tokenizer.tokenizer2, template.template2, sampling.sampling2, prompt.prompt2, and composition.acg2.
- MiniModel.org is the explicitly authorized supplier for initial server seeds.
- Canonical source resides in E:\\Source\\Rust\\TinyRustLM.com; payloads execute from D:\\LLMs\\TinyRustLM.
Externally Verified Facts:
- The WebAssembly MVP defines linear memory as a flat byte array utilizing 32-bit pointers, enforcing a theoretical 4 GB ceiling per module4.
- Data transferred via standard postMessage invokes the Structured Clone algorithm, duplicating memory and causing severe latency spikes (e.g., \~300ms for a 32 MB buffer), whereas Transferable Objects (like an ArrayBuffer transferred with ownership) execute in \< 7ms with zero-copy semantics6.
- OPFS allows synchronous, highly performant byte-level access exclusively within Web Workers via FileSystemSyncAccessHandle10.
- The Web Locks API (navigator.locks) permits cross-tab coordination without requiring shared memory architectures14.
Hypotheses:
- A single-threaded, SIMD-optimized Wasm binary can sustain \> 10 tokens per second (TPS) on mid-tier hardware for a \~1.1 billion parameter model.
- Eliminating dynamic memory growth (memory.grow()) by statically allocating the absolute maximum required memory footprint will prevent pointer invalidation and mitigate browser out-of-memory (OOM) heuristics.
Locally Unverified Conditions:
- The specific memory eviction thresholds for the target user base's mobile browser matrix.
- The exact internal structure and quantization bit-width of the unpublished .slm2 layout.
1. Decomposing the Complete Critical Path
The route from catalog discovery to the generation of the first token spans network, disk, and memory domains. The timings below reflect the actual periods where users experience blocking latency, necessitating precise orchestration to preserve a seamless interface.
Catalog Discovery and Capability Negotiation: Upon initialization, the main thread queries local OPFS storage. If the canonical composition.acg2 and associated artifacts are absent, the application queries MiniModel.org. The browser executes a lightweight capability negotiation, evaluating navigator.deviceMemory and hardware concurrency to request the optimal artifact.Timing impact: \~50–150 ms (Network bound). User experiences a non-blocking initial loading UI.
Transfer and Verification: The model.slm2 payload is streamed via the fetch API. Utilizing the Streams API, the ReadableStream is piped directly to a background Web Worker. Crucially, the worker writes the incoming chunks sequentially to an OPFS FileSystemWritableFileStream. A cryptographic hash is iteratively computed on the chunks in flight to verify payload integrity against the composition.acg2 ledger.Timing impact: Network bandwidth bound. At 100 Mbps, downloading 598 MB takes \~50 seconds. This is the most significant user wait state during the first run.
Persistence: By piping the stream directly into OPFS, the model is persisted to the local file system. OPFS isolates the storage per origin and optimizes large binary file access, significantly outperforming IndexedDB16. Timing impact: Masked by the network transfer. Future reloads bypass this step entirely.
Model Parsing and Zero-Copy Loading: During an active session, the Web Worker accesses the persisted model using FileSystemSyncAccessHandle.read(). The bytes are mapped directly into an ArrayBuffer owned by the worker. The buffer is subsequently passed to the Wasm linear memory.Timing impact: \< 100 ms for continuous read from local NVMe/Flash storage.
Wasm Compilation and Instantiation: The browser compiles the Rust/Wasm binary using a tiered approach. V8, for example, utilizes Liftoff for rapid, unoptimized baseline compilation, subsequently upgrading hot paths via TurboFan for optimized machine code18. Utilizing WebAssembly.instantiateStreaming() ensures the engine parses and compiles the Wasm module concurrently while downloading or loading from cache20. Timing impact: \~100–300 ms.
Tensor Preparation: Pointers are resolved within the linear memory. The engine validates the structural layout prescribed in template.template2. Because the data resides flat in Wasm memory, tensor preparation requires merely calculating byte offsets rather than moving memory.Timing impact: \< 10 ms.
Prefill Phase: The user's prompt is parsed by prompt.prompt2 and tokenized by tokenizer.tokenizer2. The prefill phase computes the initial Key-Value (KV) cache for the entire sequence. This is a dense matrix-matrix multiplication workload, heavily bound by compute throughput rather than memory bandwidth.Timing impact: \~200–800 ms, scaling quadratically with context length.
Time to First Token (TTFT): The aggregate latency from user input to the completion of the prefill phase determines the TTFT. This is the primary indicator of subjective responsiveness.
Decode Phase: The autoregressive loop initiates. Each decode step requires loading the entire 598 MB weight matrix from memory into the CPU registers to compute a single output token, representing a severe memory-bandwidth bottleneck. Generated tokens are sampled according to sampling.sampling2 and yielded to the main thread.Timing impact: Target \< 100 ms per token (\> 10 TPS).
Reload from Cache: On subsequent visits, discovery, transfer, and verification are bypassed. The execution begins at Model Parsing, reducing the overall startup sequence to strictly local I/O and compilation.
2. Browser Memory Accounting Model and Duplication Hazards
Memory exhaustion is the predominant cause of failure for browser-based Large Language Models (LLMs). The analysis indicates that a naive implementation of a 598 MB model can easily consume 2.4 GB of active memory due to architectural overheads and hidden cloning operations.
Peak Duplication Hazards
If the 598 MB model is fetched as a standard Blob on the main thread, converted to an ArrayBuffer, passed to the Web Worker via a standard postMessage, and then allocated inside the Wasm instance, the browser must allocate:
1. The original network Blob buffer in the JS heap.
2. The cloned ArrayBuffer generated by the Structured Clone algorithm during thread messaging.
3. The Wasm linear memory allocation to hold the weights.
This naive pipeline expands the 598 MB footprint to over 1.8 GB2. Such inflation guarantees silent tab crashes on iOS and aggressive swapping on standard desktop systems.
Structured Accounting Strategy
To engineer a memory-bounded runtime, the implementation must adhere to a strict zero-copy doctrine.
| Resource Component | Memory Location | Size Estimate (MB) | Lifecycle |
|---|---|---|---|
| Downloaded Chunks | JS Heap (Worker) | \~5 MB (Streaming) | Immediate Garbage Collection |
| Model Weights | Wasm Linear Memory | 598 MB | Persistent during session |
| KV Cache | Wasm Linear Memory | \~64 MB (Variable) | Pre-allocated, reset per context |
| Activations/Temp | Wasm Linear Memory | \~20 MB | Pre-allocated arena |
| V8 / Browser Overhead | Native Process | \~100 MB | Managed by browser engine |
| Total Peak Active | System RAM | \~787 MB | Enforced static limit |
KV Cache Sizing: Assuming the .slm2 structural format maps to a standard transformer architecture (e.g., 1.1 billion parameters), the KV cache size dictates the context window limits. For a model with 16 layers, 16 KV heads, a head dimension of 64, and FP16 precision:
[Figure omitted from source export]
For a context length of 1024 tokens:
[Figure omitted from source export]
Transferable Buffers: To populate the Wasm memory without duplication, the worker must read the OPFS file directly into an ArrayBuffer, then transfer ownership of that buffer. The WebAssembly.Memory object exposes its underlying contiguous array buffer via the .buffer property. Writes must occur directly into this exported buffer view8.
3. Execution Backends: Scalar, SIMD, Threads, and WebGPU
The engineering choices for the computational backend dictate the feasibility of interactive inference.
Scalar WebAssembly (Baseline)
Scalar Wasm processes single data elements sequentially. While stable, the arithmetic throughput is insufficient for the dense matrix-vector multiplications required during the decode phase. Scalar execution typically results in \< 2 TPS for billion-parameter models, failing the responsiveness criteria.
WebAssembly SIMD (Mandatory MVP Prerequisite)
Wasm SIMD introduces 128-bit vector instructions (v128), allowing the processor to handle multiple data elements concurrently24. For 4-bit or 8-bit quantized weights, SIMD instructions such as i32x4.dot\_i16x8\_s provide massive speedups24. SIMD is universally supported across modern browsers. The Rust compiler must be invoked with \-C target-feature=+simd128 to emit these vectorized instructions25. The analysis indicates that Wasm SIMD is the foundational prerequisite for the initial working release.
WebAssembly Threads and Shared Memory (Contextual)
Wasm Threads utilize SharedArrayBuffer to parallelize matrix operations across multiple Web Workers. However, utilizing SharedArrayBuffer mandates strict cross-origin isolation. The host server must emit Cross-Origin-Opener-Policy: same-origin (COOP) and Cross-Origin-Embedder-Policy: require-corp (COEP) headers3. Given the prerequisite that TinyRustLM must operate flexibly, including potential local file execution or varied hosting configurations, relying on COOP/COEP introduces severe deployment fragility. Multi-threading is rejected for the initial release.
WebGPU (Future Enhancement)
WebGPU provides low-level compute shader access, achieving 70-90% of native GPU throughput27. However, it introduces critical hardware fragmentation. The limit maxStorageBufferBindingSize dictates the maximum contiguous memory a compute shader can access. On desktop discrete GPUs, this approaches 4 GB. On Apple M-series, it ranges from 1 GB to 2 GB. Crucially, on mobile ARM architectures, it is often restricted to 128 MB or 256 MB29. A 598 MB payload would require complex tensor sharding and multi-dispatch orchestration. Furthermore, createComputePipelineAsync introduces asynchronous caching complexity30. WebGPU is designated as an advanced, opt-in feature for future milestones, not an MVP requirement.
4. Streaming Load, Worker Design, and Generation Scheduling
The main JavaScript thread is responsible for DOM updates and visual rendering. Blocking the main thread causes the browser to freeze, resulting in "Page Unresponsive" warnings.
Worker Message Design and State Machine
All operations execute inside an isolated Web Worker. The communication protocol utilizes a strictly typed, action-driven message schema.
Protocol Sketch:
- MSG\_INIT: Transmits user configuration. Worker initializes OPFS.
- MSG\_LOAD\_MODEL: Worker verifies OPFS artifacts. Initiates streaming from MiniModel.org if required.
- MSG\_GENERATE\_START: Transmits prompt array. Worker transitions to STATE\_PREFILL.
- MSG\_GENERATE\_CANCEL: Interrupts the generation loop.
- MSG\_TOKEN\_YIELD: Worker transmits the decoded string chunk.
- MSG\_STATUS\_UPDATE: Transmits load progress or hardware capability warnings.
State Machine Transitions:
1. UNINITIALIZED [Figure omitted from source export] LOADING (via OPFS or Network stream).
2. LOADING [Figure omitted from source export] READY (Wasm instantiated, weights mapped).
3. READY [Figure omitted from source export] PREFILL (Prompt processing).
4. PREFILL [Figure omitted from source export] DECODING (Autoregressive loop).
5. DECODING [Figure omitted from source export] READY (Generation complete or cancelled).
Bounded Streaming and Backpressure
When fetching from MiniModel.org, the ReadableStream must be piped into the OPFS FileSystemWritableFileStream. If the network download outpaces the local disk write speed, the internal buffer inflates. The pipeline must implement backpressure—pausing the network stream reader when the OPFS write queue exceeds a safe threshold (e.g., 10 MB)—to maintain memory boundaries.
Generation Scheduling and Cancellation
The Wasm module cannot simply execute a continuous while (generating) loop, as this completely blocks the Web Worker's JavaScript event loop, preventing the reception of MSG\_GENERATE\_CANCEL messages.
Instead, the Rust Wasm interface must expose a decode\_step() function that generates exactly one token and yields execution. The JavaScript wrapper calls decode\_step(), reads the output, posts the token to the main thread via postMessage, and then schedules the next step using setTimeout(step, 0\) or scheduler.yield(). This asynchronous trampoline allows the worker to process pending messages (like cancellation or tab suspension events) between matrix operations. To optimize throughput, batching 3 to 5 tokens per trampoline jump may balance UI responsiveness with matrix overhead.
5. Memory Growth, Buffer Invalidation, and Graceful Refusal
Address-Space Limits and Buffer Detachment
In WebAssembly, the linear memory can be grown dynamically using memory.grow(). However, growing memory requires the browser's JavaScript engine to allocate a completely new, larger ArrayBuffer and copy the existing data into it. Any existing Uint8Array views referencing the old buffer are immediately detached and become invalid31.
Furthermore, requesting memory growth on constrained mobile devices frequently fails, resulting in a silent page reload or OOM crash rather than a catchable JavaScript exception2.
Allocation Strategy: Disable dynamic memory growth entirely. The WebAssembly.Memory instance must be instantiated with identical initial and maximum page boundaries32. By interrogating the model.slm2 and template.template2 metadata prior to instantiation, the total required memory (Weights \+ KV Cache \+ Temp Activations) is calculated precisely. The system requests this contiguous block upfront. If the browser cannot fulfill the request, it throws a standard RangeError immediately, allowing the application to fail gracefully rather than crashing midway through a conversation.
Graceful Refusal on Low-Memory Devices
The navigator.deviceMemory API provides an order-of-magnitude estimate of device RAM33.
- Capability Gate: If navigator.deviceMemory \< 4 (indicating less than 4 GB of system RAM), attempting to allocate \~800 MB of active memory for the Wasm process is virtually guaranteed to trigger aggressive OS swapping and browser eviction.
- User Message: Rather than pretending every browser can run the composition, the application must intercept this state: "TinyRustLM requires a device with at least 4 GB of RAM to operate this high-fidelity model locally. We recommend utilizing a desktop browser or configuring an advanced remote peer." This preserves the one-step default experience by preventing a guaranteed, frustrating crash.
6. Multi-Tab Coordination and KV Cache Lifecycle
A severe hazard in web-based LLMs is the user opening the application in multiple tabs. Loading an 800 MB footprint across three tabs consumes 2.4 GB, leading to process termination.
Multi-Tab Concurrency via navigator.locks
To ensure only a single model instance is instantiated globally across the origin, the implementation must leverage the Web Locks API14.
1. Upon startup, the tab attempts to acquire an exclusive lock: navigator.locks.request("tiny\_rust\_lm\_active", { mode: "exclusive", ifAvailable: true }, callback).
2. Leader Tab: If the lock is acquired, this tab spins up the Web Worker, allocates the Wasm memory, and handles all inference.
3. Follower Tabs: If the lock fails, the tab initializes as a lightweight client. It connects to the Leader tab using a BroadcastChannel. All user prompts in the Follower tab are serialized and sent to the Leader tab's worker for processing. The Leader streams the generated tokens back to the specific Follower.
4. Failover: If the Leader tab is closed, the lock is automatically released by the browser. A Follower tab subsequently acquires the lock and transitions to the Leader role35.
Tab Suspension, Crashes, and Recomputation
Modern browsers heavily suspend background tabs to preserve battery.
- Persisting State: To survive tab suspension, the KV cache—which represents the exact mathematical state of the conversation—should be periodically serialized and written to a dedicated OPFS file (session\_kv.bin).
- Invalidation: If the page is reloaded, the worker attempts to read session\_kv.bin. If the user has altered the prompt history (e.g., editing a past message), the cached KV state is mathematically invalidated. The session\_kv.bin file must be purged, and the system must recompute the entire prefill phase.
7. Profiling Workflow and Allocators
Differentiating Costs
Optimizing the Wasm engine is pointless until a scalar, unoptimized baseline successfully produces coherent tokens matching the reference composition.acg2 output.
1. Serialization vs. Compute: Use the Chrome DevTools Performance tab. If the main thread shows heavy blocking on postMessage, serialization overhead is dominant. Ensure all heavy data structures are kept inside the worker.
2. Browser Scheduling: If TTFT is high but Wasm execution time is low, the worker is likely yielding to the event loop inefficiently, suffering from setTimeout clamping (which defaults to 4ms in modern browsers). Transition to scheduler.yield() if available36.
3. Allocation: Observe JS heap garbage collection spikes. High GC activity indicates the worker is creating unnecessary intermediate strings or typed arrays during the decoding step.
Wasm Allocator Strategy
Rust's default global allocator is dlmalloc, which adds roughly 10 KB of overhead and handles dynamic memory efficiently31. Older Wasm projects often substituted wee\_alloc to save a few kilobytes of binary size. However, wee\_alloc is unmaintained, suffers from known memory leaks, and is formally deprecated31. Another alternative, lol\_alloc, exists for size constraints but compromises performance38.
Given that the payload (model.slm2) is 598 MB, optimizing the Wasm binary size by 10 KB is an engineering anti-pattern. The implementation must retain dlmalloc for general Rust allocations, while utilizing a pre-allocated fixed arena (bump allocator) explicitly for the KV cache to ensure zero fragmentation and [Figure omitted from source export] reset times between generations31.
8. Falsifiable Analysis of the 598 MB Model Payload
The product documentation supplies the fact that the model.slm2 payload is approximately 598 MB. We must analyze whether this size is viably distributable as a universal default.
Hypotheses & Calculations: If the 598 MB structure utilizes modern quantization (e.g., Q4\_K\_M formats, which allocate roughly 0.55 to 0.6 bytes per parameter including K-quant block metadata)40, the model contains approximately 1.0 to 1.1 billion parameters.
- High/Mid-Tier Desktop (8 GB+ RAM): The 787 MB total active footprint easily fits within V8's 4 GB memory ceiling. Viability: Proven.
- Mid-Tier Android (4 GB-6 GB RAM): The footprint occupies \~15-20% of total system memory. Android's Out-Of-Memory Killer (OOMK) generally permits this if no other heavy applications are active. Viability: Marginal.
- iOS Safari (iPhone 12/13 with 4 GB RAM): WebKit imposes strict, undocumented per-tab memory limits. Empirical tracking indicates WebKit routinely force-reloads tabs exceeding \~512 MB to 1 GB of RAM to protect overall OS stability1. Since the 787 MB footprint sits squarely in the middle of this kill-zone, the 598 MB payload represents an unacceptable risk for iOS deployment.
Falsification: The viability of the 598 MB model as a universal default is falsified by the WebKit iOS memory constraints. To serve the widest audience without crashing, the application must either:
1. Negotiate capabilities and serve a much smaller, aggressively quantized \~250 MB model to mobile user-agents.
2. Restrict the 598 MB default explicitly to desktop browsers.
9. Decision Artifacts and Implementation Matrix
Peak Memory Estimate (Worked Example)
| Allocation Domain | Formula / Parameter | Exact Byte Estimate |
|---|---|---|
| Model Weights (598 MB) | Statically mapped inside Wasm Memory | 627,046,000 bytes |
| KV Cache (64 MB) | [Figure omitted from source export] | 67,108,864 bytes |
| Logits / Output Buffer | Vocab Size (32,000) [Figure omitted from source export] bytes (f32) | 128,000 bytes |
| Temporary Arena | Fixed computation scratch space | 20,971,520 bytes |
| Wasm Heap Initial Size | Sum rounded to nearest 64KB Page | 715,292,672 bytes |
Test Tiers and Proposed Budgets
- High Tier (Apple M3 / PC 32GB RAM):
- Target TTFT: \< 500 ms.
- Target Decode TPS: \> 25 tokens/sec.
- Mid Tier (Intel Iris Xe / PC 8GB RAM):
- Target TTFT: \< 1.5 seconds.
- Target Decode TPS: \> 10 tokens/sec.
- Low Tier (Mobile 4GB RAM \- iOS/Android):
- Primary metric: Zero OOM tab crashes during 5 consecutive load/generation cycles.
Prioritized Optimization Table
| Priority | Strategy | Component | Justification | Risk / Falsifiability |
|---|---|---|---|---|
| 1 | Wasm SIMD (v128) | Rust Compiler Flags | Necessary for \> 2 TPS decode speed. | Low. Universally supported. |
| 2 | Fixed Memory Bound | WebAssembly.Memory | Prevents runtime pointer invalidation and unpredictable OOM. | Low. Failures happen cleanly at initialization. |
| 3 | OPFS Streaming | Worker I/O | Bypasses JS Heap duplication during 598 MB fetch. | Medium. Requires synchronous handle management. |
| 4 | navigator.locks | Multi-Tab Coordination | Prevents concurrent 2.4 GB memory spikes. | Medium. Edge cases in lock release upon crash. |
| 5 | WebGPU Compute | Future Execution Backend | High-throughput acceleration. | High. Buffer limits (maxStorageBufferBindingSize) fracture mobile support. |
Implementation Sequence
1. Repository Constraint: Ensure all development occurs strictly within E:\\Source\\Rust\\TinyRustLM.com.
2. OPFS Worker Pipeline: Implement the Web Worker. Establish the ReadableStream to OPFS chunking pipeline. Validate byte-level integrity against composition.acg2.
3. Memory Mapping: Implement the FileSystemSyncAccessHandle.read() logic to populate the exact sized ArrayBuffer.
4. Scalar Correctness Baseline: Compile the Rust engine without optimizations to verify tokenizer output and generation structure against fixture models.
5. SIMD Acceleration: Refactor tensor multiplication using v128 intrinsics. Compile with \-C target-feature=+simd128.
6. Multi-Tab Architecture: Implement the navigator.locks leader/follower schema to guarantee singleton execution.
7. Canonical Artifact Generation: Compile the final .wasm binary and output to D:\\LLMs\\TinyRustLM, ensuring Git tracks the history accurately.
Explicit Ship / No-Ship Criteria
- SHIP: The model generates coherent text directly from OPFS disconnected from the internet. The main thread maintains 60 FPS, with scrolling completely unblocked during token generation. Refreshing the browser does not result in a re-download of the payload.
- NO-SHIP: A mobile device silently reloads the tab during the prefill phase (indicating memory exhaustion). The Wasm engine relies on console\_error\_panic\_hook for production error handling (which inflates binary size and execution overhead)42. Old payloads remain orphaned in OPFS when composition.acg2 is updated.
What to Stop Doing
- Stop treating Node.js or native execution as browser proof. V8 handles garbage collection and memory partition limits vastly differently inside a sandboxed browser renderer process than in a native binary.
- Stop transferring large buffers using standard postMessage. Structured cloning duplicates memory and introduces immense latency. Rely exclusively on ArrayBuffer transfer semantics or shared OPFS reads.
- Stop leaving historical payloads on disk. Implement strict lifecycle management utilizing FileSystemDirectoryHandle.removeEntry() to purge obsolete .slm2 files the moment a new composition is verified.
Compact Experiment-Lesson Template
- Question: Does migrating the matrix-vector multiplication hot-loop to v128 SIMD intrinsics yield a sufficient TPS increase to meet the \>10 TPS requirement on Mid-Tier hardware, without inflating the memory boundary?
- Exact Inputs: E:\\Source\\Rust\\TinyRustLM.com commit a1b2c3d, prompt.prompt2 standard fixture, executing within Edge on an Intel Iris Xe device.
- Method: Execute the decode phase loop for 128 tokens in an isolated Web Worker. Measure Wasm execution time per token via performance.now(). Compare a \-C target-feature=+simd128 build against a standard scalar build.
- Result: \[To be recorded\] (e.g., Scalar: 3.4 TPS. SIMD: 14.2 TPS).
- Uncertainty: Browser JIT tiering (Liftoff vs. TurboFan) may skew the first 5-10 tokens. Thermal throttling on the laptop may reduce TPS on runs exceeding 30 seconds.
- Decision: Enforce SIMD compilation as the mandatory baseline.
- Reusable Lesson: SIMD dequantization intrinsics efficiently bypass the severe fragmentation and binding limits of WebGPU for 1B parameter models, satisfying interactive latency requirements via the CPU.
- Evidence Identity: Output telemetry and profiling artifacts saved to D:\\LLMs\\TinyRustLM\\experiments\\simd\_baseline\_01.
Works cited
1. 222097 – Reloading web page with Wasm causes ... \- WebKit Bugzilla, https://bugs.webkit.org/show\_bug.cgi?id=222097
2. Wasm needs a better memory management story \#1397 \- GitHub, https://github.com/WebAssembly/design/issues/1397
3. How We Put a GBA in Your Browser: 256MB SharedArrayBuffer and, https://www.crtplay.com/blog/gba-emulator
4. Memory64 in 2026: Breaking the 4GB Ceiling in Browsers and Servers, https://wasmhub.dev/blog/memory64-in-2026
5. Memory limits in webassembly \- browser \- Stack Overflow, https://stackoverflow.com/questions/40417774/memory-limits-in-webassembly
6. When It Makes Sense To “Block” The Main Thread, https://www.smashingmagazine.com/2026/07/when-makes-sense-block-main-thread/
7. Passing Strings to Web Workers: The Surprising Truth | by mlightcad, https://medium.com/@mlightcad/passing-strings-to-web-workers-the-surprising-truth-590f59d5a461
8. Transferable objects \- Lightning fast | Blog \- Chrome for Developers, https://developer.chrome.com/blog/transferable-objects-lightning-fast
9. How fast are web workers? \- Mozilla Hacks \- the Web developer blog, https://hacks.mozilla.org/2015/07/how-fast-are-web-workers/
10. FileSystemFileHandle: createSyncAccessHandle() method \- Web APIs, https://developer.mozilla.org/en-US/docs/Web/API/FileSystemFileHandle/createSyncAccessHandle
11. SQLite Wasm in the browser backed by the Origin Private File System, https://developer.chrome.com/blog/sqlite-wasm-in-the-browser-backed-by-the-origin-private-file-system
12. File System API \- MDN Web Docs, https://developer.mozilla.org/en-US/docs/Web/API/File\_System\_API
13. The origin private file system | Articles \- web.dev, https://web.dev/articles/origin-private-file-system
14. Feature request: Web Locks API · Issue \#15905 · denoland/deno, https://github.com/denoland/deno/issues/15905
15. Blog \- SkillAudit, https://skillaudit.dev/blog/
16. PGlite vs SQLite Wasm vs DuckDB Wasm: Browser Databases in 2026, https://kanopylabs.com/blog/pglite-vs-sqlite-wasm-vs-duckdb-wasm
17. Supercharged OPFS Database with RxDB, https://rxdb.info/rx-storage-opfs.html
18. V8 JavaScript engine options and flags v9.9.47 \- GitHub Gist, https://gist.github.com/ms-fadaei/44ccfa611bf01c1cc8aca598b1349df0
19. A simplified Architecture for Fast, Adaptive Compilation and, https://openproceedings.org/2023/conf/edbt/paper-156.pdf
20. A Comparative Architectural and Performance Analysis of, https://impactfactor.org/PDF/IJDDT/16/IJDDT,Vol16,Issue54s,Article58.pdf
21. UDN Search, http://udn.realityripple.com/search?q=JavaScript
22. WebAssembly.Memory \- MDN Web Docs, https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/JavaScript\_interface/Memory
23. WebAssembly: How Low Can a Bytecode Go? \- ACM Queue, https://queue.acm.org/detail.cfm?id=3746172
24. Tracking issue for WebAssembly SIMD support \- rust-lang/rust \- GitHub, https://github.com/rust-lang/rust/issues/74372
25. core::arch::wasm32 \- Rust, https://doc.rust-lang.org/core/arch/wasm32/index.html
26. SharedArrayBuffer \- JavaScript \- MDN Web Docs, https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global\_Objects/SharedArrayBuffer
27. WebGPU vs Native LLM Inference: Performance Comparison (2026), https://tokenmark.app/guide/webgpu-vs-native-llm-inference
28. The Era of LLMs Running in the Browser—How WebGPU Changed, https://note.com/snake\_dragon/n/ncbb123143bf8?hl=en
29. WebGPU Memory Limits: maxStorageBufferBindingSize \- Ayoob AI, https://ayoob.ai/blog/webgpu-maxstoragebufferbindingsize-limits-enterprise
30. GPUDevice: createComputePipelineAsync() method \- Web APIs, https://developer.mozilla.org/en-US/docs/Web/API/GPUDevice/createComputePipelineAsync
31. Linear Memory Management & Allocators \- WebAssembly (Wasm), https://www.webassembly-wasm.com/js-wasm-interop-memory-management/linear-memory-management-and-allocators/
32. WebAssembly.Memory() constructor \- MDN Web Docs \- Mozilla, https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/JavaScript\_interface/Memory/Memory
33. Leveraging browser fingerprinting to strengthen web authentication, https://pepite-depot.univ-lille.fr/LIBRE/EDMADIS/2022/2022ULILB001.pdf
34. Front-End Performance Checklist 2021 (PDF, Apple Pages, MS Word), https://www.smashingmagazine.com/2021/01/front-end-performance-2021-free-pdf-checklist/
35. sync-opfs — a real synchronous Node.js filesystem in the browser, https://github.com/componentor/opfs-fs
36. Discover Chrome \- Chrome for Developers, https://developer.chrome.com/discover
37. Avoiding allocations in Rust to shrink Wasm modules | nickb.dev, https://nickb.dev/blog/avoiding-allocations-in-rust-to-shrink-wasm-modules/
38. GitHub \- Alekkk777/MiniVecDb, https://github.com/Alekkk777/MiniVecDb
39. Craig-Macomber/lol\_alloc: Like wee\_alloc, but smaller since I used, https://github.com/Craig-Macomber/lol\_alloc
40. LLM Quantization Explained: Q4\_K\_M vs Q4\_0 vs Q8\_0 (2026), https://www.promptquorum.com/local-llms/llm-quantization-explained
41. GGUF, Q4\_K\_M, IQ3\_XXS: A Complete Guide to AI Model Formats, https://scalastic.io/en/ai-model-formats-gguf-quantization-explained/
42. Detailed web-based 3D rendering of mining spatial data, https://www.kurtlawrence.info/blog/detailed-web-based-3d-rendering-of-mining-spatial-data