Portable WebAssembly Runtime Architecture for TinyRustLM

Full report

On this page

Executive Summary and Architectural Decision

The uploaded brief defines a privacy-preserving, browser-local transformer runtime written in Rust, with native Windows/Linux targets, browser WebAssembly, a normative scalar implementation, optional stable SIMD and threading, strict numerical-conformance requirements, and no assumed access to private TinyRustLM source, models, benchmarks, traces, build flags, or hardware. This report therefore distinguishes public facts from architectural recommendations and identifies every conclusion that still requires execution against authorized TinyRustLM artifacts.

Recommended architecture. Build one normative Rust inference core and model contract, but publish three separately compiled browser modules:

A module containing unsupported instructions fails validation even when the unsupported function would never be called. Consequently, a “universal” binary containing scalar, SIMD, atomics, relaxed-SIMD, and Memory64 code cannot serve as the portable baseline merely by branching around unsupported paths. WebAssembly validation covers the module’s instructions before execution, and Rust documents that unsupported Wasm target-feature instructions cause load failure rather than undefined behavior.

The loader should perform this sequence:

flowchart TD
    A[Fetch signed capability manifest] --> B[Run tiny feature-module validation probes]
    B --> C{Cross-origin isolated?}
    C -->|No| D{SIMD128 probe passes?}
    C -->|Yes| E{SIMD128, atomics, shared-memory probes pass?}
    E -->|Yes| F[Fetch threaded SIMD module]
    E -->|No| D
    D -->|Yes| G[Fetch SIMD128 module]
    D -->|No| H[Fetch scalar module]
    F --> I[Verify module SHA-256 and manifest contract]
    G --> I
    H --> I
    I --> J[Instantiate with bounded memory]
    J --> K[Run module-specific startup self-tests]
    K -->|Pass| L[Load, stream-hash, validate and activate model]
    K -->|Fail| M[Record failure and retry lower capability lane]
    L --> N[Emit capability receipt]
    N --> O[Prefill and decode]

Normative semantics. The scalar implementation defines operator order, tensor shapes, integer-overflow rules, quantization decoding, softmax behavior, pseudorandom-number generation, token filtering, and cache identity. SIMD and threaded paths implement the same operation contracts and are promoted only after differential tests demonstrate their assigned conformance tier. There should not be a permanent “legacy scalar runtime” and a separately evolving “fast runtime.” Scalar is an oracle implementation within the same architecture.

Key design decisions.

Evidence labels used below.

• Public fact means directly documented in a public specification, vendor document, paper, or public implementation.
• Inference means a conclusion derived from public facts but not directly stated by them.
• Recommendation means the proposed TinyRustLM design.
• Assumption means a temporary premise that must be checked against the private model format or product requirements.
• Local verification required means the claim cannot be established without authorized TinyRustLM code, model, browser, or physical hardware execution.

No benchmark result from another project is used as evidence of TinyRustLM performance.

Standards Status and Capability Matrix

The current live WebAssembly language is Wasm 3.0. Its 2025 completion incorporated Memory64, multiple memories, typed references, tail calls, exception handling, and relaxed SIMD; the current online Core Specification identifies itself as release 3.0 dated July 2026. Wasm 2.0 standardized fixed-width SIMD, bulk-memory operations, multivalue results, and simple reference types. The feature-status implementation table nevertheless shows that standardization does not imply universal browser support: Safari is still blank for Memory64, multiple memories, and relaxed SIMD, while threads remain at proposal phase four despite long-standing browser implementations.

Capability taxonomy as retrieved August 1, 2026.

Browser-version values and feature phases in the table are drawn from the WebAssembly project’s live feature-status matrix, not inferred from marketing claims.

Baseline Wasm. WebAssembly defines a sandboxed virtual instruction set with explicit validation, bounded linear-memory accesses, traps, and no ambient access to the host. The core specification itself does not define browser APIs or system calls; those arrive through the embedding and imported functions.

Recommendation. Define TinyRustLM’s baseline as an explicit feature manifest rather than “whatever wasm32-unknown-unknown currently enables.” Rust’s documented default feature set may evolve, while wasm32v1-none intentionally fixes a Core 1.0-oriented target and lists later proposals that must be enabled separately. A reproducible production build should therefore use either wasm32v1-none where its ABI is sufficient or wasm32-unknown-unknown -Ctarget-cpu=mvp with a reviewed allowlist of enabled features.

Stable SIMD versus relaxed SIMD. Fixed-width SIMD defines the v128 type and deterministic instruction semantics subject to WebAssembly’s floating-point rules. Relaxed SIMD explicitly permits implementation choices for operations such as fused versus unfused multiply-add, NaN behavior, and lane selection. Rust’s intrinsic documentation reflects these alternatives.

Recommendation. Production conversational equivalence should initially use only fixed SIMD128. Relaxed SIMD may be admitted later for narrowly selected kernels whose output tolerances and decoding invariants have been measured independently. A single receipt field such as "relaxed_simd": true is insufficient; it must identify the exact relaxed instructions that can execute.

Threads and isolation. Cross-origin isolation is an execution precondition, not a server-configuration intention. A page normally becomes cross-origin isolated through Cross-Origin-Opener-Policy: same-origin plus Cross-Origin-Embedder-Policy: require-corp or credentialless, provided permissions policy does not block the capability. window.crossOriginIsolated and the corresponding worker property expose the result. In that state, SharedArrayBuffer is available with fewer restrictions.

Recommendation. The threaded loader must refuse the threaded artifact unless all of the following are true:

1. self.crossOriginIsolated === true;
2. SharedArrayBuffer construction succeeds;
3. the threaded artifact validates;
4. imported shared memory can be created at the preflight size;
5. every required worker starts and acknowledges the module hash;
6. atomics and barrier self-tests complete within a bounded timeout;
7. the selected threaded kernel executes a test vector and updates a shared path counter.

Feature detection alone is not proof of execution.

Memory64. Wasm 3.0 defines 64-bit memory indices, but browser embedding limits remain. The July 2026 JavaScript Interface specifies a maximum 32-bit memory of 4 GiB and a runtime maximum for 64-bit web memory of 16 GiB, while noting that implementations can exhaust resources below normative maxima.

Recommendation. Memory64 should not be a production baseline. It should be a separate artifact and model-admission class. TinyRustLM should first determine whether any supported compact model truly needs more than the practical Wasm32 budget; segmented logical tensors or smaller admitted contexts are likely cleaner than immediately doubling pointer and index widths.

Multiple memories. Multiple memories could separate immutable weights, mutable state, and host-transfer buffers, but absent Safari support means that adopting them would require another module family. The practical value does not yet justify the packaging and test-matrix cost for a compact CPU runtime.

Exception handling and tail calls. Neither is necessary for transformer execution. A stable result-code ABI and explicit loops are easier to share with native Rust and easier to fault-inject. Compile with aborting panics and convert all expected failures into typed errors before they can panic.

WebGPU. WebGPU exposes GPU buffers, command encoders, queues, and compute pipelines, but the specification describes script-owned memory and GPU resources as distinct domains. Buffer mapping and queue writes can require staging copies, especially because the GPU driver may reside in another browser process. There is no standardized facility to bind a range of Wasm linear memory directly as a GPUBuffer.

Recommendation. Treat WebGPU as a separate backend handoff:

Wasm model contract
      |
      +-- CPU scalar
      +-- CPU SIMD128
      +-- CPU SIMD128 + threads
      +-- optional WebGPU adapter
             |
             +-- explicit weight upload
             +-- explicit activation/KV ownership rules
             +-- separate numerical mode and cache identity
             +-- device-loss and readback self-tests

A successful navigator.gpu check is no more proof of backend execution than a SIMD probe is proof of kernel use.

Packaging, Typed Boundary, Capability Receipt, and Memory

Packaging comparison.

The WebAssembly standard emphasizes modular encoding and streaming compilation, but browser feature validation still applies to each module. The Component Model’s current proposal status makes it unsuitable as the browser deployment foundation for a near-term product.

Recommended artifact manifest.

{
  "schema": "tinyrustlm.module-manifest.v1",
  "runtime_semantics": 1,
  "abi_version": 1,
  "artifacts": [
    {
      "id": "scalar-wasm32",
      "url": "runtime.scalar.wasm",
      "sha256": "<hex>",
      "required_features": ["mutable-globals"],
      "forbidden_features": [
        "simd128",
        "relaxed-simd",
        "atomics",
        "memory64",
        "multimemory"
      ]
    },
    {
      "id": "simd128-wasm32",
      "url": "runtime.simd128.wasm",
      "sha256": "<hex>",
      "required_features": ["simd128"],
      "forbidden_features": ["relaxed-simd", "atomics", "memory64"]
    },
    {
      "id": "simd128-threads-wasm32",
      "url": "runtime.simd128.threads.wasm",
      "sha256": "<hex>",
      "required_features": ["simd128", "atomics", "bulk-memory"],
      "requires_cross_origin_isolation": true,
      "memory": {"shared": true, "growth": false}
    }
  ]
}

Recommendation. Cache the manifest and modules with content-addressed filenames. Keep feature probes tiny and embedded in typed JavaScript as immutable byte arrays. Probe definitions themselves should carry hashes and version numbers so a changed probe cannot silently alter capability classification.

Capability receipt. A receipt is an auditable account of what the runtime observed and actually selected. It is not a claim about the user’s physical CPU and should not become a hardware fingerprint.

{
  "schema": "tinyrustlm.capability-receipt.v1",
  "created_utc": "2026-08-01T00:00:00Z",
  "runtime_semantics": 1,
  "abi_version": 1,

  "host_reported": {
    "browser_brand": "value exposed by browser",
    "browser_version": "value exposed by browser",
    "os_family": "optional coarse value or unknown",
    "architecture": "optional coarse value or unknown"
  },

  "module": {
    "artifact_id": "simd128-threads-wasm32",
    "sha256": "<hex>",
    "build_manifest_sha256": "<hex>",
    "rustc": "pinned exact version",
    "llvm": "pinned exact version",
    "compile_features": ["simd128", "atomics", "bulk-memory"]
  },

  "observed": {
    "cross_origin_isolated": true,
    "feature_probes": {
      "simd128": {"validates": true, "probe_sha256": "<hex>"},
      "relaxed_simd": {"validates": false, "probe_sha256": "<hex>"},
      "threads": {"validates": true, "probe_sha256": "<hex>"},
      "memory64": {"validates": false, "probe_sha256": "<hex>"}
    },
    "workers_requested": 4,
    "workers_started": 4,
    "memory_mode": "wasm32-shared-fixed",
    "memory_bytes": 1610612736
  },

  "selection": {
    "numerical_mode": "strict-f32-v1",
    "kernel_set": "simd128-threads-v1",
    "quant_layouts": ["q8-block-v1", "q4-block-v1"],
    "kv_format": "f16-or-f32-declared-by-model"
  },

  "self_test": {
    "vector_set_sha256": "<hex>",
    "scalar": "pass",
    "simd128": "pass",
    "atomics": "pass",
    "thread_participation": "pass",
    "tokenizer": "pass",
    "prng": "pass"
  },

  "executed_path_evidence": {
    "scalar_kernel_calls": 3,
    "simd_kernel_calls": 11,
    "worker_ids_participating": [0, 1, 2, 3]
  }
}

Privacy recommendation. Do not request high-entropy user-agent client hints merely to populate the receipt. Record only coarse values already exposed for normal compatibility, and write "unknown" where browser reduction hides OS or architecture. Record the worker count selected by the application, not the device’s advertised total core count. Never record CPU model, clock, GPU identifier, memory capacity, persistent installation ID, or benchmark-derived fingerprint unless a separate, explicit product requirement and privacy review authorize it.

Typed host/Wasm API. Use extern "C" exports, fixed-width integers, no Rust layout-dependent enums, no pointers retained from JavaScript without an owning handle, and no exceptions as expected control flow. For the Wasm32 ABI, all offsets and lengths are u32; a future Memory64 artifact requires an ABI-major change or separate u64 functions.

Generational handles. Encode a handle as a u64, exposed to JavaScript as BigInt, with the upper 32 bits holding a generation and the lower 32 bits an arena-table slot. All operations validate type, slot occupancy, generation, and owner runtime. The WebAssembly JavaScript API has standardized BigInt integration for i64; browser support predates the other features under consideration.

Recommendation. Never expose a raw long-lived tensor pointer to JavaScript. A pointer/length pair is valid only during the synchronous call that produced or consumed it, unless it belongs to a buffer handle whose immobility is guaranteed. This prevents stale pointers after free, arena reset, or any permitted memory growth.

Error taxonomy.

#[repr(i32)]
pub enum TrResult {
    Ok = 0,
    InvalidArgument = 1,
    UnsupportedAbi = 2,
    UnsupportedFeature = 3,
    InvalidHandle = 4,
    StaleHandle = 5,
    Bounds = 6,
    Alignment = 7,
    IntegerOverflow = 8,
    OutOfMemory = 9,
    InvalidModel = 10,
    HashMismatch = 11,
    UnsupportedQuantization = 12,
    UnsupportedShape = 13,
    ContextExceeded = 14,
    Cancelled = 15,
    WorkerFailure = 16,
    SelfTestFailed = 17,
    NumericalFailure = 18,
    TrapRecoveredByHost = 19,
    InternalInvariant = 20
}

Expected data errors must return a code. A Wasm trap or panic indicates an invariant violation or fault-injection event, not an ordinary malformed-model result.

Memory architecture comparison.

Recommended sub-arenas.

linear memory
├── guard / null page
├── runtime metadata and handle table
├── immutable model bytes and tensor directory
├── tokenizer data
├── persistent mutable state
│   ├── sampler state
│   ├── sequence metadata
│   └── KV cache
├── worker control and barriers
├── per-worker stacks and TLS scratch
├── shared operator scratch
├── logits and sampling workspace
├── host streaming-transfer window
└── trailing guard / unused reserve

Rust is memory-safe only to the extent that unsafe slices and pointer calculations are correct. WebAssembly bounds checks prevent escape from linear memory, but they do not prevent one logical tensor from overwriting another inside that memory. Therefore every model-derived product and sum must use checked arithmetic before conversion to an offset.

Memory formulas. Define:

• \(L\): layer count
• \(D\): model width
• \(F\): gated-FFN width
• \(V\): vocabulary size
• \(H_q\): query-head count
• \(H_{kv}\): key/value-head count
• \(d_h\): head dimension
• \(C\): admitted context length
• \(T\): prefill tile token count
• \(N_w\): instantiated worker count
• \(b_a\): activation bytes per element
• \(b_{kv}\): KV-cache bytes per element
• \(Q(X)\): packed bytes plus all quantization metadata for matrix \(X\)

The following formulas are engineering accounting formulas, not claims about the private TinyRustLM model.

A dense decoder layer’s logical weight elements are approximately:

\[ D(H_qd_h)+2D(H_{kv}d_h)+(H_qd_h)D+3DF, \]

where the three FFN matrices represent gate, up, and down projections. Actual packed storage is obtained by applying each matrix’s declared quantization function \(Q\), including tails and metadata.

For tiled prefill, do not allocate a full \(T \times C\) score matrix for every head. A practical upper bound is:

\[ M_{\text{attention-tile}} = N_{\text{active-head-tiles}} \cdot B_q B_k b_a + \text{row maxima} + \text{row sums} + \text{output accumulators}. \]

Peak prefill memory is:

\[ M_{\text{prefill}} = M_{\text{weights}} +M_{\text{tokenizer}} +M_{\text{KV}}(C_{\text{result}}) +M_{\text{prefill activations}}(T) +N_w(S_{\text{stack}}+S_{\text{tls}}) +M_{\text{transfer}} +M_{\text{allocator}}. \]

Steady single-token decode memory is:

\[ M_{\text{decode}} = M_{\text{weights}} +M_{\text{tokenizer}} +M_{\text{KV}}(C_{\text{used}}) +M_{\text{decode scratch}} +M_{\text{logits}} +N_w(S_{\text{stack}}+S_{\text{tls}}) +M_{\text{allocator}}. \]

Recommendation. tr_preflight must return both values and reject a model/context pair before allocating if either exceeds the configured browser budget. Memory accounting must include duplicate feature modules still resident in JavaScript or browser caches only where they are simultaneously instantiated; the loader should release references to failed module instances promptly.

Memory growth and JavaScript views. The JavaScript API refreshes a WebAssembly memory’s buffer on growth and can detach a fixed-length buffer view. Code that caches typed-array views indefinitely is therefore unsafe across growth.

Recommendation.

• Allow growth, if at all, only before model activation.
• Reacquire memory.buffer and rebuild all typed views after every successful growth.
• For the threaded artifact, instantiate shared memory at its exact admitted size with initial == maximum.
• After activation, treat memory size as immutable.
• Place 64-byte or larger padding between worker-written control records to reduce false sharing.
• Use 16-byte alignment for SIMD data and stricter alignment where a packed format requires it, even though WebAssembly permits unaligned loads.

Transformer Reference Algorithms and Accelerated Kernels

Assumed model family. This design supports dense decoder-only transformers with RMSNorm, RoPE, multi-head or grouped-query attention, gated FFNs, residual connections, tied or untied output embeddings, and declared quantization layouts. The private model header must state every dimension and optional behavior; the runtime must not infer architecture from tensor names.

Shape invariants.

\[ D = H_q d_h \]

for the common full-width query layout, unless the model declares an explicit projection width. For grouped-query attention:

\[ H_q \bmod H_{kv}=0,\qquad g=H_q/H_{kv},\qquad h_{kv}=\lfloor h_q/g \rfloor. \]

Every tensor directory entry must satisfy checked:

\[ \text{offset}+\text{stored\_bytes}\leq\text{model\_length}, \]

with no overlapping mutable tensors, no alignment violation, and no multiplication that wraps u32 or usize.

Normative execution graph.

flowchart LR
    A[Token ID] --> B[Embedding lookup]
    B --> C[Residual stream x]
    C --> D[RMSNorm]
    D --> E[Q projection]
    D --> F[K projection]
    D --> G[V projection]
    E --> H[RoPE on Q]
    F --> I[RoPE on K]
    I --> J[Append K to KV cache]
    G --> K[Append V to KV cache]
    H --> L[Causal GQA attention]
    J --> L
    K --> L
    L --> M[Output projection]
    M --> N[Residual add]
    N --> O[RMSNorm]
    O --> P[Gate projection + activation]
    O --> Q[Up projection]
    P --> R[Elementwise product]
    Q --> R
    R --> S[Down projection]
    S --> T[Residual add]
    T --> U{More layers?}
    U -->|Yes| D
    U -->|No| V[Final RMSNorm]
    V --> W[Tied embedding transpose or output projection]
    W --> X[Logits]

RMSNorm and RoPE follow the operators introduced in their original papers, while GQA maps several query heads to fewer key/value heads to reduce KV-cache requirements. Gated FFNs apply an activated projection elementwise with a second projection before the down projection.

Reference operator contracts.

Floating-point reference policy.

1. Store and return model-visible activations as the declared type, initially recommended as f32.
2. Use f32 accumulators in the strict scalar lane when matching existing f32 models, unless authorized golden evidence establishes a different reference.
3. Express accumulation as explicit sum = sum + a * b in fixed order.
4. Do not use mul_add in strict mode.
5. Do not enable fast-math, reassociation, reciprocal approximations, or flush-to-zero.
6. Canonicalize NaNs at runtime boundaries where bitwise golden vectors are required; arithmetic NaN payloads are not a sound portable identity.
7. Treat +0.0 and -0.0 as numerically equal, but specify whether serialized diagnostics canonicalize zero to positive zero.

Recommendation. Avoid dependence on platform libm for exp, sin, cos, and activation functions if native/browser raw-output reproducibility is required. Use one of:

• packed RoPE tables with exact bytes in the model;
• a versioned deterministic range-reduction and polynomial implementation shared by native and Wasm;
• a versioned lookup/interpolation scheme with golden vectors.

The coefficients, evaluation order, range clamps, and exceptional behavior become part of runtime_semantics.

Stable softmax.

For eligible scores \(s_0,\dots,s_{n-1}\):

\[ m=\max_i s_i,\qquad e_i=\operatorname{exp}_{\text{det}}(s_i-m),\qquad z=\sum_i e_i,\qquad p_i=e_i/z. \]

Required cases:

• \(n=0\): return InvalidAttentionRange.
• \(n=1\): return probability exactly 1.0 without calling exponential.
• Any permitted +\infty: distribute according to the explicitly defined infinite-score rule, recommended uniformly among positive infinities.
• NaN score: strict model execution should return a numerical error; sampling separately maps invalid logits according to its contract.
• z == 0, NaN, or infinity after the exponent stage: return NumericalFailure.

Quantized scalar kernels. A quantization descriptor should contain:

struct QuantLayout {
    layout_id: u32,
    elements_per_block: u16,
    stored_bytes_per_block: u16,
    scale_kind: ScaleKind,
    zero_point_kind: ZeroPointKind,
    code_bits: u8,
    code_signed: bool,
    code_order: CodeOrder,
    row_alignment: u16,
    tail_policy: TailPolicy,
}

The packer should canonicalize public model inputs into a small approved set, ideally one q8 and one q4 layout. The runtime should not support arbitrary executable layout logic from a model file.

For each block:

1. validate the block lies entirely within the matrix storage;
2. load scale and zero-point metadata using endian-stable byte decoding;
3. decode exactly min(block_size, remaining_elements) values;
4. accumulate with a declared integer or float equation;
5. never consume padding as a logical weight;
6. apply block scale in a specified order;
7. write the output only after the dot product completes.

Integer-overflow rule. Before choosing an i32 accumulator for integer dot products, prove:

\[ n \cdot \max|a_i| \cdot \max|w_i| \leq 2^{31}-1. \]

Otherwise subdivide the block, widen to i64, or convert partial sums at specified boundaries. Never rely on release-mode wrapping.

SIMD128 q8 design.

For q8×q8 integer dot products:

• load 16 signed 8-bit values from each operand;
• widen low and high halves to i16;
• multiply to i16 or widened i32 according to the available instruction sequence;
• pairwise widen-add into i32x4;
• maintain separate i32x4 accumulators where necessary to avoid overflow;
• reduce lanes in fixed order 0, 1, 2, 3;
• process the final incomplete block with the scalar block routine.

For q8 weights with f32 activations:

• widen groups of four or eight weights;
• convert to f32x4;
• broadcast the scale;
• multiply by activation vectors;
• accumulate with ordinary fixed SIMD multiply and add, not relaxed fused operations;
• reduce lanes in a fixed documented tree;
• accept that the SIMD reduction order differs from left-to-right scalar accumulation and assign an ULP or absolute/relative tolerance rather than claiming bit identity.

SIMD128 q4 design.

• Load packed bytes.
• Obtain low nibbles with x & 0x0f.
• Obtain high nibbles using an unsigned shift.
• Convert each nibble through the layout’s declared signed-code or zero-point rule.
• Widen before subtraction where needed to prevent 8-bit wrap.
• Pair decoded weights with the exact activation positions defined by code_order.
• Use the same accumulator proof as q8.
• Route an odd terminal element through scalar decoding.
• Include vectors whose final dimension is every value from one through at least twice the SIMD block size, ensuring every tail length is exercised.

SIMD floating-point design.

Alignment and bounds. WebAssembly permits unaligned loads, but canonical packer alignment improves implementation freedom and avoids crossing guarded regions. Each SIMD load must be justified by the stored byte extent, not only by the logical tensor dimension. Over-reading up to a vector boundary is forbidden unless the model format declares and hashes that padding.

Local verification required.

• Actual TinyRustLM matrix orientation and quantization layouts.
• Whether activations are f32, f16-converted, or dynamically quantized.
• Exact RMS epsilon and activation function.
• RoPE base, scaling, partial rotary dimension, and long-context policy.
• Whether tied output embeddings must be accessed in a transposed packed layout.
• Existing native output behavior, if any, that should become the initial oracle.

Threads, Scheduling, Cache Identity, and Deterministic Sampling

Thread-pool architecture. Instantiate a persistent pool only after the threaded module and atomic self-tests pass. Each worker receives:

• the same compiled module or byte-identical module hash;
• one shared linear memory;
• a stable worker index;
• a fixed stack/TLS region;
• command and completion slots separated to avoid false sharing;
• an atomic cancellation epoch;
• no authority to allocate from global persistent arenas during a kernel.

The WebAssembly thread feature is widely implemented, but browser access to shared memory depends on cross-origin isolation and real worker creation. Background tabs and mobile resource constraints can suspend or throttle workers, so worker availability is an observed runtime condition rather than a permanent capability.

Partitioning decision.

Recommendation. No floating-point accumulator should be atomically updated by multiple workers. Assign each final output element to exactly one worker. This makes the result independent of worker completion order.

Cancellation. The host increments an atomic cancellation epoch. Long operations snapshot the epoch and check it:

• between transformer layers;
• between matrix row tiles;
• between attention query/head tiles;
• before and after each worker barrier;
• during long tokenizer loops;
• before sampling and state commit.

Cancellation must leave persistent state either entirely unmodified or rolled back to the last committed token. A partially written KV position is marked invalid until a commit flag is atomically published.

Worker failure. A barrier includes generation, expected participant count, completion count, and error code. If a worker terminates or fails to acknowledge a command within the product’s bounded liveness policy, cancel the operation, invalidate the pool, discard uncommitted sequence state, and reinstantiate in a lower lane. Do not continue with fewer workers inside the same reduction or barrier generation.

Prefill versus decode.

Recommendation. Implement two schedulers rather than forcing decode through a batched-prefill path.

For prefill:

1. choose a bounded token tile \(T\);
2. project Q/K/V for the tile;
3. apply RoPE and publish K/V;
4. compute causal attention using tiled online softmax;
5. run output and FFN projections;
6. commit the tile;
7. yield or report progress between tiles.

For decode:

1. compute one normalized residual vector;
2. launch Q/K/V row partitions;
3. append one K/V position;
4. partition attention by query heads;
5. synchronize once before output projection;
6. partition output and FFN rows;
7. produce logits and sample.

Token-by-token prefill is the simplest oracle but should not remain the primary optimized prefill path. Batched/tiled prefill must nevertheless be differentially compared with token-by-token reference prefill because different matrix shapes and softmax tiling can alter arithmetic order.

Fusion policy. Fuse only operations that have a clear normative composite contract and useful memory-traffic reduction. Good candidates include:

• RMSNorm plus projection input scaling, if the fused operation reproduces the declared reference equation;
• q4/q8 decode plus dot product;
• gate activation plus elementwise multiply;
• attention score scaling plus maximum accumulation.

Avoid giant fused layers whose intermediate states cannot be independently tested. Every fused kernel needs an unfused differential oracle.

Cache identity. A reusable prefix cache must be indexed by the entire state-producing identity:

cache_identity =
    hash(
        model_bytes_sha256,
        model_format_version,
        ordered_adapter_identity_list,
        tokenizer_bytes_sha256,
        tokenizer_normalization_version,
        chat_template_sha256,
        exact_prompt_token_prefix,
        BOS/EOS insertion policy,
        positional_encoding_policy,
        context_scaling_policy,
        runtime_semantics_version,
        numerical_mode,
        selected_backend_family,
        KV_element_type,
        KV_layout_version
    )

Exact-prefix rule. A cache entry is reusable only where its stored prompt tokens are an exact prefix of the requested tokens and every other identity field matches. Text equality is insufficient because tokenization, template insertion, normalization, or special-token policy can differ.

Composition-bound rule. Ordered adapters are part of the identity. [adapter_A, adapter_B] is not the same as [adapter_B, adapter_A], and adding, removing, or changing any adapter invalidates reuse. The same applies to a change from scalar strict mode to a backend whose numerical contract may produce different K/V values.

Recommendation. Do not reuse K/V across CPU and WebGPU modes unless a separate conformance program proves that their serialized K/V representations and arithmetic contract are cache-compatible. Default to backend-bound cache identity.

Deterministic sampling contract. Use a fully specified, integer-only PRNG rather than an operating-system RNG after initialization. PCG32 is a suitable compact choice, but its exact transition and output function must be frozen in TinyRustLM’s contract.

Example state:

#[repr(C)]
struct Pcg32State {
    state: u64,
    increment: u64 // must be odd
}

Transition:

old = state
state = old * 6364136223846793005 + increment  (mod 2^64)
xorshifted = ((old >> 18) XOR old) >> 27
rot = old >> 59
output = rotate_right(u32(xorshifted), u32(rot))

Seed interpretation:

state = 0
increment = (stream_id << 1) | 1
advance once
state += seed interpreted as little-endian u64
advance once

Convert a random word to a portable f32 uniform value using the high 24 bits:

\[ u = (r \gg 8)\cdot2^{-24}. \]

This conversion is exactly representable and yields \(0 \le u < 1\).

Sampling order.

1. Copy or reference logits in token-ID order.
2. Apply forbidden-token mask by replacing values with negative infinity.
3. Apply repetition penalty under an exact formula:

• recommended common rule: positive repeated logits divide by penalty; negative repeated logits multiply by penalty;
• reject nonpositive penalty.

1. Canonicalize all NaNs to negative infinity.
2. Canonicalize negative zero to positive zero for comparison and diagnostics.
3. If temperature == 0, choose the greatest logit with lowest token ID as the tie-break.
4. Otherwise divide by positive finite temperature.
5. Apply top-k using descending logit and ascending token ID for ties.
6. Compute stable probabilities over surviving tokens.
7. Sort or retain candidates in descending probability, ascending token ID.
8. For top-p, retain the minimal ordered prefix whose cumulative probability is at least \(p\); always retain at least one token.
9. Draw using a cumulative scan in that same order.
10. Return the final token and serialized PRNG state.
11. If the token is in the declared EOS set, emit the token according to the stream contract and then mark the sequence complete.

Local verification required. The project must decide whether the initial native behavior already has a PRNG and sampling convention that users expect. If so, changing it can alter conversation behavior even when the new policy is “more deterministic”; migration requires explicit product approval and raw-output comparison.

Compiler Controls, Instruction Verification, Startup Tests, and Numerical Conformance

Toolchain pinning. Freeze exact Rust, LLVM, linker, Binaryen if used, and inspection-tool versions in a build manifest. The Rust compiler documents target features through -C target-feature and #[target_feature]; its current Wasm target documentation distinguishes baseline, SIMD, atomics, exception handling, multiple memories, relaxed SIMD, and tail calls.

Recommended production profiles.

[profile.release]
opt-level = 3
lto = "fat"
codegen-units = 1
panic = "abort"
overflow-checks = true
debug = 0
strip = "symbols"
incremental = false

Use explicit wrapping_* only where modulo arithmetic is part of the contract, such as the PRNG. Use checked_* for model-derived indices and lengths. Global overflow checks may later be performance-reviewed, but disabling them must not change any validation-path behavior.

Recommended artifact feature controls.

scalar:
  target = wasm32v1-none or frozen wasm32-unknown-unknown configuration
  target-cpu = mvp
  enabled = only reviewed baseline features

simd128:
  scalar features
  +simd128

simd128-threads:
  simd features
  +atomics
  +bulk-memory
  imported shared memory
  initial == maximum

Do not compile the scalar artifact with SIMD and merely promise not to execute it. Conversely, assert that the SIMD artifact actually contains expected vector opcodes.

Native strict lane.

• Build explicit targets such as x86_64-pc-windows-msvc, x86_64-unknown-linux-gnu, and aarch64-unknown-linux-gnu.
• Do not enable CPU-wide -C target-cpu=native for reproducible release artifacts.
• Keep scalar oracle code free of platform intrinsics.
• Avoid mul_add and fast-math in strict mode.
• Verify that the final native code did not contract strict multiply/add sequences or reassociate reductions.
• Give any native SIMD path a capability and numerical-mode identity separate from the scalar oracle.

Reproducible paths and metadata.

• Use --remap-path-prefix or equivalent to remove developer-machine paths.
• Set a controlled SOURCE_DATE_EPOCH where tooling honors it.
• Generate the public build ID as a SHA-256 of the canonical manifest plus final module bytes.
• Keep names, DWARF, and source maps out of the public production module.
• Publish debug artifacts only through access-controlled channels, with their own hashes.
• Review source maps for repository paths, source text, model names, and unpublished feature identifiers.

Post-link optimization. Binaryen provides deterministic Wasm transformation and optimization tools, including wasm-opt, but some options can change floating behavior or assumptions. It also notes that its optimizer has fast-math-related controls and transforms the final module.

Recommendation. Start without a post-link optimizer. Admit a pinned wasm-opt pipeline only after:

1. final bytes are separately hashed;
2. feature sections and opcode inventory remain correct;
3. scalar and accelerated golden vectors remain within their tiers;
4. traps and bounds behavior remain unchanged;
5. load, compile, and execution measurements justify the added tool;
6. the pipeline contains no fast-math option.

Instruction verification. CI should parse final modules, not rely on compiler flags.

Use at least two independent tools where practical: one to validate and disassemble, and another small in-repository parser that inventories section types, target-feature declarations, imports, memories, and opcodes. This guards against misunderstanding a single tool’s feature defaults.

Startup self-tests. Self-tests must be bounded, deterministic, and specific to the selected artifact.

Bound startup cost by using tiny dimensions, for example \(D \le 32\), a few quant blocks, and one short worker command. Record duration but do not fail merely because a slow device exceeds an aggressive performance expectation; fail only on a generous liveness timeout or incorrect result.

Conformance hierarchy.

Recommended assignments.

Golden-vector record.

{
  "schema": "tinyrustlm.golden-vector.v1",
  "runtime_semantics": 1,
  "operator": "q4_dot_f32",
  "layout": "q4-block-v1",
  "shape": {"rows": 3, "cols": 19},
  "input_sha256": "<hex>",
  "expected": {
    "kind": "f32_bits",
    "values": ["0x...", "0x...", "0x..."]
  },
  "alternate_tolerance": {
    "abs": 1e-6,
    "rel": 1e-5,
    "max_ulp": 4
  },
  "provenance": {
    "oracle": "scalar-rust-v1",
    "source_revision": "<commit>",
    "generated_utc": "<date>"
  }
}

Golden vectors must include raw inputs or content-addressed fixtures, not only expected summaries.

Measurement, Platform Matrix, Fault Injection, and Replacement Roadmap

Measurement principle. Performance claims are valid only for the exact module, model, browser build, operating system, architecture, worker count, context, decoding policy, and measurement procedure represented in a receipt. WebAssembly’s portability goals do not make throughput portable between machines, and browser implementations may impose resource limits below specification maxima.

Required phases.

Browser timing caveat. Browser wall time is observable through web performance APIs; true per-process CPU time and thermal state are generally not available consistently. Cross-origin isolation can expose higher-resolution timing, but the receipt should state whether timing precision was reduced.

Report these conditions rather than hiding them:

• foreground or background page state;
• plugged-in versus battery if voluntarily known, without fingerprinting;
• warm-up count and run order;
• module and model cache state;
• context length and prompt token count;
• exact sampler configuration;
• actual worker count;
• whether the run was interrupted, throttled, or experienced page visibility changes;
• raw generated token IDs and final text;
• peak Wasm memory by planned arena accounting;
• host-side transfer buffers and concurrently live module instances.

Do not claim CPU utilization or thermal stability when the browser does not expose trustworthy measurements.

Cross-browser and platform matrix.

Cross-compilation in CI establishes that an artifact was produced. It does not establish that a browser accepted it, that a worker started, that the intended instructions executed, or that a physical arm64 target produced conforming outputs.

Test dimensions.

• Context: zero/invalid, one token, ordinary, maximum admitted, and one beyond maximum.
• Shapes: dimensions below, equal to, and above each SIMD width; every quant block tail.
• Model: tied/untied embeddings, MHA/GQA, each supported quant format, each supported context policy.
• Memory: exact fit, one byte/page short, fragmented sequence-state churn, repeated load/free, and browser restart.
• Scheduling: one worker, maximum admitted workers, failed worker creation, worker death, and cancellation at every barrier.
• Cache: exact prefix, near-prefix mismatch, tokenizer change, adapter reordering, numerical-mode change, and KV-format change.
• Sampling: all ties, all masked, NaNs, infinities, negative zero, EOS alternatives, temperature zero, and top-p boundary equality.

Fault-injection matrix.

Public-safe diagnostics. Expose codes, runtime and ABI versions, artifact hash prefix, feature observations, self-test identifier, selected lane, memory plan, worker count, and failing operator/test ID. Do not expose stack contents, model tensors, prompt text, token history, raw source paths, or detailed device identifiers by default.

TDD and clean replacement sequence.

gantt
    title TinyRustLM Runtime Replacement Program
    dateFormat  YYYY-MM-DD
    axisFormat  %b %d

    section Contract
    Freeze model and ABI schemas           :a1, 2026-08-03, 10d
    Build scalar tensor and tokenizer oracles :a2, after a1, 15d

    section Browser Baseline
    Wasm32 scalar host boundary            :b1, after a1, 15d
    Model validation and memory preflight  :b2, after b1, 10d
    Scalar end-to-end differential lane    :b3, after a2, 15d

    section Acceleration
    SIMD q8/q4 and floating kernels        :c1, after b3, 20d
    SIMD startup tests and promotion gate  :c2, after c1, 10d
    Thread pool and deterministic scheduler :c3, after c2, 20d

    section Qualification
    Browser and native matrix              :d1, after c3, 20d
    Fault injection and memory-pressure tests :d2, after c3, 15d
    Performance and raw-output qualification :d3, after d1, 15d

    section Cleanup
    Default-lane promotion                 :e1, after d3, 5d
    Delete superseded unpublished paths    :e2, after e1, 5d

The dates are illustrative, not a promise of future asynchronous work.

Promotion gates.

Rollback thresholds. Immediately disable a lane when any of these occurs:

• invalid model input reaches a trap rather than a typed rejection;
• a startup self-test fails;
• selected-path evidence contradicts the receipt;
• memory accounting underestimates peak usage;
• deterministic fixtures differ outside their tier;
• raw conversation output exceeds the approved regression budget;
• worker failure can leave apparently valid but partially modified state;
• a browser update changes capability or numerical behavior without a receipt/version distinction.

Deletion rule. Superseded unpublished paths should be deleted after:

1. the new scalar core handles every admitted model shape;
2. end-to-end native and browser differential tests pass;
3. SIMD and threads have independent rollback flags;
4. at least one full browser-process-restart qualification cycle passes;
5. raw evidence and build artifacts are archived;
6. no packer emits the old format;
7. the release branch contains no fallback call into the old execution path.

Do not retain an undocumented legacy implementation “just in case.” Retain fixtures and archived source revisions, not live divergent semantics.

Effort ranges. These are engineering-planning estimates based on scope, not measured TinyRustLM productivity.

Unknown private format complexity, tokenizer behavior, quantization diversity, and availability of physical Safari/arm64 test machines can materially change these ranges.

Deliverable formats.

A useful memory chart should show separate stacked values for weights, tokenizer, KV, activations, workers, transfer chunks, and allocator overhead at increasing contexts. A latency chart should report prefill and per-token decode separately rather than combining them into one tokens-per-second value.

Unknowns, Quality Checklist, and Annotated Primary Sources

Unknowns requiring authorized local evidence.

Research and source-priority table.

Quality, bias, and reproducibility checklist.

• [ ] Every claim is labeled public fact, inference, recommendation, assumption, or local-verification requirement.
• [ ] Standardized status is separated from browser implementation and from product enablement.
• [ ] Browser support is confirmed by direct execution, not only a compatibility table.
• [ ] Capability probes are versioned and content-hashed.
• [ ] A successful probe is not treated as proof that the selected kernel executed.
• [ ] The receipt records module hash, build features, self-test results, and path counters.
• [ ] Threads are never claimed when crossOriginIsolated is false.
• [ ] Worker creation, shared allocation, barriers, and participation are independently tested.
• [ ] Memory totals include weights, metadata, tokenizer, KV, scratch, stacks, TLS, transfers, duplicate live instances, and allocator overhead.
• [ ] Peak prefill and steady decode are measured separately.
• [ ] Every shape multiplication and offset addition is checked.
• [ ] Every supported quant layout has all tail lengths in its fixture set.
• [ ] Scalar loop order, exceptional values, and approximation functions are versioned.
• [ ] No fast-math or relaxed SIMD enters a strict artifact.
• [ ] Final module opcodes are inspected independently of build flags.
• [ ] Native scalar binaries are checked for contraction/reassociation assumptions.
• [ ] Tokenizer, chat template, adapter order, numerical mode, and KV layout are in cache identity.
• [ ] Sampling specifies PRNG state, tie order, masks, NaNs, signed zero, EOS, top-k, and top-p.
• [ ] Raw token IDs are retained in differential evidence.
• [ ] Performance results identify browser, OS, architecture, module/model hashes, context, worker count, and cache state.
• [ ] No third-party benchmark is presented as TinyRustLM performance.
• [ ] Cross-compilation is never represented as physical execution.
• [ ] Failure-injection tests prove no partially committed token or KV state remains valid.
• [ ] Source maps and debug artifacts are reviewed for private paths and source disclosure.
• [ ] Every release can be reconstructed from pinned tools, manifests, and source revision.
• [ ] Superseded unpublished paths are removed after promotion rather than maintained indefinitely.

Sample annotated bibliography entry format.

WebAssembly Working Group. “WebAssembly Core Specification, Release 3.0.” July 2026. Retrieved August 1, 2026.

Type: Normative/editorial primary specification.

Use in this report: Core instruction, validation, execution, memory, vector, trap, and implementation-limit semantics.

Key limitation: The online editor’s draft can change and does not alone establish that a particular browser implements every feature. Pair with the implementation matrix and direct execution.

Pinned evidence: Archive the exact retrieved HTML or repository revision and its SHA-256 in the project research ledger.

Annotated primary-source bibliography.

1. WebAssembly Working Group, WebAssembly Core Specification, Release 3.0, July 2026; retrieved August 1, 2026. The normative foundation for module validation, execution, numerical instructions, memories, vectors, references, exceptions, and traps. The site labels the current online material as release 3.0, while the W3C-facing editor copy cautions that an editor’s draft can change.

1. WebAssembly Project, Wasm 3.0 Completed, September 17, 2025; retrieved August 1, 2026. Primary project announcement summarizing the integration of Memory64, multiple memories, typed references, tail calls, exception handling, and relaxed vector instructions. Useful for release context, not a substitute for opcode-level specification.

1. WebAssembly Project, Wasm 2.0 Completed, March 20, 2025; retrieved August 1, 2026. Primary project account of fixed-width SIMD, bulk memory, multivalue, reference types, and related additions.

1. WebAssembly Project, Feature Status, live table retrieved August 1, 2026. The principal cross-engine implementation matrix used here. It distinguishes proposal phases and reports first browser versions, but still requires local browser tests because live support can depend on build, platform, flags, and resource constraints.

1. WebAssembly Working Group, WebAssembly JavaScript Interface, July 2026; retrieved August 1, 2026. Primary source for WebAssembly.validate, modules, instances, memories, BigInt integration, buffer refresh and detachment, limits, and JavaScript/Wasm value conversion.

1. WebAssembly Working Group, WebAssembly Web API, July 2026; retrieved August 1, 2026. Primary source for browser streaming integration, MIME type, security context, and web embedding.

1. Rust Project, @@MKREPORTTOKEN0@@ Target Documentation, retrieved August 1, 2026. Primary compiler documentation for a conservative Wasm target and its relationship to later target features. It supports the recommendation to freeze the baseline rather than inherit evolving defaults.

1. Rust Project, Code Generation Attributes and Wasm Target Features, retrieved August 1, 2026. Primary language/compiler reference for #[target_feature], SIMD128, relaxed SIMD, and the load-failure behavior of unsupported Wasm features.

1. Rust Project, rustc Codegen Options, retrieved August 1, 2026. Primary reference for LTO, embedded bitcode, overflow checks, panic behavior, optimization, symbols, and other release controls.

1. MDN Web Docs, @@MKREPORTTOKEN0@@, updated March 2025; retrieved August 1, 2026. Browser-platform documentation consolidating the observable isolation state, COOP/COEP requirements, permissions policy, and SharedArrayBuffer access. It should be read with the underlying HTML and security specifications and confirmed against production responses.

1. Chrome for Developers, SharedArrayBuffer Updates and Cross-Origin Isolation, January 18, 2021, with later update; retrieved August 1, 2026. Browser-vendor explanation of why SharedArrayBuffer requires cross-origin isolation and how COOP, COEP, CORS, and CORP affect deployment.

1. GPU for the Web Working Group, WebGPU Specification, editor’s draft retrieved August 1, 2026. Primary specification for GPU devices, queues, buffers, mapping, validation, errors, security, and timing. It supports treating WebGPU as a distinct backend with explicit resource movement rather than a Wasm target feature.

1. W3C, WebGPU Publication History, retrieved August 1, 2026. Primary standards-status record showing WebGPU’s 2026 Candidate Recommendation Draft publications.

1. Biao Zhang and Rico Sennrich, Root Mean Square Layer Normalization, 2019; retrieved August 1, 2026. Original RMSNorm paper. It defines the normalization family but does not determine TinyRustLM’s exact epsilon, accumulator precision, or exceptional-value policy.

1. Jianlin Su et al., RoFormer: Enhanced Transformer with Rotary Position Embedding, 2021; retrieved August 1, 2026. Original RoPE paper. Local metadata must still specify rotary dimension, base, scaling, table generation, and long-context behavior.

1. Joshua Ainslie et al., GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints, 2023; retrieved August 1, 2026. Original GQA work, useful for the query-to-KV-head relationship and KV-cache motivation. It does not prescribe TinyRustLM tensor layout.

1. Noam Shazeer, GLU Variants Improve Transformer, 2020; retrieved August 1, 2026. Original paper covering gated FFN variants including SwiGLU-related forms. The model header must identify the exact activation and bias policy.

1. WebAssembly Project, Binaryen, revision to be pinned at adoption; retrieved August 1, 2026. Primary tool repository for wasm-opt, parsing, printing, transformations, and deterministic tool behavior. Use only through a pinned and numerically qualified pipeline.

1. WebAssembly Project, Portability, retrieved August 1, 2026. Primary project documentation on hardware assumptions, endianness, addressing, floating point, atomics, and the separation between the core ISA and host APIs.

1. WebAssembly Project, Security, retrieved August 1, 2026. Primary project overview of sandboxing, bounds checks, structured control flow, and embedding policies. It is important to distinguish Wasm sandbox safety from logical memory safety inside TinyRustLM’s own arenas.