WebGPU Quantized Transformer Backend for TinyRustLM: Architecture, Admission, Memory Planning, and Recovery

Full report

On this page

Executive Summary

The attached technical brief resolves the otherwise unspecified research scope: this report evaluates an optional WebGPU execution backend for qualified TinyRustLM .slm models, while preserving the existing artifact, tokenizer, chat-template, sampling, model-composition, and behavioral contract shared by native and browser-CPU paths. No private TinyRustLM source code, artifacts, shaders, traces, browser profiles, hardware, or benchmark receipts were available; every TinyRustLM-specific design below is therefore an engineering recommendation or assumption requiring local verification, not an observation about the unpublished implementation.

Adoption recommendation. TinyRustLM should proceed with WebGPU only as a strictly admitted, replaceable acceleration backend, not as an alternative model runtime. The recommended first production scope is deliberately narrow:

1. Retain Rust/WASM control of artifact validation, tokenizer and chat-template processing, prompt-token construction, sampling state, output commitment, cancellation epochs, and user-visible generation state.
2. Give WebGPU ownership only of validated tensor buffers, explicitly planned activation and KV-cache allocations, trusted precompiled shader families, command encoding, and numerical results.
3. Support a conservative f32-accumulating path first, followed by q8 and q4 weight-only kernels. Treat shader-f16 and subgroups as optional optimization tiers rather than prerequisites.
4. Admit a model only after deterministic resource planning, adapter-feature and limit checks, atomic pipeline preparation, bounded operator self-tests, allocation probes, and registration of device-loss and error handling.
5. Make CPU fallback a visible state transition governed by a token-commit ledger. Never race CPU and GPU answers, silently restart a response, or choose whichever output appears more attractive.
6. Delete, rather than preserve, shader variants that cannot meet correctness, security, memory, or end-to-end performance gates.

This recommendation reflects WebGPU’s current maturity: as of August 1, 2026, it is broadly shipped across Chromium on mainstream Windows, macOS, ChromeOS, substantial Android configurations, and selected Linux configurations; Firefox ships it on Windows and Apple-silicon macOS but has narrower stable support elsewhere; Safari 26 ships WebGPU on current Apple platforms. Nevertheless, feature exposure, worker support, limits, drivers, and backend behavior remain browser- and platform-dependent, and WebGPU and WGSL remain W3C Candidate Recommendation Drafts rather than final Recommendations.

Prioritized research and implementation workstreams

Scores are on a five-point scale. “Feasibility” estimates implementation tractability without private TinyRustLM evidence; “novelty” measures project-specific research value rather than academic originality.

Principal conclusion. The project’s greatest risk is not whether a q4 multiply can be written in WGSL. It is whether TinyRustLM can preserve one validated model identity and one output-commit contract across asynchronous GPU errors, heterogeneous numerical behavior, browser resource limits, cancellation, and loss. Kernel optimization should begin only after that contract exists.

Relevant five-year development timeline

timeline
    title WebGPU and transformer-inference developments relevant to TinyRustLM
    2021 : RoFormer formalizes rotary position embedding
    2022 : FlashAttention introduces IO-aware exact attention
         : GPTQ advances practical low-bit transformer weight quantization
    2023 : Chrome 113 ships WebGPU on major desktop configurations
         : GQA paper formalizes grouped-query attention tradeoffs
    2024 : W3C publishes a WebGPU Candidate Recommendation Snapshot
         : WebLLM documents production-oriented in-browser WebGPU inference
    2025 : Firefox 141 ships WebGPU on Windows
         : Safari 26 enables WebGPU across current Apple platforms
         : Chromium exposes subgroup functionality on supported configurations
    2026 : Chromium expands stable Linux support
         : Compatibility-mode deployment expands on GLES-class devices
         : WebGPU and WGSL Candidate Recommendation Drafts continue to evolve

The academic milestones are documented by the original RoFormer, FlashAttention, GPTQ, and GQA papers. Browser rollout dates come from the GPUWeb implementation-status record and browser-vendor publications; the timeline should not be interpreted as proof that every optional feature is available on every listed platform.

Standards, Browsers, Platforms, and Capability Matrix

Standards baseline and normative implications

Observed public fact. The latest retrieved W3C WGSL publication is a Candidate Recommendation Draft dated June 5, 2026. The WebGPU publication history records multiple Candidate Recommendation Drafts through May 21, 2026. These documents are active standards-track specifications, and implementation details can still change; TinyRustLM should therefore pin a backend protocol version, shader-bundle hash, CTS revision, and browser test baseline rather than assuming “WebGPU” denotes one permanently stable implementation.

WebGPU is exposed in secure contexts and may be available through both Navigator.gpu and WorkerNavigator.gpu; worker availability remains a browser-deployment question rather than a universal assumption. Adapter features and limits are deliberately controlled or bucketed by user agents for privacy, and underlying hardware capability does not guarantee that a browser exposes a corresponding feature.

Optional capabilities relevant to this backend include shader-f16, subgroups, and timestamp-query. They must be discovered from the adapter and explicitly requested when needed; only requested features appear on the resulting device. Unsupported required capabilities cause device creation to fail.

WebGPU’s default capability profile includes a 256 MiB maxBufferSize, 128 MiB maxStorageBufferBindingSize, four bind groups, 256 compute invocations per workgroup, and at least 16 KiB of workgroup storage. Browsers may expose larger values, but applications must use the admitted device’s actual limits and must not assume the physical GPU’s native maximum.

Browser, operating-system, and API-backend matrix

The following matrix records the public implementation status retrieved August 1, 2026. “Runtime-gated” means TinyRustLM must query the adapter and run its self-test rather than infer support from browser name.

Engineering recommendation. The product compatibility table should contain only broad shipping expectations. The executable source of truth must be the versioned admission receipt, because browser features may be withheld even where the hardware supports them, and numeric limits may be privacy-bucketed.

Feature policy

Admission, Privacy Receipt, and Typed Runtime Boundary

Versioned backend-admission algorithm

Engineering recommendation. navigator.gpu is only the beginning of discovery. Admission should be a transaction in which no GPU session becomes active until the artifact, memory plan, shaders, allocations, pipelines, and numerical self-tests have all passed.

Admission protocol: tiny-gpu-admission-v1

Input:
    ValidatedModelManifest manifest
    BackendPolicy policy
    CancellationEpoch epoch

Output:
    AdmittedGpuSession | AdmissionFailure

Procedure:
    1. Reject if the secure-context WebGPU entry point is absent.
    2. Validate the entire .slm artifact in Rust/WASM:
         - magic/version/checksum
         - tensor type, rank, shape, offsets, lengths, alignment
         - quantization descriptor and every tail block
         - architectural invariants
         - RoPE/GQA/context parameters
         - checked arithmetic for all byte and element counts
    3. Derive immutable:
         ModelIdentity
         ExactGpuRequirements
         DeterministicMemoryPlan
         RequiredPipelineSet
    4. Request one adapter:
         featureLevel = policy.feature_level
         forceFallbackAdapter = false
         powerPreference = absent unless user policy explicitly selects one
    5. Reject null, expired, disallowed fallback, or cancelled adapter.
    6. Compare requirements against adapter.features and adapter.limits.
       Select the lowest capability kernel tier satisfying the model:
         Tier A: f32 baseline
         Tier B: f16 storage with f32 accumulation
         Tier C: admitted subgroup optimization
    7. Request exactly the selected features and required limits.
    8. Immediately register:
         device.lost handler
         uncapturederror handler
         generation/session epoch
    9. Push validation and out-of-memory error scopes.
    10. Allocate all mandatory buffers according to the plan.
        Use bounded upload chunks and no unbounded "find free memory" loop.
    11. Compile every mandatory pipeline asynchronously from trusted WGSL.
    12. Run bounded golden tests:
          buffer addressing and odd tails
          q8 and q4 unpack/dequantization
          f32 and optional mixed-precision dot products
          RMSNorm
          RoPE
          stable/online softmax
          GQA head mapping
          GEMV and small GEMM
          NaN, infinity, signed zero, and cancellation handling
    13. Compare with an independent CPU oracle using versioned tolerances.
    14. Pop all scopes and reject on any captured or uncaptured error.
    15. Atomically publish the session handle only if:
          epoch is current
          model identity is still active
          all resources and pipelines exist
          device is not lost
          every mandatory self-test passed
    16. Emit a privacy-minimized receipt.

WebGPU’s design supports this pattern: unsupported required features fail device creation; devices enforce requested limits; allocation failure can generate an out-of-memory error and invalid resource; asynchronous pipeline creation can fail; and the device-loss promise must be handled because a device can be lost at creation or later.

Exact admission outcomes

Privacy-minimized capability receipt

Engineering recommendation. Keep detailed information in memory for scheduling, but persist only coarse evidence:

CapabilityReceiptV1 {
    backend_protocol_version
    runtime_build_hash
    shader_bundle_hash
    self_test_suite_hash
    model_artifact_hash              // local identity, not transmitted by default
    model_layout_profile             // coarse q4/q8/fp profile
    browser_engine_major_bucket
    operating_system_family_bucket
    feature_level                    // core | compatibility
    enabled_feature_bits             // only features actually requested
    admitted_limit_profile_id        // project-defined bucket, not raw full limits
    fallback_adapter                 // boolean
    outcome_code
    failure_stage
    receipt_day                      // day-level, not high-resolution timestamp
}

The receipt must contain no prompts, generated text, token IDs, raw adapter description, driver string, precise device identifier, high-resolution timestamps, exact memory-probe frontier, or shader/compiler diagnostic text. The WebGPU specification deliberately restricts machine-specific capability distinguishability and cautions that device-loss messages should not be parsed or disclose unnecessary system information.

Ownership and command boundary

flowchart LR
    UI[Browser UI] -->|User action only| HOST[Rust/WASM Host]
    HOST -->|Validated opaque commands| GPU[WebGPU Worker Backend]
    GPU -->|Numerical result tagged with epoch| HOST
    HOST -->|Committed text events| UI

    ART[Untrusted .slm bytes] --> VALID[Artifact validator]
    VALID -->|Immutable manifest and tensor descriptors| HOST
    VALID -. no raw offsets from JS .-> GPU

    HOST --> TOK[Tokenizer and chat template]
    HOST --> SAMP[Sampler and RNG state]
    HOST --> LEDGER[Token commit ledger]
    GPU --> RES[Weights, activations, KV cache, pipelines]

A minimal typed interface should resemble:

struct ModelHandle(u64);
struct SessionEpoch(u64);
struct GenerationEpoch(u64);
struct SubmissionSeq(u64);

enum GpuCommand {
    LoadValidatedModel {
        model: ModelHandle,
        manifest_hash: [u8; 32],
        plan_hash: [u8; 32],
        epoch: SessionEpoch,
    },
    BeginGeneration {
        model: ModelHandle,
        session: SessionEpoch,
        generation: GenerationEpoch,
        prompt_tokens: Box<[u32]>,
        sampler_checkpoint_hash: [u8; 32],
    },
    PrefillChunk {
        generation: GenerationEpoch,
        start_position: u32,
        tokens: Box<[u32]>,
        submission: SubmissionSeq,
    },
    DecodeOne {
        generation: GenerationEpoch,
        committed_position: u32,
        input_token: u32,
        submission: SubmissionSeq,
    },
    Cancel { generation: GenerationEpoch },
    DestroySession { session: SessionEpoch },
}

enum GpuResult {
    SessionAdmitted { session: SessionEpoch, receipt: CapabilityReceiptV1 },
    PrefillComplete { generation: GenerationEpoch, through_position: u32 },
    Logits {
        generation: GenerationEpoch,
        position: u32,
        submission: SubmissionSeq,
        values: Box<[f32]>,
    },
    Cancelled { generation: GenerationEpoch },
    Failed {
        generation: Option<GenerationEpoch>,
        code: GpuFailureCode,
        last_completed_submission: Option<SubmissionSeq>,
    },
}

Security invariant. The GPU worker may accept an opaque tensor ID and operation ID, but never a JavaScript-provided byte offset, row stride, quantization block size, dispatch count, or shader fragment.

Transformer Graph and Kernel Architecture

Complete decoder-only graph

For each layer, the recommended operation sequence is:

\[ x_0 \rightarrow \operatorname{RMSNorm} \rightarrow (Q,K,V) \rightarrow \operatorname{RoPE}(Q,K) \rightarrow \operatorname{KVAppend} \rightarrow \operatorname{CausalGQAAttention} \rightarrow W_O \rightarrow \operatorname{Residual} \]

\[ \rightarrow \operatorname{RMSNorm} \rightarrow \left[\operatorname{SiLU}(W_{\mathrm{gate}}x)\odot (W_{\mathrm{up}}x)\right] \rightarrow W_{\mathrm{down}} \rightarrow \operatorname{Residual} \]

The final sequence is final RMSNorm, tied or untied output projection, logits extraction, and return to the canonical sampler. RMSNorm, RoPE, and GQA are grounded in the corresponding original publications; FlashAttention and online-softmax work provide the conceptual basis for tiled, numerically stable attention, although a WebGPU implementation should not claim equivalence to any named CUDA kernel without direct evidence.

Validation rules before dispatch should include:

\[ d_{\text{model}} = n_Q d_{\text{head}}, \qquad n_Q \bmod n_{KV} = 0, \]

plus exact compatibility among every projection matrix, RMSNorm vector, FFN width, vocabulary size, RoPE dimension, context limit, and tied-embedding shape. For GQA, query head \(h_q\) maps to:

\[ h_{kv} = \left\lfloor \frac{h_q n_{KV}}{n_Q} \right\rfloor . \]

The mapping must be computed from validated unsigned dimensions using checked arithmetic; malformed dimensions must be rejected rather than clamped.

Prefill and decode require different kernels

A single “universal” matrix kernel is unlikely to be competitive. Prefill benefits from two-dimensional output tiling and reuse of both activations and dequantized weight blocks. Decode typically has one input row, so aggressive matrix tiles leave most lanes idle and can spend more time coordinating than computing.

Recommended fusion boundaries. Fuse operations only when doing so eliminates a full memory round trip without causing unacceptable registers, workgroup storage, code size, or variant explosion. Good early candidates are residual-plus-RMSNorm, Q/K RoPE after a fused QKV projection, gate-plus-up projection, and online-softmax attention. Avoid a monolithic full-layer shader: it would produce large live ranges, impede cancellation boundaries, increase compiler risk, and complicate numerical localization.

Quantized tensor descriptors

No universal .slm q4 or q8 encoding should be invented. The validator should map each artifact tensor into a canonical descriptor:

struct QuantTensorDesc {
    tensor_id: u32,
    encoding: QuantEncoding,       // F32, F16, Q8, Q4
    rows: u32,
    cols: u32,
    block_elements: u32,
    block_stride_bytes: u32,
    row_stride_bytes: u64,
    payload_offset: u64,
    payload_length: u64,
    scale_layout: ScaleLayout,     // inline, side array, per-row, per-block
    scale_type: ScaleType,         // f16 or f32
    zero_point_layout: ZeroPointLayout,
    signedness: QuantSignedness,
    nibble_order: Option<NibbleOrder>,
    tail_elements: u32,
    required_alignment: u32,
    content_hash: [u8; 32],
}

A q8 block may be modeled generally as integers \(q_i\), scale \(s_b\), and optional zero point \(z_b\):

\[ w_i = s_b(q_i-z_b). \]

For q4, the descriptor must explicitly identify whether a byte stores low-nibble-first or high-nibble-first values, and whether the four-bit code is offset-binary, signed two’s-complement-like, or interpreted through another artifact-defined map. The final partial block must have an authenticated tail count; padding bits must never be interpreted as weights.

Storage layout recommendation. Normalize metadata, not payload semantics. It is acceptable to repack a validated artifact into a GPU-specific buffer layout during loading, provided the operation is deterministic, covered by a checksum, and does not change quantized values or scales. Record both the artifact hash and the normalized GPU-layout hash.

Quantized GEMV and GEMM families

A viable vendor-neutral family should contain multiple fixed shader variants rather than runtime-generated WGSL:

Load and reduction policy. Use aligned vec4<u32> or equivalent loads only when the descriptor proves alignment. Dequantize as close as practical to multiply-accumulate so that expanded floating-point weights are not written to global memory. Reuse each scale across its block. Accumulate in f32 by default. Apply bounds masks to rows, columns, and K tails, and guarantee that every workgroup barrier is reached uniformly.

Workgroup selection. Start with fixed 64-, 128-, and 256-invocation variants. Admission microtests should choose only among a small allowlisted set; they must not expose arbitrary shader generation. Workgroup-storage demand must remain below the admitted maxComputeWorkgroupStorageSize, and invocation dimensions must fit every per-axis and total-workgroup limit. The standards minimum for compute invocations is 256 and for workgroup storage is 16 KiB, but larger values must not be assumed.

Register pressure and bank behavior. WGSL does not provide a portable register-count or shared-memory-bank model. Therefore, bank-conflict avoidance and occupancy claims require compiler inspection and physical measurements for each browser/backend family. Keep tile sizes as specialization constants selected from an allowlist, inspect generated backend code where authorized, and maintain a conservative variant with smaller accumulator arrays.

Floating-point policy

WGSL does not promise cross-vendor bit identity for ordinary floating-point shader arithmetic. Implementations may differ in contraction, intermediate rounding, subnormal handling, NaN representation, and optimization, especially when f16 is enabled. The WGSL standard defines allowed behavior but does not convert heterogeneous GPUs into a single deterministic floating-point machine.

Recommendation. Do not promise bit-exact GPU logits or hidden states. Promise an explicitly versioned hierarchy of numerical and behavioral conformance. Integer unpacking, layout mapping, token bookkeeping, model identity, and commit ordering can and should remain exact.

RoPE and attention kernels

RoPE parameters—including base frequency, rotary dimension, interleaving convention, position offset, and any context-scaling variant—must be supplied by the validated artifact contract. The backend must not infer a “standard” formula from model dimensions. RoPE can use either a validated precomputed table or on-device generation; table storage improves repeatability, while generation reduces static memory but depends more directly on GPU transcendental behavior.

For decode attention, use an online stable softmax:

\[ m_j = \max(m_{j-1}, s_j), \]

\[ l_j = l_{j-1}e^{m_{j-1}-m_j}

\]

• e^{s_j-m_j},

while updating the weighted value accumulator with the same rescaling. This avoids materializing an entire score vector and is based on established online-normalizer work; blockwise IO-aware attention follows the same general motivation as FlashAttention.

Every attention dispatch must validate:

• position \(<\) admitted context capacity;
• no addition or multiplication overflow in KV offsets;
• query-to-KV-head mapping within range;
• causal end position and any sliding-window start;
• RoPE table coverage;
• finite or explicitly handled score values;
• dimensions compatible with tile and tail logic.

KV Cache, Deterministic Memory Planning, and Binding Strategy

KV-cache layout

Static weights and mutable KV state should never share identity or lifetime assumptions. The first production layout should be per-layer contiguous K and V storage, with explicit strides:

\[ \text{KVBytes} = L \times C \times H_{KV} \times D_H \times 2 \times B_e \times B, \]

where \(L\) is layer count, \(C\) context capacity, \(H_{KV}\) KV-head count, \(D_H\) head dimension, \(B_e\) bytes per element, and \(B\) batch or verification width. Alignment and shard padding must be added separately.

A layer-major layout minimizes binding changes when layers execute sequentially. Within a layer, a recommended logical index is:

\[ [\text{token}][\text{kv-head}][\text{head-element}], \]

with separate K and V regions or an explicitly interleaved descriptor. Separate regions simplify attention access and sharding; interleaving may reduce binding count but can worsen access locality. The choice requires local profiling.

Contiguous versus paged.

GPU KV operation should remain readback-free during normal generation. The CPU must retain committed token history and sampler state, but need not mirror the entire KV cache. After device loss, CPU continuation can reconstruct KV by replaying committed tokens; the latency of that replay is product-visible and must be benchmarked.

Illustrative KV-memory trend

For an illustrative model with 24 layers, eight KV heads, head dimension 64, one batch, and two-byte KV elements:

\[ \text{bytes per token} =24\times8\times64\times2\times2 =49{,}152\text{ bytes} =48\text{ KiB}. \]

This is a computed example, not a claim about any TinyRustLM model, and excludes alignment, paging, staging, activations, weights, metadata, and driver allocations.

xychart-beta
    title "Illustrative GPU KV memory versus context length"
    x-axis "Context tokens" [1024, 2048, 4096, 8192, 16384]
    y-axis "KV memory MiB" 0 --> 800
    line [48, 96, 192, 384, 768]

The linear trend means context capacity can become the decisive memory term even when q4 substantially reduces static weights. GQA reduces KV storage relative to full multi-head attention by using fewer key/value heads, one reason its exact head count must be included in admission.

Deterministic memory planner

The planner should produce one immutable allocation graph before requesting large buffers:

\[ M_{\mathrm{owned,GPU}} = M_W + M_{\mathrm{meta}} + M_{\mathrm{KV}}

\]

• M_{\mathrm{arena}} + M_{\mathrm{scratch}}
• M_{\mathrm{logits}} + M_{\mathrm{staging,GPU}}
• M_{\mathrm{readback,GPU}}
• M_{\mathrm{reserved}}.

CPU claims should be reported separately:

\[ M_{\mathrm{owned,CPU}} = M_{\mathrm{artifact}}

\]

• M_{\mathrm{repack}}
• M_{\mathrm{upload,CPU}}
• M_{\mathrm{readback,CPU}}
• M_{\mathrm{shadow}}
• M_{\mathrm{runtime}}.

Pipeline objects and browser/compiler internal allocations cannot be precisely sized through WebGPU. The receipt should therefore distinguish exactly owned buffer bytes, known CPU bytes, and an unmeasured implementation-overhead category rather than pretending every byte is observable.

Activation aliasing. Construct a liveness interval \([b_i,e_i]\) for every temporary. Two tensors may share a region only if:

\[ e_i < b_j \quad\text{or}\quad e_j < b_i. \]

The planner should store a machine-checkable proof record naming the producer, final consumer, offset, size, and non-overlap reason. Garbage collection is not a liveness proof.

A typical layer schedule permits reuse of:

• normalized activation space after all Q/K/V projections consume it;
• Q scratch after attention output has been produced;
• score/softmax tile scratch between heads or blocks;
• gate and up intermediate space after elementwise multiplication;
• layer-local projection output after the residual is committed.

Safety reserve. Because WebGPU does not expose precise free VRAM and implementations may reject allocations under pressure, TinyRustLM needs a versioned policy reserve plus a bounded allocation test. An initial research value might be:

\[ M_{\mathrm{reserve}} = \max(64\text{ MiB},\,0.15 M_{\mathrm{owned,GPU}}) \]

for desktop-class trials, with a separately calibrated mobile profile. This is an assumption for experimentation, not a standards-derived threshold. Promotion requires physical measurements demonstrating an acceptable allocation-failure rate without excessive false rejection.

Large tensors and binding limits

A buffer’s total size and a single storage binding’s visible size are separate limits. With default limits, a 200 MiB buffer could fit the default 256 MiB maxBufferSize but could not be exposed as one default-size 128 MiB storage binding.

Recommended resolution order:

Reject the GPU plan if any tensor cannot be represented by a prevalidated combination of these strategies. Do not split tensors dynamically in response to allocation failures, because that would create an untested execution composition after admission.

Suggested memory and resource metrics

Pipeline Identity, Cancellation, Device Loss, and Security

Pipeline and bind-group identity

A pipeline-cache key must bind every fact capable of changing shader semantics:

PipelineKey {
    backend_abi_version
    runtime_build_hash
    complete_wgsl_source_hash
    entry_point
    specialization_constant_map
    quant_descriptor_version
    tensor_layout_profile_hash
    enabled_feature_bits
    requested_limit_profile
    feature_level
    browser_engine_major_bucket
    coarse_native_backend_class
    shader_compiler_pipeline_version_if_public
}

Do not key only on browser name or model name. Conversely, do not persist raw adapter identifiers that unnecessarily increase fingerprinting. Browser pipeline objects are generally process/device objects rather than portable serialized binaries, so durable caching should focus on model bytes, normalized payloads, shader hashes, and compile-result metadata; pipelines must be recreated for a new device.

Use asynchronous pipeline compilation for all mandatory pipelines. Candidate resources remain isolated until compilation and self-tests complete. createComputePipelineAsync() can reject with validation or internal pipeline errors, so a model must never become active merely because some early pipelines compiled.

Bind-group caches require the device identity, pipeline-layout identity, exact buffer-object identity, offset, size, model/session epoch, and buffer generation. A buffer reused at the same JavaScript variable name after model switching is not the same resource.

Submission and cancellation

WebGPU command buffers already submitted to the queue cannot be preempted by application JavaScript. Cancellation therefore operates at submission boundaries, not arbitrary instruction boundaries.

Recommended policy:

• split long prefill into bounded token or layer chunks;
• check the cancellation epoch before encoding, before submission, and after completion callbacks;
• never block every decode token on queue.onSubmittedWorkDone() unless needed for lifecycle correctness;
• use mapped staging/readback buffers only in legal states and unmap before GPU reuse;
• tag every result with generation epoch, committed position, and submission sequence;
• explicitly destroy large buffers once no outstanding submission can reference them;
• on worker termination or page closure, abandon user-visible completion and let browser cleanup proceed, while persisting no prompt-bearing diagnostics.

The standard specifies that explicit buffer destruction unmaps the buffer and permits memory reclamation after prior submitted operations complete. Explicit destruction should be preferred over waiting for garbage collection.

Device-loss and fallback state machine

stateDiagram-v2
    [*] --> CPUReady
    CPUReady --> Probing: user enables GPU or model requests it
    Probing --> Preparing: adapter/device admitted
    Probing --> CPUReady: admission failure
    Preparing --> GPUReady: all allocations, pipelines, tests pass
    Preparing --> CPUReady: validation/OOM/compile/loss
    GPUReady --> Generating: begin generation
    Generating --> GPUReady: generation completes
    Generating --> CancelPending: user/model/page cancellation
    CancelPending --> GPUReady: submitted work retires, result discarded
    Generating --> Lost: device.lost or fatal GPU error
    Lost --> CPURebuild: committed tokens and sampler state are intact
    CPURebuild --> CPUGenerating: deterministic replay succeeds
    CPURebuild --> VisibleFailure: exact continuation cannot be established
    CPUGenerating --> CPUReady: generation completes
    GPUReady --> Destroying: model switch or explicit free
    Destroying --> CPUReady: resources retired and destroyed
    VisibleFailure --> CPUReady: user acknowledges or retries

WebGPU devices can be lost because the application destroys them, the browser revokes access, the GPU process or driver changes state, or an implementation encounters a serious problem. The loss message is implementation-defined and must not be parsed as a stable error protocol; no new errors are generated from an already lost device.

Product semantics by failure point:

The output-commit ledger is the central anti-duplication mechanism:

CommitRecord {
    generation_epoch
    token_position
    token_id
    sampler_state_after_token_hash
    raw_text_delta_hash
}

Only the Rust/WASM semantic layer may advance the committed position. GPU completion alone never means a token was shown.

Security and malicious-artifact threat model

WebGPU includes validation, resource initialization, and robust-access protections intended to reduce memory-safety exposure, but these mechanisms are not substitutes for application-level shape and bounds validation. The standard also explicitly recognizes timing, fingerprinting, GPU-memory pressure, watchdog, and denial-of-service concerns.

Numerical Conformance, Benchmarking, Fault Injection, and TDD

Conformance hierarchy

A WebGPU backend should be promoted only after passing each lower level:

Bit-exact targets should include artifact hashes, quantized code unpacking, integer address calculations, tokenization, chat-template bytes, sampler RNG, cancellation epochs, token-commit order, and model identity. Floating-point hidden states and logits should use tolerances.

Suggested starting, not production-approved, numerical envelopes are:

These numbers are provisional engineering hypotheses. Final values require actual .slm models, CPU-reference outputs, multiple GPU vendors, and representative context lengths.

Adversarial vectors must include all-zero rows, constant rows, maximum and minimum quantized codes, alternating signs, scales near zero, very large scales, NaNs and infinities where the input contract permits them, signed zero, subnormals, odd K dimensions, single-element tails, non-power-of-two head sizes, long positions, and malformed descriptors that must fail before dispatch.

Benchmark protocol and formulas

Report cold and warm behavior separately:

Approximate lower-bound operation and transfer accounting should accompany measured results:

\[ \operatorname{MACs}_{\mathrm{GEMM}} \approx MKN, \qquad \operatorname{FLOPs} \approx 2MKN \]

under the conventional multiply-plus-add accounting.

For decode projection GEMV, the weight-read lower bound is approximately the packed tensor payload per token unless cache behavior reduces physical traffic. For attention at context length \(S\), the logical KV read is approximately:

\[ B_{\mathrm{KVread}} \approx 2 L S H_{KV}D_H B_e \]

per generated token, excluding caching, padding, metadata, and repeated passes. These are planning formulas, not hardware bandwidth measurements.

All performance claims must state:

• browser and exact version;
• OS version;
• physical GPU and driver where authorized;
• native backend or translation layer if known;
• power mode;
• model artifact hash and exact quantization;
• prompt/context/output lengths;
• cold or warm state;
• shader feature tier;
• all owned CPU, GPU, staging, readback, and shadow bytes;
• whether the adapter is software, integrated, or discrete;
• raw-output identity or measured divergence.

Published WebLLM and Transformers.js projects demonstrate that substantial in-browser LLM inference systems can be built on WebGPU, while llama.cpp documents a WebGPU build path tied to Dawn/Emscripten components. Their results and architecture are useful comparative evidence, but they do not validate TinyRustLM’s artifact, kernels, memory behavior, or output contract.

Fault and browser test matrix

The WebGPU CTS should be run or tracked at a pinned revision for API and shader conformance, but passing CTS is necessary rather than sufficient: it does not establish TinyRustLM model correctness, speed, memory fit, or product recovery semantics.

Staged TDD backlog and promotion gates

Stop and deletion rules. Delete a shader path when, after reasonable tuning, it cannot beat the CPU/WASM reference on its declared target cohort after including adapter acquisition, compilation, upload, readback, and fallback overhead; when it exceeds the numerical or behavioral budget; when it requires unbounded device-specific exceptions; when its memory planner cannot provide a deterministic admission proof; or when it cannot survive cancellation and loss without output-contract violations. Because TinyRustLM is pre-publication, these deletions should remove code, cache keys, tests intended solely for that path, and undocumented API surface rather than retaining a “legacy” fallback.

Unknowns Requiring Private Evidence and Annotated Primary Sources

Required local or authorized verification

The following facts cannot be resolved from public sources:

Recommended deeper-study sequence

The next research cycle should first freeze the .slm validation schema and semantic command contract. It should then create a tiny public or internally shareable fixture model, because that fixture enables independent shader work without exposing production weights. The third phase should collect operator golden vectors and memory plans from the CPU reference. Only then should q8/q4 kernels, full layers, and real models enter performance work.

In parallel, build a physical test pool spanning at least D3D12, Metal, and Vulkan; one integrated and one discrete desktop GPU; an Apple-silicon Mac; Android Vulkan and compatibility-mode devices; and Safari on iOS. Browser software adapters and headless execution remain valuable for validation automation but cannot substitute for this pool.

Short annotated primary-source bibliography

All sources below were retrieved August 1, 2026 unless noted otherwise.

W3C WebGPU Candidate Recommendation Draft and publication history. The normative basis for devices, adapters, limits, buffers, error behavior, loss, synchronization, resource destruction, security, privacy, and capability negotiation. The publication history shows continuing 2026 Candidate Recommendation Draft updates.

W3C WebGPU Shading Language Candidate Recommendation Draft, June 5, 2026. The primary source for WGSL types, floating-point behavior, shader validation, f16, workgroup rules, and language semantics.

GPUWeb Editor’s Draft. The most current consolidated WebGPU text retrieved, including optional-capability negotiation, privacy bucketing, timing risks, robust access, allocation failure, and feature definitions. Because it is an editor’s draft, implementation should pin the exact revision used for each release.

WebGPU CTS documentation and sources. The conformance suite associated with the standards effort, including default limits and shader/API tests. Pin a commit in the project lockfile and record the executed subset.

GPUWeb implementation-status record, edited May 28, 2026. Cross-vendor public shipping matrix for Chromium, Firefox, Safari, OS configurations, and implementation libraries such as Dawn and wgpu.

WebKit, Safari 26 WebGPU documentation. Vendor evidence that Safari 26 enables WebGPU on macOS, iOS, iPadOS, and visionOS and implements it over Apple’s graphics stack.

Mozilla Firefox 141 release material. Vendor release evidence associated with Firefox’s initial stable Windows WebGPU rollout in July 2025; the GPUWeb status record is more specific about continuing platform coverage.

Chrome “What’s New in WebGPU” series. Vendor evidence for subgroup deployment, Linux expansion, compatibility mode, compiler changes, and other Chromium/Dawn developments. Performance figures in these posts should be treated only as observations of the named configurations.

Su et al., “RoFormer: Enhanced Transformer with Rotary Position Embedding,” 2021. Original paper underlying the RoPE mechanism and its positional formulation.

Ainslie et al., “GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints,” 2023. Primary description of grouped-query attention and the relationship among query and KV-head counts.

Zhang and Sennrich, “Root Mean Square Layer Normalization,” 2019. Original RMSNorm formulation used to specify normalization behavior and test oracles.

Milakov and Gimelshein, “Online Normalizer Calculation for Softmax,” 2018. Basis for maintaining a running maximum and normalization term without materializing the entire score vector.

Dao et al., “FlashAttention,” 2022. Primary IO-aware exact-attention work motivating tiled attention and reduction of external-memory traffic. It is an algorithmic reference, not evidence that a WebGPU kernel inherits CUDA performance.

Frantar et al., “GPTQ,” 2022. Primary low-bit post-training quantization reference useful for understanding quantized transformer weight distributions and validation, although TinyRustLM’s .slm encoding must be defined independently.

MLC WebLLM. Public open-source evidence of a substantial in-browser WebGPU LLM runtime, useful for comparison of worker integration, model loading, shader compilation, and browser deployment. It does not validate TinyRustLM’s format or results.

llama.cpp WebGPU build documentation. Public implementation evidence for use of Dawn and Emscripten WebGPU tooling in a cross-platform inference project; the documentation’s pinned dependency information illustrates why TinyRustLM should record exact toolchain revisions.