TinyRustLM Privacy-Preserving Diagnostics and Reproducible Performance Laboratory

Full report

On this page

Executive measurement and privacy recommendation

Research resolution. The uploaded brief resolves the formerly unspecified research topic: this report addresses the diagnostics, benchmarking, and release-evidence system for TinyRustLM rather than the earlier placeholder topic concerning remote work. The analysis assumes local browser WebAssembly inference, an optional native companion, optional WebGPU, no project-server receipt of prompts or model bytes, and no access during this research to TinyRustLM’s private code, models, telemetry, profiles, test receipts, or execution hardware.

The recommended system is a local-first, receipt-producing performance laboratory, not a conventional telemetry pipeline. Full traces remain on the user’s machine. Public or support-facing artifacts are generated only through an explicit export action that applies schema validation, deterministic redaction, path normalization, identifier re-keying, bounded-field enforcement, and secret scanning.

flowchart LR
    A[Browser or native operation] --> B[Versioned local event recorder]
    B --> C[In-memory bounded trace]
    C --> D[Local benchmark and quality gates]
    C --> E[User-requested diagnostic export]
    E --> F[Deterministic redaction]
    F --> G[Secret and policy scan]
    G -->|Pass| H[Bounded support bundle]
    G -->|Fail| I[Export rejected locally]
    D --> J[Signed or hashed release receipt]
    H --> K[Optional user-mediated sharing]
    J --> K

The core recommendation has eight consequences.

Web timing must be based on monotonic timing domains. High Resolution Time defines a time origin and a current-time mechanism intended not to be affected by wall-clock adjustments; Performance Timeline and User Timing provide the observation and application-marker mechanisms around that clock. Browser timing precision may be reduced for security and privacy, so the observed resolution must be captured rather than assumed.

Evidence-label convention used below

The initial deliverable should be treated as complete when TinyRustLM can produce, for one admitted real model, a reproducible local trace, a privacy-safe bundle, a quality-coupled comparative receipt, and physically executed results on at least one Windows x64, Linux x64, and Linux arm64 host plus current Chromium, Firefox, and WebKit lanes. The full release laboratory, including independent repeats and calibrated wall-power measurements, is a subsequent maturity level.

Claim taxonomy and evidence matrix

A claim is valid only when its subject, workload identity, metric definition, evidence class, environment, sample structure, and uncertainty are explicit. A benchmark artifact should reject claim text that exceeds the underlying evidence level.

REC. Claim generation should be mechanical. Each receipt should contain claim_capabilities, an array computed from evidence, not written by a benchmark operator. The public report generator must refuse adjectives such as “native,” “cross-platform,” “energy-efficient,” “memory-efficient,” or “quality-neutral” unless the receipt satisfies the corresponding predicate.

The WebGPU "timestamp-query" capability is optional. The specification notes that timestamp-query values are expressed in nanoseconds but that their determination is implementation-defined, and it discusses deliberate precision reduction because GPU timing can expose high-resolution information. Therefore, GPU timestamps support scoped GPU-pass duration claims only when the feature is present and calibrated; they do not establish CPU/GPU synchronization or whole-operation latency by themselves.

Platform evidence requirements

REC. The following are minimum promotion requirements, not estimates of statistical sufficiency for every workload. Pilot variance may require more runs.

Current browser memory interfaces are uneven. performance.memory is non-standard and Chromium-specific. measureUserAgentSpecificMemory() is experimental, requires a secure cross-origin-isolated context, and its values are implementation-dependent and unsuitable for comparisons across browser families or even unqualified browser-version changes.

Firefox’s official profiler supports thread and allocation analysis, while WebKit’s Timelines tooling can organize work by frame and task. Chrome’s Performance panel records runtime activity, frame behavior, CPU work and long tasks. These tools are diagnostic corroboration lanes; they should not remain attached during primary promotion measurements unless their overhead has been separately quantified.

Versioned events, traces, clocks, and performance metrics

REC. Adopt trl.diag.event/1.0.0 as the sole event envelope. JSON is suitable for exchange and diagnostics, but the in-memory recorder may use a compact binary representation. Integer nanoseconds should be serialized as decimal strings when a consumer cannot safely represent 64-bit integers.

{
  "schema_version": "trl.diag.event/1.0.0",
  "event_id": "random-per-run-128-bit-id",
  "operation_id": "random-per-run-128-bit-id",
  "parent_id": "random-per-run-128-bit-id-or-null",

  "event_kind": "span_end",
  "phase": "decode_step",

  "clock": {
    "domain": "browser-performance",
    "origin_id": "random-per-realm-id",
    "monotonic_ns": "18429374000",
    "resolution_ns": "100000",
    "reduced_precision": true,
    "sync_uncertainty_ns": null,
    "discontinuity_epoch": 0
  },
  "duration_ns": "19230000",

  "counts": {
    "bytes_in": 0,
    "bytes_out": 0,
    "bytes_allocated": 4194304,
    "bytes_freed": 0,
    "input_tokens": 0,
    "output_tokens": 1
  },

  "identity": {
    "model_composition_id": "sha256:...",
    "tokenizer_id": "sha256:...",
    "runtime": "tinyrustlm-wasm",
    "runtime_version": "semver-or-revision",
    "backend": "wasm-simd-threads",
    "build_id": "sha256:...",
    "workload_id": "private-local-id"
  },

  "status": "ok",
  "error_code": null,
  "cancelled": false,
  "cancellation_stage": null,

  "optional": {
    "thread_class": "decode-worker",
    "memory_snapshot_id": null,
    "gpu_scope_id": null,
    "quality_receipt_id": "sha256:..."
  }
}

Schema rules and redaction classes

REC. Permit local-only encrypted attachments for authorized quality investigations, but keep them outside the diagnostics schema and outside normal bundle creation. This prevents an accidental schema evolution from turning payload material into telemetry.

Operation trace contract

Every operation is a parent span with explicit child phases. An operation is not complete until its terminal status and cleanup disposition are recorded.

Performance Timeline exposes PerformanceObserver so applications can receive new and, for supported entry types, buffered entries without polling. User Timing measures are available in workers and can carry structured detail, but TinyRustLM should place only allowlisted non-payload metadata in that detail.

Clock semantics

Windows documents QueryPerformanceCounter as the primary native API for high-resolution interval measurements and distinguishes it from UTC-synchronized wall-clock APIs. Linux monotonic timekeeping similarly distinguishes CLOCK_MONOTONIC-style elapsed-time sources from realtime clocks.

REC. Clock correlation algorithm. Perform at least nine ping-pong exchanges between producer and collector. For each exchange, calculate round-trip time and midpoint offset. Select the minimum-RTT observation as the offset estimate, retain half its RTT plus observed timer granularity as sync_uncertainty_ns, and repeat after backgrounding, process migration, suspend/resume, or a long benchmark. Do not merge event ordering across clock domains when uncertainty intervals overlap.

REC. Discontinuity detection. Increment discontinuity_epoch and terminate or segment the run when any of these occur: a monotonic reading decreases; a delta exceeds a registered suspension threshold; worker correlation offset changes beyond its uncertainty budget; the browser process restarts; the GPU device is lost; or wall-to-monotonic correlation changes sharply. A run crossing discontinuity epochs cannot support one continuous latency claim.

Latency and throughput definitions

Define:

\[ \text{engine tokens/s} = \frac{N_{\text{included output tokens}}} {t_{\text{last decode end}}-t_{\text{first decode start}}-\text{registered pauses}} \]

\[ \text{visible tokens/s} = \frac{N_{\text{rendered tokens}}} {t_{\text{last visible commit}}-t_{\text{first visible commit}}} \]

Do not remove sampling, garbage collection, scheduling, or rendering time from an end-to-end metric. If an engine-only number excludes these costs, report a second user-visible number.

Responsiveness measurements

The Long Tasks API is designed to surface UI-thread work that monopolizes the thread for an extended period, while Event Timing exposes input-processing and next-render timing for qualifying interactions. Event Timing durations are intentionally coarsened and have support-specific thresholds, so capability detection and observed granularity belong in each receipt.

REC. Record the following without capturing UI text:

Primary responsiveness gates should include: no composer input loss, no post-cancel output, no duplicate output events, no permanent scroll lock, bounded long-task regression, and no substantial deterioration in input-tail latency. Performance work that increases tokens per second while freezing the chat surface fails.

Memory, energy, state control, and instrumentation overhead

Memory-accounting model

No single browser or operating-system metric represents “TinyRustLM memory.” The laboratory should expose several non-overlapping accounting views and explicitly identify nested or unobservable components.

Linux /proc documentation distinguishes virtual size, RSS, high-water RSS and related fields; smaps_rollup provides summed mapping information including RSS and proportional-set-size fields. These are process-level observations, not direct tensor or model-ownership measurements.

On Windows, GetProcessMemoryInfo returns process memory counters, while GetProcessTimes and QueryProcessCycleTime expose processor-time or cycle information rather than elapsed latency. The receipt must name the exact selected counter instead of relabeling it as total memory or wall time.

REC. Non-overlap formula.

Let application-owned host categories be disjoint:

\[ M_{\text{owned-host}}(t)= M_{\text{input}}(t)+ M_{\text{wasm-allocator-live}}(t)+ M_{\text{external-js-buffers}}(t)+ M_{\text{other-tagged}}(t) \]

Then:

\[ M_{\text{owned-host-peak}}=\max_t M_{\text{owned-host}}(t) \]

Do not add WASM committed pages to allocator live bytes if allocator live memory is contained within those pages. Instead report:

\[ M_{\text{WASM-slack}}(t) = M_{\text{WASM-linear-extent}}(t) - M_{\text{tagged-live-in-WASM}}(t) \]

where the value is labeled an accounting difference, not necessarily reclaimable memory.

For GPU objects:

\[ M_{\text{GPU-known}}(t)= \sum_{\text{live buffers}} \text{declared size} + \sum_{\text{live textures}} \text{estimated logical footprint} \]

This is known requested allocation, not physical GPU-memory consumption, residency, available VRAM, or driver overhead.

Artifact and runtime memory should remain separate:

\[ F_{\text{deployment}} = B_{\text{persisted-artifact}} \quad\text{and}\quad M_{\text{runtime-peak}} \]

A combined “footprint” may be shown only as a visibly labeled sum of unlike storage and runtime concepts.

Energy and thermal evidence levels

Linux powercap exposes power zones and, where supported, Intel RAPL domains through sysfs. Linux’s power-supply class may expose energy, charge and battery properties, but it explicitly allows unavailable properties to be omitted and distinguishes energy from charge and percentage capacity. These interfaces support named-platform evidence, not cross-platform equivalence.

Arm PMUs expose performance events rather than universal joule counters. Counter availability varies by core and platform; Arm’s tooling can combine CPU, GPU, scheduler and hardware-counter observations when the target platform supports them. Therefore, generic PMU events may explain energy changes but cannot by themselves be labeled energy.

REC. Energy protocol.

\[ E_{\text{net}} = E_{\text{active-window}} - P_{\text{idle-matched}}\times T_{\text{active-window}} \]

Report both gross and idle-subtracted energy. Idle subtraction is valid only when the idle run uses the same display state, browser/native process state, model residency policy, network state, power mode, background-service policy and thermal starting band.

The protocol should fix or record: meter model and calibration date; sampling frequency; voltage and meter range; synchronization uncertainty; screen brightness/state; network state; power plan/governor; fan/cooling policy; ambient temperature; initial package/device temperature; battery/AC state; frequency scaling policy; run duration; idle procedure; and post-run thermal state.

SPEC power rules are useful methodological references: they require controlled environmental conditions and, for compliant wall-power work, calibrated measurement devices with specified uncertainty. They should guide TinyRustLM’s method without implying that TinyRustLM results are SPEC-certified.

REC. Thermal controls. Begin a run only when the selected temperature signals are inside a preregistered band for a minimum stabilization interval. Reject a block if temperature drifts beyond the registered band, sustained frequency falls outside policy, thermal throttling is observed, or the candidate and baseline begin from materially different thermal conditions. Alternate baseline and candidate instead of running all baseline trials first.

Cold, warm, steady and restart states

Reset mapping

A receipt must therefore record the reset procedure, not merely the word “cold.”

Instrumentation-overhead calibration

REC. Calibrate the following modes independently:

These thresholds are REC, not externally established constants. VERIFY must confirm they are attainable for TinyRustLM.

Use paired runs with identical workload and state, randomized instrumentation order, and separate effect estimates for latency, throughput, memory and long tasks. For production diagnostics, aggregate repetitive decode events into bounded windows, such as eight or sixteen steps, unless an explicit full-trace diagnostic session is enabled. Full per-step events are appropriate in fixture and laboratory modes, not as an always-on telemetry path.

The WebKit documentation explicitly notes that some timelines can affect inspected-page performance, and browser profiling tools generally change what is observed. This supports requiring profiler-on versus profiler-off calibration rather than treating profiler output as primary benchmark evidence.

Workloads, preregistration, statistics, and quality coupling

Workload identity contract

Each benchmark operation must bind to an immutable identity record:

workload_identity_version
model_release_or_private_local_id
model_composition_hash
quantization_and_numeric_tier
adapter_hashes_and_order
tokenizer_hash
chat_template_hash
private_exact_prompt_hash
private_prompt_token_sequence_hash
prompt_token_count
context_token_count_before_request
output_token_cap
sampling_algorithm_and_parameters
seed_or_seed_policy
prefix_and_kv_cache_state
artifact_cache_state
runtime_and_version
backend_and_feature_path
thread_count_and_worker_topology
browser_engine_and_exact_build
operating_system_and_build
build_artifact_hash
measurement_policy_hash
quality_policy_hash

The exact prompt hash and token-sequence hash remain in private local evidence. A public receipt substitutes a registered workload-class ID, token-count bucket or disclosed count, context band, and privacy-preserving fixture identity. Hashes should not be assumed anonymous: hashes of low-entropy prompts or private models may permit guessing, so public export should omit or key them unless the underlying object is already public.

Workload suite

A workload is inadmissible when its identity cannot be reproduced privately, its quality oracle is undefined, it relies on mutable remote content without a snapshot, or it would expose payload material in a public receipt.

Preregistration contract

The preregistration and its hash should be written before candidate labels are unblinded in the comparison report.

Statistical treatment

Performance observations are hierarchically dependent: tokens occur within turns, turns within processes or browser profiles, and processes within physical hosts. Treating every token as an independent replicate would create falsely narrow intervals.

REC. Experimental unit. For release decisions, the principal replicate should usually be a paired operation or session within a randomized block, not an individual token. Token-level data may characterize inter-token distributions but should not inflate the number of independent release replicates.

REC. Run structure. Use randomized AB/BA pairs within host-state blocks. For expensive energy tests, use longer alternating blocks and independently measured idle windows. Preserve all valid registered runs.

REC. Warm-up. Use fixture and pilot evidence to define the warm-up boundary before comparative analysis. Report startup separately rather than discarding it when startup matters to users.

REC. Autocorrelation. Examine run-order plots and autocorrelation of paired residuals. When dependence remains, use block bootstrap or cluster-level aggregation. Do not randomly reshuffle dependent token observations as though they were independent.

REC. Heavy tails and heteroscedasticity. Report median, p90, p95 and p99 when sample size supports them; show empirical distributions; estimate paired ratios or differences with bootstrap intervals; and avoid relying only on an arithmetic mean. Use quantile uncertainty or hierarchical bootstrap for tail decisions.

REC. Multiple comparisons. Declare one primary endpoint. Apply Holm or another preregistered family-wise procedure to confirmatory secondary tests. Exploratory findings remain labeled exploratory.

REC. Effect size. Report both relative and absolute change:

\[ \Delta_{\text{abs}} = M_C-M_B \]

\[ \Delta_{\text{rel}} = \frac{M_C-M_B}{M_B} \]

For latency, a negative value is an improvement; the report should state that direction explicitly. Include confidence intervals and the practical-regression threshold.

Criterion is a useful Rust benchmarking reference but must not be assumed to be TinyRustLM’s adopted harness. Its documented defaults include warm-up, repeated measurement, bootstrap resampling, confidence and significance parameters, illustrating why harness configuration belongs in receipts rather than remaining implicit.

Systems-benchmarking research warns that nondeterminism, insufficient repetition and favorable-run selection can produce misleading conclusions. Kalibera and Jones emphasize accounting for uncertainty at multiple execution levels, while Georges, Buytaert and Eeckhout show why best-run or otherwise statistically weak methodologies can reverse conclusions in managed-runtime experiments. The exact runtime differs, but the methodological risks apply to browser, WASM, allocator, GPU and process-state variability.

Promotion and regression decisions

REC. Provisional default budgets

These values are starting recommendations. VERIFY must calibrate them to TinyRustLM’s quality suite and release risk. A statistically detectable improvement below the minimum worthwhile effect does not justify complexity; a statistically uncertain regression whose interval crosses the registered failure threshold does not pass.

Every comparison stores a private quality receipt containing the model identity, private output hash, authorized token-sequence hash where permitted, numeric-tier result, semantic-test result, error counts and cancellation result. Public receipts retain only pass/fail statuses, policy hashes, bounded aggregate counts and approved model/composition identity.

Privacy-safe bundles, dashboards, stop rules, and implementation plan

Local diagnostics and export policy

REC. Full traces are local, ephemeral by default and user-controlled. There is no implicit upload. Enabling diagnostics changes local retention only; sharing requires a separate explicit action showing the bundle contents and omission report.

Privacy engineering should follow data minimization and purpose limitation: collect only data required to diagnose or substantiate the declared claim, keep payloads outside the observability channel, bound high-cardinality fields, and make retention and sharing explicit. NIST’s Privacy Framework provides a general risk-management structure rather than a TinyRustLM-specific telemetry design, so it should be used as a governance reference, not as proof of compliance.

Allowed support-summary fields

Forbidden content

Prompts, output text, token IDs or token strings, raw model bytes, model excerpts, credentials, cookies, authorization headers, invitation/enrollment URLs, query strings, private absolute paths, browser-storage dumps, hidden tenant/account IDs, persistent device IDs, MAC addresses, serial numbers, exact stable hardware fingerprints, unrestricted exception text, environment-variable dumps, or arbitrary profiler metadata.

Diagnostic-bundle format

tinyrustlm-diagnostic-bundle/
  manifest.json
  schema/
    event-schema.json
    receipt-schema.json
    redaction-policy.json
  receipts/
    benchmark-receipt.json
    quality-receipt-public.json
    platform-evidence.json
  traces/
    bounded-events.jsonl.zst
  tests/
    summary.json
  tools/
    versions.json
  redaction/
    report.json
    secret-scan-report.json
  notes/
    user-note.txt                 # optional, user-authored and separately warned
  SHA256SUMS

manifest.json should include bundle schema version, creation-tool version, normalized creation date, file list, file lengths, SHA-256 hashes, evidence class, retention warning and explicit forbidden-content declaration. The manifest must not contain the machine username or original filesystem roots.

REC. Bounded trace rules

Redaction and secret scanning

Export proceeds through these mandatory stages:

1. Parse against the exact schema; reject unknown fields.
2. Delete fields classified as forbidden or local-only.
3. Normalize separators and redact absolute paths, usernames and profile directories.
4. Strip URL query, fragment, user-info and tokens; preferably omit URLs entirely.
5. Re-key event and operation identifiers.
6. Apply count bucketing and hardware-class coarsening.
7. Scan canonical decoded content and compressed/uncompressed members for credentials, cookies, private keys, bearer tokens, known invitation patterns, prompt/output markers and high-entropy suspicious strings.
8. Reparse the completed bundle and compare it to the allowlist.
9. Emit a redaction report containing rule IDs and counts, not the redacted values.
10. Reject export on scan failure, malformed archive, unknown schema, hash mismatch or nondeterministic redaction result.

Run the redactor twice on the same canonical input and require byte-identical output except for an explicitly normalized bundle creation field. This makes the privacy transformation testable and reproducible.

Dashboard and release receipt

The live local dashboard should show only operational metadata:

REC. Stop or reject the run when:

TDD backlog and clean logging replacement

All unversioned diagnostic formats, arbitrary console serialization of internal objects, payload-bearing logs, hidden analytics hooks and implicit network telemetry paths should be deleted rather than bridged. CI should scan for removed event names, forbidden payload fields, generic “log context” maps, unrestricted URL/path emission and unauthorized network destinations.

Delivery timeline and estimated effort

The dates below are REC planning assumptions beginning Monday, August 3, 2026. Hardware procurement, browser automation maturity and access to release-intended models can extend the schedule.

gantt
    title TinyRustLM diagnostics and performance laboratory
    dateFormat  YYYY-MM-DD
    axisFormat  %b %d

    section Foundations
    Schema, enums and claim predicates           :a1, 2026-08-03, 8d
    Redaction and secret-scan TDD                :a2, 2026-08-03, 12d
    Delete unsafe and unversioned logging        :a3, after a1, 6d

    section Instrumentation
    Browser clock and trace adapters             :b1, 2026-08-11, 12d
    Native Windows and Linux adapters            :b2, 2026-08-11, 15d
    Memory and GPU object accounting             :b3, 2026-08-17, 15d

    section Benchmarking
    Deterministic fixture harness                :c1, 2026-08-17, 10d
    One admitted real-model lane                 :c2, after c1, 10d
    Preregistration and statistical pipeline     :c3, 2026-08-24, 12d
    Instrumentation-overhead calibration         :c4, after b3, 8d

    section Evidence
    Browser and x64 physical matrix              :d1, 2026-09-07, 12d
    Physical Linux arm64 lane                    :d2, 2026-09-07, 15d
    Energy and thermal lane                      :d3, 2026-09-14, 15d

    section Release
    Bundle, dashboard and receipt hardening      :e1, 2026-09-07, 15d
    Independent repeat and release review        :e2, 2026-09-28, 10d

With parallel engineering, an initial credible laboratory is approximately eight to ten calendar weeks and twenty to thirty person-weeks. A smaller first milestone—schema, redaction, deterministic fixture, one browser lane and one native x64 lane—can fit roughly four to five calendar weeks and ten to fifteen person-weeks. These are engineering estimates, not observed TinyRustLM delivery rates.

Research protocol, verification unknowns, and primary-source bibliography

Reusable research protocol

The uploaded brief supplies the concrete subject, so this filled protocol uses TinyRustLM rather than the earlier remote-work placeholder.

Unknowns requiring private or physical verification

The following cannot be established through public research:

Recommended visualizations

The release report should generate the same small set of interpretable graphics from receipt data:

Annotated primary-source bibliography

All web sources were retrieved on August 1, 2026.