Fifty-Round Fresh-Browser Dogfood Harness And Conversation UX Reliability

Full report

On this page

1. Public TinyRustLM Observation Log

The public observation of the TinyRustLM architecture, conducted on July 14, 2026, against the deployed artifact at GGUF.MiRust.com and TinyRustLM.com, establishes the baseline for the browser-automation harness. The interface was evaluated across a strict viewport matrix comprising desktop dimensions at 1920 by 1080 pixels, narrow desktop dimensions at 1024 by 768 pixels, and mobile dimensions at 375 by 812 pixels. The architectural reality of the application differs significantly from conventional remote-inference web clients, necessitating a highly specialized reliability engineering approach. The observed application operates exclusively through a 114 KiB WebAssembly (WASM) module compiled from a zero-dependency Rust workspace1. This module exposes a raw C-style application binary interface (ABI) that manages model loading, memory allocation, and token generation directly on the browser's main thread, completely lacking Web Worker delegation, WebGPU acceleration, or background thread pools1. The system enforces a strict 128 MiB single-transfer ceiling for memory allocation, requiring the entire model artifact to be copied into a contiguous WASM memory block prior to initialization2. Furthermore, despite the project's domain nomenclature, the active runtime contains no GGUF loader; it exclusively parses a custom .slm v1 binary container1. This format is characterized by a 108-byte header, 64-byte aligned tensor directory entries, and an embedded internal tokenizer utilizing either BPE1 or BTOK algorithms1. The memory footprint is statically bound to mathematical formulas derived from the artifact dimensions, requiring equal counts for attention and key-value heads, which inherently rejects Grouped-Query Attention (GQA) and Multi-Query Attention (MQA) architectures1. Network activity is restricted to fetching loopback static assets, ensuring that model bytes remain strictly local or operate via direct peer-to-peer protocols without exposing proprietary prompts or weights to external ingestion endpoints.

2. Campaign Manifest and State Machine

To produce statistically significant evidence regarding user experience reliability and memory continuity, the dogfood campaign must execute precisely 50 complete rounds. This execution is orchestrated by a cryptographically secured campaign manifest that defines the exact state machine, binds the application origin, and schedules the randomized yet strictly distributed evaluation matrix. The campaign manifest operates as an immutable JSON ledger binding the overarching campaign ID to exactly 20 cryptographically distinct, quality-qualified .slm artifacts. The schedule guarantees that every model is exercised in at least two independent, clean-profile restarts, satisfying 40 of the required rounds. The remaining 10 rounds are distributed among the heaviest models in the manifest to induce maximal memory pressure on the WASM allocator. The manifest dictates the prompts by private hash rather than plaintext, ensuring that the automation harness does not inadvertently leak intellectual property into system logs if a crash occurs. Exactly 25 of these 50 rounds are explicitly designated for memory-continuity testing, requiring the state to be synchronized through the MultiAgentMemory MATM protocol at MemoryEndpoints.com. Model aliases or duplicate bytes are universally rejected during the manifest validation phase. This cryptographic enforcement is critical to prevent the automation harness from spoofing a legitimate conversation round by repeatedly loading the documented 4.8 KiB or 8.6 KiB deterministic runtime smoke fixtures3. A fixture of this size fails to stress the 128 MiB transfer ceiling, circumvents the mathematical limits of the forward scratch buffer, and is fundamentally incapable of generating genuine multi-turn conversational syntax2. By demanding 20 distinct, verified models in f32, q8\_0, or q4\_0 formats, the campaign guarantees that the browser process experiences realistic memory fragmentation, tensor materialization delays, and garbage collection pressure1. The orchestrator transitions through a rigid state machine for each round. The sequence begins with the preparation phase, verifying the manifest and local file hashes. It advances to profile initialization, generating an isolated directory root on the host filesystem. The execution phase manages the file injection, multi-turn conversation, and visual assertions. If the round is flagged for memory continuity, the machine branches into the sync phase, awaiting the immutable revision receipt from the external endpoint before proceeding to the teardown phase.

3. Fresh-Profile and Deletion Protocol

Achieving a verifiably clean browser state on every round requires abandoning superficial isolation methods. The campaign strictly prohibits the use of localStorage.clear(), simple page reloads, or Playwright's browser.newContext() for data restarts, as these methods fail to guarantee the erasure of orphaned IndexedDB structures, persistent service-worker caches, or network-level isolation keys4. The only acceptable method is the creation and subsequent physical deletion of an owned, fresh user profile directory. Every round initiates a browser session utilizing the chromium.launchPersistentContext(userDataDir) method6. The orchestrator asserts safe temporary-root ownership by dynamically generating a highly entropic path within the operating system's temporary directory, prefixed specifically for the campaign. This ensures that the headless browser instance constructs an entirely new set of SQLite databases for cookies, local storage, and profile preferences without inheriting state from previous test executions or the host machine's default user profile. The teardown and deletion phase is notoriously complex due to operating system filesystem locks. On Linux, Chromium utilizes a ProcessSingleton lock, while Windows environments frequently retain file handles on the userDataDir even after the primary process signals termination8. A naive deletion command will result in an EBUSY error on Windows or leave zombie instances in the background on Linux, corrupting subsequent runs9. Process closure is managed by explicitly awaiting the closure of all pages, followed by the context, and finally the browser instance, forcing the underlying databases to flush their write-ahead logs4. Profile deletion incorporates an exponential backoff retry mechanism. If the operating system rejects the initial deletion request, the orchestrator pauses and retries up to five times to allow lagging kernel file handles to release10. To make recursive deletion impossible outside the owned root, the teardown function programmatically verifies that the absolute path strictly begins with the expected temporary prefix and explicitly rejects paths resolving to root directories, user home directories, or system folders. Proof of deletion is ultimately asserted by confirming the directory no longer exists on the disk before the state machine permits the commencement of the next round.

4. Real-Model Import and Conversation Procedure

The ingestion of large local models into the browser context represents a significant reliability hurdle, particularly because typical automation protocols impose payload limitations. The Playwright Model Context Protocol (MCP), for instance, enforces a 50 MiB upload limit for files transferred over the bridge12. To bypass these restrictions and simulate true user interaction, the campaign relies on the native FileChooser API to select large local .slm files13. The orchestrator executes a trusted click on the interface's upload button concurrently with page.waitForEvent('filechooser'), allowing the harness to intercept the OS-level dialog and programmatically invoke setFiles() with the absolute path of the target model13. Following the file selection, the WASM and native load status are deeply monitored. The system tracks the 108-byte header validation, the decoding of the 64-byte tensor directory entries, and the custom non-cryptographic checksum verification1. Strict timeout budgets are enforced during this phase; the allocation of the forward scratch and KV cache must complete within fifteen seconds, or the round is flagged as a failure due to memory pressure or a WASM Out-Of-Memory (OOM) trap1. Once the model transitions to the ready state, the conversation execution mandates a genuine multi-turn interaction spanning two to six specific turns. This matrix requires the model to navigate casual chat to establish basic attention, ambiguity to evaluate response branching, practical planning to stress sequential reasoning, source conflict to test context window retention and contradiction checking, current preference tracking, untrusted memory handling, summarization, rewriting, structured output generation, and finally, code formatting. A single prompt is entirely insufficient to validate the KV cache's durability across the generation transaction1. During this execution, the automation preserves the raw model output while enforcing strict user interface contracts. Mutation observers are deployed to guarantee that each assistant answer renders exactly once on the DOM, preventing bugs where the streaming decoder duplicates assistant text blocks. The orchestrator asserts scroll stability by calculating bounding boxes, ensuring the viewport remains anchored to the active generation node without chaotic jumping. Furthermore, cancellation behavior is evaluated by interrupting the generation mid-stream, proving that the WASM state safely resets and the tokenizer clears the transient arrays without leaking handles or causing a main-thread panic1.

5. Browser Restart Semantics

Differentiating between superficial state resets and genuine cryptographic isolation is a core tenant of the campaign's reliability engineering. The framework strictly delineates between a page reload, a context reset, a browser restart, and an actual persistent-profile deletion. A page reload preserves all disk-backed storage, IndexedDB tables, and active Service Worker registrations, making it entirely unsuitable for isolation testing15. A context reset clears in-memory cookies but leaves the underlying persistent profile databases completely intact4. A standard browser restart flushes memory but reconnects to the existing userDataDir, recovering the previous session's storage7. Therefore, a true data restart can only be achieved by executing the full process closure, verifying that no zombie handles remain, and completely deleting the physical directory from the filesystem9. For the 25 rounds designated for memory continuity, the restart semantics are uniquely complex. At the conclusion of the conversation, the first browser process is cleanly closed and its profile deleted. The orchestrator then creates a second, entirely distinct fresh profile with a new temporary directory7. This forces the application to boot into an environment utterly devoid of local storage, cookies, or cached tensor fragments. It proves that the subsequent restoration of the conversational state relies exclusively on the successful retrieval of data through the MultiAgentMemory MATM protocol, rather than relying on lingering local artifacts17. Once the continuity verification is complete, this second profile is also subjected to the strict deletion protocol.

6. Secret and Privacy Scanning Contract

The architecture of modern browser storage partitioning—such as Mozilla's Total Cookie Protection (TCP) and Chrome's Cookies Having Independent Partitioned State (CHIPS)—double-keys client-side state based on both the origin of the resource and the top-level site18. The campaign leverages an understanding of these boundaries to execute a comprehensive secret and privacy scanning contract before any profile is deleted. The automation injects an evaluation script into the page context to scrape and serialize all local storage mediums. Because IndexedDB data does not update synchronously in external observation tools, the script utilizes page.evaluate() routines to interact directly with the database cursors, pulling ArrayBuffers and Blob data from the object stores22. The scanner also intercepts Cache Storage, Service Worker states, localStorage, sessionStorage, and examines Playwright network traces16. This serialized output is subjected to heuristic decoding, accounting for UTF-8, UTF-16, Base64 encodings, and compressed payloads2. The contract mandates an immediate, unrecoverable campaign failure if any sensitive data is discovered in unencrypted, persistent layers. This includes enrollment URLs, raw agent tokens, workspace bearer keys, user prompts, or local model bytes17. The scanning logic is designed to mitigate false alarms from legitimately hashed values, such as the FNV-1a name hashes utilized within the SLM1 tensor directory or the SHA-256 base hashes used for local collaboration overlays, by verifying payload lengths, entropy signatures, and structural context1.

7. Network Assertions

The campaign operates under a zero-trust network policy enforced via Playwright's route interception capabilities (browserContext.route())4. The network contract permits only a strictly defined allowlist of outbound and inbound requests. The orchestrator permits expected public static assets required for the application shell, catalog metadata for discovery, direct peer-to-peer loopback traffic for companion synchronization, and the specific MemoryEndpoints MATM API operations required for the 25 continuity rounds17. Any deviation from this allowlist triggers an immediate round failure. The harness explicitly monitors for, and blocks, unauthorized prompt uploads to project sites, implicit telemetry gathering, public requests for model bytes, and any credential-bearing URLs attempting to pass workspace keys via query parameters rather than secure headers17. By isolating the network, the campaign guarantees that the TinyRustLM execution remains fundamentally local and private.

8. UI Verification at Desktop, Narrow Desktop, and Mobile Widths

Functional fixes and optimizations to the WASM runtime or memory allocator must not casually redesign or break the approved user interface. The visual language of the application is locked, and the campaign verifies this stability across desktop, narrow desktop, and mobile viewports. Instead of relying on fragile CSS selectors or inconsistent pixel-matching algorithms, the automation heavily utilizes Playwright's accessibility tree snapshots (page.ariaSnapshot()) to guarantee semantic integrity25. The orchestrator verifies that introductory prompt cards, identified by specific ARIA roles, are fully visible upon initial load but completely disappear from the accessibility tree the moment the chat sequence is initiated25. The center column is asserted to become the primary, expansive conversation workspace. Responsive checks ensure that overlapping columns do not obscure text on narrow viewports and that the fixed composer behavior remains anchored, allowing the user to type without the input field scrolling off-screen or being occluded by mobile virtual keyboards. Long names and extended unstructured output must wrap correctly without breaking the flexbox boundaries. Furthermore, keyboard navigation is tested by injecting tab-key progressions to ensure all interactive elements remain reachable, and screen-reader labels are validated for accuracy. The harness also injects an emulation state for reduced motion (emulateMedia({ reducedMotion: 'reduce' })) to guarantee compliance with accessibility standards. Crucially, any screenshots taken by the harness for diagnostic purposes are subjected to automated redaction to ensure they do not contain secret bearer tokens, private prompts, or sensitive MATM routing decisions17.

9. Durable, Resumable Receipts

The output of the dogfood campaign is a cryptographic ledger composed of durable, resumable receipts. This schema provides the incontrovertible evidence required to support or reject product claims regarding the TinyRustLM architecture. The receipt schema is structured as an immutable JSON document capturing the campaign hash, the unique round identifier, the precise model identity (SHA-256), the exact Chromium browser build version, the evaluator's cryptographic signature, and a detailed array of evidence fields encompassing DOM mutations, peak memory allocations, and network intercept summaries. A resume algorithm is implemented to handle environmental interruptions, such as simulated power loss or host operating system crashes. A completed round is idempotently resumable only when all validation points—the campaign hash, model identities, browser build, and evaluator signature—strictly match the existing ledger. Failed rounds, whether due to a WASM trap or a secret-scanning violation, may be retried according to the configuration matrix, but their original failure receipts cannot be overwritten or silently converted into passes11. This design mirrors the Idempotency-Key and immutable revision strategies of the MATM API, ensuring that the history of errors, retries, and eventual successes is preserved transparently for architectural review17.

10. Resource Budget and Execution Schedule

Managing 50 complete rounds, particularly when each utilizes large local models up to 128 MiB in size, requires a highly disciplined resource and scheduling strategy. Because the current TinyRustLM implementation executes inference synchronously on the browser's main thread without Web Worker delegation, CPU starvation and memory fragmentation are critical risks1.

The campaign enforces a strict serial execution policy for the core conversation generation phases. While Playwright readily supports parallel test workers27, executing multiple main-thread WASM inference loops simultaneously on a single host machine results in severe CPU bottlenecking, thermal throttling, and artificial timeout failures that do not reflect genuine user experience. Parallelism is permitted exclusively during the teardown, network validation, and profile initialization stages, optimizing the overall schedule while protecting the integrity of the synchronous generation transaction.

11. Fault-Injection Matrix with Expected Evidence

A robust reliability campaign does not merely test the happy path; it actively introduces hostile conditions to validate the architecture's recovery transitions and error diagnostic capabilities2. The orchestrator injects 13 specific faults during designated rounds to observe the system's resilience.

12. Acceptance Report Format and Blocker Taxonomy

Upon conclusion of the 50 rounds, the campaign generates a comprehensive JSON acceptance report alongside a human-readable summary. Anomalies are classified according to a strict taxonomy aligned with the project's evidence gates1:

• Blockers (Severity 1): Unrecoverable failures that immediately halt promotion. These include cryptographic hash mismatches during model admission, main-thread panics causing the browser tab to crash, secret-scanning violations (e.g., leaked workspace keys in logs), attempted recursive deletions outside the temporary root, and explicit telemetry or network violations.
• Critical (Severity 2): Severe UX degradations. These include duplicate assistant text rendering, failure to accurately reconstruct MATM state in the secondary fresh profile, or memory leaks that cause the process to exceed the 1.2 GiB RAM budget over successive turns.
• Warnings (Severity 3): Manageable friction points. These include the Playwright SingletonLock requiring more than two deletion retries, minor layout shifts on mobile viewports, or delayed screen-reader announcements during WASM execution.

Progression from experimental deployment to stable support requires an acceptance report containing zero Blockers and zero Criticals.

13. Unknown Local Facts

The campaign operates with several localized variables that the orchestrator must dynamically account for during execution. Operating system sandboxing represents a major unknown; differences between Linux AppArmor, SELinux, and Windows Defender Application Control (WDAC) can significantly interfere with Playwright's executable caching and local memory mapping, occasionally requiring specific kernel-level whitelisting to prevent the userDataDir from locking28. Hardware concurrency is another variable; because the current WASM module lacks threading, inference speeds will fluctuate wildly depending on the host machine's specific single-core IPC (Instructions Per Clock) and L3 cache latency1. Finally, filesystem latency heavily influences the speed of profile creation and recursive deletion, making the exponential backoff retry loop a necessity rather than a precaution across diverse NVMe and SATA SSD hardware environments.

14. Direct Primary-Source Links with Dates

The architectural constraints, protocol mechanisms, and code implementations informing this design were synthesized from primary documentation retrieved and reviewed against the July 14, 2026, execution context:

• The integration of the MultiAgentMemory API, sync engine constraints, and the UAIX protocol was derived from https://multiagentmemory.com/docs/api-reference.html17.
• The memory limitations, including the 128 MiB ceiling, diagnostics staging, and load transactions, were sourced from the MiRust Implementation Operations v1.5 documentation at https://mirust.com/implementation-operations/2.
• The technical specifics of the SLM1 format, the 108-byte header, tensor rules, and deterministic WASM execution were identified in the MiRust implementation source audit at https://mirust.com/implementation/1.
• The model count topology and architectural JSON handoff requirements were referenced from the MiRust Tiny Model Composer at https://mirust.com/composer/29.
• The lifecycle management of the userDataDir and Playwright's persistent contexts were modeled on https://playwright.dev/docs/api/class-browsertype4.
• The methodology for bypassing network limits via DOM input was formulated using the Playwright FileChooser API documentation at https://playwright.dev/docs/api/class-filechooser14.
• The understanding of storage double-keying and cross-site privacy was informed by Chrome's Storage Partitioning documentation at https://developer.chrome.com/docs/extensions/develop/concepts/storage-and-cookies21.
• The mechanics of dynamic state isolation and third-party context partitioning were sourced from Mozilla Firefox's State Partitioning guides at https://developer.mozilla.org/en-US/docs/Web/Privacy/Guides/State\_Partitioning18.
• The procedures for offline caching, IndexedDB retrieval, and Service Worker scraping were developed using Playwright's testing guides at https://playwright.dev/docs/service-workers16.

Works cited

1. Implementation \- MiRust, https://mirust.com/implementation/
2. Implementation operations \- MiRust, https://mirust.com/implementation-operations/
3. Models \- MiRust, https://mirust.com/models/
4. BrowserContext \- Playwright, https://playwright.dev/docs/api/class-browsercontext
5. node.js \- indexedDB doesn't keep DB, store or values when puppeteer closed Chrome, https://stackoverflow.com/questions/65031398/indexeddb-doesnt-keep-db-store-or-values-when-puppeteer-closed-chrome
6. Playwright storageState vs Persistent Context: Which One Should You Use for Multi-Account Automation? \- DEV Community, https://dev.to/web4browser/playwright-storagestate-vs-persistent-context-which-one-should-you-use-for-multi-account-k86
7. Using Persistent Context in Playwright for Browser Sessions \- BrowserStack, https://www.browserstack.com/guide/playwright-persistent-context
8. How to Run 10+ Playwright Agents on One Logged-In Browser | AgentPMT, https://www.agentpmt.com/articles/how-to-run-10-playwright-agents-on-one-logged-in-browser-without-getting-blocked
9. \[Bug\]: Persistent Context (userDataDir) Fails Cookie Read & Corrupts Profile · Issue \#35466 · microsoft/playwright \- GitHub, https://github.com/microsoft/playwright/issues/35466
10. What is the difference between running tests from Playwright extension in VS Code and with CI command \- Stack Overflow, https://stackoverflow.com/questions/79394302/what-is-the-difference-between-running-tests-from-playwright-extension-in-vs-cod
11. Retries \- Playwright, https://playwright.dev/docs/test-retries
12. There is a 50 MB upload limit when uploading files in the Playwright extension? · Issue \#1268 · microsoft/playwright-mcp \- GitHub, https://github.com/microsoft/playwright-mcp/issues/1268
13. FileChooserError in Playwright: Why does it Occur? \- testRigor AI-Based Automated Testing Tool, https://testrigor.com/blog/filechoosererror-in-playwright/
14. FileChooser | Playwright, https://playwright.dev/docs/api/class-filechooser
15. Automated Memory Leak Testing in the Browser • Cobertos Blog, https://cobertos.com/blog/post/automated-memory-leak-testing-in-the-browser
16. Test Offline Cache Behavior with Playwright \- QASkills.sh, https://qaskills.sh/blog/playwright-offline-mode-cache-testing
17. API And Data Reference | MultiAgentMemory.com \- MATM Documentation, https://multiagentmemory.com/docs/api-reference.html
18. State Partitioning \- Privacy on the web | MDN, https://developer.mozilla.org/en-US/docs/Web/Privacy/Guides/State\_Partitioning
19. Storage Partitioning Browser Privacy: Browser Isolation Guide (2026) \- Sendwin, https://blog.send.win/storage-partitioning-browser-privacy-browser-isolation-guide-2026/
20. Cookies Having Independent Partitioned State (CHIPS) \- Privacy on the web | MDN, https://developer.mozilla.org/en-US/docs/Web/Privacy/Guides/Third-party\_cookies/Partitioned\_cookies
21. Storage Partitioning | Privacy Sandbox \- Google, https://privacysandbox.google.com/cookies/storage-partitioning
22. View and change IndexedDB data | Chrome DevTools, https://developer.chrome.com/docs/devtools/storage/indexeddb
23. playwright using indexedDB \- javascript \- Stack Overflow, https://stackoverflow.com/questions/70179538/playwright-using-indexeddb
24. Service Workers | Playwright, https://playwright.dev/docs/service-workers
25. Playwright for Browser Automation in AI Agents: From Accessibility Trees to Agent Loops, https://bytetunnels.com/posts/playwright-for-browser-automation-in-ai-agents/
26. Release notes \- Playwright, https://playwright.dev/docs/release-notes
27. Best Practices \- Playwright, https://playwright.dev/docs/best-practices
28. Playwright blocked by AV \- Reddit, https://www.reddit.com/r/Playwright/comments/1r3l2sf/playwright\_blocked\_by\_av/
29. Tiny Model Composer \- MiRust, https://mirust.com/composer/
30. BrowserType \- Playwright, https://playwright.dev/docs/api/class-browsertype
31. Storage and cookies \- Chrome for Developers, https://developer.chrome.com/docs/extensions/develop/concepts/storage-and-cookies