Compact Model Pareto Frontier for TinyRustLM

Full report

On this page

Public runtime constraints

TinyRustLM’s public site describes the project as a browser-first Rust runtime focused on compact local models, and the companion MiniModel site as the place for browsing TinyRustLM-ready candidates and publishing verified footprint information. The public TinyRustLM materials also indicate that the browser path loads model and tokenizer data client-side, verifies bytes, and then copies verified model bytes into WASM linear memory; in practice that means first-run browser peak memory is not just the quantized model size, but the transient sum of JavaScript-resident bytes, WASM-resident bytes, tokenizer bytes, KV cache, and runtime scratch. The public MiniModel notes also matter operationally because they imply a static, footprint-oriented registry rather than a hosted-weight distribution service, which is exactly the right constraint model for a TinyRustLM.com-compatible browser catalog.

The public TinyRustLM materials also make one architectural point especially important for model selection: the currently advertised browser/runtime support is centered on compact decoder-only families such as Llama-family and Qwen2-family layouts, with grouped-query or multi-query attention and either tied or untied embeddings. That makes SmolLM2 and TinyLlama straightforward fits because they are Llama-family models, and Qwen2.5-0.5B-Instruct a straightforward fit because it is a Qwen2-family model. By contrast, OLMo 2 is attractive on license and quality grounds, but its distinct olmo2 architecture is not part of the public TinyRustLM support statement I could verify from the site, so it should be treated as a conversion-and-runtime-extension project rather than a drop-in TinyRustLM browser candidate today.

I therefore screened the space in two passes. The first pass kept only instruct/chat-capable models with public model cards and commercially redistributable licenses. The second pass favored families that line up with the public TinyRustLM browser/runtime story. That leaves three true finalists for present-day TinyRustLM browser deployment: SmolLM2-135M-Instruct, SmolLM2-360M-Instruct, and Qwen2.5-0.5B-Instruct. TinyLlama-1.1B-Chat-v1.0 remains a useful reference point but sits slightly above the user’s nominal 1B ceiling and is dominated on context and structured-output behavior by Qwen2.5. OLMo-2-0425-1B-Instruct is high quality and Apache-2.0, but today it is more a “future runtime target” than a practical TinyRustLM browser default.

Normalized candidate table

The table below normalizes the commercially viable instruct/chat candidates I judge relevant for TinyRustLM browser deployment, plus two reference candidates that are important for comparison. Architecture, tokenizer, license, and file-size fields come from official model cards, config/tokenizer files, and official file listings. The q8/q6/q5/q4/q3/q2 download figures, KV cache math, peak-browser-memory figures, and latency bands are my estimates from the equations in the next section; they are not published vendor claims. SmolLM2-135M’s BF16 weight size is estimated from nominal parameter count because I did not have a direct official safetensors size line in the captured source bundle for that row.

Two frequently requested families should be explicitly screened out, not silently ignored. Apple’s OpenELM instruct models are released under the Apple Machine Learning Research license and are limited to research purposes, explicitly excluding commercial exploitation, product development, and commercial product or service use. Meta’s MobileLLM and MobileLLM-R1 weights are released under FAIR noncommercial licenses, which likewise block commercial redistribution and commercial use. Because the user asked to exclude licenses that prevent commercial redistribution, neither family belongs on the commercial TinyRustLM deployment frontier despite their technical interest.

Footprint equations

The right way to estimate browser-local footprint here is not “parameter count times bits per parameter” in isolation. TinyRustLM’s public browser story implies at least five materially different buckets: quantized weight bytes, tokenizer bytes, transient JavaScript copies during fetch/verification, persistent WASM linear-memory copies after import, and live decode-state memory such as KV cache and small operator scratch. That is why a 230 MB q4 artifact does not behave like a 230 MB browser app at first load.

I used the following normalized equations:

\[ D_{\text{total}} \approx D_{\text{model,quant}} + D_{\text{tokenizer}} \]

\[ D_{\text{model,quant}} \approx S_{\text{bf16}} \times r_q \]

where \(S_{\text{bf16}}\) is the official BF16/FP16 weight artifact size from the public file listing and \(r_q\) is the implied byte ratio relative to 2-byte weights. For TinyRustLM-style block quantization, the public docs describe q8, q6, and q4 in terms of fixed groups with per-block scale overhead. That yields effective bytes-per-weight of roughly 1.125 for q8, 0.875 for q6, and 0.625 for q4, corresponding to ratios \(r_q\) of 0.5625, 0.4375, and 0.3125 versus BF16/FP16. I then extrapolated q5, q3, and q2 with the same 32-weight-plus-scale logic, giving 0.75, 0.50, and 0.375 bytes per weight, or BF16 ratios of 0.375, 0.25, and 0.1875. Those q5/q3/q2 numbers are my estimates, not published TinyRustLM figures.

For live decode-state memory, I used the standard decoder-only KV formula:

\[ KV_{\text{bytes/token}} = 2 \times L \times H_{kv} \times d_{\text{head}} \times b \]

where \(L\) is layer count, \(H_{kv}\) is the number of KV heads, \(d_{\text{head}} = \frac{\text{hidden size}}{\text{Q heads}}\), and \(b\) is bytes per cached element. I assumed FP16 KV entries in browser/WASM, so \(b = 2\). That produces approximately 22.5 KiB/token for SmolLM2-135M, 40 KiB/token for SmolLM2-360M, 12 KiB/token for Qwen2.5-0.5B, 22 KiB/token for TinyLlama-1.1B, and 128 KiB/token for OLMo-2-1B. OLMo’s very large KV footprint is the direct result of having 16 full KV heads with 128-d head size rather than aggressive grouped-query attention.

For browser memory, I used two levels:

\[ M_{\text{steady,WASM}} \approx D_{\text{model,quant}} + D_{\text{tokenizer}} + KV_{\text{live}} + M_{\text{scratch}} + M_{\text{runtime}} \]

\[ M_{\text{peak,browser}} \approx 2 \times (D_{\text{model,quant}} + D_{\text{tokenizer}}) + KV_{\text{live}} + M_{\text{scratch}} + M_{\text{runtime}} \]

The factor of two in the peak estimate is the important browser-local penalty: one copy still lives in JavaScript-managed memory while another has already been imported into WASM linear memory. I used small fixed scratch/runtime allowances of roughly 40–80 MB total depending on model width, which is conservative but realistic for a one-token-at-a-time decode loop. These are not official TinyRustLM numbers; they are engineering estimates anchored to the public “copy verified bytes into WASM memory” loading model and typical WebAssembly allocator/scratch needs.

Latency is the least source-verifiable dimension because the public model cards do not publish TinyRustLM-browser token/sec numbers for these exact models. My desktop and mobile latency bands are therefore engineering estimates for q4–q5 decode on contemporary CPU-class browser environments. As an approximate rule, SmolLM2-135M can feel instant on desktop and tolerable on modern phones, SmolLM2-360M is still interactive on desktop and borderline-but-usable on higher-end phones, and Qwen2.5-0.5B is best thought of as a good desktop / flagship-mobile model rather than a universal low-end-mobile default. Those ranges should be validated empirically in the browser with the exact TinyRustLM build before publishing a MiniModel-ready footprint claim.

Quality and licensing evidence

SmolLM2’s official positioning is “on-device” and “compact,” with Apache-2.0 licensing. The 135M instruct model’s official Hugging Face README reports improved instruction-following over the prior SmolLM-135M-Instruct, including IFEval 29.9 versus 17.2 and MT-Bench 19.8 versus 16.8, but those same official numbers still imply a low ceiling for demanding reasoning or careful developer-oriented explanation. Meta’s official MobileLLM-R1 comparison table is also useful here: it places SmolLM2-135M-Instruct at 2.4 GSM8K and 0.0 LiveCodeBench, and SmolLM2-360M-Instruct at 8.1 GSM8K and 0.7 LiveCodeBench. Those are not TinyRustLM-specific results, but they are published benchmark claims and they fit practical experience: 135M is a “micro-assistant” ceiling, while 360M is the first size that starts to feel reliably helpful across several consumer tasks.

Qwen2.5-0.5B-Instruct is the strongest compact commercial candidate I found once Whisper-sized browser practicality and license filtering are both enforced. Its official model card is Apache-2.0, states that the architecture uses RoPE, SwiGLU, RMSNorm, QKV bias, tied embeddings, 24 layers, 14 Q heads with 2 KV heads, and a 32,768-token model context, while also emphasizing improved instruction following, long-text generation, table/structured-data handling, and especially structured outputs including JSON. That last point matters a lot for TinyRustLM.com because “genuinely useful” local assistants usually live or die on whether they can return short, valid structured outputs and clean rewrites rather than just conversational fluff. The tokenizer template is also richer than SmolLM2’s because it carries a tools branch in addition to the normal <|im_start|>system/user/assistant dialog path, so any TinyRustLM packaging flow should preserve at least the non-tool chat branch exactly.

TinyLlama-1.1B-Chat-v1.0 remains a valid Apache-2.0 baseline and is still easy to use because it follows Llama-tokenizer conventions and a Zephyr-style chat format. The official model card explicitly says the project uses the same architecture and tokenizer as Llama 2 and that the chat model was then tuned with UltraChat and UltraFeedback using a Zephyr-like post-training recipe. The problem is not that TinyLlama is “bad”; the problem is that in 2026 it is not on the relevant browser-local Pareto frontier for this use case. Its context is only 2K, its q4 local footprint is still large, and it does not offer an obvious quality-per-MB advantage over Qwen2.5-0.5B or SmolLM2-360M in the TinyRustLM browser setting.

OLMo-2-0425-1B-Instruct is the best “if browser memory were not the bottleneck” model in this report. Its official model card is Apache-2.0, documents a full post-training path through Tülu 3, DPO, and RLVR, and publishes a strong comparative performance table in which OLMo 2 1B materially improves over earlier OLMo 1B and competes surprisingly well against larger small models across GSM8K, IFEval, MATH, and MMLU-like mixtures. It also has a clear and simple chat template. If TinyRustLM were a native runtime with looser memory budgets, OLMo 2 1B would be one of the most interesting fully open small-model targets. In the browser, though, its untied embeddings, large tokenizer, and especially enormous KV cache make it a poor fit for current TinyRustLM.com-style deployment.

The biggest licensing gotcha in this whole compact-model space is that a number of technically appealing candidates do not actually satisfy “commercial redistribution.” OpenELM is explicitly restricted to research purposes and excludes commercial exploitation, product development, and commercial products or services. MobileLLM and MobileLLM-R1 are explicitly FAIR noncommercial research models. That makes them unsuitable for the distribution target the user described even though they are otherwise serious compact-model efforts and worth benchmarking in internal R&D.

Pareto frontiers and deployment tiers

For the under 80 MB download frontier, the only commercially viable instruct/chat candidate that even approaches the line is SmolLM2-135M-Instruct at an estimated q3 first-run download of roughly 76 MB including tokenizer bytes. I do not recommend treating that as the “smallest genuinely useful” target for the full TinyRustLM.com task bundle, because q3 at 135M is much too fragile for consistent code explanation and short reasoning. My practical conclusion is that the under-80-MB frontier exists, but it is more of a demo frontier than a product frontier.

For the under 160 MB download frontier, there are two non-dominated estimated points: SmolLM2-135M-Instruct around q4 at roughly 94 MB total and SmolLM2-360M-Instruct around q2 at roughly 139 MB total. I would only ship the first one. The second point is mathematically attractive but operationally suspect: q2 usually gives away too much quality. In real product terms, the under-160-MB frontier is best read as “SmolLM2-135M is the only sane commercial option, but it is still a constrained assistant.”

For the under 300 MB download frontier, the interesting pair is SmolLM2-360M-Instruct at q4, roughly 230 MB total, versus Qwen2.5-0.5B-Instruct at q3, roughly 258 MB total. This is the most informative frontier in the report. SmolLM2-360M q4 is the safer product choice because q4 usually preserves small-model behavior more gracefully than q3. Qwen2.5 q3 is the higher-ceiling choice if you are willing to trade some quantization risk for better base-model capability and longer context. In other words: under 300 MB, SmolLM2-360M-q4 is the conservative optimum; Qwen2.5-0.5B-q3 is the aggressive optimum.

For the under 512 MB peak browser memory frontier, the story tightens sharply because TinyRustLM’s load path implies JS-plus-WASM duplication during import. SmolLM2-135M-Instruct at q4 fits comfortably. SmolLM2-360M-Instruct at q4 is a near-threshold candidate that lands around half a gigabyte in my estimate and could fit or narrowly miss depending on how quickly JavaScript buffers are released and how much scratch the exact runtime build needs. Qwen2.5-0.5B-Instruct only gets under this peak budget at very aggressive low-bit settings like q2, which undermines the reason to choose Qwen in the first place. So for a hard 512-MB browser peak budget, SmolLM2-360M-q4 is the practical upper frontier and SmolLM2-135M-q4 is the safe frontier.

For the under 1 GB peak browser memory frontier, Qwen2.5-0.5B-Instruct becomes the clear winner. Its q4, q5, and even q6 operating points fit under or around a 1-GB peak-browser budget in my model, and every one of those points dominates TinyLlama’s corresponding low-bit settings on practical value because Qwen gives more context, better compact instruction tuning, and explicitly better structured-output behavior. OLMo-2-1B does not make this frontier in a robust browser sense because even its aggressively quantized settings still drag heavy tokenizer bytes and unusually large KV-state costs.

That leads to three deployment tiers. The ultra-compact tier is SmolLM2-135M-Instruct q4 for “always fits, feels instant, but keep prompts and expectations small.” The balanced browser tier is SmolLM2-360M-Instruct q4 for mainstream TinyRustLM.com usage. The high-quality local tier is Qwen2.5-0.5B-Instruct q4 by default, with q5 as an optional desktop-class upgrade and q3 as a sub-300-MB alternative if download size matters more than output stability.

Recommendations and .slm conversion plans

The smallest candidate likely to be genuinely useful for the full TinyRustLM.com bundle is SmolLM2-360M-Instruct, not the 135M model. The 135M model is still useful for lightweight chat, rewrite, and short summaries, but once “code explanation,” “structured JSON,” and “short reasoning” become non-negotiable, 360M is the first size that crosses from “cute demo” to “real local assistant.” Its Llama-family architecture is publicly aligned with TinyRustLM’s current browser/runtime story, its tokenizer is small, its 8K context is enough for real summarization and rewrite tasks, and its q4 operating point is still within reach for serious browser deployment.

The best quality candidate with a still-reasonable local footprint is Qwen2.5-0.5B-Instruct, with q4 as the default TinyRustLM browser target and q5 as the premium desktop-class variant. Qwen2.5’s official model card makes it the cleanest fit for structured output and JSON-heavy local workflows, and its grouped-query attention keeps KV memory unexpectedly low for the quality you get. The only real downside is tokenizer weight: its tokenizer bundle is much heavier than SmolLM2’s, so first-load bandwidth and peak import memory are materially worse even before generation begins. If the product wants the best “local model that still feels like a real assistant,” this is the right answer.

For SmolLM2-135M-Instruct, the concrete .slm plan should target q4 as the shipping profile and q3 as an optional “micro” profile. Pull the official config.json, tokenizer.json, vocab.json, merges.txt, tokenizer_config.json, and special tokens, preserve the exact <|im_start|>/<|im_end|> template and default-system behavior, convert the Llama-family tensors into TinyRustLM’s .slm layout, then verify parity with a fixed greedy-decoding harness over at least six prompts: short chat, rewrite, short summary, tiny code explanation, JSON extraction, and one easy reasoning prompt. Verification should compare first-token logits or first-16-token greedy trajectories against a trusted PyTorch reference before doing browser-level acceptance. For this tier, reject any quantization that materially breaks JSON validity on a 20–50 prompt schema-extraction set because that task is far more fragile than casual chat.

For SmolLM2-360M-Instruct, the conversion plan is the same Llama-family path but with q4 as the primary artifact and q3 as the “download-sensitive” fallback. Because this is the first model I would actually productize, I would add a second verification layer beyond token-parity: browser telemetry. Measure first-load bytes, time-to-first-token, tokens/sec, steady-state WASM memory, and peak memory during import on a representative desktop browser and one representative high-end phone. Publish those measurements through MiniModel rather than only the raw artifact size, because the site’s public role is clearly to provide verified fit information rather than vague model descriptions. This is also the right tier to create task-specific eval gates: valid JSON rate, rewrite preference rate, and code-explanation clarity score on a short internal rubric.

For Qwen2.5-0.5B-Instruct, the conversion plan should ship two browser profiles: q4 as the standard build and q3 as the sub-300-MB build. Preserve the official Qwen2 tokenizer exactly, including <|im_start|> and <|im_end|> IDs and the non-tool chat path from the official tokenizer template. If TinyRustLM does not implement the tool-call branch from the tokenizer Jinja template, do not approximate it loosely; instead, ship a simplified officially derived chat template limited to system, user, and assistant turns and document that choice in the MiniModel metadata. Verification should include a structured-output suite that checks both JSON syntax and schema adherence, because Qwen2.5’s differentiator is not just “better quality” but “better quality on structured local assistant tasks.” This is the finalist where I would insist on golden-prompt regression tests every time the tokenizer or template changes, because small prompt-format drift can waste the very advantage Qwen brings.

If TinyRustLM’s public runtime expands beyond its currently advertised architecture focus, then OLMo-2-0425-1B-Instruct is the first model I would revisit. But for the current public browser-local TinyRustLM and MiniModel ecosystem, the deployment answer is simpler than the size range might suggest: SmolLM2-360M-Instruct q4 is the practical default, Qwen2.5-0.5B-Instruct q4 is the quality leader with a still-reasonable local footprint, and SmolLM2-135M-Instruct q4 is the emergency-small tier rather than the recommended mainstream tier.