Runtime

Trustworthy Metadata and Verification Framework for .slm Model Packages

Report summary

The current public .slm description in TinyRustLM already points in the right direction: the format is intentionally narrow, uses strict header and tensor-directory validation, binds artifacts with checksums, includes tokenizer identity checks, and is designed for bounded local loading rather than a

Status
Research archive item
Category
Runtime
Length
5,243 words
Reading time
24 minutes
Report type
evaluation

Key topics

  • Runtime
  • AI
  • Python
  • Rust
  • GGUF
  • Privacy
  • Semantic Systems
  • Research Archive

Research provenance

Archive status
Research archive item
Content identity
sha256:89267f397512fab7b5cab14025c9b517ed9ef07eb9e0f8d57c71b57c3798ba21

For citation, use the report title and canonical URL. Archival presence does not establish authorship or promote report statements into portfolio evidence.

Source availability: 41 citation markers in the source export have no recoverable source links. Those markers are omitted from this reader; any supplied bibliography and ordinary links remain. Check the original sources before relying on the cited claims.

This page renders the archived Markdown as safe, formatted HTML. It is background research and does not become a portfolio claim without evidence review.

Full report

On this page

Executive summary

The current public .slm description in TinyRustLM already points in the right direction: the format is intentionally narrow, uses strict header and tensor-directory validation, binds artifacts with checksums, includes tokenizer identity checks, and is designed for bounded local loading rather than a broad, permissive “load anything” runtime. TinyRustLM’s public UI and developer docs also show that MiniModel interoperability is already treated as metadata-first, with local .slm admission, metadata-only catalogs, peer-offer proof, chunk hashes, Merkle fields, and local re-verification before a model becomes runnable. That combination strongly suggests that the next step for .slm should not be a looser all-in-one metadata blob, but a fail-closed trust envelope around a strict binary artifact.

This report therefore proposes a two-layer design. The first layer is the existing or minimally extended .slm binary ABI, which should remain compact and loader-critical: magic, format version, byte count, tensor layout, tokenizer digest, quantization facts, and artifact checksum. The second layer is a signed slm.manifest.v1.json sidecar, validated with JSON Schema Draft 2020-12, canonically hashed for signatures, and bound to the .slm blob through in-toto subjects and digest equality. This split preserves TinyRustLM’s bounded loader contract while allowing richer claims such as capability tier, tokenizer compatibility, chat template, provenance, evaluation evidence, machine-readable license restrictions, MiniModel compatibility, and P2P verification metadata.

The central trust decision should be policy-based and fail-closed. If a package claims to be instruction-tuned but omits a compatible chat template, loading should stop. If it claims “quality” but has no reproducible provenance or no evaluation sidecar, it should not be loadable as a quality model. If tokenizer identity is unknown, if native code is required but undeclared, if signatures do not chain to a trusted root, or if chunk/Merkle verification fails for P2P transfer, the runtime should refuse admission rather than silently downgrade behavior. This approach aligns with Hugging Face’s separation of model config, tokenizer files, and chat templates, with in-toto’s subject-binding model, with SLSA provenance expectations, and with TUF/Sigstore trust-root practice.

The clearest operational distinction is between three policy tiers. A demo fixture is safe to load for smoke-testing the loader and UI, but is explicitly non-claiming: no capability marketing, no “assistant quality” implication, and no ambiguous instruction-tuned status. An experimental model is runnable, but evidence is incomplete: maybe provenance is partial, evals are self-reported, or chat-template compatibility is declared but not independently reproduced. A quality model is only one that has complete loader metadata, tokenizer and template agreement, signed provenance, reproducible or at least auditable build details, and evaluation reports specific enough that a third party could rerun them. TinyRustLM’s own public docs already distinguish runtime-smoke admission from true quality claims, stating that assistant-quality claims require evaluation sidecars with exact case evidence.

Design basis and packaging model

TinyRustLM’s public .slm description makes two architectural choices that are worth preserving. First, the runtime is designed around strict browser loading, bounded validation, and compact local execution. Second, the format records explicit structural invariants: non-zero critical header fields, in-file tensor entry bounds, payload-count checks, checksums, and “no NaN or Infinity payloads.” TinyRustLM also explicitly states that tokenizer data belongs inside the model artifact or a verified sidecar path so runtime generation does not depend on Python-only tokenizer files. Those are not cosmetic details; they are the basis for a load-time trust boundary.

At the same time, adjacent ecosystems show why richer metadata cannot live only in the raw tensor container. Hugging Face commonly spreads semantically important information across config.json, generation_config.json, tokenizer.json, tokenizer_config.json, and special_tokens_map.json. Chat templates are stored with the tokenizer, and the wrong control tokens can “drastically” degrade performance even for models derived from the same base. Safetensors improves on pickle-based formats by providing a safer tensor container with parseable metadata and optional text-only header metadata, but even safetensors still does not solve higher-level questions like license restrictions, provenance, or P2P policy by itself.

For comparison, GGUF demonstrates the benefits of an extensible metadata key-value store inside a model file: single-file deployment, extensibility, mmap-friendly loading, and “full information” sufficient to load the model. But the same GGUF documentation also exists because generalized extensibility can otherwise produce ambiguity. TinyRustLM’s .slm posture is intentionally tighter than that. The best synthesis is therefore not “turn .slm into GGUF,” but “keep .slm narrow and add a formally validated, signed manifest layer around it.”

The proposed package model is:

model.slm
slm.manifest.v1.json
slm.attestations.intoto.jsonl
eval/
  capability-report.v1.json
  safety-report.v1.json
tokenizer/
  tokenizer.json
  tokenizer_config.json
  special_tokens_map.json
p2p/
  chunk-list.v1.json
  peer-offer.v1.json
license/
  use-policy.odrl.json

This structure is intentionally metadata-centric. The .slm blob remains the immutable subject. The manifest describes it. Attestations authenticate the manifest or the .slm blob itself. Tokenizer, eval, and P2P files are all digest-addressed sub-artifacts. The design follows in-toto’s statement/envelope model, where artifacts are bound by digest and are assumed to be immutable, while richer claims live in typed predicates.

erDiagram
    SLM_PACKAGE ||--|| SLM_BINARY : contains
    SLM_PACKAGE ||--|| SLM_MANIFEST : describes
    SLM_MANIFEST ||--o| TOKENIZER_BUNDLE : references
    SLM_MANIFEST ||--o| CHAT_TEMPLATE : embeds_or_points_to
    SLM_MANIFEST ||--o{ EVAL_REPORT : links
    SLM_MANIFEST ||--o{ ATTESTATION : authenticated_by
    SLM_MANIFEST ||--o| LICENSE_POLICY : constrains
    SLM_MANIFEST ||--o| MINIMODEL_PROFILE : maps_to
    SLM_MANIFEST ||--o| P2P_SHARE_METADATA : publishes
    SLM_BINARY ||--o{ ADAPTER_COMPAT : permits

The signature-bearing manifest should itself be canonicalized before hashing. JSON Schema Draft 2020-12 is the right validation basis for the manifest because it is current, stable, and expressly designed for vocabularies, dynamic references, and strict validation. For canonical signing of JSON, RFC 8785 JSON Canonicalization Scheme is a pragmatic fit because it exists precisely to make hashing and signature verification repeatable.

Proposed metadata schema

The proposed manifest should be structured, namespaced, and explicitly typed. A useful top-level shape is:

{
  "$schema": "https://example.org/slm/manifest/v1/schema.json",
  "manifest_version": "1.0.0",
  "package": {},
  "artifact": {},
  "runtime": {},
  "capability": {},
  "tokenizer": {},
  "chat": {},
  "quantization": {},
  "eval": {},
  "provenance": {},
  "license": {},
  "security": {},
  "minimodel": {},
  "p2p": {},
  "attestations": []
}

This proposed layout is grounded in four existing patterns. TinyRustLM exposes loader-critical facts such as checksums, tokenizer import, validation, quality, eval cases, and admission; Hugging Face separates model, tokenizer, generation, and chat-template metadata; in-toto shapes authenticated claims around typed statements and immutable subjects; and SPDX plus ODRL together provide a workable split between license expression and machine-readable restrictions.

The table below is normative for this proposal. “Fail-closed” means the loader must refuse admission, not merely warn.

FieldTypeRequiredValidation and cross-field rulesFail-closed behavior
package.schema_versionstring semverYesMust equal supported major version, e.g. 1.x.yReject manifest on unsupported major
package.manifest_idstring URI-safeYesStable identifier for manifest instanceReject if missing
package.created_utcRFC 3339 UTC stringYesMust end in Z; cannot be in distant future beyond policy skewReject if malformed
artifact.kindenumYesmodel, adapter, tokenizer_only, demo_fixtureReject unknown kinds
artifact.idstringYesReverse-DNS or MiniModel-compatible stable IDReject if empty
artifact.versionstring semverYesRequired for cache and policy pinningReject if malformed
artifact.file_namestringYesMust match actual file name loadedReject mismatch for P2P/import
artifact.size_bytesuint64YesMust equal actual bytes on diskReject mismatch
artifact.digest.sha25664-hex stringYesMust equal actual .slm digest and all attestation subjectsReject mismatch
artifact.layout_checksum64-hex stringYesMust equal runtime-computed tensor/layout checksumReject mismatch
runtime.formatenumYesExample: slm.v1; must match binary headerReject mismatch
runtime.architecturestringYesMust match actual tensor naming/layout family understood by runtimeReject unknown architecture
runtime.context.prompt_max_tokensintegerYes1..runtime.context.total_max_tokensReject if invalid
runtime.context.total_max_tokensintegerYesPositive; cannot exceed runtime or policy cap unless explicitly allowedReject or policy-block
runtime.weights_dtypeenumYesMust agree with quantization fields and tensor payloadsReject mismatch
capability.tierenumYesdemo_fixture, experimental, qualityReject unknown tier
capability.demo_onlybooleanYesMust be true when tier is demo_fixture; must be false for qualityReject inconsistency
capability.instruction_tunedbooleanYesIf true, chat.template or documented prompt format is requiredReject if template missing
capability.chat_capablebooleanYesIf true, chat template + required special tokens must validateReject if unresolved
capability.eval_statusenumYesnone, smoke_only, self_reported, reproduced, verifiedDowngrade or reject depending on tier
tokenizer.family_idstringYesStable compat family, e.g. llama3-bpe-v1; must match runtime expectationsReject if absent
tokenizer.primary_artifactdescriptor objectYesDigest, size, media type, pathReject if missing or mismatch
tokenizer.formatenumYestokenizer.json, sentencepiece, bpe_merges_vocab, tiktoken_converted, byte_fallback, custom_compiledReject unknown formats unless runtime plugin explicitly allowed
tokenizer.digest.sha25664-hex stringYesMust match actual tokenizer artifact or embedded tokenizer regionReject mismatch
tokenizer.vocab_sizeintegerYesMust agree with tokenizer artifact and model embeddings policyReject hard mismatch
tokenizer.special_tokensobjectYesIDs and strings must be unique and consistentReject duplicate or conflicting IDs
tokenizer.compatibility.strictbooleanYesIf true, exact digest match required; if false, family match can be allowed by policyReject if strict and digest mismatches
chat.template_formatenumConditionalRequired when chat_capable or instruction_tuned is trueReject if required and absent
chat.template_sourceenumConditionalembedded, inline, sidecar, tokenizer_configReject if source missing
chat.template_digest.sha25664-hex stringConditionalRequired unless fully inline and signed with manifestReject if mismatch
chat.required_rolesarray enumConditionalTypically user, assistant, optional systemReject if template cannot render declared roles
chat.eos_token_idsarray intConditionalMust exist in tokenizer vocab and not collide with forbidden control IDsReject mismatch
quantization.schemestringYesExamples: f32, q8_0, q4_0, gptq, awq, bnb-nf4Reject unknown scheme
quantization.weight_bitsintegerYes1..32Reject invalid
quantization.activation_bitsinteger or nullNoNull if not activation-quantizedReject if out of range
quantization.group_sizeinteger or nullNoPositive when scheme requires grouped quantizationReject missing required group size
quantization.per_channelbooleanYesIf true, channel axis metadata must be present where applicableReject if scheme needs axis and it is absent
quantization.toolchain.namestringYesConverter or library nameReject for quality if absent
quantization.toolchain.versionstringYesVersion string or commitReject for quality if absent
quantization.dequantization_risksarray enumYesFrom controlled vocabularyReject if omitted for experimental/quality
eval.summary.claimed_qualitybooleanYesIf true, tier cannot be demo_fixtureReject inconsistency
eval.reportsarray descriptorsConditionalRequired for quality; optional otherwiseReject quality if absent
provenance.authorstringYesHuman or service identityReject for quality if absent
provenance.organizationstring or nullNoRecommendedWarning only unless required by policy
provenance.source_repoURIYesSource or parent artifact originReject for experimental/quality if absent
provenance.source_commitstringYesImmutable revision or digestReject for experimental/quality if absent
provenance.build_pipelineobjectYesBuilder ID, run ID, pipeline URIReject for quality if absent
provenance.container_image_digeststring or nullNoOCI digest if build used a containerWarning for experimental; reject for quality if pipeline says containerized but digest absent
provenance.reproducible_buildbooleanYesIf true, reproduce info and attestation must existReject inconsistency
provenance.slsa_attestationdescriptorConditionalStrongly required for qualityReject by policy for quality if absent
license.expressionstringYesValid SPDX expression or LicenseRef-* formReject invalid expression
license.custom_refsobjectConditionalRequired if expression uses LicenseRef-*Reject missing referenced texts
license.use_policy_odrldescriptor or inline objectNoRecommended for non-SPDX usage restrictionsWarning unless policy requires
license.restrictionsarray enumYesControlled vocabulary mirrors ODRL/SPDX profile semanticsReject if tier/claim requires restrictions but this field is absent
security.requires_native_codebooleanYesMust be false for browser-local TinyRustLM-style safe load unless sandbox policy says otherwiseReject if true and policy disallows
security.custom_opsbooleanYesIf true, list of ops and libraries requiredReject if undeclared or disallowed
security.external_networkenumYesnone, loopback_only, declared_endpoints, unrestrictedReject any value above policy ceiling
security.filesystem_accessenumYesnone, read_package_only, read_user_selected_only, broadReject above policy ceiling
security.serialization_riskenumYesnone, pickle_like, native_plugin, custom_loaderReject unsafe or undeclared modes
security.sandbox_requiredbooleanYesMust be true if high-risk execution features are declaredReject inconsistency
minimodel.profile_versionstringNoDeclares mapping profile versionWarning if absent
minimodel.model_idstringNoMust equal or deterministically derive from artifact.idReject mismatch when MiniModel import path is used
minimodel.catalog_stateenumNometadata_only, proof_ready, p2p_readyReject impossible states
p2p.chunk_size_bytesintegerConditionalRequired for chunked P2P transferReject if chunked mode and absent
p2p.chunk_hashesarray hashesConditionalRequired if transfer is chunkedReject mismatch
p2p.chunk_list_sha25664-hex stringConditionalMust hash canonical chunk listReject mismatch
p2p.merkle_root64-hex stringConditionalRequired for resumable or multi-peer importReject mismatch
p2p.peer_offer_attestationdescriptorConditionalRequired for peer importReject if P2P import and absent
attestationsarray descriptorsYesAt least one manifest or artifact attestation required outside fully local trust modeReject if policy requires signatures and none exist

Tokenizer metadata deserves special treatment because it is one of the most common hidden failure modes. Hugging Face documents that tokenizer.json stores the learned vocabulary, tokenizer_config.json stores special tokens and often the maximum input length, and special_tokens_map.json maps special tokens. Chat templates are also stored with tokenizer metadata and can be written back into tokenizer_config.json. At the same time, not every tokenizer family is fully captured by a single file: some models are fully described by tokenizer.json, while some model-specific settings are not; and tiktoken’s tokenizer.model lacks added tokens and pattern strings unless converted to tokenizer.json.

Accordingly, this report recommends that .slm packages either embed a fully compiled tokenizer representation inside the .slm file or include a verified tokenizer bundle with at least tokenizer.json, tokenizer_config.json, and special_tokens_map.json whenever the provenance chain originates from Hugging Face-style assets. The manifest should carry both a strict digest and a compatibility family identifier. Exact digest match should be required for instruction-tuned or quality packages; family-level compatibility should only be permitted for explicitly marked experimental packages.

Validation and fail-closed policy

The loader should make trust decisions in three phases: integrity, semantic compatibility, and policy admission. Integrity verifies bytes, checksums, chunk hashes, Merkle roots, and signatures. Semantic compatibility verifies that architecture, tokenizer, special tokens, context limits, and chat template agree with one another. Policy admission then applies the local trust policy: whether native code is allowed, whether network access is acceptable, whether the package can occupy the requested tier, and whether the user’s trust anchors validate the signatures present. This separation keeps the runtime honest about what is true, what is compatible, and what is permitted.

Fail-closed behavior is particularly important for chat models. Hugging Face’s own documentation explains that chat models are still causal LMs expecting a token sequence, that different models use different control tokens, and that applying the wrong chat format can severely degrade results. It also notes that chat templates live with the tokenizer and that special control tokens should be added as special tokens so they are not split incorrectly. That means a package cannot safely claim instruction_tuned=true or chat_capable=true unless tokenizer, special tokens, and chat template are all present and mutually validated.

The same logic applies to context length. In common ecosystems, “how long the model can really accept” is spread across architecture config and tokenizer metadata: config.json describes the model blueprint, while tokenizers expose model_max_length, and tokenizer config may also surface the maximum acceptable input length. A trustworthy .slm package should therefore declare its own explicit runtime limits rather than forcing the loader to reconstruct them from multiple conventions. If context length is missing or contradictory, the loader should reject quality admission and at minimum refuse automatic loading.

Criteria for demo fixture, experimental, and quality tiers

A demo fixture is a model package whose purpose is to validate the loader, UI, conversion pipeline, or P2P pathway. It may be structurally valid and safe to load, but it must not imply usable assistant quality. That means capability.tier=demo_fixture, demo_only=true, and eval_status limited to none or smoke_only. TinyRustLM’s public UI already surfaces “Tiny Fixture” variants alongside tiny quantized forms, which is a strong precedent for making this tier explicit instead of leaving users to infer it.

An experimental package is a runnable artifact with enough metadata to validate structure and compatibility, but with incomplete trust evidence. Typical reasons include self-reported evals instead of reproduced evals, missing reproducible-build evidence, family-level rather than exact tokenizer compatibility, or incomplete safety reporting. TinyRustLM’s own public docs reflect this intermediate state by distinguishing “runtime-smoke” and admission from stronger “assistant-quality” claims.

A quality package should satisfy all of the following: successful loader-critical integrity validation; exact tokenizer and chat-template agreement when instruction tuning is claimed; complete provenance including source revision and build pipeline identity; signed attestation for artifact or manifest; valid license expression plus any extra restrictions; and at least one evaluation report with identifiable benchmarks, datasets, metrics, run date, and execution environment. The exact benchmark suites are open-ended and should remain policy-configurable, but the reporting structure must be fixed enough that a third party can replay or at least audit the claim.

Diagnostics and user-facing messages

Diagnostics should explain why a model was trusted, downgraded, or refused. The messages should expose evidence rather than generic “invalid package” text.

CodeConditionDecisionUser-facing message
SLM-I100All structural checks passedInformational“Package integrity verified: header, tensor layout, tokenizer digest, and artifact checksum all match.”
SLM-W220Tokenizer family matches but exact digest missingDegrade to experimental“Tokenizer appears compatible by family, but exact tokenizer bytes were not verified. Chat quality claims are downgraded.”
SLM-E230Instruction-tuned claim without valid chat templateReject“This package claims instruction tuning, but no compatible chat template could be verified. Loading is blocked to avoid misleading output quality.”
SLM-E310quality tier with no signed eval reportReject quality admission“Quality tier requested, but reproducible evaluation evidence is missing or unsigned.”
SLM-E420P2P chunk, chunk-list, or Merkle mismatchReject import“Peer transfer verification failed. At least one chunk hash or Merkle proof did not match the declared package.”
SLM-E510Native code, custom ops, or network access above local policyReject“This model requires execution privileges beyond your trust policy. It was not loaded.”
SLM-W610SPDX custom license reference without ODRL policyDegrade“License terms were partially machine-readable. Usage restrictions may require manual review.”
SLM-I700Demo fixture with successful smoke passInformational“Demo fixture loaded successfully. This package is for validation and demonstration only; capability quality is not asserted.”

These messages should also be reflected in a machine-readable decision record so that P2P imports, local catalogs, and MiniModel snapshots can preserve the reasoning chain across systems. TinyRustLM’s public UI already exposes diagnostic, provenance, validation, admission, quality, and eval fields, which makes a decision ledger a natural extension rather than a new concept.

MiniModel compatibility and migration

There does not appear to be a fully public standalone MiniModel specification page indexed on the public web. What is public is TinyRustLM’s developer documentation and runtime UI, both of which describe MiniModel as a metadata-first exchange layer using local models.v0.json, peer-source announcements/feeds, “proof-ready” and “P2P-ready” states, prepare-peer-share, import-peer-request, and a local share-metadata bundle carrying artifact SHA-256, fixed-size chunk hashes, chunk-list SHA-256, and Merkle roots. Because of that, the mapping below should be treated as a compatibility profile inferred from current public TinyRustLM materials, not as a claim about a formal independent MiniModel standard.

TinyRustLM or MiniModel-facing conceptProposed .slm manifest fieldMigration rule
model_idartifact.id / minimodel.model_idPreserve exact ID; if absent, derive from reverse-DNS + artifact version
models.v0.json local catalog entryminimodel.catalog_entryExport metadata only; never embed runnable bytes in a catalog snapshot
“proof-ready” stateminimodel.catalog_state=proof_readyRequire signed artifact/manifest + eval or provenance proof, but no peer source yet
“P2P-ready” stateminimodel.catalog_state=p2p_readyRequire proof-ready plus at least one verified peer source
Share metadata JSONp2p.share_metadataNormalize artifact hash, chunk hashes, chunk-list hash, Merkle root
Peer-source announcement/feed JSONp2p.peer_sources[]Treat as availability metadata only, never as proof of artifact integrity
Peer-offer proofp2p.peer_offer_attestationRequire signature and subject digest binding
Conversion request / receiptprovenance.conversionPreserve source model ID, source snapshot, toolchain, output digest
TinyRustLM UI “Quality” / “Eval Cases” / “Eval Checksum”eval.summary, eval.reports[]Map to structured eval reports, not free text
TinyRustLM UI “Tokenizer Import” / “Source Validation”tokenizer.*, security.*Normalize tokenizer identity and source review into strict fields

Migration rules

For existing MiniModel-facing inventories, the safest migration policy is additive and conservative. A metadata-only models.v0.json entry should migrate into the new schema even if evaluation and provenance are sparse, but such entries should default to experimental, never to quality. A peer-source announcement should never upgrade a package past proof_ready; only verified chunk/Merkle and signature checks should do that. And any imported local .slm without exact tokenizer identity should be eligible for “local structural validation” but not for quality or chat-capable admission. This mirrors TinyRustLM’s own public status buckets: P2P-ready, proof-ready-needs-peer, local-.slm-needed, external-.slm-review, and Rust-conversion-needed.

A practical migration recommendation is to preserve old MiniModel files as source documents and generate a new slm.manifest.v1.json plus a translation receipt. That receipt should record: source file names, source digests, translation time in UTC, fields that mapped exactly, fields inferred by policy, and fields left unresolved. This keeps the trust model auditable and avoids “silent enrichment” of a legacy catalog entry. The underlying idea is borrowed from supply-chain attestation practice, where transformed metadata should remain traceable back to immutable source subjects.

P2P import verification and provenance

P2P import is the place where ambiguity is most dangerous, so it should use the strictest path. TinyRustLM’s public docs already establish the key pattern: peer networking remains metadata-gated, users must consent before transfer or import, rows are not runnable until evidence exists, empty announcement feeds do not create P2P availability, and local Rust validation remains the final authority before admission. The right generalization is that availability metadata and integrity metadata must never be conflated. A peer URL says where bytes might be fetched; only digests, chunk proofs, signatures, and policy validation say whether the resulting package can be trusted.

The recommended verification protocol is:

flowchart TD
    A[Acquire manifest and peer-offer metadata] --> B[Validate JSON Schema]
    B --> C[Verify manifest signature or attestation]
    C --> D[Verify trust root and signer identity]
    D --> E[Check artifact digest subject binding]
    E --> F[Download chunk list and pieces]
    F --> G[Verify per-chunk hashes]
    G --> H[Verify chunk-list hash and Merkle root]
    H --> I[Reconstruct .slm artifact]
    I --> J[Recompute artifact sha256 and layout checksum]
    J --> K[Validate tokenizer and chat template compatibility]
    K --> L[Apply license and security policy]
    L --> M[Apply tier policy and eval gates]
    M --> N[Admit or reject package]

For signatures and attestations, the most interoperable choice is an in-toto attestation framework statement in a DSSE envelope, because in-toto is expressly designed for authenticated metadata about software artifacts, with statements that bind digests in subject, and DSSE-style envelopes that authenticate both payload and type. Sigstore can then be used as the operational signature system where online or organizational trust infrastructure exists, while preserving offline verification through signed bundles and pinned trust roots.

For trust anchors, the best current model is TUF-style root management. TUF’s root metadata names trusted keys and signature thresholds; targets metadata lists hashes and sizes; snapshot and timestamp metadata protect against freeze and mix-and-match attacks; and all signed metadata expires. Sigstore itself distributes its root of trust via TUF and documents both offline bundle verification and air-gapped trust-root scenarios. That makes TUF a strong template for .slm P2P import, especially for catalogs or organizations that need threshold trust, expiration, and key rotation.

For offline verification, the report recommends two modes. In the organizational mode, use Sigstore-compatible bundles plus a pinned TUF root snapshot distributed out-of-band. In the portable-share mode, allow detached Ed25519 signatures over the canonical manifest and over the chunk list, with an out-of-band keyring that is itself signed or otherwise pinned. The key insight is that offline verification must not require live access to a transparency log. Sigstore explicitly supports bundle-based proof of log inclusion without querying the log, and its tooling supports offline verification with trusted roots.

Provenance requirements

The provenance section should include at minimum: author, organization, source_repo, source_commit, build_pipeline.id, build_pipeline.run_uri, container_image_digest when containerized, builder_platform, reproducible_build, and a pointer to a SLSA provenance attestation. SLSA specifically defines provenance as verifiable information describing where, when, and how an artifact was produced. That aligns almost perfectly with the needs of a trustworthy .slm package.

If reproducibility is claimed, the manifest should also include a deterministic build recipe digest and the exact converter/toolchain versions used for tokenization and quantization. This matters because quantization is often not a semantically invisible step, and because even adjacent local-model ecosystems have already observed practical reproducibility drift across machines and toolchains. The standard posture should therefore be: reproducible if demonstrably reproducible; otherwise explicitly non-reproducible, with exact builder evidence still required for experimental and quality claims.

Evaluation, quantization, licensing, and security declarations

Evaluation reporting format

Model cards remain the best high-level precedent for evaluation reporting. Google’s model-card paper recommends documentation of intended uses, performance characteristics, and details of evaluation procedures. Hugging Face’s model-card docs similarly call out intended uses and limitations, training parameters, datasets used, and evaluation results, and treat model cards as metadata-bearing documentation essential for discoverability and reproducibility. Those principles should be kept, but .slm packages need a stricter machine-readable evaluation sidecar for gating trust decisions.

A recommended eval.report.v1.json schema should include:

  • suite_name
  • suite_version
  • benchmark_ids[]
  • dataset_ids[]
  • dataset_versions[]
  • task
  • metric_name
  • metric_value
  • higher_is_better
  • confidence_interval
  • sample_count
  • random_seed
  • date_utc
  • runner.name
  • runner.version
  • hardware
  • software_stack
  • prompt_template_digest
  • tokenizer_digest
  • artifact_digest
  • evaluator_type such as exact_match, human, LLM_judge, unsafe_content_classifier
  • notes

The exact suites should remain open-ended. That is the right policy choice because benchmark ecosystems are changing quickly, and TinyRustLM itself leaves room for local quality evidence rather than prescribing a single benchmark canon. What should not be open-ended is the evidence envelope: every quality claim should say what was measured, on what data, with what metric, under what runtime conditions, on what date in UTC, and against which exact artifact and tokenizer digests. Where a suite supports it, confidence intervals should be reported instead of point estimates alone; contemporary benchmark ecosystems such as MLCommons explicitly discuss tightening confidence intervals and versioning evaluators to preserve comparability.

For safety reporting, one capability-focused report and one safety-focused report should both be allowed, because “quality” is multidimensional. A package can be strong on summarization or coding yet poorly characterized on hazardous content behavior. MLCommons’ AILuminate family is a useful example of a benchmark family that distinguishes hazard categories and publishes public results with hazard-specific grading. This report does not propose AILuminate as mandatory, only as an example of the kind of structured safety reporting that quality-grade packages should link or embed.

Quantization metadata

Quantization metadata should be more explicit than “q4” or “q8.” TinyRustLM already names f32, q8_0, and q4_0 and explains that q4_0 saves bytes but requires unpacking and dequantization work in WASM. Elsewhere in the serving ecosystem, Hugging Face documents a wide range of quantization schemes, including GPTQ, AWQ, bitsandbytes, EETQ, Marlin, EXL2, and fp8, and distinguishes pre-quantized from on-the-fly quantization. Bitsandbytes further exposes 8-bit and 4-bit modes, including NF4 and FP4, and documents hardware compatibility. A trustworthy package therefore needs to declare not only the nominal precision, but also the operational form of quantization.

The minimal recommended quantization fields are:

  • scheme
  • weight_bits
  • activation_bits
  • group_size
  • block_size
  • per_channel
  • mixed_precision_components[]
  • toolchain.name
  • toolchain.version
  • quantized_from_digest
  • calibration_dataset when applicable
  • on_the_fly boolean
  • dequantization_required_for_runtime boolean
  • known_risks[]

known_risks[] should come from a controlled vocabulary such as: quality_loss, kernel_specific_behavior, requires_prequantized_weights, on_load_requantization, nondeterministic_repack, reduced_numerical_reproducibility, hardware_limited, and dequantize_before_eval. The purpose is not to scare users away from quantized models, but to prevent a “quality” label from hiding quantization details that materially affect the user’s trust decision.

License expression and machine-readable restrictions

License metadata should use SPDX expressions as the primary field because SPDX expressions already define single identifiers, AND, OR, WITH, and LicenseRef-* syntax for custom terms. That makes them the best baseline for machine-readable licensing. However, AI model licenses often include operational restrictions not easily captured by plain SPDX alone. For those, the best complementary layer is ODRL, which is expressly designed to model permissions, prohibitions, and duties over assets.

The recommended license block is therefore:

"license": {
  "expression": "Apache-2.0 OR LicenseRef-Model-Use-Restricted",
  "custom_refs": {
    "LicenseRef-Model-Use-Restricted": {
      "text_digest": "sha256:...",
      "uri": "..."
    }
  },
  "use_policy_odrl": {
    "...": "..."
  },
  "restrictions": [
    "no_military_use",
    "no_biometric_surveillance",
    "research_only"
  ]
}

Validation rules should be strict. If expression is syntactically invalid, reject. If it references LicenseRef-* entries that are not defined, reject. If restrictions[] is non-empty but no ODRL or equivalent structured policy is present, downgrade to experimental and require manual review for sensitive deployment paths. This prevents a package from carrying legal restrictions only in prose while still being auto-labeled “safe to load.”

Security checks

Security policy should assume that model packages can be dangerous even when weights themselves are inert. PyTorch’s documentation warns that torch.load() uses an unpickler under the hood and says to “never load data from an untrusted source.” PyTorch’s security guidance goes further, stating that models are effectively programs and that running untrusted models is equivalent to running untrusted code, recommending secure isolated environments such as containers or virtual machines. ONNX Runtime similarly exposes custom operators and execution-provider plugins, which are legitimate extension points but also an attack-surface signal if they are declared or required by a model package. Safetensors, by contrast, exists specifically as a safer tensor serialization alternative to pickle-like formats.

For .slm, the security declaration should therefore minimally include:

  • requires_native_code
  • custom_ops
  • custom_op_libraries[]
  • external_network
  • filesystem_access
  • serialization_risk
  • sandbox_required
  • sandbox_profile
  • unsafe_capabilities[]
  • declared_endpoints[]
  • loopback_only
  • telemetry_behavior
  • dynamic_code_loading

The default safe .slm posture for TinyRustLM-like browser-local loading should be:

{
  "requires_native_code": false,
  "custom_ops": false,
  "external_network": "none",
  "filesystem_access": "read_user_selected_only",
  "serialization_risk": "none",
  "sandbox_required": false,
  "dynamic_code_loading": false,
  "telemetry_behavior": "none"
}

Any package that deviates upward from that baseline should not be eligible for silent auto-load. If requires_native_code=true, custom_ops=true, serialization_risk!=none, or external_network exceeds local policy, admission should fail closed unless the loader is running in a compatible sandbox profile and the user has explicitly consented. TinyRustLM’s own public posture—local-only inference, no project-hosted model bytes, loopback-only endpoint checks for local control, and “Local Connect asks before app control”—supports exactly this conservative direction.

Example quality package manifest excerpt

The example below is illustrative and proposed, not an existing standard.

{
  "manifest_version": "1.0.0",
  "artifact": {
    "kind": "model",
    "id": "org.example.tinychat",
    "version": "1.2.0",
    "file_name": "tinychat-16m-q8_0.slm",
    "size_bytes": 16777216,
    "digest": { "sha256": "8f4c..." },
    "layout_checksum": "b731..."
  },
  "capability": {
    "tier": "quality",
    "demo_only": false,
    "instruction_tuned": true,
    "chat_capable": true,
    "eval_status": "reproduced"
  },
  "tokenizer": {
    "family_id": "llama3-bpe-v1",
    "format": "tokenizer.json",
    "digest": { "sha256": "aa92..." },
    "compatibility": { "strict": true }
  },
  "chat": {
    "template_format": "jinja",
    "template_source": "tokenizer_config",
    "template_digest": { "sha256": "de45..." },
    "required_roles": ["system", "user", "assistant"],
    "eos_token_ids": [128009]
  },
  "quantization": {
    "scheme": "q8_0",
    "weight_bits": 8,
    "per_channel": false,
    "toolchain": { "name": "tinyrustlm-packer", "version": "0.9.3" },
    "dequantization_risks": ["quality_loss"]
  },
  "license": {
    "expression": "Apache-2.0",
    "restrictions": []
  },
  "security": {
    "requires_native_code": false,
    "custom_ops": false,
    "external_network": "none",
    "filesystem_access": "read_user_selected_only",
    "serialization_risk": "none",
    "sandbox_required": false
  }
}

What matters here is not the exact field spelling, but the fact that the trust decision can be made without guessing. That is the central design goal of the framework. It is also exactly the opposite of the failure mode seen in older or broader ecosystems, where a model’s actual runtime behavior depends on unstated Python code, unstated tokenizer conventions, or unstated template assumptions.

Final recommendations

The most defensible way to evolve .slm is to treat the .slm blob as the immutable executable weight artifact and to treat trust metadata as a signed, versioned, schema-validated manifest and attestation bundle. That preserves what .slm already does well—strict loading, bounded validation, tokenizer checks, and compact local execution—while adding enough structured evidence for users and policy engines to answer the questions that actually matter: Is this only a demo? Is it really instruction-tuned? Does the tokenizer actually match? Is the claimed context length grounded? Was the package built from a known source? Is it safe to load under this runtime policy?

The most important normative rules are these: never infer capability from file name alone; never infer instruction tuning without a verified chat template; never infer tokenizer compatibility from architecture family alone when strict=true; never allow peer availability metadata to stand in for integrity proof; never allow “quality” without signed provenance and structured eval evidence; and never silently load a package that broadens execution privileges beyond the local policy baseline. Those rules are consistent with prevailing model-packaging practice, with modern software supply-chain verification, and with the explicitly conservative direction already visible in TinyRustLM’s public design documents.

In short, the recommended .slm trust model is: strict binary ABI, rich signed sidecar metadata, explicit compatibility declarations, reproducible provenance, policy-aware tiering, and fail-closed admission. That combination gives users a practical and explainable answer to the question they are really asking at import time: not merely “can this file be parsed,” but “should I trust what this package claims to be?”