Executive Summary

Full report

On this page

The Embedded Semantics site describes a registry-backed, multilingual concept identity system where stable ConceptCodes are assigned to meanings, and human-language expressions (with locale and provenanced review) serve as evidence. The current resolver only handles exact, reviewed expressions deterministically, abstaining (returning unknown_expression or ambiguous_expression) otherwise. Embedding vectors are explicitly not used as identity evidence. Concept definitions are versioned and provenance-aware, and multilingual synonyms map to the same ConceptCode.

To scale this to Internet-wide stable identities, a new architecture must combine the best of established systems: the global uniqueness and resolvability of URIs, the hierarchical delegation of DNS, and the decentralized trust of emerging identifier frameworks (e.g. DID).

Key requirements include persistent uniqueness, authority, decentralization trade-offs (Zooko’s triangle), robust versioning/deprecation (so identities remain valid even as meanings evolve), a clear owner/trust model for namespaces, support for provenance and cryptographic signing, and mechanisms for offline caching, federation, and conflict resolution. In practice, this suggests a hybrid namespace: ConceptCodes as URI-like identifiers (e.g. concept://<namespace>/<localName> or domain-style codes), owned by organizations (like DNS zones). A root registry (or meta-registry) can delegate subspaces to independent publishers but publish global records (similar to how IANA or identifiers.org maintain namespace prefixes). Each concept record would carry publisher, revision, and cryptographic signatures; multiple mirrored “registry” services (logs or distributed databases) would synchronize changes (possibly via append-only logs akin to Certificate Transparency). Relations (equivalence, hierarchy, etc.) would use semantic-web standards (e.g. SKOS/OWL properties).

We recommend a phased approach: Year 1 to define the ConceptCode URI scheme, governance model, and prototype a federated registry (experimentally using CRDT or blockchain-like logs for sync). Year 3 to expand to dozens of domains/languages and build trust services (digital signatures, reputation). Year 5 to have a robust ecosystem of registries (including offline clients) with measurable adoption (e.g. millions of concepts, API usage) and clear “success criteria” (e.g. ≤1% lookup ambiguity).

Key decision points will test whether stable semantic IDs reduce ambiguity and scale (via pilots) before standardizing. Critical open questions include the best way to encode “contradiction” or composition in concept relationships, and how to align independently created ontologies without a single authority. Our detailed analysis below draws on facts from EmbeddedSemantics.com, relevant standards (DNS, URIs, SKOS, etc.), and engineering inference.

Current Public-System Interpretation

Facts (EmbeddedSemantics.com): The site repeatedly emphasizes authority by registry, not by model similarity. It is “registry-backed multilingual semantic representation” where “persistent concept records define meaning” and “language expressions provide governed evidence”. The ConceptCode (e.g. mt.representation.machine_readable) is the canonical ID for a concept. Reviewed expressions (with language tags) attach to a ConceptCode along with locale, review status, equivalence strength, and provenance. The current “production resolver” accepts only these reviewed exact expressions: it “maps reviewed exact expressions to stable concept identifiers” and abstains otherwise. Unseen or ambiguous inputs yield special tokens (unknown_expression, ambiguous_expression). The site clearly states: “Stable concept identifiers … are authoritative. Embedding vectors … do not define concept identity”.

Facts from site: Embedded Semantics thus guarantees that (a) each ConceptCode is the single, stable handle for a meaning; (b) multiple reviewed translations can map to the same ConceptCode; (c) ConceptCodes and their definitions are immutable records (with version history); (d) resolution is deterministic exact-matching based on the registry. There is no current “learning” or fuzzy matching: unreviewed expressions are considered unknown.

Inference: In summary, the site describes Embedded Semantics as a controlled registry of concept identities (ConceptCodes) with attached multilingual evidence. Its core design is “Separate the probability from the meaning”: concept identity is independent of any probabilistic model. Thus any large-scale evolution must retain this invariant: a ConceptCode is authoritative (like a database key) and not derived from or changed by a statistical algorithm.

Core Invariants a Semantic Identity System Must Preserve

Any global semantic identity system must guarantee the following invariants:

• Global Uniqueness: Each concept has one and only one identifier (and vice versa) across the system. (Analogous to “One thing = one identifier”.) This prevents ambiguity and ensures interoperability.
• Persistence: Once assigned, an identifier must remain permanently tied to the same concept. It should never be reassigned to a different concept. Historical mappings (versions, deprecated IDs) may be preserved, but IDs are not reused. This matches practices like ISO standards and identifiers.org, which keep retired codes marked but never repurposed.
• Resolution: An identifier should be resolvable to authoritative information (e.g. concept definition, provenance). Even if offline or moved, the system must allow retrieval of the concept record. URIs exemplify this: they are strings that identify an entity (possibly abstract). A semantic ID system should support HTTP/URIs or URNs so that ConceptCodes can be looked up globally.
• Authority & Governance: The system must ensure trust in concept definitions. Registry entries should record who defined or reviewed the concept, with trusted validation (e.g. signed by authority). This aligns with the site’s emphasis that definitions and registry entries are authoritative.
• Immutability of Meaning: A concept’s meaning (its definition) should not change silently. If consensus about a concept’s meaning shifts, either the concept is versioned/deprecated, or a new identifier is created. This echoes SNOMED CT practice: “every SNOMED concept identifier retains its semantic integrity over time”.
• Multilingual Synonymy: Different language expressions for the same concept should unify to one ID; the ID’s meaning transcends language. The site affirms this: “Reviewed expressions in different languages can map to the same stable ConceptCode”.
• Transparency: Changes to concept records should be inspectable. Version history, audit logs, and provenance metadata must be accessible to ensure the system’s trustworthiness (similar to recordkeeping in W3C PROV standards). The site notes that records are versioned and provenance-aware.
• Avoiding Opaque Conflicts: The system should not rely on similarity measures that can merge or split concepts unpredictably. Instead, it should use explicit equivalence assertions (skos:exactMatch etc.) to represent when two concept codes refer to the same meaning. The site explicitly forbids silent overrides by vector similarity.

These invariants echo best practices from identifier registries: for example, Identifiers.org requires that each namespace prefix “is globally unique and stable” and resolves reliably. A URI is by definition unique and immutable for its resource. A UUID offers decentralized uniqueness, but by itself provides no meaning or versioning—here the chosen ConceptCode scheme must embed authority info or rely on registry governance.

Comparison with Relevant Systems

We compare these requirements to existing schemes:

• DNS (Domain Name System): DNS is a hierarchical, distributed namespace. Authority is delegated: top-level zones (like .com) are centrally managed (by ICANN/IANA), but subdomains can be administered by organizations. DNS names are human-readable and globally unique; changes propagate via caching. Analogy: One could assign each publisher a DNS-like prefix (or use real domains) under which they issue ConceptCodes. As with DNS, avoiding a single central database increases fault tolerance. DNS also supports DNSSEC for authenticity, hinting that concept codes could similarly use cryptographic signatures.
• URI/IRI: URIs provide unique, global identifiers for any resource. For example, http://example.org/concept/mt.representation.machine_readable could be a ConceptCode. A URI can be a name (URN style) or also retrievable (URL). The system should probably use URIs so concepts are Web-resolvable. Importantly, URIs do not imply central control: any organization can mint a URI under its own domain. The trade-off is coordination to avoid collisions unless namespaces (domains) are controlled.
• UUID/GUID: UUIDs are 128-bit identifiers that are “for practical purposes unique” without a central authority. They solve collision risk but are opaque (not human-friendly) and provide no built-in provenance. One could use UUIDs for concept IDs to decouple generation, but this sacrifices readability and meaning. In practice, many registries combine a human-readable namespace with a random suffix (e.g. mt:123e4567-e89b-12d3-a456-426614174000). The key point: decentralization vs meaningfulness (Zooko’s triangle).
• IANA registries (media types, etc.): IANA centralizes registration of identifiers (e.g. MIME types). Each entry lists name, description, status, and references. Example: the media types registry has a Standards Tree plus Vendor/Personal trees. This shows how expert review is used for standard identifiers. For semantic IDs, a similar body could vet new concept namespaces or high-level categories. However, IANA is centralized and can be a bottleneck.
• Unicode and ISO Codes: Unicode assigns codepoints to characters; ISO standards assign codes (e.g. country codes). These are centralized assignments with versioning and deprecation (old characters remain, named, etc.). They ensure stability (code points never change meaning). They illustrate central assignment and backwards compatibility. Semantic IDs could similarly tag deprecated concepts but not reuse them.
• SKOS / OWL (Semantic Web): SKOS provides properties to link concepts: skos:exactMatch, skos:broadMatch/narrowMatch for equivalence/hierarchy. OWL has owl:sameAs (strong identity) and disjointness for contradictions. In a registry, we would use these standards to record relationships between ConceptCodes. For example, if two codes are synonyms, a mapping entry (using skos:exactMatch) asserts their equivalence. This separates identification (the code itself) from relationships (the mappings).
• Wikidata/Q-numbers: Wikidata uses numeric IDs (Q1, Q2, …) as persistent, opaque identifiers for entities. These are resolved via the wikidata.org API, and labels in many languages are attached. This provides a large-scale, multilingual registry with a single centralized system. It shows: concept IDs can be simple (just numbers) but need a central lookup. We should learn from Wikidata’s scale (millions of items) but note that it is one organization (Wikimedia) controlling the IDs.
• FHIR Terminology (e.g. SNOMED CT): Medical code systems like SNOMED have their own URIs (e.g. http://snomed.info/sct) and each concept has a numeric code. They version by date. FHIR’s CodeSystem defines a canonical URI for a code system; each concept has a code, display, definition. These systems are centralized, with strict editorial control, and support concept maps between systems. They illustrate best practices: each code has a human-readable definition and optional multilingual terms.
• Package Registries (npm, PyPI, etc.): These allow many publishers to upload packages under namespaces (e.g. @org/pkg). The registry (npmjs.com) enforces uniqueness of names and versions. It offers lessons: namespace scoping (a prefix owned by a user/org), versioning, and record immutability (old versions remain). However, package registries are centralized services, not federated. A federated concept registry might emulate package registries by using scoped namespaces (like concept:library/Value).
• Content-Addressable Systems (e.g. Git, IPFS): Git uses SHA hashes of content as commit IDs; Docker uses content hashes for image layers. Such systems ensure integrity (you can recompute the hash). We could similarly hash concept definitions and store these hashes in records (with the code as a pointer). Multihash schemes allow algorithm agility. The challenge is ensuring canonical serialization (like JSON canonicalization) so identical semantics yield the same hash.
• Certificate Transparency (CT): CT uses public append-only logs of certificates so that mis-issuance is detectable. By analogy, a concept registry could append each new concept (or update) to a public log (or blockchain). This would allow anyone to audit all issued ConceptCodes and detect conflicting definitions. E.g., if two parties create “mt.core.foo” with different meanings, observers could see the conflict. CT’s use of Merkle trees and signed tree heads suggests a model for transparency logs of concept issuance.
• Decentralized Identifiers (DIDs): W3C DIDs allow entities to have decentralized identifiers not tied to any single registry. A DID is a URI that resolves to a DID Document (with public keys, etc.). DIDs achieve decentralization and self-sovereign control, but often at the cost of human readability. In concept IDs, we might borrow the idea of decoupling an ID from a registry: e.g. a DID-style URI could identify a concept, with resolution via any compatible DID method. This meets Zooko’s challenge for decentralization and security, but would require strong key-management for concept authorities.

Summary: No existing system by itself fits all needs. DNS and IANA illustrate delegation but are human-readable and centralized at the top. UUIDs solve uniqueness but not usability. Semantic Web standards cover relationships, not governance. Certificate Transparency and DIDs inspire decentralized trust. Our solution must integrate these lessons: hierarchy for delegation, cryptographic logs for transparency, standard vocabularies for relationships, and namespaces with owners to avoid collisions, while preserving the invariants above.

Centralized vs Federated Architecture

Choosing a centralized vs. federated model is a core decision:

• Centralized (“Single Registry”): One global authority (or small council) assigns all ConceptCodes and hosts the master registry. This ensures consistency and ease of conflict resolution, but creates a bottleneck and single point of failure. It risks forming a semantic monopoly. For example, IANA maintains all IANA registries centrally. Centralization simplifies governance but limits scalability and diversity.
• Federated/Distributed: Many independent registries co-exist, each under its own namespace (like DNS name servers or blockchains). No single point of trust: participants cross-publish or synchronize entries. This enhances resilience and avoids central control. The drawback is conflict resolution: without a central arbiter, two authorities could mint overlapping or equivalent concepts. Zooko’s Triangle highlights this trade-off: achieving secure, decentralized, human-friendly naming is non-trivial. A fully decentralized system (like Namecoin or DIDs) can generate collision-free IDs securely, but human-meaningful names then require extra infrastructure (like mapping services).

Inference: A hybrid approach is likely best. For instance, a federation of registries that agree on root namespaces (like top-level domains). Each organization could control a namespace (analogous to a DNS zone or Git namespace). A central consortium might only oversee the root list, not every entry. This parallels identifiers.org (a central registry of namespace prefixes, with each prefix’s content managed by the domain owner). Such a model balances distribution (each namespace owner publishes their concepts) with a known root of trust (the prefix registry).

We must also consider governance. A federated model needs policies for how new namespaces or log operators are accredited (like choosing trusted DNS root servers or CT log operators). We might adopt federated trust: operators publish signed metadata (e.g. a registry’s public key, policies) that clients can verify. Similar to how web PKI allows multiple CAs (federated trust) but requires cross-signing.

Proposed Global Namespace Architecture

ConceptCode format: We suggest a URI-like scheme for ConceptCodes. For example, use a custom URI scheme or HTTP(S) URIs under a controlled domain:

• A namespace prefix identifies the issuer (e.g. an organization or context). This could be a DNS name or a defined code. For instance, cs: (short for “concept scheme”) or http://concept.org/<namespace>/.
• The local name is a human-readable term (often dotted hierarchy) unique within that namespace (e.g. mt.representation.machine_readable). The example site’s codes appear namespace-like (mt, semantic_registry). These local names should follow identifier naming rules (alphanumeric, separators) and not include unpredictable randomness (to remain stable and memorable).

Delegation and Prefix Ownership: Each namespace prefix (like mt or a domain) is owned by some entity. Registry operations (creating new local names) are managed by that owner. Transfers of ownership (e.g. a project is handed to a new maintainer) should be possible via recorded registry updates (much like domain name transfers). A global namespace registry (could be a distributed ledger or a web of registries) maintains mappings of prefix→owner and policies (similar to identifiers.org prefix registry).

Central vs. Local Resolution: There could be a root resolver service (analogous to DNS root servers) that knows where to find each namespace’s registry. However, concept resolution need not query a single server; clients might directly query the authoritative registry for a given namespace. E.g. a ConceptCode URI like concept://mtsys/representation.machine_readable might resolve via DNS SRV or via a well-known HTTP(S) endpoint managed by “mtsys”. We might also allow content-addressable resolution (e.g. publishing concept definitions in IPFS and resolving via a content hash), but DNS/HTTP is simpler for now.

Name Collisions: To prevent duplicates, once a namespace is assigned, its owner must guarantee uniqueness of local names. Collisions across different namespaces are okay (e.g. mt.foo vs acme.foo) since the full identifier includes the namespace.

Analogy: This is like using DNS+URN: e.g. “urn:myorg:mt.representation.machine_readable” or “http://myorg.concept/mt.representation.machine_readable”. The key is namespace isolation.

Authority and Ownership Model

Each ConceptCode namespace must have a defined owner/maintainer. We envision:

• Namespace Registration: A global directory (federated or centralized) where entities register a namespace (and optionally pay or meet criteria). Similar to IANA language subtags or ICANN TLD allocation, the registry records owner identity (e.g. DID or TLS cert), contact info, namespace pattern, and status. The prefix owner is responsible for minting concept IDs in that namespace.
• Ownership Transfer: The system should allow transfers (e.g. when an organization changes). This can be done by signed registry updates: the current owner signs a message naming the new owner, which is published to the root. Transfers are logged so clients can verify continuity (like DNS transfer logs or blockchain transactions).
• Authority Certificates: Each namespace owner holds a cryptographic key pair. They sign all concept definitions they publish (see Trust Model below). The public key could be stored in the namespace registry entry or in a WebPKI certificate (much like DANE). Ownership thus equates to control of the key.

This model is akin to namespace certificates. For instance, in CAs we have certificate chains; here we would have “concept authority” chains. If the owner is a recognized organization (e.g. W3C, ISO body), that lends confidence. To avoid central monopoly, we allow many independent namespace owners (like commercial publishers, academic groups, etc.), as long as they register and agree to the system’s policies. Namespaces could be for domains (e.g. acme.widgets) or for topics (e.g. fhir, gs1).

A challenge is renaming: if an organization rebrands, concept codes including its old name might need aliasing. We recommend immutable codes: even if namespace is renamed, old URIs should redirect or alias to new form. The registry’s provenance history can record such renames without breaking the ID itself.

Versioning and Deprecation Model

ConceptCodes themselves are identifiers for a meaning, not for a version. We recommend concept definitions have versions, but the identifier remains constant. E.g. each registry entry includes a versionNumber or date. Clients can query a code “latest” or a specific version. This follows FHIR’s CodeSystem approach where each CodeSystem URL remains same but a version tag indicates edition.

• Immutable History: All past definitions remain recorded. If a meaning evolves, the owner publishes a new version (with an incremented version or timestamp) but under the same ConceptCode. The older version is marked deprecated or superseded in the metadata.

• Deprecation Policy: If a concept is rendered invalid (e.g. merged, obsolete), it should be marked deprecated but not deleted. The record can include pointers to replacement concepts. Users can then map old data to new codes. The code still resolves, but its status indicates “do not use” and possibly a replacedBy link.

• Immutable IDs: The system forbids reusing a deprecated code for a new meaning. Instead, a new ConceptCode must be issued. This preserves data integrity. (For example, SNOMED keeps old codes active with “inactive” flags and never recycles them.)

• Version Access: A ConceptCode URI might have a query parameter or path for version (e.g. ?version=2024-03-12) allowing retrieval of specific definitions. Alternatively, each version might get a sub-identifier (like a suffix). But it’s simpler to keep the code clean and specify version separately.

• Schema Migration: Because the identifier is stable, migrating stored data is easier: old records can still reference the same ConceptCode after an update. Clients should always fetch the concept’s definition via the registry rather than embed full semantics. If changes are backward-incompatible, the registry’s provenance log explains the change.

Provenance Model

Provenance (the “who/when/how” of a concept) is crucial for trust. Each concept record in the registry should include:

• Authoritative Provenance: Fields like createdBy, createdAt, modifiedBy, modifiedAt. These record which namespace owner or reviewer made changes. This aligns with the site’s emphasis on “reviewed expressions … with explicit provenance”.
• Source References: If the concept is based on an existing standard or publication, cite it. E.g. if mt.representation.machine_readable is defined in a standard, link that standard’s URI.
• Review History: Maintain an audit trail of changes (like a changelog), perhaps in version control (Git, database log, or blockchain). This could mirror W3C PROV (e.g. prov:wasGeneratedBy with a process and agent).
• Namespace Trust Anchors: The namespace registry should itself have provenance (who issued the namespace, etc.).

Inference: We can adopt linked-data provenance vocabularies (PROV-O) for this. E.g. each concept record could be an RDF graph with prov:Entity as the ConceptCode, prov:wasAttributedTo the org/person, and prov:generatedAtTime the timestamp. This way, any consumer can trace exactly who signed off on the concept.

Cryptographic Trust Model

To secure the system, critical data should be signed or hashed:

• Signed Concept Records: The concept definition (name, definition text, relationships, etc.) should be digitally signed by the authority’s private key. For example, a registry API response or JSON record could include a signature (e.g. JWS) covering all fields. Clients verify with the publisher’s public key stored in the namespace registry. This prevents undetected tampering.
• Content-Addressing: The content (definition) could be hashed (e.g. SHA-256) and the hash stored in the record or incorporated in the ID (multihash). This provides integrity checking. For instance, concept attachments (images, long text) could be on IPFS with their CID stored. The site’s separation of meaning implies content should be canonicalized before hashing. JSON-LD canonicalization or RDF N-Quads sorting might be needed.
• Audit Logs (Transparency): All registry updates could go into an append-only log (modeled on Certificate Transparency). Each log entry includes a timestamp, hash of the new or updated record, and a signature by the logging service. Clients can fetch a Merkle tree head and verify inclusion of entries. This ensures that once a concept is published, it cannot be retroactively altered or deleted without detection. Auditors or other registries can run monitors to ensure consistency across mirrors.
• Trust Anchors: We should establish root-of-trust public keys (similar to TLS root CAs) for the namespace registry. Alternatively, use DIDs for namespace owners, where the DID document (verifiable credential) lists the owner’s key for signing.

In sum, concept records are cryptographically anchored. The namespace signing key ensures only the legitimate owner can assert a definition, and logs ensure accountability. This prevents “hostile registries” from quietly introducing bad concepts: any registrar’s entries are public and can be cross-checked.

Federation and Replication Model

To scale and avoid central points of failure, multiple registry instances should synchronize:

• Multi-Master Replication: Concept registries can operate in a distributed way. Each namespace owner runs a registry server for its concepts. These servers periodically publish changes (new/updated concepts) to peers. Conflicts within one namespace are avoided by single-writer (the owner). Across namespaces, no conflict (different prefixes).
• Inter-Registry Sync: Registries could subscribe to each other’s updates (e.g. via publish/subscribe or pulling from logs). This resembles how DNS zone transfers work or how distributed git repos sync commits. Technologies like distributed databases (CouchDB/PouchDB) or CRDTs could implement eventual consistency.
• Caching and CDNs: For read-heavy access, concept records should be cacheable. A CDN or local mirror could cache frequently used concept definitions. TTLs should be set so updates propagate reasonably fast but allow offline access.
• Offline Operation: Clients should be able to operate on a local copy of needed concepts. For example, mobile apps could embed a snapshot (RDF or JSON) of relevant concepts and use it without connectivity. Once online, they sync missed updates. This is similar to how blocklists or hash lists in browsers are shipped and updated. The registry should publish static dumps or use peer-to-peer (IPFS) so clients can download sets of concept records.

Inference: This federated model combines aspects of DNS (delegated zones) and blockchain logs (verifiable append-only). It avoids a single centralized API by allowing each publisher to host their namespace. Mirror or aggregator nodes can service queries for multiple namespaces.

Conflict-Resolution Model

In a distributed system, conflicts can occur. We foresee two main cases:

1. Duplicate/Synonymous Concepts: Two namespaces might inadvertently define concepts with essentially the same meaning. This is not a “conflict” in the sense of data corruption, but a semantic equivalence. We handle this by mapping, not merging. Publishers should declare equivalence (e.g. skos:exactMatch or a federation-level “oneID” system) between concept codes across namespaces. Consumers can then choose one as canonical. If there is debate, it must be resolved by semantic alignment processes (human curation or machine inference, but explicitly logged as a mapping, not an internal ID conflict).
2. Direct ID Collisions: If two registry owners accidentally use the same ConceptCode string in a shared namespace, governance must prevent it. With namespace prefixes, collisions across prefixes are harmless. If using a single flat registry, collisions would require a conflict resolution policy (e.g. first-come wins, or priority by registry UUID). We avoid this by design: each namespace is disjoint.
3. Contradictory Information: If two authoritative sources publish logically inconsistent assertions about the same ConceptCode (e.g. they both claim authoritative meaning but disagree), clients must detect this. Signatures and provenance help: a consumer can see that concept “C” has two signed definitions by different entities. The system should not silently choose one; it should flag a conflict. Potential resolution strategies include: rely on a trust hierarchy (a “primary authority” namespace overrides others), or require human adjudication.
4. Concurrent Updates: If a namespace owner’s systems diverge (e.g. two replicas make offline edits), the owner’s key can sign whichever is chosen as final. Replay logs or distributed consensus (Raft, etc.) within the owner’s infrastructure can prevent split-brain.

Inference: In short, conflicts are managed at the level of mapping and policy, not by automatic merging. We encourage explicit mapping relationships (exactMatch, narrowMatch) and versioning to clarify intent. The model follows SKOS’s guidance that only logically consistent relations are allowed (SKOS disallows chaining exact with broadMatch etc.).

Concept Relationship Model

ConceptCodes form a graph. We propose using standard semantic relations:

• Equivalence: Use skos:exactMatch (or owl:sameAs for class-level identity) to assert that two concepts are essentially the same. This could link concepts across different registries or across versions. (SKOS defines exactMatch as a high-confidence interchangeability.)
• Broader/Narrower: Use skos:narrower/skos:broader (or OWL subClassOf if concepts are classes) to represent specialization. For example, concept “Animal” broader, “Dog” narrower.
• Related: Use skos:relatedMatch or a custom relatedTo for associative links without hierarchy.
• Contradiction: If two concepts are mutually exclusive, this can be modeled via OWL disjointness: e.g. mark them as owl:disjointWith or with a custom property incompatibleWith. This at least makes contradictions explicit.
• Composition: For complex concepts (e.g. “Color or Shape”), one could use RDF collections or OWL class expressions. For registry purposes, we might allow defining composite concepts by referencing other ConceptCodes (e.g. a concept’s definition text: “Combination of C1 and C2”). The registry might not need a formal operator if such concepts are rare, but it should allow linking to component codes.
• Provenance as Relationships: States like skos:definition, skos:note can include references and provenance to literature or authority statements.

Using SKOS and RDF/OWL ensures interoperability and tooling support. The site’s insistence on “explicit equivalence evidence” suggests we should avoid fuzzy links; all these should be asserted by curators and recorded.

Scalability Considerations

To handle millions of concepts and billions of observations:

• Sharding by Namespace: Since namespaces are independent, we naturally shard data by owner. Each registry server need only hold its portion. This divides load.
• Efficient Storage: Use scalable database technologies (NoSQL, graph DB, or optimized SQL) to store concepts. For read-heavy workloads, materialized indexes (by concept code, label, etc.) are needed. Full-text search (for finding concepts by expression) may use Elasticsearch or similar.
• Caching: Leverage CDNs or edge caches for concept lookups. For example, an HTTP-based API could have intermediate caches for hot concepts.
• Batch Syncing: Rather than querying across registries on every lookup, publish a global index or manifest (like the “Public Registry Bundle Manifest” mentioned on the site) to help clients discover where concepts live. The manifest can be periodically fetched and cached.
• Parallel Processing: Validation tasks (e.g. automated mapping proposals) can be farmed out to many nodes or GPUs. Semantic matching (e.g. embedding-based evidence) should remain auxiliary, not in the critical path.
• Capacity Planning: The architecture should plan for horizontal scaling (adding more servers) when concept count grows. Similar to how Wikidata scaled to 100M+ statements. Using cloud infrastructure and container orchestration can help.
• Monitoring and Metrics: Instrument query volume (requests/sec), registry growth (concepts added/day), and latency. Autoscaling policies can react to load.

Offline and Disconnected Operation

Clients (e.g. mobile apps, field devices, embedded systems) must sometimes use the registry without connectivity:

• Local Snapshots: The registry should publish periodic dumps (RDF/XML, JSON, etc.) of all or subsets of concepts. Clients can download a “bundle” of relevant concepts. The site already mentions a “Registry Pack” schema. These static packs can be versioned (with checksums).
• Peer Caching: Clients can also request data from peer devices (mesh networks). If content-addressed (e.g. IPFS), then offline peers can share caches.
• Fallback Codes: If lookup fails offline, clients should still use the stored ConceptCode as-is. The code itself may be human-readable enough to be understandable. (For example, even without definitions, mt.representation.machine_readable hints at meaning.)
• Deferred Sync: Once back online, clients reconcile local usage with updated registry: e.g. if a code was deprecated, flag for an update.
• API Simplicity: Provide simple lookup APIs (e.g. HTTP GET by code) that can be stubbed with local file or embedded DB. For instance, the resolver API could be mirrored locally with a subset of data.

Thus, the system’s core (IDs and local language labels) should be as self-contained as possible; online services are needed only for advanced features (full definition, validation).

Failure Modes

Key potential failure scenarios:

• Namespace Hijacking: If an attacker obtains a namespace owner’s key or credentials, they could publish bogus concept definitions. Mitigation: use multi-factor auth for key updates, allow rapid revocation (via the registry log), and require multiple approvals for critical namespaces.
• Registry Partition: In a federated model, network splits could lead two parts to diverge. Recovery requires reconciling logs and resolving any duplicate concept creation by the same owner. Using append-only logs with conflict detection helps.
• Concept Collision: If two owners accidentally agree on the same ConceptCode syntax without knowing, their concepts collide semantically. Namespace separation largely prevents this; rules should forbid reuse of another’s namespace.
• Data Corruption: Erroneous definitions (bugs, bad data) could get inserted. Reviews and cryptographic signing reduce silent corruption; clients should check signature validity.
• Performance Overload: A DDoS on a registry or API could cripple lookups. Mitigation: rate limiting, caching, and distributed endpoints. The federation model spreads the load.
• Stale Caches: Caches may serve outdated concept definitions after updates. Setting sensible TTLs and using eTags or content hashes can signal updates to clients.

Abuse Cases

Consider malicious or low-quality behavior:

• Spam Concepts: A malicious publisher might mint thousands of trivial or misleading concepts (like a scammer pushing irrelevant product codes). To counter this, the system could require minimal metadata (definition text, provenance) and optionally community voting or rating for concept quality. Registry monitors can flag abnormal creation rates.
• Fake Equivalences: An attacker could claim innocent concepts are equivalent to malicious ones (skos:exactMatch with a bad concept). We should restrict who can assert mappings for a concept (e.g. only the owner or trusted partners) and require provenance on mappings. Conflicting equivalences can be discovered via reconciliation logs.
• Private Namespaces: If someone registers a namespace but never publishes public records, others might unintentionally create concepts that should belong to it. The namespace registry should deprecate unused registrations.
• Service Monopolization: A dominant platform might try to centralize clients (e.g. “Only use our resolver”). We avoid this by open protocols and multiple resolver implementations.
• Privacy/Provenance Leakage: Careful: publishing where and when concept was used could reveal sensitive data. The registry should limit what usage data it logs publicly. Only metadata about definitions should be public.
• Sybil Attacks: In a federated trust model, an attacker could spin up many fake registries or monitors. A reputation mechanism or requirement of proof-of-identity (e.g. linked to legal entities) can mitigate sybils. Certificate Transparency-style monitoring also helps detect unusual patterns (like an organization suddenly issuing tons of concepts).

Recommended Architecture

Based on the above, we recommend a federated registry architecture with these features:

1. ConceptCode URIs: Define ConceptCodes as URIs of the form cs:<namespace>:<localName> or https://concepts.org/ns/<namespace>/<localName>. The exact scheme can be standardized (e.g. a new URI scheme concept: or use https with a reserved domain). The <namespace> is a short owner code, and <localName> is a dot-separated name.
2. Namespace Registry: A root directory lists all namespaces, their owners, and public keys. This could be a distributed ledger or a centralized but publicly archived service. Namespaces are vetted and controlled.
3. Authority Keys: Each namespace owner has a key pair. Concept definitions (JSON or RDF) published under that namespace are signed. The namespace registry stores the owner’s public key (like DNSSEC KSKs) for verification.
4. Append-Only Logs: Every change (new concept or update) is appended to a global log (or per-namespace log) with a timestamp and signature. Merkle trees enable monitors to audit consistency (drawing on CT).
5. Semantic Relations: Use RDF/OWL/SKOS to encode relationships. Provide standard ontology (or reuse SKOS) for fields like definition, deprecated, version, source, etc. The registry API and data format are based on JSON-LD or RDF so that it's easy to consume and link.
6. APIs and Feeds: Public HTTP APIs allow lookups by code or by exact reviewed expression. Bulk download endpoints (concept dumps, changefeeds) enable syncing. The Resolver API would accept expressions+language and return a ConceptCode if known (like the current exact resolver). The Status API could report overall health/metrics.
7. Federation Protocol: Define how registry servers discover and trust each other (e.g. via the namespace registry and log roots). We may use a protocol similar to certificate log mirror or blockchain consensus (not necessarily proof-of-work, could be proof-of-authority among known servers).
8. Client Behavior: Clients (software needing meaning resolution) must accept that a ConceptCode stands for a concept if signed by its namespace key or included in the log. They can fetch definitions via the API or trust locally cached data. For unseen expressions, they should use the standard unknown/ambiguous tokens (as now).

This architecture aims to be modular: one could implement it gradually. The root namespace registry and naming scheme can be specified first, then working registries can be built as pilot projects.

Alternatives Rejected and Why

• Purely Centralized Registry: Not chosen because it risks monopoly, slow responsiveness, and central failure. (We want many publishers to innovate independently.)
• UUID-only identifiers: Rejected because they are not human-readable and could complicate on-device caching (users wouldn’t know what concept the code refers to). Also, UUIDs alone don’t solve trust or collisions beyond statistical uniqueness.
• Blockchain for Everything: A public blockchain for concept management could ensure immutability, but it incurs high overhead and latency (and often secrecy concerns). Instead, we prefer simpler append-only logs that can be monitored but not mined.
• Exclusive Ontology Reasoning: Some systems use OWL reasoning to assert identity (e.g. linking classes). But automatic ontology merging is error-prone. Our model requires explicit evidence-based mapping rather than inferred identity.
• AI/Embedding-Driven ID: One could imagine using AI to assign meaning-based IDs (embedding-based clustering). The site explicitly rejects this for trust reasons. We follow that: any model-based suggestion would only be “candidate” evidence, not authoritative.
• Proprietary or Paid-only Access: To avoid “semantic lock-in”, the architecture should be open. Proprietary access models (e.g. private API requiring fees) would hinder wide adoption. Instead, use open standards and, if desired, minimal cost-sharing among stakeholders.

These rejections align with keeping the system decentralized, evidence-based, and transparent, rather than opaque or monopolistic.

1-Year / 3-Year / 5-Year Evolution Path

• Year 1 (Prototype):
• Define the ConceptCode URI scheme and governance rules (e.g. via a standards working group).
• Implement a pilot registry service: possibly for a limited domain or a few organizations. For example, a “medical” namespace (using FHIR/SNOMED terms) and a “general English” namespace. Publish the API and data format (likely JSON-LD).
• Build basic tooling: a web resolver (APIs to resolve codes and expressions), a registry editor UI, and simple digital signing support.
• Test offline use cases: provide static dumps of the prototype concepts and a client library for lookups.
• Metrics: number of concepts added, resolution correctness on test inputs.
• Year 3 (Expansion and Federation):
• Expand to dozens of namespaces (multi-lingual, multi-domain). Onboard independent publishers.
• Deploy multiple registry mirrors and ensure they sync correctly (testing CRDT or log-based synchronization).
• Implement provenance logging and digital signatures fully, and release monitoring tools (like CT logs) to catch discrepancies.
• Optimize performance for scale (database sharding, caching). Support billions of expression lookups per day.
• Start discussions on standardization bodies (W3C/ISO) for chartering official processes.
• Metrics: number of concepts (target millions), number of active namespaces, uptime (target ≥99.9%).
• Year 5 (Stabilization and Adoption):
• Solidify a governance consortium or standards organization for oversight (ensuring no single vendor can hijack).
• Widespread adoption: integration into AI/ML pipelines, databases (e.g. RDF triple stores), and applications (so that “Semantic ID” fields appear in data schemas).
• Federated trust: alternative resolution methods in place (e.g. peer-to-peer, DNS-based concept lookup) so no one service point can go down.
• Automation: machine-assisted suggestions for new concept creation or linking (subject to review).
• Success criteria: e.g. achieve 90% coverage of critical terms in targeted domains, <1% lookup failure for known expressions, and community consensus around the model’s value (measured by surveys or commits to the project).

Experiments to Run Next

• Pilot Concept Packs: Develop “starter packs” of concepts (as the site has) for different domains and languages, then test them with real AI/agent systems to see if using ConceptCodes improves coherence over raw strings.
• Offline Caching Test: Build a client that uses a downloaded concept subset; verify it functions correctly with no internet. Measure sync time when reconnecting.
• Conflict Simulation: Simulate two independent registries publishing similar concepts (like a “temperature” concept). See how easily conflicts are detected and resolved via our logs/mappings.
• Performance Benchmarking: Stress-test the resolver and API with millions of queries. Examine latency, throughput, and failure modes.
• Trust Model Trials: Set up a “grassroots” registry (no central authority) using DIDs and see if creators trust it; compare with a managed registry. Also experiment with CT-like logs and an auditor verifying completeness.
• Semantic Mapping QA: Use ontology matching tools (e.g. SSSOM standard) to automatically generate potential concept equivalences; feed these into the system for human review, evaluating if the workload is manageable.

Decision Gates and Success Criteria

To proceed at each stage, we define clear gates:

• Gate 1 (Proof-of-Concept): Can we resolve expressions to ConceptCodes reliably in a small domain? Success = >95% correct resolution on test set; stable retrieval of concept definitions by code.
• Gate 2 (Scalability): Can the system handle 10× projected concept count? Success = performance metrics (e.g. <200ms API latency under load, error rate <0.1%).
• Gate 3 (Trustworthiness): Is the provenance/crypto model workable? Success = all concept records validated against signatures/logs with no breaches in tests; community trust (measured by adoption in one real project).
• Gate 4 (Federation): Can multiple registries interoperate without loss? Success = two independently hosted registries syncing consistently, no data loss, and concept equivalences recognized.
• Gate 5 (Adoption): Are external organizations using ConceptCodes? Measure by integration in at least 3 independent projects (e.g. one from government, one from industry, one open-source).

Success criteria also include user-facing results: e.g. AI systems that use these codes should have demonstrably lower semantic ambiguity errors.

Open Research Questions

• Expressing Contradiction: How best to formally represent mutually exclusive concepts (and ensure users respect that)? SKOS/OWL provide disjointWith, but this is rarely used in practice. We need a user-friendly way to say “concept A cannot co-occur with B”.
• Ontology Alignment at Scale: Without central control, how to systematically align overlapping domain ontologies? Research in ontology matching suggests partial automation, but human oversight is needed. Can we crowdsource alignments?
• Namespace Incentives: What motivates organizations to publish/share their concepts in this common system? Possibly a search of shared semantics or credibility. Defining clear benefits (e.g. better interoperability with others’ data) is key.
• Economic Model: Will there be cost to register a namespace or run a registry node? Funding this infrastructure (like IETF does for standards development) is an open question.
• Security Threats: Detailed threat modeling is needed. For instance, could someone manipulate concept logs to disrupt trust (e.g. eclipse attacks on log mirrors)?
• Content Normalization: For content-addressed hashing of definitions, what canonicalization suffices? JSON-LD normalization (like RDF Dataset canonicalization) is complex. This needs more research to apply in practice.

Bibliography

• EmbeddedSemantics.com (site content, FAQ, About): concepts of registry-backed identity.
• Uniform Resource Identifier (URI) [RFC3986/Wikipedia]: URIs identify abstract resources including concepts.
• UUID (Universally Unique Identifier) [Wikipedia]: decentralized unique IDs, “for practical purposes unique”.
• Identifiers.org: principles for stable, global IDs and namespace prefixes.
• DNS [Wikipedia]: hierarchical distributed naming, authoritative delegation.
• IANA Media Types [IANA]: centralized registry with expert review (see RFC6838).
• SKOS Reference: mapping properties (exactMatch, broadMatch, etc.).
• FHIR CodeSystem [HL7]: canonical URIs for code systems, versioning concept (snip).
• UMLS Metathesaurus [NLM]: Concept Unique Identifiers (CUIs) for shared meaning.
• Wikidata [Meta-Wiki]: stable entity URIs (e.g. Q42) in a linked-data model.
• Zooko’s Triangle [Wikipedia]: naming trade-offs (human-meaningful vs decentralized).
• Content-Addressable IDs (Sean Gillies): hashing objects (git, docker).
• Certificate Transparency [Wikipedia]: public append-only logs for auditing issuance.
• Embedded Semantics llms.txt: lists machine endpoints and registry packs (implicit in research, but site context).

(Additional references on standards and practice were reviewed but not directly cited above.)