Semantic Systems / Language / Glyphs
Embedded Semantics: Design and Justification of Stable Semantic Concept Identifiers and Registry-Backed Meaning
Report summary
The rapid proliferation of large language models (LLMs) and vector-based retrieval systems has precipitated a foundational crisis in knowledge representation: the conflation of mathematical similarity with semantic identity. Modern artificial intelligence systems inherently represent concepts as hig
Key topics
- Semantic Systems / Language / Glyphs
- Semantic Systems
- Language
- Glyphs
- AI
- Runtime
- Research Archive
- Strategy
- Audit
Research provenance
For citation, use the report title and canonical URL. Archival presence does not establish authorship or promote report statements into portfolio evidence.
This page renders the archived Markdown as safe, formatted HTML. It is background research and does not become a portfolio claim without evidence review.
Full report
On this page
1. Executive Summary
The rapid proliferation of large language models (LLMs) and vector-based retrieval systems has precipitated a foundational crisis in knowledge representation: the conflation of mathematical similarity with semantic identity. Modern artificial intelligence systems inherently represent concepts as high-dimensional continuous vectors, known as embeddings, which capture spatial relationships based on statistical co-occurrence. While immensely powerful for probabilistic matching and similarity search, these vectors are mathematically volatile. They drift dynamically during model fine-tuning, shatter entirely across dimensional architectural changes, and diverge unpredictably across different algorithmic vendors. The core philosophy of Embedded Semantics addresses this crisis by instituting a strict architectural separation: an embedding model may suggest the probabilistic likelihood of a meaning, but a persistent, immutable registry determines the absolute identity of that meaning. An embedding vector is merely replaceable, model-dependent evidence of a concept. The concept identifier itself must serve as the permanent anchor. Concept identity must remain utterly stable independently of the embedding model, vector dimensions, retraining cycles, index implementations, language variations, wording, model vendors, or underlying storage engines. This exhaustive report provides a definitive investigation into the precedents, architecture, and governance required to implement stable semantic concept identifiers for Embedded Semantics. By synthesizing decades of research across clinical terminologies, scientific taxonomies, linguistic registries, and semantic web standards, this document outlines a robust framework for managing concept lifecycle, evolution, and provenance. The resulting architecture ensures that semantic identity remains impervious to infrastructural shifts and model iterations, providing a permanent, trustworthy foundation for next-generation artificial intelligence systems.
2. Why Semantic Identity Needs Persistence
The modern artificial intelligence ecosystem frequently attempts to utilize continuous embedding vectors as functional identifiers for information retrieval, database primary keys, and semantic grounding. This approach is epistemologically and mathematically flawed. Vector embeddings represent statistical features and correlations, not discrete identity1. Establishing true semantic identity requires a persistent, registry-backed identifier to survive the continuous turbulence of algorithmic evolution. The volatility of vector embeddings manifests through two distinct phenomena: representation drift and semantic drift2. Representation drift occurs when an embedding model is updated, retrained, or subjected to domain-specific fine-tuning. The entire vector space is mathematically remapped, causing the numerical coordinates of a concept to shift unpredictably, even if the human understanding of the concept remains static2. Semantic drift, conversely, captures how the association between inputs and outputs evolves over time as cultural and linguistic contexts change2. For instance, the embedding for the word "cloud" in a corpus from 1990 is vastly different from its embedding in 2026, shifting from meteorology to distributed computing3. If a system relies on a vector as a primary identifier, it becomes subject to catastrophic historical amnesia. A registry-backed identifier resolves this by mapping both historical and modern vectors to a stable Concept ID, treating the vectors as temporal evidence rather than the entity itself. The recommendation of this report is unequivocal: under no circumstances should a model-generated embedding vector be utilized as a permanent identifier. To illustrate the systemic danger of treating vectors as persistent identifiers, the following detailed examples demonstrate why a vector cannot safely replace a persistent concept ID: The Dimensionality Fracture occurs when an enterprise upgrades its underlying model infrastructure. Suppose a knowledge graph maps millions of documents using a 1536-dimensional embedding model. A year later, a more computationally efficient 384-dimensional model is adopted to reduce inference costs. Because vectors cannot be deterministically translated across disparate dimensional spaces without severe information loss, every conceptual link, cluster, and edge in the database is instantly severed. If the system had utilized a stable Concept ID (e.g., ES000123), the database administrator could have simply appended the new 384-dimensional vector as secondary evidence to the concept record, preserving all downstream relational logic and historical graph edges. The Polysemy Collapse demonstrates the failure of vectors to handle discrete ontological branching. A single embedding vector for the text string "bank" averages the semantic spaces of a financial institution, a river edge, and an aircraft maneuver4. If a system relies on the text's vector to identify the concept, downstream applications cannot reliably distinguish these senses without constant contextual recalculation. A persistent registry issues ES000401 for the financial institution and ES000402 for the river edge. This allows the vector evidence to be analyzed against distinct, non-overlapping semantic anchors, completely preventing polysemous collapse. Cross-Lingual Asymmetry highlights the mathematical discrepancies in multilingual models. The German term "Schadenfreude" and its multi-word English explanation ("pleasure derived by someone from another person's misfortune") produce vectors that occupy similar, but not identical, regions in a multilingual vector space. Without a universal identifier, merging analytical data across these languages relies on fuzzy mathematical thresholding (e.g., cosine similarity greater than 0.85), which frequently introduces false positives into the dataset. A registry assigns a single, language-agnostic ID to the concept, eliminating probabilistic guessing and ensuring flawless cross-lingual data aggregation.
3. Lessons from Existing Identifier Systems
The challenge of naming, persisting, and tracking abstract entities across time and technological paradigms has been confronted by librarians, scientists, and web architects for decades. Analyzing these legacy systems provides foundational architectural rules for Embedded Semantics. Persistent Identifiers (PIDs) form the backbone of modern scientific literature. The Digital Object Identifier (DOI), backed by the Handle System architecture, revolutionized digital persistence by strictly separating the identifier from the object's physical location or current state5. A DOI resolves to a central metadata registry, which then points to the resource. This indirection is critical: if a publisher goes bankrupt or changes domain names, the DOI remains mathematically stable; only the registry's location mapping is updated5. For Embedded Semantics, a Concept ID must act identically—resolving to a stable metadata record representing the "meaning," regardless of which vector database engine or neural index currently stores the embeddings. Uniform Resource Identifiers (URIs) and Internationalized Resource Identifiers (IRIs) provide namespace-based uniqueness, which is essential for the Semantic Web. However, standard URLs are notorious for link rot. UUIDs (Universally Unique Identifiers) guarantee mathematical uniqueness without a centralized registry, but they are entirely opaque and carry no semantic weight. Embedded Semantics should utilize IRIs for global interoperability, with the final path segment containing a specialized, semi-opaque alphanumeric identifier (e.g., https://embeddedsemantics.com/id/ES104857). Accession numbers utilized in bioinformatics databases, such as GenBank, offer another critical precedent. Accession numbers are issued sequentially and are never reused. If a genetic sequence is later found to be a duplicate, contaminated, or erroneous, the accession number is deprecated and mathematically routed to a new number or permanently flagged as obsolete. This strict "no deletion, no reuse" policy is mandatory for maintaining the integrity of cited scientific literature, a principle that Embedded Semantics must adopt to maintain trust in enterprise AI systems.
4. WordNet
WordNet pioneered the computational lexicon by organizing words into "synsets" (cognitive synonyms), each expressing a distinct concept. However, early versions of WordNet lacked persistent identifiers across version updates. A synset offset utilized in WordNet 2.0 might point to a completely different concept in WordNet 3.0, causing massive mapping failures in natural language processing pipelines worldwide. To rectify this architectural flaw, the Global WordNet Association introduced the Collaborative Interlingual Index (CILI)6. CILI provides a single, flat, interlingual index of concepts. When Princeton WordNet or the Open Dutch WordNet maps a synset to CILI, they use a persistent CILI identifier6. This allows concepts to exist independently of language-specific lexicalizations. The structural lesson for Embedded Semantics is clear: concept IDs must transcend specific languages, and language-specific expressions are merely labels attached to the universal, persistent identifier.
5. Wikidata
Wikidata utilizes "Q identifiers" (e.g., Q42 for the author Douglas Adams) that are entirely language-agnostic and inherently opaque. The meaning of Q42 is established exclusively by its properties, relationships within the graph, and accompanying metadata, not by an English label. Wikidata's massive global success demonstrates that human curators and software agents can easily adapt to opaque identifiers in editing interfaces, provided the tooling automatically resolves the ID to a localized, human-readable label instantly. Embedded Semantics must ensure that its stable IDs are treated as primary keys at the database layer, while Canonical Names and text expressions are strictly treated as presentation-layer attributes.
6. SKOS/RDF
The Simple Knowledge Organization System (SKOS), built upon the Resource Description Framework (RDF), is the W3C standard for representing controlled vocabularies and taxonomies. SKOS enforces the strict philosophical and technical separation of the abstract concept from the lexical string.
| SKOS Paradigm | Embedded Semantics Equivalent | Function |
|---|---|---|
| skos:Concept | Concept ID (ES000123) | The atomic, abstract unit of meaning. |
| skos:prefLabel | Canonical Name | The primary human-readable text expression. |
| skos:altLabel | Expressions / Rendering Candidates | Alternative text strings, synonyms, and translations. |
| skos:exactMatch | Relations | Graph-based mappings to external ontologies. |
An Embedded Semantics concept record directly aligns with the skos:Concept paradigm. It treats continuous embeddings as an entirely new class of attribute (e.g., an es:hasVectorEvidence property) that supplements, rather than replaces, traditional lexical labels. By adhering to the RDF/SKOS model, Embedded Semantics ensures seamless interoperability with legacy knowledge graphs.
7. Medical Terminology Systems
Healthcare terminologies manage life-or-death data, requiring the most rigorous change-management frameworks in the world. Their architectural choices regarding concept inactivation, supersession, and append-only versioning form the strongest precedents for Embedded Semantics. The Systematized Nomenclature of Medicine, Clinical Terms (SNOMED CT) is the most comprehensive clinical terminology globally7. To manage massive rates of change—including merging, splitting, and deprecating hundreds of thousands of concepts—SNOMED CT utilizes the Release Format 2 (RF2)7. The RF2 architecture is an event-sourced, append-only ledger12. Every row in the database contains strict metadata columns.
| RF2 Column | Embedded Semantics Equivalent | Operational Purpose |
|---|---|---|
| id | Concept ID | Uniquely identifies the component across all time. |
| effectiveTime | Version / Timestamp | Specifies the exact date the state became valid12. |
| active | Status (boolean) | Specifies whether the component is currently active or deprecated12. |
| moduleId | Provenance / Source | Identifies the authoring organization or module13. |
In SNOMED CT, a concept record is never deleted. If a concept changes, a new row is appended with the same id, a new effectiveTime, and updated properties12. If a concept is deprecated, a row is appended with the active flag set to 0 (false)12. This means it is no longer part of the current active snapshot, but it remains forever accessible for historical medical records, ensuring that a patient file from a decade prior remains perfectly interpretable13. The National Library of Medicine’s Unified Medical Language System (UMLS) integrates over 200 source vocabularies into a single Metathesaurus. It assigns a Concept Unique Identifier (CUI) to a cluster of synonymous terms15. Every unique string from a source vocabulary gets an Atom Unique Identifier (AUI), and these AUIs are mapped to a single CUI19. The UMLS maintains a rich history table (MRCUI.RRF and MRAUI.RRF)20. If concept editors determine two CUIs actually mean the same thing, they are merged. One CUI is retained, and the other is removed from the active file, but its history is permanently tracked in MRCUI.RRF with a relationship (REL='SY') pointing to the surviving CUI22. A CUI is permanently retired and never reused20. LOINC (Logical Observation Identifiers Names and Codes) and the OMOP Common Data Model manage complex laboratory vocabularies subject to a continuous lifecycle of generation, deprecation, merging, and splitting23. The OMOP model handles this by assigning valid\_start\_date and valid\_end\_date25. When a concept is deprecated, the valid\_end\_date is updated, and an invalid\_reason (e.g., 'D' for deleted without successor, 'U' for updated/replaced) is applied25. The updated concepts feature a record in the CONCEPT\_RELATIONSHIP table indicating their active replacement concept25. This explicit routing from deprecated concepts to their replacements is vital for preventing systemic "version drift" and the massive "maintenance tax" that cripples downstream data engineering teams23.
8. Scientific Identifier Systems
Scientific identifier systems prioritize historical immutability. The Gene Ontology (GO) standardizes the representation of gene and gene product attributes across species26. As the structural biological understanding of the genome evolves, a previously accepted GO term may be deemed scientifically unsuitable. When this occurs, the term is not deleted. Instead, the term is marked as "obsolete," its ID is deprecated into a specialized obsolete category, and computational reasoners are explicitly instructed on how to handle the dead ID via metadata mappings26. This design pattern guarantees that historical research papers citing the deprecated GO ID do not become unresolvable or point to 404 errors.
9. Authority Records
Library of Congress (LoC) authority records establish recognized forms for names, subjects, and titles, differentiating between an entity's primary identity and its contextual variations. For instance, the authors Mark Twain and Samuel Clemens are managed through a unified authority record that establishes the primary identity while linking all pseudonyms and aliases as subordinate text strings. Embedded Semantics must similarly distinguish the core Concept ID from its myriad "rendering candidates" and textual expressions, treating the identifier as the authority and the text as contextual variations.
10. Concept versus Label
A recurring, catastrophic failure in naive taxonomy systems is the conflation of a concept with its label. A label is merely a string of alphanumeric characters; a concept is the semantic intent and ontological boundary behind that string. Embedded Semantics strictly enforces this boundary. The concept is an abstract node in the semantic space, represented strictly by the Concept ID (ES000123). It has no inherent language, formatting, or capitalization. The label, or expression, is a human-readable instantiation of the concept (e.g., "Heart Attack", "Myocardial Infarction", "Crise cardiaque"). Because textual definitions, character strings, and vector embeddings can all shift over time due to cultural and linguistic evolution, identifier stability must be prioritized above all else. Label stability is desirable for human user interfaces but is never structurally guaranteed. If a canonical label changes from "Manic Depression" to "Bipolar Disorder," the Concept ID remains mathematically identical, preserving all historical data linkages, vector distances, and graph edges.
11. Concept Evolution
Concept evolution refers to the ontological shifting of a meaning over time. Philosophically, the Ship of Theseus paradox directly applies to terminology management: if every textual expression, embedding vector, and graph relation of a concept changes over a ten-year period, is it still the same concept? In a registry-backed system, concept identity relies on continuous, documented lineage. Small refinements in a textual definition—such as clarifying the boundary scope without altering the core extension of the concept—constitute standard evolution and maintain the same Concept ID. However, if an epistemological shift fundamentally alters what the concept refers to in physical or theoretical reality, it crosses the boundary from evolution to the necessity of splitting or deprecating the concept entirely. The identifier only remains stable if the core meaning remains stable.
12. Versioning
To safely manage semantic evolution, Embedded Semantics requires a rigorous versioning architecture built upon event sourcing. Following the SNOMED RF2 model, the database state is never mutated in place. Every modification—whether adding a label, changing a text definition, or uploading a newly calculated embedding vector—generates a new immutable record linked to the Concept ID, marked with a timestamp and a strict version number12. Concept payloads should adopt Semantic Versioning (SemVer) paradigms:
- Patch (x.x.1): Typographical corrections in labels, updating rendering candidates, or fixing metadata typos. There is zero semantic impact.
- Minor (x.1.x): Adding new positive or negative training examples, appending a new language translation, or updating a vector embedding from a new model. The core meaning is unchanged, but new evidence is available.
- Major (1.x.x): A tightening or expansion of the human-readable definition that could theoretically invalidate previous edge-case classifications. Downstream systems can use major version bumps to trigger re-indexing operations.
13. Splitting and Merging
Splitting and merging are the most complex operations in terminology governance, often necessitated by discoveries in science, linguistics, or logic. Merging is triggered when two concepts are discovered to be duplicates28. For example, ES000100 "Hepatoma" and ES000200 "Hepatocellular Carcinoma" are determined by medical curators to be semantically identical. One concept is selected as the survivor, usually the older or more heavily referenced ID. The other is deprecated. A mapping is created—equivalent to UMLS REL='SY' or OMOP invalid\_reason='U'—from the deprecated ID directly to the surviving ID22. The surviving concept inherits all expressions, translations, and vector evidence of the deprecated concept, ensuring no data loss. Splitting is triggered when a single concept is found to contain two distinct, irreconcilable senses. For instance, a legacy system may have created a "Bank" concept that mistakenly aggregated both financial and geographic data. In this scenario, a "pure split" occurs22. The original Concept ID is deprecated entirely to prevent future ambiguity. Two brand new Concept IDs are minted. A historical linkage is preserved pointing from the dead parent ID to the two new child IDs, allowing legacy data to be flagged for human review or contextual disambiguation.
14. Deprecation
Deprecation is the act of marking a concept as no longer valid for new classifications or indexing operations, while strictly preserving its resolvability for historical data. A concept cannot be deleted; doing so violates the fundamental integrity of the data lineage. When a concept is deprecated, it must possess a status flag (e.g., status='deprecated') and a deprecation\_date. Crucially, a deprecated concept continues to host its canonical name and historical embedding vectors. This ensures that downstream legacy systems can still execute similarity searches against historical archives, and machine learning models can still map old data to the deprecated ID to understand the original intent of the text.
15. Supersession
Supersession is the evolutionary mechanism that follows deprecation. When a concept is deprecated due to structural inaccuracy or a merger, the registry must provide a systemic path forward via a replaced\_by or superseded\_by field. This directs machine agents and human curators to the correct, active Concept ID. Supersession chains must be logically valid and mathematically protected against circular loops. A circular loop occurs when Concept A is superseded by Concept B, which is superseded by Concept C, which erroneously points back to Concept A. This phenomenon occasionally corrupts SNOMED extension packs if poorly governed, causing infinite loops in reasoner engines14. Graph-based acyclic validation scripts must run prior to any deprecation commit, ensuring directed acyclic graphs (DAGs) for all supersession linkages.
16. Aliases
An alias is a systemic mechanism allowing multiple external identifiers to resolve seamlessly to the same semantic concept payload. While local aliases are primarily handled through the "expressions" array of a single concept, identifier-level aliasing becomes necessary when importing or federating external ontologies. For instance, if an Embedded Semantics ID (ES001000) is mapped to a Wikidata ID (Q12345), the registry treats Q12345 as a systemic alias. If a user queries the registry for Q12345, the system provides seamless cross-registry resolution, returning the payload for ES001000.
17. Provenance
Provenance is the verifiable, cryptographically secure history of how a concept and its evidence were generated. As artificial intelligence models become increasingly generative and autonomous, distinguishing between human-curated ground truth and machine-generated probabilities is a matter of systemic trust. Embedded Semantics must adopt the W3C PROV Ontology (PROV-O) framework, tracking Entities (the concept record, the specific vector embedding, the definition text), Activities (the act of proposing, reviewing, computing the vector, or merging), and Agents (the human reviewer, the specific LLM such as gpt-4-0613, or the embedding model such as text-embedding-3-large). To guarantee rigorous data lineage, every piece of evidence within a concept must retain the following specific provenance parameters:
| Provenance Field | Description |
|---|---|
| Source Identity | The organization, user, or system that provided the evidence. |
| Source Version | The specific semantic version of the external ontology or data source. |
| Retrieval Timestamp | The exact ISO-8601 timestamp when the evidence was indexed. |
| Reviewer | The unique identifier of the human curator who approved the payload. |
| Review State | Current state in the governance flow (e.g., draft, approved, rejected). |
| Evidence Type | Classification of the payload (e.g., lexical\_definition, vector\_embedding). |
| Positive/Negative Examples | Retained strings used to contextually anchor the embedding space. |
| Translation Provenance | Tracking if a label was translated manually or via neural machine translation. |
| Machine vs. Human | A strict boolean or enum flag differentiating human intent from AI generation. |
| Model/Version Used | The explicit architecture (e.g., OpenAI text-embedding-ada-002). |
| Confidence | A float (0.0 to 1.0) indicating the statistical reliability of machine evidence. |
| Supersession | Tracking the lineage of which ID replaced this evidence historically. |
18. Review Workflows
The governance of meaning requires human oversight to validate algorithmic assertions. The review workflow must enforce a strict, one-way state machine to maintain ontological purity:
1. Draft/Proposed: A new concept is submitted, often automatically generated by an LLM parsing an unstructured corpus. Initial embeddings are calculated and attached.
2. Under Review: Human curators analyze the proposed definitions, review the positive and negative examples, and evaluate the vector space proximity to existing concepts to prevent duplication.
3. Published/Active: The concept is minted as official. It becomes immutable in its current version and is pushed to the production index.
4. Revised/Deprecated: Triggered by community feedback, scientific advancement, or automated semantic drift detection scripts identifying anomalies in the vector space.
19. Human versus Machine Evidence
The registry must explicitly and structurally partition its payload into model-independent evidence (human-authored truth) and model-generated evidence (statistical vectors). Human evidence includes canonical names, textual definitions, strict ontological relations (is-a, part-of), and manually curated positive/negative textual examples. This evidence is the definitive ground truth. Machine evidence includes high-dimensional embedding vectors, probabilistically generated synonyms, and semantic prototype coordinate mapping. This evidence is inherently replaceable. When a model architecture changes, the human evidence remains utterly static. The machine evidence is entirely recalculated. This architectural segregation ensures that AI hallucinations or vector space collapse do not corrupt the foundational ontology. Confidence scores must be assigned to machine evidence (e.g., similarity\_confidence: 0.92), while human evidence is treated as absolute (confidence: 1.0).
20. Registry Governance
A decentralized but strictly governed registry is paramount. The registry acts as the ultimate arbiter of semantic truth. Governance policies must explicitly define authority (who has the cryptographic clearance to mint, merge, or deprecate IDs), dispute resolution (how conflicting conceptual boundaries are mediated by editorial boards), and data lineage preservation. Cryptographic hashing of historical records, utilizing Merkle trees or similar ledger structures, must be implemented so that earlier states of the registry can be audited for regulatory compliance and model-training transparency.
21. Failure and Abuse Scenarios
Robust registry design must anticipate adversarial manipulation and systemic failures. Vector Poisoning and Semantic Attacks present a critical modern vulnerability. Malicious actors can manipulate textual data streams to force a model to generate embeddings that collapse distinct concepts into the same vector space, or push related concepts apart4. Because the Embedded Semantics Concept ID is structurally distinct from the vector, vector anomalies can be detected via continuous semantic drift monitoring—such as checking Mahalanobis distances against trusted centroids or tracking output volatility4. If an embedding is flagged as poisoned, it is isolated and recalculated without affecting the Concept ID or historical graph relationships. Edit Wars occur during disagreements over the canonical definition of a term, particularly in political, cultural, or evolving scientific contexts. To defend against this, the system must implement concept branching. If consensus fails, the core concept can be locked, and users can rely on subset metadata or branch-specific views (similar to Git version control workflows) to apply contextual overlays.
22. Recommended Embedded Semantics Lifecycle
The lifecycle of an Embedded Semantics Concept is a linear state machine. To operationalize this lifecycle, the following concrete guidance is provided for eight specific evolutionary scenarios:
| Scenario | Concrete Guidance and Action |
|---|---|
| 1\. Definition becomes more precise | If the new precision does not invalidate previously correct uses, issue a Minor Version update. If the precision excludes historical data, issue a Major Version update. Consider splitting if the broader sense is still required by legacy systems. |
| 2\. Concepts are duplicates | Deprecate the newer/less-used ID. Set status to deprecated\_merged. Set superseded\_by to the retained ID. Transfer all valid vector evidence and labels to the survivor. |
| 3\. Concept contains distinct senses | Deprecate the original ID entirely (status='deprecated\_split'). Mint two new IDs representing the distinct senses. Set split\_into=\[NewID1, NewID2\] on the deprecated record. |
| 4\. Public ID interpreted incorrectly | Deprecate the ID immediately. Mark as deprecated\_erroneous. Do not change the definition to the "correct" one, as this violently alters history. Mint a new, correct concept ID. Add an explanatory note in the provenance payload. |
| 5\. Translations disagree | If the translated term possesses a distinct cultural meaning violating the core definition, split the concept into two culturally specific concepts. If it is merely a bad translation, update the localized label (Patch version). |
| 6\. Rendering is removed | If an expression is deemed inappropriate or inaccurate, soft-delete it by updating its active status to false. It remains in the event-sourced history for provenance, but is excluded from active search indices. |
| 7\. A model changes | The Concept ID remains untouched. The registry iterates over all active concepts, recalculates the embedding vectors using the new model, and appends them to the evidence array with the new model provenance ID. |
| 8\. Concept is culturally contentious | Apply a contentious metadata flag. Lock the human definition to prevent edit wars. Allow domain-specific namespaces to fork the definition, mapping back to the parent ID via a SKOS broadMatch. |
23. Proposed Registry Status Model
To support the append-only lifecycle and ensure API consumers can correctly route legacy queries, the following statuses are mandatory:
| Status Code | Description | Actionable By APIs |
|---|---|---|
| proposed | Draft state, awaiting human/algorithmic validation. | Excluded from production index. |
| active | Fully vetted, immutable in current version. | Available for vector search and graph traversal. |
| deprecated\_merged | Absorbed into another concept. | Routes automatically to the superseded\_by ID. |
| deprecated\_split | Fractured into multiple new concepts. | Returns a 300 Multiple Choices response. |
| deprecated\_erroneous | Fundamentally flawed or conceptually incorrect. | Returns a 410 Gone / Obsolete flag. |
24. Proposed Evidence Model
The concept payload must strictly separate semantic identity, lexical labels, and mathematical evidence. The following JSON structure represents the optimal architecture for an Embedded Semantics record, isolating human truth from machine probabilities.
JSON { "concept\_id": "ES104857", "status": "active", "version": "2.1.0", "human\_evidence": { "canonical\_name": "Semantic Drift", "definition": "The gradual evolution of a concept's meaning or vector representation over time...", "positive\_examples": \["Representation drift", "Lexical shift"\], "negative\_examples": \["Data loss", "Hardware failure"\] }, "machine\_evidence": \[ { "type": "vector\_embedding", "model": "text-embedding-3-large", "dimensions": 3072, "vector\_data": \[0.012, \-0.045, 0.881\], "generated\_at": "2026-08-22T09:53:31Z", "confidence": 0.98 }, { "type": "vector\_embedding", "model": "bert-base-uncased", "dimensions": 768, "vector\_data": \[\-0.112, 0.543, \-0.198\], "generated\_at": "2024-01-10T12:00:00Z", "confidence": 0.85 } \], "provenance": { "created\_by": "Curator\_Alice", "creation\_date": "2024-01-10T12:00:00Z", "last\_modified\_by": "System\_Agent\_Omega", "last\_modified\_date": "2026-08-22T09:53:31Z" } }
Notice how multiple vectors from entirely different models, with vastly different dimensionalities, coexist peacefully under the same immutable ID, preserving the system against infrastructure upgrades.
25. Website Explanation Strategy
To effectively communicate this complex architecture to enterprise users, data engineers, and AI developers on EmbeddedSemantics.com, the strategy should focus on the visceral analogy of the Anchor and the Sail. The Concept ID is the Anchor: Immutable, heavy, and permanently fixed to the seabed. It ensures the system knows exactly where the meaning lies, regardless of the surface weather. The Embeddings and Vectors are the Sail: Dynamic, mathematically capturing the shifting winds of language, and easily swapped out when better materials (newer, larger AI models) are invented. The website should feature interactive, visual widgets demonstrating a database fracture. Users can simulate migrating a dataset from "Model A" to "Model B," watching the relational edges snap, and then activate the "Embedded Semantics Registry" to watch the Concept IDs instantly repair and map the two disparate vector spaces together.
26. Proposed Educational Articles
To build industry authority and educate the AI ecosystem on the necessity of semantic stability, Embedded Semantics should author and publish the following technical whitepapers:
1. "The Illusion of Vector Identity: Why Math is not Meaning." An exploration of representation drift versus semantic drift, detailing why vector databases fail as systems of record.
2. "Lessons from the EMR: How SNOMED CT and LOINC Solved Data Rot." Explaining RF2 append-only architectures and how AI engineers can learn from decades of clinical terminology governance.
3. "The Ship of Theseus in the Vector Space." A deep dive into the philosophy of concept evolution, detailing the mechanics of splitting, merging, and supersession.
4. "Future-Proofing RAG Architectures with Persistent Identifiers." A practical guide for enterprise AI, showing how decoupled semantic identifiers prevent vector database lock-in.
27. Diagrams
Lifecycle Diagram: Standard Concept Generation
\[ Unstructured Text \] \--\> \[ LLM Processing \] \--\> ( Generates Vector & Proposed Definition ) | v \[ STATUS: PROPOSED \] | ( Human Curator Review ) / \[ REJECTED \] \[ STATUS: PUBLISHED \] | ( Continuous Monitoring ) | ( Semantic Drift Detected / Definition Refined ) | \[ STATUS: REVISED \] ( Version bumped ) | ( Model Architecture Changes ) | \[ STATUS: DEPRECATED/SUPERSEDED \]
Lifecycle Diagram: Splitting a Concept
\[ STATUS: ACTIVE \] (Concept ID: ES0010) | ( Discovery of Dual Meaning / Polysemy Collapse ) | \+---\> \[ STATUS: DEPRECATED\_SPLIT \] (ES0010 becomes historical read-only) | \+---\> \[ MINT: ES0011 \] (Sense A: Financial) \--\> \[ STATUS: ACTIVE \] | \+---\> \[ MINT: ES0012 \] (Sense B: Geographic) \--\> \[ STATUS: ACTIVE \]
28. Frequently Asked Questions
Q: Why can't I just use the generated vector as the primary key in my database? A: Vectors change mathematically when models are upgraded, and they drift spatially as language evolves. If a vector is your primary key, a single model upgrade will sever every relational link in your system, permanently corrupting your database's integrity. Q: Does assigning a permanent ID prevent machine learning models from learning new linguistic nuances? A: No. The immutable ID points to a metadata record that contains an array of machine evidence. As models learn and language shifts, you simply append new, updated vectors to the concept record without changing the ID, thereby preserving historical backward compatibility while embracing new AI insights. Q: What happens if a concept is discovered to be entirely wrong or useless? Is it deleted? A: Concepts are never deleted in an event-sourced architecture. They are deprecated. This ensures that a system retrieving a 5-year-old medical, legal, or financial document will still be able to resolve the identifier to understand what the AI model meant at the exact time the document was originally processed.
29. Research Gaps
While this architectural framework is highly robust, several research gaps remain for the field at large that Embedded Semantics could pioneer: Automated Splitting Resolution remains a major hurdle. While algorithms can detect semantic drift and cluster divergence, generating the precise topological and ontological boundaries required to autonomously split a concept into two distinct ideas—without human intervention—remains an unsolved challenge in artificial intelligence. Cross-Dimensional Translation lacks deterministic perfection. Developing mathematically sound bridging functions to translate vector coordinates between entirely disparate architectures (e.g., Transformer-based models versus Diffusion models) without utilizing a lexical intermediary is currently highly lossy. Decentralized Registry Consensus is required for global scale. Creating Byzantine fault-tolerant consensus mechanisms for semantic definitions in decentralized, trustless networks (akin to a blockchain architecture tailored explicitly for semantic truth) will be necessary to prevent any single corporate vendor from monopolizing the definition of human knowledge.
30. Annotated Sources
The foundational architecture of Embedded Semantics draws upon the following systems, studies, and research precedents outlined in the original query and research materials:
- SNOMED CT RF2 Specification7: Provides the definitive global standard for event-sourced, append-only terminology ledgers utilizing effectiveTime and active states to prevent data destruction.
- UMLS Metathesaurus19: Establishes the structural precedent for Concept Unique Identifiers (CUIs) and the tracking of atomic units of meaning (AUIs) through historical merge/split files (MRCUI.RRF).
- OMOP Common Data Model25: Demonstrates the practical, database-level implementation of lifecycle dates (valid\_start\_date, valid\_end\_date) and strict invalidation routing to curb the "maintenance tax" of stale vocabularies.
- WordNet CILI6: Highlights the mathematical necessity of flat, interlingual indexes that persist beyond language-specific expressions or local version offsets.
- Semantic Drift and Vector Security Research2: Delineates the critical boundary between representation drift (the underlying math changing) and semantic drift (human meaning changing), definitively proving the necessity of decoupling vectors from identity to prevent model collapse and adversarial vector poisoning.
Works cited
1. What Are Vector Embeddings: Types, Use Cases, & Models | Airbyte, https://airbyte.com/data-engineering-resources/vector-embeddings
2. Autonomous Drift Learning in Data Streams: A Unified Perspective \- arXiv, https://arxiv.org/html/2605.01295
3. Measuring Semantic Drift Across Generational Corpora: A Framework Using Pretrained Embeddings \- Preprints.org, https://www.preprints.org/manuscript/202509.0661/v1/download
4. LLM Embedding Security: How to Defend Against Them \- Galileo AI, https://galileo.ai/blog/llm-embedding-security-risks-defenses
5. PERSISTENT IDENTIFICATION: A KEY COMPONENT OF ... \- GovInfo, https://www.govinfo.gov/content/pkg/GOVPUB-Y3-PURL-gpo156216/pdf/GOVPUB-Y3-PURL-gpo156216.pdf
6. cili/README.md at master \- Collaborative Interlingual Index \- GitHub, https://github.com/globalwordnet/cili/blob/master/README.md
7. Rates of Change in a Large Clinical Terminology: Three Years Experience with SNOMED Clinical Terms \- PMC, https://pmc.ncbi.nlm.nih.gov/articles/PMC1560592/
8. Practical Guide to Reference Sets \- NRCeS, https://www.nrces.in/download/files/pdf/doc\_ReferenceSetPracticalGuide\_Current-en-US\_INT\_20170516.pdf
9. sct-rs 0.21.0 \- Docs.rs, https://docs.rs/crate/sct-rs/latest/source/docs/uk-edition-structure.md
10. SNOMED International Release Management \- SNOMED Spaces, https://conf.spaces.snomed.org/wiki/spaces/RMT/pages/131958774
11. SNOMED International Release Management \- SNOMED Spaces, https://conf.spaces.snomed.org/wiki/spaces/RMT/pages/131958772/General+Practitioner+Family+Practitioner+GP+FP+Reasons+for+Encounter+Health+Issues+SNOMED+CT+package+Release+Notes+-+January+2025
12. Relationship File Specification \- SNOMED CT Document Library, https://docs.snomed.org/snomed-ct-specifications/snomed-ct-release-file-specification/component-release-file-specification/4.2-file-format-specifications/4.2.3-relationship-file-specification
13. Module Dependency Reference Set | Specifications SNOMED CT Release File Specification, https://docs.snomed.org/snomed-ct-specifications/snomed-ct-release-file-specification/reference-set-release-file-specification/5.2-reference-set-types/5.2.4-metadata-reference-sets/5.2.4.2-module-dependency-reference-set
14. Import Extensions not working(SNOMED CT India Extensions) · Issue \#496 · IHTSDO/snowstorm \- GitHub, https://github.com/IHTSDO/snowstorm/issues/496
15. NLM Office Hours: Unified Medical Language System (UMLS) \- National Library of Medicine, https://www.nlm.nih.gov/oet/ed/umls/08-24\_oh-umls.html
16. Glossary \- National Library of Medicine \- NIH, https://www.nlm.nih.gov/research/umls/new\_users/online\_learning/glossary.html
17. medgen \- KG-Registry, https://kghub.org/kg-registry/resource/medgen/medgen.html
18. Unified Medical Language System® (UMLS®) – Basics \- National Library of Medicine, https://www.nlm.nih.gov/research/umls/pdf/UMLS\_Basics.pdf
19. A GCN-Based Approach To Uncover Misaligned Synonymous Terms In The Umls Metathesaurus \- DigitalCommons@TMC, https://digitalcommons.library.tmc.edu/cgi/viewcontent.cgi?article=1133\&context=uthshis\_docs
20. Metathesaurus \- UMLS® Reference Manual \- NCBI Bookshelf, https://www.ncbi.nlm.nih.gov/books/NBK9684/
21. UMLS Database Query Diagrams \- National Library of Medicine, https://www.nlm.nih.gov/research/umls/implementation\_resources/query\_diagrams/index.html
22. UMLS Database Query Diagrams: How to perform searches in the UMLS history files (MRAUI.RRF and MRCUI.RRF) \- National Library of Medicine, https://www.nlm.nih.gov/research/umls/implementation\_resources/query\_diagrams/er7.html
23. Vocabulary Lifecycle Management \- OMOPHub.com API, https://docs.omophub.com/guides/use-cases/vocabulary-lifecycle-management
24. Chapter 5 Standardized Vocabularies | The Book of OHDSI, https://ohdsi.github.io/TheBookOfOhdsi/StandardizedVocabularies.html
25. documentation:cdm:concept \[Observational Health Data Sciences and Informatics\] \- OHDSI, https://www.ohdsi.org/web/wiki/doku.php?id=documentation:cdm:concept
26. Representing Ontogeny Through Ontology: A Developmental Biologist's Guide to The Gene ... \- PMC, https://pmc.ncbi.nlm.nih.gov/articles/PMC2830379/
27. OBO and OWL: A Logical Framework for Biomedical Knowledge, https://www.bohrium.com/en/sciencepedia/feynman/keyword/obo\_and\_owl
28. Concepts and Synonymy in the UMLS Metathesaurus \- PMC, https://pmc.ncbi.nlm.nih.gov/articles/PMC2850250/