Language-Agnostic Embeddings for IOTA-1 and JustAnIota

Full report

On this page

Core answer

Yes, this is achievable in a useful engineering sense, but not in the strongest literal sense of “every translation automatically produces the exact same raw embedding numbers.” Current multilingual embedding research can place many languages into a shared semantic space, and recent work can explicitly separate language-specific signal from more language-agnostic semantic signal. At the same time, multilingual representations are still only moderately language-neutral by default, not perfectly language-free. That matters because Protocol5’s own IOTA-1 documentation already makes the right conceptual move: it separates visible expression from inferred concept, ranks evidence for the expression-to-concept relation, and already compares vectors against glyph candidates rather than treating glyphs as meaning by themselves.

For IOTA-1 and JustAnIota, the most practical design is therefore concept canonicalization rather than hoping a base encoder will magically emit identical vectors for every equivalent phrase in every language. In other words: use a multilingual encoder to map expressions into a shared semantic neighborhood, then map those outputs into a canonical concept registry that stores one stable concept ID and one canonical vector for the meaning you want to preserve. That is how “Hello,” “Hola,” and “Bonjour” can end up with the same downstream numbers in your system: not because the first-pass encoder is perfectly language-free, but because all of those expressions resolve to the same registered concept object. This conclusion follows directly from the way multilingual models are trained toward shared-space similarity, and from IOTA-1’s explicit expression-versus-concept architecture.

What language-agnostic embeddings can actually do

Modern cross-lingual sentence encoders are specifically built to place semantically related text from different languages into one space. Google’s multilingual Universal Sentence Encoder embedded 16 languages into a shared semantic space through translation bridge tasks. LaBSE extended the idea much further, combining masked language modeling, translation language modeling, dual-encoder translation ranking, and additive margin softmax; the published model covers 109+ languages and reported 83.7% bi-text retrieval accuracy over 112 languages on Tatoeba, far above the 65.5% reported for LASER in that comparison. More recent open models continue the trend: multilingual E5 was contrastively pre-trained on 1 billion multilingual text pairs, and BGE-M3 reports uniform support for more than 100 working languages while supporting dense, sparse, and multi-vector retrieval in one model.

Sentence-level meaning is also a better target than word-level meaning for your use case. A recent ACL paper on linear cross-lingual mapping argues that sentence semantics is less ambiguous than single-word semantics and that, if multilingual sentence embeddings really represent semantics, similarity between embeddings should be invariant under translation. That fits your “throw away the language, keep the meaning” goal much better than word-level dictionaries or one-to-one translation tables.

The important limitation is that language cannot be assumed to disappear completely. Research on multilingual contextual embeddings found they are more language-neutral than aligned static embeddings, but still only moderately language-neutral by default; that work specifically proposed per-language centering and explicit projection as ways to increase neutrality. A newer 2025 paper goes further and shows that multilingual sentence encoders can be disentangled into language-specific and language-agnostic components for improved cross-lingual semantic similarity. This is strong evidence that “meaning without language” is partly achievable, but only if the system is explicitly designed to remove or quarantine the language signal.

There is one more practical caution. Translation equivalence at the sentence level is usually better than at the word level, but pragmatic meaning is not automatically preserved in a simple, lossless way. Translation scholarship emphasizes that pragmatic meaning depends on context and shared background knowledge, and research on multilingual translation formality explicitly shows that languages differ in grammatical formality markers. So a production system should distinguish core semantic intent from pragmatic features such as politeness, register, role relationship, or time-of-day specificity.

Why IOTA-1 and JustAnIota are already structurally aligned with this idea

The strongest part of the current IOTA-1 design is that it already rejects the naive equation “glyph equals meaning.” The Protocol5 documentation says the converter is better when it does not pretend a glyph is the meaning, and instead treats the visible symbol as expression or signifier while ranking the supported concept or signified. It also says the signified is not guaranteed by the glyph shape and is inferred from metadata, descriptors, vectors, ontology fit, and review. That is almost exactly the conceptual precondition needed for a language-agnostic concept layer.

The current hosted converter is also already vector-aware and evidence-oriented. Protocol rules state that for English-to-IOTA conversion, input is segmented, matched against stored categories, and the resulting vector is compared against glyph candidates in Category.ISO10646. The Evidence Workbench exposes ranked candidates, trace evidence, vector evidence, ranking-lane counts, semantic similarity, and provenance. The public language converter further states that each result is an approximate semantic neighbor, not an exact translation, and that stored public evidence remains primary even when live AI fills gaps.

That means the missing piece is not the philosophical model; it is the breadth and neutrality of the concept space. Right now, the hosted converter says English is the active human-language lane. JustAnIota, meanwhile, describes itself as a compact, structured AI-message profile built on Unicode constraints, registries, canonicalization, and validation. Taken together, those public claims point toward an obvious next step: keep the existing evidence-first converter architecture, but move its center of gravity from an English-centric lane toward a multilingual concept registry with canonical, language-independent concept vectors.

A practical architecture for concept-first embeddings

The clean pattern for your system is:

expression in any language
    -> multilingual encoder
    -> language-neutralizing layer
    -> concept registry lookup
    -> canonical concept vector
    -> renderer to English / Spanish / IOTA glyphs / other outputs

This is the architecture that best fits both the research and the current Protocol5/JustAnIota semantics. The base encoder gives you cross-lingual neighborhood structure; the neutralizing layer reduces language-specific residue; the concept registry converts “nearby meanings” into one stable semantic object; and the renderer turns that object back into a target expression or glyph set while preserving provenance and confidence. That approach also matches IOTA-1’s published distinction between visible expression and inferred concept.

The concept registry should use opaque IDs, not English labels as the system of record. Wikidata is a useful model here: it is multilingual, fully structured, and its items are uniquely identified by a Q number while carrying labels, descriptions, and aliases in many languages. For IOTA-1, an internal identifier like C0001842 is better than using an English key such as HELLO as the authoritative identity. English, Spanish, French, IOTA glyph sequences, and any other renderings should all be metadata attached to the concept record, not the root identity itself.

This is also the point where you can satisfy the user-visible requirement of “same embedding numbers.” Keep two vectors in the system: a raw expression embedding and a canonical concept vector. The raw embedding is used for retrieval, clustering, and evidence. The canonical concept vector is what you return once the expression has been resolved. If “Hello,” “Hola,” and “Hej” all map to concept C0001842, they can all inherit that concept’s single canonical vector. This is the most reliable way to make the numbers identical downstream while still preserving the traceability that Protocol5 already exposes.

The language-neutralizing layer should not be skipped. The 2020 language-neutrality paper found that multilingual contextual representations are only moderately neutral by default and proposed two practical improvements: centering the representation per language and fitting an explicit projection on small parallel data. More recent disentanglement work shows that extracting a dedicated language-agnostic component can further improve cross-lingual semantic comparison. In practice, that gives you a natural “middle stage” between base embeddings and concept canonicalization.

Model and training choices

If you want an open, alignment-first base model, LaBSE remains a serious reference point. It was designed specifically for multilingual sentence embeddings, covers 109+ languages, and reported strong Tatoeba bitext retrieval numbers. Multilingual E5 is a more recent open alternative with a very large contrastive training recipe over 1 billion multilingual text pairs, and it is a strong candidate if your main use cases are semantic retrieval, clustering, and multilingual RAG-style lookup.

If you want a base model that fits Protocol5’s emphasis on inspectable evidence, BGE-M3 is especially attractive. It supports more than 100 languages, handles long inputs up to 8,192 tokens, and unifies dense retrieval, sparse retrieval, and multi-vector retrieval in one model family. Its own documentation explicitly recommends hybrid retrieval plus reranking in RAG pipelines. For an evidence-driven converter, that matters because sparse lexical evidence and dense semantic evidence can be shown together instead of hiding everything in a single opaque vector score.

If you want to design for a future beyond human language alone, SONAR and OmniSONAR are worth attention. Meta’s SONAR project describes itself as a multilingual and multimodal fixed-size sentence embedding space with speech and text encoders and decoders, and says it substantially outperforms LASER3 and LaBSE on xSIM and xSIM++ multilingual similarity search tasks. The newer OmniSONAR preprint extends the idea into a single semantic space spanning text, speech, code, and mathematical expressions, reporting strong results across thousands of language varieties. That direction is very close to your long-term ambition of representing meaning without privileging one human language. The main caveat is deployment maturity and licensing: the SONAR repository warns that some released models are under non-commercial licenses even though the code is MIT-licensed.

If you want a managed API baseline before building your own registry layer, the mainstream commercial options are already multilingual enough for prototyping. OpenAI’s text-embedding-3-small and text-embedding-3-large are documented as having higher multilingual performance than earlier generations, and OpenAI exposes a dimensions parameter for shortening vectors while preserving concept-representing properties. Cohere’s embed-v4.0 supports more than 100 languages and multiple output dimensions through matryoshka-style shortening. Voyage’s 4-series models are explicitly positioned for multilingual retrieval quality and offer compatible embeddings across the series. These products are strong proving-ground encoders, but the language-agnostic identity layer would still need to live in your own registry if exact canonical vectors matter.

For your specific Protocol5 / JustAnIota use case, the most defensible near-term choice is usually BGE-M3 or multilingual-E5 as the self-hosted base, followed by a custom neutralizer and a registry-backed concept layer. A fast proof of concept could use OpenAI or Cohere embeddings first, but the final system should not rely on provider vectors alone if your goal is stable, language-independent concept identity. That is because the real asset is not just multilingual similarity; it is canonical semantic resolution.

How the greeting set should be represented

Your greeting list is a good example of why the system should separate core semantic act from pragmatic traits. If the application-level meaning you care about is simply “opening greeting,” then English “Hello,” Spanish “Hola,” French “Bonjour,” German “Hallo,” Dutch “Hallo,” Swedish “Hej,” Danish “Hej,” Norwegian “Hei,” Finnish “Hei,” and many others can absolutely be forced onto one concept ID and one canonical concept vector. But if you care about politeness, formality, honorific force, relational stance, or context-specific usage, then a one-vector collapse can hide distinctions that translation studies treats as real and important. Pragmatic meaning is context-dependent, and multilingual formality research shows languages differ in how such distinctions are marked.

A good design is therefore to store one shared core concept plus side-channel pragmatics. In a registry, that looks less like “every greeting is the same word” and more like “every greeting can point to the same speech act, while carrying optional metadata about register, relational work, or cultural framing.” That approach also fits IOTA-1’s own public claim that the glyph is only visible evidence and the concept must be inferred and ranked.

An illustrative record could look like this:

{
  "concept_id": "C.GREETING.OPENING",
  "canonical_vector": "[shared concept vector]",
  "labels": {
    "en": ["hello"],
    "es": ["hola"],
    "fr": ["bonjour"],
    "de": ["hallo", "guten tag"],
    "ja": ["こんにちは"],
    "ko": ["안녕하세요"]
  },
  "pragmatics": {
    "register": "neutral_or_polite",
    "speech_act": "greeting",
    "time_of_day": "unspecified_or_contextual"
  }
}

If your business rule really is to discard all pragmatic substructure and preserve only the broad greeting act, then all of those inputs can resolve to C.GREETING.OPENING and inherit one canonical vector. If later you decide that “Bonjour,” “Guten Tag,” or honorific forms should not collapse so aggressively, you can split the registry into sub-concepts without changing the broader architecture. That ability to change semantic granularity over time is exactly what you want when languages are mutable but concept identity needs to remain stable.

How to evaluate whether language has really been removed

The first test is concept consistency across languages. For each concept in your registry, collect multilingual expressions, embed them, run them through the neutralizer, and check whether they resolve to the same concept ID with a tight cosine cluster around the canonical vector. The 2024 ACL findings paper gives you a useful philosophical benchmark here: if sentence embeddings truly preserve meaning under translation, semantic similarity relations should be invariant under translation. The 2020 multilingual distillation paper gives you the corresponding training objective: translated sentences should map to the same location in vector space.

The second test is language leakage. Train a simple language classifier on the vectors you claim are language-agnostic. If it can easily guess the language, then your registry is still carrying too much language-specific signal. This is not a hypothetical risk: the language-neutrality study found that multilingual contextual embeddings are only moderately neutral by default, and it explicitly discusses stronger neutrality through centering and projection.

The third test is task quality on standard multilingual benchmarks. MIRACL covers 18 languages with more than 726,000 relevance judgments for 78,000 queries, making it useful for multilingual retrieval behavior. MTEB spans 8 tasks, 58 datasets, and 112 languages. MMTEB expands the landscape much further, to more than 500 evaluation tasks across more than 1,000 languages. Even if your final product is not generic retrieval, these benchmarks are still helpful sanity checks for whether your base encoder is separating semantic signal from language and surface form in a robust way.

The fourth test is converter-specific round-trip evaluation. Protocol5 already exposes approximation status, ranked candidates, vector evidence, semantic similarity, and provenance. Preserve that ethos. When a concept match is weak or ambiguous, the system should abstain, lower confidence, or surface multiple concept candidates rather than silently forcing an exact semantic collapse. That is not a bug; it is consistent with IOTA-1’s own published boundary that the system proposes approximate semantic neighbors rather than lossless translations.

Strategic recommendation

The best path for Protocol5 and JustAnIota is to make the concept layer, not the language layer, the authoritative object. Concretely, that means keeping the current expression-to-concept philosophy; replacing the English-only vector lane with a multilingual encoder; adding a language-neutralizing stage through centering, projection, or disentanglement; resolving expressions into an opaque concept registry with multilingual labels and aliases; and returning the canonical concept vector as the stable semantic identity for downstream ranking, glyph selection, and message transport. That recommendation follows directly from Protocol5’s current expression-versus-concept design, from JustAnIota’s registry/canonicalization posture, and from the current state of multilingual embedding research.

If you want the shortest route to a working prototype, start with a strong multilingual encoder and build the registry first. If you want the strongest medium-term system under your own control, start with multilingual-E5 or BGE-M3, add a small neutralization head trained on domain-specific parallel and paraphrase pairs, and make concept IDs the system-of-record. If you want the broadest long-term research direction for non-language modalities, keep an eye on SONAR and OmniSONAR, because they push the concept of a shared semantic space beyond text alone.

The deepest takeaway is this: a truly language-agnostic embedding system for IOTA-1 will not come from a single miracle model. It will come from a layered design in which multilingual embeddings provide the candidate semantic neighborhood, and a registry-backed concept layer provides the stable, language-independent meaning object. That is the architecture most consistent with both the external research literature and the public design principles already visible in Protocol5 and JustAnIota.