Open Symbol Architecture for JustAnIota and Protocol5

Full report

On this page

Framing the experiment correctly

For the JustAnIota.com version of this idea, the cleanest and most defensible architecture is not a private encoding profile and not an exact translation system. It is an approximate semantic retrieval experiment over assigned Unicode and emoji symbols. That distinction matters because Unicode explicitly says private-use characters only have meaning by private agreement, while RFC 3629 confirms that Unicode and ISO/IEC 10646 remain synchronized on repertoire and code-point assignments. If your public claim is “no secret dictionary, no private profile, public international and emoji characters only,” then the public-facing system should operate on assigned Unicode symbols, not on private-use code points.

It also needs to be stated plainly that this is not exact conversion. Embeddings are vector representations whose distances correlate with semantic similarity; they are useful for search, clustering, and relatedness, but they do not create a mathematically exact text equivalence the way a codec does. SQL Server 2025’s vector features reinforce that same distinction: VECTOR_DISTANCE can compute an exact distance between vectors, but the underlying semantic relationship is still an approximate model of meaning, and VECTOR_SEARCH is explicitly approximate nearest-neighbor search.

The most important public statement for both sites should therefore be this: JustAnIota and Protocol5 do not use a secret or proprietary dictionary. They use public Unicode characters, emoji, and public Unicode data to compute approximate semantic similarity. Results are best-fit approximations, not exact translations. That wording is aligned with how Unicode, CLDR, Unihan, and embedding systems actually work. Unicode supplies the public symbol inventory, CLDR supplies public names and keywords used for search and predictive typing, Unihan supplies public Han ideograph data, and embeddings supply similarity math. None of that is a hidden lexicon.

There is one crucial caveat you should make explicit on the site so it cannot be “argued with” later: assigned Unicode characters are public and cross-script, but they are not a formally universal semantic language. Unihan itself says Han ideographs are formally defined through mappings rather than through a universal semantic definition, and CLDR’s short names and keywords are locale data used for search and predictive typing. Emoji guidance in UTS #51 is about structure and interoperability of emoji characters and sequences, not about a single culture-free semantic ontology. So the right claim is public-symbol approximation, not “Unicode itself is a universal meaning layer.”

Building an open symbol corpus without a secret dictionary

The way to make “no proprietary dictionary” operationally true is to build an open symbol atlas from public Unicode data sources and store provenance for every symbol, every metadata fragment, and every embedding. The Unicode Character Database is the base layer for character properties and names; CLDR contributes public short names, annotations, keywords, and TTS labels used for search and prediction; Unihan contributes Han ideograph mappings, readings, dictionary-like data, and variants; and UTS #51 defines the structure of emoji characters and sequences. Unicode’s licensing materials also make clear that most Unicode data files and software are made available under the Unicode License v3, even though some publication materials have different restrictions, which is why the implementation should rely on data files and derived metadata, not on republishing code chart artwork.

A practical first-release corpus should be open but curated. It should include: RGI emoji and stable emoji sequences from the emoji standard; Han ideographs that have useful Unihan metadata; and technical or conceptual symbols such as arrows, math symbols, and geometric shapes where UCD properties and names are useful. It should exclude private-use code points, surrogate code points, noncharacters, controls, and most isolated format or combining characters because those contribute transport and rendering problems rather than stable public semantics. Unicode’s own materials explain the difference between public assigned characters and private-use areas, UTS #51 defines the emoji inventory structure, and the UCD provides the public property layer for assigned characters.

The most important implementation insight is that you should not embed raw code points as if the code point number itself were the meaning. Instead, the system should embed a public metadata gloss assembled from public Unicode sources. For emoji, CLDR annotations are especially important because the CLDR guidance explicitly says annotations are used for specific character features and predictive typing, while also noting that immutable Unicode names are unique identifiers and often are not the best descriptive names for emoji. For Han ideographs, Unihan is the right enrichment layer because it contains readings, dictionary-like data, semantic variants, and related support data for languages using the Han script.

A good symbol atlas schema for v1 is shown below. This is the point where the “no secret dictionary” rule becomes auditable:

This also gives you a strong product message: the system is not taking an English sentence and looking it up in a hidden bilingual map. It is comparing English text to a public corpus of Unicode-derived symbol glosses and returning the nearest public-symbol candidates. That is a very different claim, and it is much easier to defend technically.

Enterprise architecture for WordPress and C#

The cleanest architecture is to keep WordPress as the publication and interaction layer while putting the actual conversion engine in a reusable C# service. That preserves the requirement that the C# code be easy to consume and test from other projects, while also letting JustAnIota.com remain a WordPress-managed site. On the .NET side, the right baseline is .NET 10 LTS, which Microsoft lists as an active LTS release supported through November 2028.

flowchart LR
    A[Browser] --> B[WordPress block or page]
    B --> C[WP REST endpoint]
    C --> D[JustAnIota .NET API]
    D --> E[Facade]
    E --> F[Application and Logic Layer]
    F --> G[ADO.NET SQL Repository]
    F --> H[LM Studio Adapter]
    G --> I[(SQL Server 2025)]
    H --> J[LM Studio localhost]

    K[Population Worker] --> E
    L[Unicode / CLDR / Unihan loaders] --> K

Inside the C# solution, the best shape is still a Facade-led modular monolith. The public surface should stay small and stable, for example IJustAnIotaFacade, with calls like EnglishToSymbolsAsync, SymbolsToEnglishAsync, RoundTripAsync, PopulateAtlasAsync, and HealthAsync. Behind that surface, keep clear project boundaries for Contracts, Application, Domain, Infrastructure.SqlServer, Infrastructure.LMStudio, and a Worker. That gives you enterprise-level testability without forcing JustAnIota.com into a distributed system. Microsoft’s current guidance around the options pattern and IHttpClientFactory fits this architecture well because it supports configuration-bound services, typed HTTP clients, logging, and resilient outbound calls.

On the WordPress side, the best UI shape is a server-registered custom block or dynamic page component. WordPress recommends registering blocks on the server using block.json metadata, and the Interactivity API introduced in WordPress 6.5 is the right tool for building a responsive front-end without dragging a separate SPA framework into the site. For REST endpoints, WordPress requires route registration on rest_api_init, and permission_callback is required on registered routes. Same-origin browser interactions should use WordPress cookie authentication and nonces, exactly as the REST API handbook describes.

The WordPress plugin should generally not contain the semantic engine. Its job should be to validate requests, enforce permissions, render the block, and forward requests to the .NET backend. WordPress’s own HTTP API supports this through wp_remote_post(), and if the endpoint is configurable or not fully trusted, wp_safe_remote_post() is the safer choice because it validates URLs and redirects to reduce SSRF risk.

For background processing, do not rely on regular page requests to drive population and reindexing. WordPress documents that wp_schedule_event() triggers when someone visits the site after the scheduled time has passed, which is fine for lightweight publication tasks but not ideal for deterministic ETL or embedding work. For this project, the enterprise-grade answer is a dedicated .NET worker or system cron job for atlas population, model warmup, re-embedding, and cache rebuilds.

SQL Server 2025 and ADO.NET design

SQL Server 2025 is a strong fit for this experiment because it can keep public symbol metadata, English anchors, and vectors in the same database instead of forcing you into a separate vector store. Microsoft’s vector type is native, stored efficiently in binary form, exposed as JSON arrays for convenience, and supports dimensions from 1 to 1998 by default. SqlClient adds native support through SqlVector<T>, and in .NET 10 the SqlDbType.Vector enumeration is available for vector parameters. That means your ADO.NET repository can remain low-level and efficient without needing an ORM for the retrieval-critical path.

A recommended schema for this system looks like this:

The database design should remain conventionally relational around the vector columns. Microsoft is explicit that vector columns do not support key constraints such as primary keys or foreign keys, do not support DEFAULT or CHECK column constraints, do not support B-tree or columnstore indexes directly, and should not be treated as uniqueness or join keys. So the correct design is ordinary scalar keys plus vector attributes.

The retrieval strategy should start simple. Microsoft recommends exact search with VECTOR_DISTANCE when the filtered candidate set is small enough, and gives a general recommendation of exact search when the number of vectors involved in a search is below roughly 50,000. Approximate search is available through VECTOR_SEARCH() and CREATE VECTOR INDEX, but those features are preview-oriented in SQL Server 2025 and the current vector index type is DiskANN. For a public demo site, exact search over a filtered candidate set is the lowest-risk starting point, with approximate indexing added only if corpus size or latency requires it.

For DatabaseOnly mode, SQL Server full-text search is as important as vector search. Microsoft’s full-text documentation makes a useful distinction: CONTAINSTABLE supports precise or fuzzy matching with ranking, weighting, and proximity, while FREETEXTTABLE is intended to match meaning rather than exact wording. That is exactly what you want for a no-live-AI fallback over English anchors and public symbol metadata. The practical pattern is: full-text search narrows the candidate set, then vector similarity reranks within that set.

SQL Server 2025’s new AI features are interesting, but they should be treated as optional. Microsoft now supports CREATE EXTERNAL MODEL, AI_GENERATE_EMBEDDINGS, and local ONNX Runtime models in SQL Server 2025. The documented API formats include OpenAI, Azure OpenAI, Ollama, and ONNX Runtime. Because LM Studio exposes OpenAI-compatible /v1/embeddings and /v1/chat/completions endpoints, it is plausible in principle that SQL external models could be pointed at an LM Studio-compatible endpoint in an OpenAI-style configuration. However, Microsoft’s documentation does not explicitly name LM Studio as a supported target, so the safer baseline is still app-side embedding generation in C# with vectors persisted through ADO.NET. That inference keeps the architecture robust and avoids coupling the whole database story to an undocumented integration edge.

One operational detail is worth carrying into the repository design: SqlClient’s internal retry providers do not retry commands that execute inside an open transaction. So retry policy should be applied around transaction boundaries or in repository operations that are safe to repeat, not blindly within a transaction scope.

Population and query flows

The population pipeline should be built around public-source reproducibility. First, load the current UCD baseline and selected Unicode data sets. Next, ingest CLDR emoji short names and keywords and Unihan enrichment for Han ideographs. Then assemble a public gloss string for each symbol from those metadata fragments. After that, generate one or more embeddings per symbol using LM Studio and persist them along with the source-version metadata. The UCD is versioned and published through Unicode’s public data directories, so the entire build can be pinned to specific Unicode versions rather than drifting silently.

For LM Studio, the clean local-first default is to run it on localhost and front it with a typed .NET client. LM Studio documents OpenAI-compatible endpoints for /v1/embeddings and /v1/chat/completions, and also documents a headless mode through llmster for background service deployments. Just as importantly, LM Studio also documents that authentication is off by default unless API tokens are enabled, which means a public demo deployment should enable API tokens and ensure only the server-side .NET application talks to LM Studio directly.

For the default embedding model, EmbeddingGemma is a very strong fit. Google’s model card says it is a compact multilingual embedding model trained on 100+ spoken languages, with a 2K input context, 768-dimensional output that can be truncated to 512, 256, or 128 via Matryoshka representation learning, and a footprint suitable for on-device or local deployment. The same model card also notes that prompt instructions can be prepended for different retrieval use cases, which means your ModelRegistry needs to store not only the model name and dimensions but also the exact prompt profile used during population.

The runtime should expose three user-visible modes. In AI-assisted mode, the incoming English text is normalized, sent to LM Studio for embeddings, then searched against SymbolVector with SQL candidate generation and reranking. In DatabaseOnly mode, the system does not pretend to create fresh semantic embeddings without AI; instead, it normalizes the input, uses full-text search over EnglishAnchor and public symbol metadata, optionally uses deterministic lexical vectors generated in C#, and then reranks against stored vectors and cached query embeddings. In RoundTrip mode, the system can go English → public symbols → English by retrieving the top public-symbol candidates and then the nearest English anchor explanations attached to those symbols. That satisfies your “gist of it” requirement honestly: it is a recovered approximation, not an exact reversal. This is an architectural inference from the cited sources and your no-exactness requirement, but it is the most truthful way to deliver a no-live-AI fallback.

The response contract should make the approximation status impossible to miss. Every result should include: the top symbol candidates, the similarity score, whether live AI was used, the source data provenance, the Unicode code points or emoji sequence, and one of four mode labels such as AiAssisted, CachedSemantic, DatabaseOnly, or RoundTripApproximate. That is good product design for a public research demo because it matches the actual guarantees of Unicode metadata and embedding similarity rather than implying false precision.

Public wording, risks, and open questions

The public wording on JustAnIota.com should be extremely explicit. A good homepage statement would be: “JustAnIota uses public Unicode characters and emoji as an open symbol field. It compares English text and public symbols using public Unicode metadata and embedding similarity. It does not use a secret or proprietary dictionary. Results are approximate semantic matches, not exact translations.” That phrasing is directly aligned with the role of UCD, CLDR, Unihan, and embedding vectors.

There are also real risks that should be disclosed. Unicode’s security guidance warns about mixed-script and visually confusable characters, which matters on any public site that surfaces international characters or mixed-symbol outputs. Emoji presentation can vary by platform even when the underlying sequence is standardized. CLDR short names and annotations are localized, which is good for public search but also means the same symbol can be glossed differently by locale. SQL Server 2025 vector indexes and some AI features are still preview-oriented, and LM Studio’s server is open by default unless you enable authentication. Those are manageable risks, but they should be acknowledged, not hidden.

The best final recommendation is this: keep the C# enterprise core, but redefine IOTA-1 for this public site as an open public-symbol approximation profile rather than a private encoding profile. WordPress should host the educational and interactive surface. The C# service should own the Facade, ranking logic, LM Studio integration, and SQL Server access. SQL Server 2025 should store the open symbol atlas, vectors, and provenance. And the site should repeatedly and plainly say that the system works by public-symbol similarity, not by a hidden translation dictionary and not by exact conversion. That architecture is the one most consistent with your stated goal and the underlying standards.

The open questions that still need a product decision are limited but important. The first is whether v1 should return single symbols, short symbol sequences, or both. The second is how broad the initial English anchor corpus should be for DatabaseOnly mode, because no-live-AI quality will depend heavily on that corpus. The third is whether SQL Server 2025 preview features such as vector indexes and float16 storage are acceptable on a public demo host, or whether v1 should stay with exact VECTOR_DISTANCE, full-text search, and standard float32 vectors until the preview surface matures.