Steganography Encoder and Decoder Design for Protocol5 Iota

Full report

On this page

Executive summary

Protocol5’s IOTA-1 language-converter is not a lossless translator and should not be treated as a stable symbolic codec. The official documentation repeatedly defines it as an approximate public-symbol semantic converter: English input is segmented, mapped to concept evidence, and then ranked against public Unicode or ISO 10646 candidate glyphs; the visible glyph is not the semantic authority by itself. The converter also uses phrase-first segmentation, NFC normalization, scalar and grapheme handling, public Unicode safety checks, and private-use rejection. Those design choices matter more than the exotic appearance of the output, because they sharply constrain which steganographic methods can survive without damaging Iota’s own semantics or violating its rules.

From an engineering standpoint, the safest conclusion is this: methods that modify the visible Iota text itself are rarely the best fit. The best fit is a sidecar or wrapper method applied after conversion, so the Iota glyph string remains semantically inspectable and Protocol5-compliant. The most practical options are, in order, markup-sidecar steganography for structured transports, post-conversion zero-width insertion for plain-text transports that you have round-trip tested, and cross-modal image or audio steganography when the Iota output is being distributed as a rendered asset rather than as raw text. By contrast, homoglyph substitution, private-use characters, and most pre-conversion linguistic or syntactic stego are poor matches for Iota because they either collide with public-symbol boundary rules or destabilize phrase and concept ranking.

The highest-confidence recommendations are therefore: best — keep the Iota output unchanged and store encrypted payload in adjacent markup or structured metadata; pragmatic — append zero-width symbols only after final Iota output and only on channels you have tested for preservation; experimental — use controlled paraphrase or semantic bucketing only when both parties can observe the pre-conversion text or trace, because Iota’s approximate mapping makes such schemes fragile as pure output-only channels.

Protocol5 Iota constraints that matter for steganography

The converter documentation establishes a precise operating model. On the public Language Converter page, English is the active human-language lane; the converter exposes directions for English to ɩ≃1 and ɩ≃1 to English, with modes including Database only, Hybrid fallback, and LLM assisted, plus tokenization views for Single glyph and Component parts. The page states that conversion is approximate, traceable, and evidence-backed, with views for input expression, Unicode safety, concept anchors, candidate signs, semantic drift, and Developer JSON.

The Protocol Rules page adds the most important mechanics. For English-to-Iota, input is split at paragraph and sentence boundaries; within each sentence the engine tries the longest stored English segment first from Category.Categories, then falls back to Category.Words, and finally ranks public glyph candidates from Category.ISO10646. Paragraph breaks and sentence punctuation are explicit boundaries meant to prevent lookup from spanning unrelated thoughts. The same page says the logic layer owns NFC normalization, System.Text.Rune scalar handling, grapheme grouping, semantic segmentation, candidate ranking, evidence summaries, and private-use rejection. It also states that IOTA-1 must use assigned ISO/IEC 10646 / Unicode characters and public standard sequences, and that private-use areas are prohibited.

The Expression-Concept documentation is especially relevant because it separates visible expression from inferred concept. That means any encoder that mutates the visible string is not merely changing a cover layer; it is tampering with an input the system explicitly treats as evidence. The docs even provide concrete phrase-level examples, such as expression concept mapping to the expected IOTA-1 gist 名意, signifier and signified also mapping to 名意, and AI expression concept mapping to 智名意. Those examples are useful because they show that multiple English expressions can converge on similar Iota gist outputs, but they also show why exact decoding from visible glyphs alone is not guaranteed.

The public API descriptors expose a partial machine contract. The status endpoint reports supported directions english-to-iota and iota-to-english, modes database-only, llm-assisted, hybrid, and semantic-hybrid, alongside an approximation notice. The ranking-demo endpoint documents JSON request fields input, direction, mode, tokenizationMode, and resultLimit, with resultLimit constrained to 1 to 50. The search endpoint likewise uses JSON and accepts input or an embedding, plus target values both, categories, words, or iso-10646, again with resultLimit 1 to 50. What the docs do not specify is an input-character maximum for the converter itself, nor any exact whitespace normalization beyond sentence and paragraph boundaries.

The symbol-atlas pipeline described by Protocol5 is also important. The docs say the symbol layer is built from public metadata including the Unicode Character Database, CLDR, Unihan, and emoji data, with public provenance retained. That strengthens Iota’s auditability, but it also makes hidden-character tricks easier to detect if they exploit obscure or security-sensitive Unicode behavior rather than semantically justified public symbols.

A compact extraction of the Iota-facing contract is below.

The table is synthesized directly from the public converter, rules, evidence, and endpoint descriptors.

Steganography techniques and what they imply for Iota

Text steganography falls into several families with very different operating assumptions. The classic distinction in the literature is between format-based methods, linguistic methods, and generated/statistical methods; broader steganography also includes markup, image, audio, and network carriers. Across those families, the persistent trade-off is among capacity, detectability, and robustness: higher hidden bandwidth usually creates stronger statistical or structural artifacts, while more robust schemes often reduce payload. That trade-off is fundamental in the steganography literature, not specific to Iota.

Whitespace systems are the canonical format-based text method. The SNOW tool hides data by appending spaces and tabs at line ends, and its own manual states that this usually stores 3 bits every 8 columns of appended whitespace. That is simple and historically important, but its robustness is weak in modern web and document pipelines because editors, mail clients, diff tools, and browsers often trim or normalize trailing whitespace.

Zero-width Unicode systems hide bits in invisible characters. The long-running unicodeSteganographer project documents a 4-character default alphabet using U+200C, U+200D, U+202C, and U+FEFF, implying a payload of about 2 bits per inserted invisible symbol; it also notes that U+200B can be deleted by Gmail in browser mail composition. A recent open-source tool, StegZero, makes the capacity trade-off explicit: an 8-character zero-width alphabet yields 3 bits per symbol, while a 2-character binary alphabet using U+200B and U+200C yields 1 bit per symbol but is more likely to survive aggressive sanitization.

Homoglyph techniques encode data by substituting visually similar characters from different scripts or Unicode blocks. Unicode’s own security standard, UTS #39, treats these as single-script, mixed-script, and whole-script confusables, and provides confusable-detection mechanisms precisely because they are security-relevant. In practice, this means homoglyph stego can be easy for humans to miss but relatively straightforward for software to detect with confusable skeletons and script-mixing rules. Open-source libraries such as codebox/homoglyph exist specifically to detect such substitutions.

Linguistic schemes use meaning-preserving substitutions or paraphrases rather than invisible formatting. Atallah and colleagues’ early natural-language watermarking work embeds watermark bits in syntactic structure, while Chang and Clark’s work develops more practical synonym-based linguistic steganography with contextual filtering and vertex coding. More recent neural and distributional methods can push capacity upward; for example, a 2026 CBOW-based linguistic stegosystem reports up to 3 bits per word by selecting among the top eight predicted candidates, though that is not specific to Iota and should be viewed as a cover-text result rather than a converter guarantee. These schemes are often more semantically natural than format hacks, but they are also vulnerable to paraphrase, editing, and language-model steganalysis.

Generated linguistic steganography goes further by using a language model itself as the carrier generator. Ziegler, Deng, and Rush show that arithmetic coding with a neural language model can generate cover text while matching the model distribution, which is attractive from a security perspective because the generation process aims to stay on-distribution. But implementation complexity is high, and this family is conceptually orthogonal to Iota: it creates a new language model channel, rather than using Iota itself as a stable deterministic codec.

Markup-based steganography uses the fact that some source-level structure is not visible in the rendered page. SOAP and webpage steganography papers show payloads hidden through element ordering or other source-level transformations that leave the rendered page looking the same. This makes markup methods especially relevant whenever Iota output is carried inside HTML, XML, or a structured protocol surface. The upside is that the visible Iota string can remain untouched; the downside is that reserialization, minification, canonicalization, or copy-paste to plain text may destroy the payload.

Image and audio steganography are not native to Iota, but they become relevant whenever the Iota output is rendered into another medium. Classical image methods include high-capacity LSB embedding and more resistant schemes such as F5, which uses matrix encoding and permutative straddling to reduce detectable changes. MP3-oriented audio schemes likewise exploit compressed-audio structure. These are stronger fits when the intended transport is a screenshot, a poster, a video overlay, or audio narration of Iota output, rather than raw Iota text itself.

The following comparison table is the report’s engineering synthesis of the public sources. Capacity is intentionally heterogeneous because the underlying carriers differ; where source material gives units like bits per symbol or bits per appended columns, those are preserved rather than forced into a misleading common denominator.

These comparative judgments are grounded in the cited method descriptions and in Iota’s explicit public-symbol, normalization, and segmentation rules.

Mapping each method to the Iota converter

The central design question is where to place the covert layer relative to Iota’s own pipeline. Because Iota splits input into sentences and segments, ranks concepts approximately, and treats visible glyphs as evidence rather than authority, pre-conversion steganography is fragile unless the covert information is carried in a way that does not perturb stored-segment matching or concept ranking. By contrast, post-conversion steganography can preserve the visible Iota result and therefore respects the converter’s own interpretability model.

A useful design rule is therefore:

flowchart TD
    A[Need covert channel around Iota] --> B{Must visible Iota text stay unchanged?}
    B -->|Yes| C{Carrier can be structured markup?}
    C -->|Yes| D[Use markup sidecar or source-level wrapper]
    C -->|No| E{Plain-text channel preserves zero-width chars?}
    E -->|Yes| F[Use post-conversion zero-width encoding]
    E -->|No| G[Render Iota into image or audio and hide there]
    B -->|No| H{Can both parties inspect pre-conversion text or trace?}
    H -->|Yes| I[Experimental semantic or syntactic encoding]
    H -->|No| J[Avoid linguistic stego with Iota]

That decision rule follows directly from Iota’s documented approximate semantics and Unicode handling, together with the known robustness profile of text-stego methods.

Applied method-by-method, the fit looks like this. Markup sidecar is strongly feasible when the Iota output lives in HTML, XML, UAI-like envelopes, or other structured records, because the visible string can remain exactly the same while the payload rides in source-level structure. Preprocessing is minimal: convert English to Iota first, then wrap the returned glyphs. Postprocessing extracts metadata or source-level order and verifies that the displayed text still matches the original Iota glyph sequence. Capacity is effectively limited by the surrounding transport rather than by Iota itself, which is why this is the best fit.

Zero-width steganography is feasible only as a post-conversion step. If you inject invisible characters into the English input before conversion, you risk changing phrase segmentation, glyph safety, or ranked concept evidence. If you inject them after the converter returns the Iota string, you can preserve the visible text while carrying a small payload. A practical estimate is easy to compute: for an Iota output of length N grapheme clusters, using one hidden symbol after each cluster gives N bits with a binary alphabet or 3N bits with an 8-character alphabet. For example, the documented Iota gist 智名意 has three visible grapheme clusters, so a postfix scheme gives about 3 bits in binary mode or 9 bits with an 8-character alphabet if the channel preserves those characters. The main preprocessing step is encrypting the payload first; the main postprocessing step is channel-specific round-trip verification.

Whitespace steganography has poor fit. The SNOW family encodes at line ends, but Iota’s own parser assigns meaning to paragraph and sentence boundaries, and modern web forms or copy-paste channels frequently collapse or delete exactly the whitespace patterns SNOW depends on. You can still use whitespace stego after conversion in a plain-text file or email you fully control, but it is not a strong match for a browser-facing converter workflow. Sample capacity is carrier-specific: if you have 24 columns of trailing slack in a formatted cover file, SNOW’s published rate implies roughly 9 bits of payload. That is workable in a static text file, but fragile through browser submission or web rendering.

Homoglyph substitution is a bad fit both before and after conversion. Before conversion it can perturb English terms that the category and word vectors depend on. After conversion it alters the very public glyph identity that Iota wants kept inspectable through assigned Unicode characters and public metadata. It is also the easiest family to flag with modern Unicode security tooling. In practice, the only reason to discuss it here is to recommend against it.

Semantic or synonym substitution is theoretically attractive because Iota is about concept relations, but that is exactly why it is risky. The expression-concept documentation shows that different English phrases can map to similar Iota gists, such as expression concept and signifier and signified both yielding 名意. That proves there is some room for semantic binning. But if the hidden payload must be recoverable from the final Iota output alone, then synonym-based pre-conversion choices are unreliable because the converter is explicitly approximate and not a one-to-one codebook. This method only becomes feasible when both parties can observe the original English input, the converter trace, or a stable candidate identifier beyond the top visible glyph string.

Syntactic transformations suffer from the same problem, but more strongly. They can hide bits in active/passive voice or other sentence-level alternations, as the natural-language watermarking literature shows, yet Iota’s sentence and phrase segmentation means seemingly harmless syntax changes may cause different longest-segment matches or ranking behavior. That makes the converter part of the threat model, not just the cover channel. For Iota-specific engineering, syntactic stego belongs in the experimental category only.

Image or audio steganography is feasible when the final artifact is not raw Iota text but a rendering of it. A screenshot containing 名意 or 智名意 can carry much more payload than the glyph string alone, and modern image or audio schemes do not interfere with Iota’s own semantics because the converter step has already finished. This is not text stego anymore, but it is often the right answer if the distribution channel is visual media, signage, PDF, or multimedia.

Security, detection, and mitigations

The biggest security risk is not that covert data exists; it is that covert data may alter meaning or auditability in a system whose own documentation insists on inspectable public evidence. For Iota, that means the primary threat is any hidden layer that changes visible glyph choice, script identity, segmentation, or safety checks. A second threat is Unicode abuse more broadly: invisibles, bidirectional controls, and confusables can create strings that render one way and parse another, which is why Unicode Security Mechanisms and Trojan Source-style defenses exist.

Detection should therefore happen at multiple layers. At the text layer, render all whitespace, hex-dump suspicious strings, and scan for unexpected Cf-class format behavior, zero-width characters, bidirectional controls, and private-use code points. At the Unicode layer, compute a confusable skeleton and apply mixed-script or restriction-level checks in the style of UTS #39. At the Iota layer, run the converter’s own Unicode safety checks, preserve provenance, and compare the visible expression against a canonicalized version that strips disallowed covert carriers. These are standard defensive measures and align with Iota’s public-symbol rules.

For linguistic stego, statistical and ML steganalysis remain relevant. Taskiran and colleagues showed that lexical natural-language steganography could be detected with language-model and SVM-based methods, reporting 84.9% classification accuracy in one experimental setting. More recent NLP-based linguistic stego work continues to report detection figures against neural detectors rather than assuming invisibility. The lesson for Iota is straightforward: if the covert scheme depends on pre-conversion synonym or syntax selection, assume it is detectable and only use it when the channel goal is deniability against casual observers, not resistance against deliberate steganalysis.

The right mitigations are conservative. First, encrypt and authenticate the payload before hiding it; steganography should hide the presence of the payload, not replace cryptography. Second, keep the visible Iota string canonical and unchanged whenever possible. Third, allowlist channels rather than assuming Unicode preservation: StegZero’s documentation shows that some platforms strip invisibles aggressively, while others preserve them. Fourth, cap payload length because detectability rises with artifact density. Finally, log both a visible canonical form and a raw transport form if auditability matters, so decoding does not depend on an editor’s rendering quirks.

Recommended approaches and pseudocode

The three approaches below are ordered by how well they fit Iota’s documented behavior.

Best approach — markup sidecar around unchanged Iota output. Use this when the Iota string is carried in HTML, XML, or another structured envelope. The visible Iota expression remains untouched, and the hidden payload rides in adjacent metadata or source-level structure rather than in the glyph sequence itself. This is the strongest Iota-specific recommendation because it preserves semantic inspectability.

function encode_with_markup_sidecar(english_text, payload_bytes, iota_converter):
    iota = iota_converter.convert(
        input=english_text,
        direction="english-to-iota",
        mode="database-only"
    )

    canonical_iota = NFC(iota.output_visible_text)
    ciphertext = AEAD_Encrypt(key, payload_bytes, aad=canonical_iota)
    token = Base64Url(ciphertext)

    carrier = RenderStructuredEnvelope(
        visible_text=canonical_iota,
        hidden_token=token,
        profile="p5-sidecar-v1"
    )

    assert VisibleText(carrier) == canonical_iota
    return carrier

function decode_with_markup_sidecar(carrier):
    canonical_iota = NFC(VisibleText(carrier))
    token = ExtractHiddenToken(carrier, profile="p5-sidecar-v1")
    payload = AEAD_Decrypt(key, Base64UrlDecode(token), aad=canonical_iota)
    return { visible_iota: canonical_iota, payload: payload }

Pragmatic approach — zero-width postfix after final Iota output. Use this only on channels you have tested end-to-end. It works well for short payloads and plain text, but it should be treated as fragile and channel-specific. Binary alphabets using U+200B and U+200C are usually safer than larger alphabets; larger alphabets are shorter but more easily stripped.

function encode_with_zero_width(iota_visible_text, payload_bytes):
    canonical_iota = NFC(iota_visible_text)
    ciphertext = AEAD_Encrypt(key, payload_bytes, aad=canonical_iota)

    bits = ToBits(ciphertext)
    zw_symbols = MapBitsToZeroWidth(bits, alphabet=["U+200B", "U+200C"])  # binary mode

    clusters = GraphemeClusters(canonical_iota)
    assert len(bits) <= len(clusters)   # one hidden symbol per grapheme cluster in this simple variant

    out = ""
    for i in range(len(clusters)):
        out += clusters[i]
        if i < len(bits):
            out += zw_symbols[i]

    return out

function decode_with_zero_width(stego_text):
    raw = stego_text
    visible = StripConfiguredZeroWidth(raw)
    canonical_iota = NFC(visible)

    zw = ExtractConfiguredZeroWidth(raw)
    bits = ZeroWidthToBits(zw, alphabet=["U+200B", "U+200C"])
    ciphertext = BitsToBytes(bits)

    payload = AEAD_Decrypt(key, ciphertext, aad=canonical_iota)
    return { visible_iota: canonical_iota, payload: payload }

Experimental approach — controlled semantic bucketing before conversion. Use this only if both parties can observe the source-English phrase or stable converter trace, and only after building a vetted phrase inventory that behaves consistently under Iota. This is not suitable when only the final Iota output is transmitted. The attraction is that it aligns with Iota’s concept orientation; the weakness is that approximation makes decoding unstable.

function encode_with_semantic_bucket(bitstring, bucket_map, iota_converter):
    phrase = bucket_map[bitstring]             # e.g., vetted paraphrase inventory
    result = iota_converter.convert(input=phrase, direction="english-to-iota")
    return {
        source_phrase: phrase,
        visible_iota: result.output_visible_text,
        trace_id: result.trace_or_candidate_id
    }

function decode_with_semantic_bucket(observed_source_phrase, inverse_bucket_map):
    return inverse_bucket_map[observed_source_phrase]

A concrete end-to-end example, using a documented Iota phrase rather than a live POST call, looks like this. The Expression-Concept page says that AI expression concept has expected IOTA-1 gist 智名意. Suppose the hidden payload is the 3-bit value 101, and the post-conversion channel is known to preserve U+200B and U+200C. With the rule U+200C = 1 and U+200B = 0, the encoder emits the visible Iota string with one invisible symbol after each grapheme cluster:

source English: "AI expression concept"
documented Iota gist: "智名意"

payload bits: 1 0 1
mapping: 1 -> U+200C, 0 -> U+200B

transport form:
智<U+200C>名<U+200B>意<U+200C>

human-visible rendering:
智名意

The decoder reads grapheme clusters, extracts the configured zero-width characters, reconstructs 101, and authenticates the payload against the visible string 智名意. This is a workable proof of concept, but it is not the best production pattern for Iota because any sanitizer, normalizer, messaging platform, or re-submission into the converter may strip or flag the hidden characters.

Open questions and limitations

Some parts of the public Iota contract remain unspecified. The documentation I reviewed clearly describes approximation, phrase-first grammar order, public Unicode boundaries, tokenization labels, JSON request fields, and resultLimit ranges on the documented read-only descriptors, but it does not specify a maximum converter input length or a formal allowlist or denylist for all invisible Unicode characters submitted through the public UI. That means any production encoder should treat zero-width and other invisible-character handling as channel- and deployment-specific, not guaranteed by the docs.

I also could not rely on a live conversion POST trace in this report, so concrete examples use documented examples from the public pages rather than a direct runtime interaction with /api/justaniota/convert. That is sufficient for architectural analysis, but it means candidate rankings, SQL-backed versus seed-backed behavior, and any edge-case Unicode rejection behavior could differ on the live host. The public status endpoint itself is explicitly a snapshot and may change.

The overall conclusion, however, is stable. Given Protocol5 Iota’s explicit commitment to approximate meaning, public assigned characters, inspectable evidence, NFC and grapheme handling, and private-use rejection, the most rigorous design is to treat Iota as a semantic surface that should remain visible and canonical, and to carry hidden payloads in a transport layer around it rather than inside its visible glyph choices whenever possible.