Data-Driven UAIX Persona and Memory (File-First and Advanced)

Full report

On this page

This report defines a single semantic model for UAIX Advanced Persona profiles and memory, with two lossless projections:

• File-First (human-readable): A portable package of .uai files (and .uaix archives) that can be edited, reviewed, backed up, and versioned in source control without databases.
• Data-Driven (service/API): A normalized database or memory-store model for large-scale, transactional, multi-agent services, that exactly implements the same semantics as the file format.

Both projections share identities, logical paths, ordering, hashes, provenance, and validation rules. No semantics may be invented, lost, or auto-merged in either projection. The file workflow remains the default “Quick Start” path, and the database/API path is documented as an alternative view of the same data.

We draw on UAIX public standards for Advanced Persona, memory, and package formats, plus standards for JSON canonicalization, SHA-256 hashing, and database ACID/transactionality to guide the design.

1. Executive Decision and Terminology

• One Semantic Model: All data (persona, memory, metadata) is one abstract model. .uai files, .uaix ZIPs, and database tables are views onto this model.
• Package: A bundle of files (or DB records) declaring a persona and related memory state. Each package has a unique packageId (UUID or similar) and a declared profile (e.g. “assistant profile” or “advanced_persona_profile” as per UAIX). Packages carry metadata such as creation timestamp (createdUtc), issuer, review status, and support boundary.
• Persona Source vs Projection: We distinguish the immutable persona source (approved content describing the persona) from any runtime projection (in-memory state or conversation context). The source is versioned and preserved without alteration.
• Record: A discrete semantic element (e.g. one .uai file’s content). Each record has:
• a logical path (e.g. .uai/persona.uai or .uai/personality/voice.uai),
• a stableId (internal unique ID, e.g. GUID, that is independent of path),
• an immutable revisionId (e.g. a content-derived hash or sequential revision code),
• and links to its parent revision (for version history).
• Required-State: UAIX classifies each file/record as universal required, profile-required, content-triggered required, configuration-specific required, or out-of-scope. The same classification applies to database records.
• Ordering: Package files have a declared startup order (e.g. startup-packet.uai first) and persona dimensions follow a fixed closed ordering (see Section 3).
• Hashing/Integrity: Each file’s raw bytes are hashed with SHA-256 (per NIST FIPS 180-2) and recorded in the manifest. We also compute a semantic hash (e.g. JSON canonical hash per RFC 8785) if needed. Both the original byte stream and the canonicalized content are preserved.
• Provenance & Disclosure: Records include source authority and evidence links. Sensitive fields have disclosure flags; UAIX forbids silent inference or revealing private notes. Everything (reviews, consent, blockers) is explicitly recorded.

Decision: We adopt the UAIX definitions. A package is the unit of identity, with a packageId and profileId assigned at creation (portable across import/export). Each record within the package has a logical path and a separate immutable ID. The revisionId is a deterministic content hash or version number. On export/import, packageId and persona identity fields (names, orientation, etc.) remain the same only in an “identity-preserving restore” operation; if the persona is treated as a template for a new entity, new IDs and identities must be generated with provenance.

(Sources: Manifest spec; persona package rules; JSON canonicalization; SHA-256; ACID transactions.)

2. Normative Storage-Independent Schema

We define the abstract record schema (in JSON-like form) for every entity (packages, persona dimensions, memory records, etc.). Here are key fields (others can be added as needed):

• packageId (string): Unique package identifier (e.g. GUID). Survives export/import of the same package; must be changed for new persona creation.
• profileId (string): Package profile identity (e.g. “advanced_persona_profile”). Part of manifest.
• agentId / npcId (string): Bound to the specific agent or NPC (non-player character) that owns or uses this package.
• path (string): Logical file path, e.g. "persona.uai" or "personality/voice.uai". This is the human-readable key.
• stableId (string): Internal stable identifier for this record (e.g. a GUID or content-address). Distinct from path. Used for cross-reference (especially in DB).
• revisionId (string): Immutable revision identifier (e.g. semantic content hash or sequential version string) for this record content.
• parentRevisionId (string): The prior revisionId (if any). Used to chain version history.
• manifestCheckpoint (string): Checkpoint or manifest version for this record’s package (the version of the manifest.uaix.json at time of change).
• requiredStatus (enum): One of UAIX closed-set values (universal-required, profile-required, etc.). Matches the declared status in manifest.
• startupOrder (int): Sort index for startup assembly (lower = load earlier). The manifest declares entrypoint order.
• sizeBytes (integer): Byte length of the raw content.
• sha256 (string): Hex-encoded SHA-256 of the raw file bytes (all files) or content (for JSON canonical). From manifest.
• canonicalHash (string, optional): If we apply JSON canonicalization (RFC 8785) to get a deterministic form, hash that. This ensures semantic hash is insensitive to trivial differences.
• mediaType (string): MIME type (e.g. text/uaix) of content. Required for manifest.
• charset (string): Character encoding (typically "utf-8").
• sourceAuthority (string): Authority or origin (e.g. username, URL, or organization) for this record’s source.
• provenance (object): Links to source documents, evidence, reviews that gave rise to this content (content-address, reference ID, etc.).
• disclosureLevel (enum): E.g. public, private, sensitive, or flags for non-inference. From UAIX persona rules.
• confidence (number or enum): Source confidence or review state (e.g. reviewed, draft, expert-verified).
• reviewStatus (enum): e.g. review_required, reviewed, approved, etc. The manifest uses review_required until an importer validates content.
• status (enum): Active, Superseded, Conflict, Tombstone, etc. Indicates if this record is current or retired.
• author / createdUtc: Who created and when (UTC). For provenance.
• modifiedBy / modifiedUtc: For later revisions (UTC).

All IDs and names must be case-sensitive in DB. On import/export, we preserve raw bytes and semantic content. We evidence this by storing both the raw hash and the canonical semantic hash. (See [11] on manifest fields and [20] on canonical JSON.)

Fields portability: packageId and persona identity fields (name, orientation, etc.) survive direct restore. If the package is installed as a new persona, new IDs must be generated and lineage recorded. In either case, the linkage (parentRevisionId, source references) is recorded so provenance is never lost.

3. Advanced Persona Dimension Registry

An Advanced Persona profile is decomposed into exactly 18 ordered .uai/personality/… files (plus persona.uai as entrypoint). This closed set of dimensions, in fixed order, is:

1. Identity (identity.uai): Persona’s personal identity fields (realistic name, summary, career, etc.; open-text plus numeric detail level). Depth level controls detail/privacy.
2. Religion/Worldview (religion-worldview.uai): Faith or secular worldview, with convictions, engagement, sources, and strict no-inference boundaries.
3. Political Leaning (political-leaning.uai): Civic ideology profile (positions on issues, strength, engagement rules).
4. Sense-of-Style (sense-of-style.uai): Overview of persona’s general style and demeanor (open text plus depth_level_0_to_10).
5. Sense-of-Winter (sense-of-winter.uai): Seasonal/winter clothing and style (content-triggered by sense-of-style).
6. Sense-of-Spring-Fall (sense-of-spring-fall.uai): Spring/fall fashion style.
7. Super-Fashion-Style (super-fashion-style.uai): Any additional style motif beyond seasons (culture, era influences).
8. Jewelry-Choices (jewelry-choices.uai): Accessories preferences.
9. Physical Appearance (physical-appearance.uai): Descriptive traits (body, hair, distinguishing marks).
10. Birth-Age-Generation (birth-age-generation.uai): Date of birth, exact age (as of UTC date), and generation cohort. (Use one of these as public label and keep precise age date-locked.)
11. Romantic Orientation (romantic-orientation.uai): Romance/relationship orientation. Use this as the only public intimacy label; do not substitute other categories without explicit source evidence.
12. Voice (voice.uai): Style of spoken or written voice (tone, accent, pace).
13. Values (values.uai): Core values and priorities (open description).
14. Temperament (temperament.uai): Personality temperament (e.g. sanguine, phlegmatic, etc.) and emotional profile.
15. Reasoning Style (reasoning-style.uai): Cognitive approach (logical, intuitive, etc.).
16. Emotional Patterns (emotional-patterns.uai): Typical emotional responses or affect.
17. Relationship Style (relationship-style.uai): How persona interacts socially, forms relationships.
18. Behavioral Patterns (behavioral-patterns.uai): Habitual behaviors or routines.
19. Boundaries (boundaries.uai): Declared limits on conversation or topics (user consents, taboo topics).
20. Examples/Dialogue (examples-dialogue.uai): Example utterances or dialogues for role modeling.
21. Adaptation Rules (adaptation-rules.uai): How persona may adapt to user preferences or new info.
22. Preservation Rules (preservation-rules.uai): Rules to preserve persona authenticity (what never to change or sanitize).

(This ordering is the startup/loading order. Fields in each file depend on the dimension but generally include: an open-text description; optional enumerated values; a depth_level_0_to_10 numeric field to express detail level; and explicit disclosure: none/private/public tags. Each dimension file may also carry an semantic_content section and references to reviewed source excerpts. Importers must not invent or infer unprovided categories – e.g. if religion is “none”, do not assume any hidden faith.)

Handling missing/hidden values: If a dimension is absent or explicitly “not disclosed” in persona source, the importer should leave it blank or flagged private, but never guess or create a default. For example, if values.uai is missing, the persona simply has no listed values. If a field is declared private, it must not be shown to external APIs but preserved in the record. “None” or “mixed” values must be recorded exactly (with depth_level indicating low confidence).

Every persona package also includes the source persona file (.uai/persona.uai) which ties together voice, purpose, and switching cues. The importer must keep a copy (or content hash) of the original reviewed persona source for full fidelity and for computing provenance with derived normalized records.

References: UAIX persona decomposition.

4. File Projection Specification

The file-first projection uses UTF-8 text files and a canonical ZIP/“.uaix” archive. Key rules:

• Encoding: All .uai files are UTF-8 encoded, with Unix-style LF newlines (to maximize portability). Paths use UTF-8 and forward slashes. No BOM.
• Paths: Use a relative namespace under the root, e.g. persona.uai, personality/identity.uai. No parent-directory (..), absolute, or Windows drive paths. Paths are case-sensitive; duplicates (after normalizing case) are forbidden.
• Line Endings: Use LF. Tools must accept either CRLF or LF when reading, but write using LF.
• Manifest: Every package must include exactly one manifest at .uai/manifest.uaix.json. The manifest lists uaixVersion, packageFormat, packageId, profileId, status, createdUtc, issuer, declaredScopes, entrypoints, files, integrity, and supportBoundary. Each files entry includes path, role, requiredStatus, scope, mediaType, sizeBytes, sha256, reviewState, and source.
• Closed Inventory: The manifest’s files array must enumerate every file in the package, and all required files for the declared profile must be present. Undeclared files cause a lint failure.
• Safety: The manifest uses only relative paths and no user-specific data or credentials. For example, no hardcoded API keys, no local file system secrets. Support boundary and review data are metadata, not code.
• Validation: A correct .uaix is a ZIP with extension .uaix. Paths must not traverse or duplicate. The validator checks manifest schema, closure of required sets, declared scopes, file hashes, and trusted media types.
• Archive Rules: Use standard ZIP (no proprietary features). Do not compress or encrypt (storing as UTF-8). The archive should be reproducible; e.g. sort entries by path, normalize timestamp.
• Hashing: The manifest’s sha256 for each file is the SHA-256 of that file’s original bytes. (All-zero 000… is used only as a placeholder before import; then the importer recomputes and the record remains review_required until hashes match.)
• Normalization: The package must preserve both raw bytes and a normalized form. E.g. for JSON content, the importer should canonicalize JSON (per RFC 8785) to compute semantic hashes. The manifest (and DDL model) should record both the raw sha256 and, if needed, a canonicalHash.

Evidence of Preservation: Both the raw content and a canonical version (if applicable) are kept. The manifest and any receipts/exports include these hashes so a round-trip can verify both byte-level and semantic fidelity.

5. Data-Driven Projection Specification

We now map the same model into a normalized storage schema. This is a logical model – one can implement it in SQL or in a document DB. The key idea is no opaque blobs: every persona dimension and memory field is queryable.

Logical Tables and Documents

• Packages (table Packages): (packageId PK, profileId, status, createdUtc, issuer, declaredScopes, supportBoundary, agentId, npcId, /*auth fields*/ ).
• Records (table Records): (recordId PK, packageId FK, stableId, path, mediaType, sizeBytes, sha256, canonicalHash, depthLevel, disclosureLevel, sourceAuthority, /*etc*/ ). One row per .uai file (or memory record). Unique constraint on (packageId,path). stableId is unique across package (and may incorporate path).
• Revisions (table Revisions): (revisionId PK, recordId FK, content JSONB or CLOB, createdUtc, author, parentRevisionId FK, changesetHash). Immutable history; content holds the file text or structured data. One-to-many: each record has many revisions.
• Heads (table Heads): (recordId PK, headRevisionId FK, packageId FK). Denotes the active revision of each logical record in a package. Each record has one current head.
• Manifest/Checkpoints (table Checkpoints): (packageId, manifestJson JSONB, createdUtc, createdBy, checkpointId PK). Stores each exported manifest. The latest row is the current checkpoint; older ones form history.
• SourceArtifacts (table Artifacts): (artifactId PK, recordId FK, data JSONB/blob, artifactType, checksum, reviewStatus). E.g. raw source excerpts, evidence docs, persona seed. Linked to records.
• Provenance (edges): A table (fromRevisionId, toRevisionId) for “derived-from” links. E.g. when importing a .uai file, create a Provenance edge from the source artifact revision to the new record revision.
• Reviews (table Reviews): (reviewId PK, recordId FK, reviewer, reviewAction, reviewNotes, reviewTime). For tracking human approvals or annotations.
• Conflicts (table Conflicts): (conflictId, packageId, recordId, conflictDetails, detectedTime). To record concurrent edit conflicts if any.
• Tombstones (table Tombstones): (recordId, tombstoneTime, reason). Marks logical records that are deleted/inactive.
• ImportExportReceipts (table Receipts): (receiptId PK, packageId, operationType, timestamp, details JSONB). E.g. “import success”, “export failure (missing dimension)”, with diffs/hashes.
• Entitlements (table Entitlements): (packageId, principalId, role, scope, grantTime). Who (agent or user) can read or modify this package/records. E.g. agent-self-only.
• AgentBindings (table AgentBindings): (agentId PK, packageId, bindTime). Which agent/NPC instance the package is installed in.
• RuntimeLeases (table Leases): (leaseId PK, packageId, agentId, startUtc, endUtc, credentialsIssued). Represents when a persona is active in an agent’s session.
• IdempotencyKeys (table Idempotency): (key, operationId, timestamp). To safely retry writes (the expectedRevision pattern).
• IndexesWatermarks (table IndexWatermarks): (searchIndexId, indexedThroughUtc). Tracks how far any search/index system has processed this package’s records.

All PKs are normally GUIDs. FKs enforce package→records→revisions constraints. Add NOT NULL on all logically required fields (e.g. path). Unique constraints on (packageId, stableId) and (packageId, path) to prevent duplicates. Bounded columns (e.g. varchar lengths) can match mediaType or paths max length.

Transaction Boundaries: Each operation that mutates multiple records (e.g. a new persona import writes many dimension records) is one ACID transaction. Use optimistic concurrency: updates require matching the current head revisionId (see Section 9). Serializable or at least Repeatable Read isolation should be used to avoid lost updates. (Modern RDBMS allow snapshot isolation for better concurrency.)

Document-DB Equivalent: In a document store (e.g. CouchDB or MongoDB), we would store each Record or Revision as a document with fields as above. The invariants (unique stableId, sequential revisions, etc.) must be enforced at the application layer or using write concerns. We can index on packageId, path, and revisionId. Instead of joins, we might nest revision docs inside record docs, but still keep revision entries immutable. The same constraints (no silent merges, explicit parent pointer, authorizations) must be coded. For example, use MongoDB transactions or CouchDB revisioning to simulate the revision chain. The primary point is that the logical schema (packages, records, revisions, entitlements) is identical; how it’s stored (tables vs nested docs) does not change semantics.

Example Relational DDL (PostgreSQL-style)

CREATE TABLE Packages (
    PackageId UUID PRIMARY KEY,
    ProfileId TEXT NOT NULL,
    Status TEXT NOT NULL,
    CreatedUtc TIMESTAMP WITH TIME ZONE NOT NULL,
    Issuer TEXT NOT NULL,
    DeclaredScopes TEXT[],
    SupportBoundary JSONB,
    AgentId UUID,
    NPCId UUID,
    UNIQUE(PackageId)
);
CREATE TABLE Records (
    RecordId UUID PRIMARY KEY,
    PackageId UUID NOT NULL REFERENCES Packages(PackageId),
    StableId UUID NOT NULL,
    Path TEXT NOT NULL,
    MediaType TEXT NOT NULL,
    Charset TEXT NOT NULL,
    SizeBytes INTEGER NOT NULL,
    Sha256 CHAR(64) NOT NULL,
    CanonicalHash CHAR(64),
    DepthLevel INT,
    DisclosureLevel TEXT,
    SourceAuthority TEXT,
    Status TEXT NOT NULL,
    UNIQUE (PackageId, Path),
    UNIQUE (PackageId, StableId)
);
CREATE TABLE Revisions (
    RevisionId UUID PRIMARY KEY,
    RecordId UUID NOT NULL REFERENCES Records(RecordId),
    Content JSONB NOT NULL,
    CreatedUtc TIMESTAMP WITH TIME ZONE NOT NULL,
    Author TEXT NOT NULL,
    ParentRevisionId UUID REFERENCES Revisions(RevisionId),
    Checksum CHAR(64) NOT NULL  -- e.g. hash of Content
);
CREATE TABLE Heads (
    RecordId UUID PRIMARY KEY REFERENCES Records(RecordId),
    HeadRevisionId UUID NOT NULL REFERENCES Revisions(RevisionId)
);
CREATE TABLE Entitlements (
    EntitlementId UUID PRIMARY KEY,
    PackageId UUID NOT NULL REFERENCES Packages(PackageId),
    PrincipalId UUID NOT NULL,
    Role TEXT NOT NULL,
    GrantTime TIMESTAMP WITH TIME ZONE NOT NULL,
    UNIQUE(PackageId, PrincipalId, Role)
);
CREATE TABLE AgentBindings (
    AgentId UUID PRIMARY KEY,
    PackageId UUID NOT NULL REFERENCES Packages(PackageId),
    BindTime TIMESTAMP WITH TIME ZONE NOT NULL
);
-- (Other tables follow similarly)

Each field above should have a C# class property with a [Display(Name = "...")] attribute matching the field name (minus “Id”). For example:

public class Package {
    [Key]
    [Display(Name="packageId")]
    public Guid PackageId { get; set; }
    [Display(Name="profileId")]
    public string ProfileId { get; set; }
    [Display(Name="status")]
    public string Status { get; set; }
    [Display(Name="createdUtc")]
    public DateTime CreatedUtc { get; set; }
    // ...
}

(Properties omit “Id” suffix in the Display name, per instructions.)

Transactions and Constraints: Enforce UNIQUE(PackageId, Path) and foreign keys as above. Use transactions for multi-table writes. Define indexes on (PackageId, Path) and (RecordId, RevisionId). Use low null counts: most fields are NOT NULL (e.g. sha256, path). Delete is logical (using Tombstones); never physically delete for audit.

References

UAIX manifest schema; best practices for DB transactions.

6. Revision, Conflict, and Branch Semantics

• ExpectedRevision: All update operations (via API or DB) must specify an expectedRevisionId (the current head for that record). This enforces optimistic concurrency: if the stored head differs, the update fails with a conflict. (Like an HTTP If-Match header.)
• Create vs Update: Creating a new record (file) in a package requires that no head exists (expectedRevisionId = null). Otherwise, updates require matching the exact parent revision.
• Idempotency: Clients include an idempotency key for each operation to allow safe retries. If the same operation with same key repeats, it must not create duplicates. For example, use the combination of packageId+path+expectedRevisionId to detect retries.
• Lost-Response Recovery: If a request times out after commit, the client can fetch the current revision and compare. Receipt schemas (see below) are used to confirm success.
• Concurrent Writes: If two clients write conflicting edits to the same record (different expectedRevisionId), one must get an error. The system can record a Conflict candidate but must not merge. The client must resolve manually (or by policy) and retry.
• Conflict Resolution: Only human or policy-driven merges are allowed. For persona-affecting data (identity, values, taboo, etc.), automatic merges are forbidden. If a conflict arises, one branch can be kept active and the other tombstoned or made an alternate version. Each branch gets a new packageId if needed (true branch creates new identity lineage).
• Immutable History: Once a record revision is created, it is never altered. Superseding it means writing a new revision and updating the head pointer. Historical revisions stay accessible (for audit).
• Supersession and Tombstone: Mark a record as superseded by creating a tombstone record or status. A tombstone is an explicit flag (status="Tombstoned") in the record row. Required files cannot be tombstoned without making the package invalid.
• Checkpoints: A package checkpoint (export) is itself versioned. Branching means copying a package as new, not merging.

Disallowed: No “last-write-wins” by timestamp, no score-based auto-merge, no silent drop of one branch. Persona identity, Totem/Taboo/Talisman rules, and preservation rules can never be silently changed.

References

This follows UAIX’s “Persona Preservation Rule” and standard optimistic concurrency principles.

7. Authority, Isolation, and Entitlement

We define a multi-dimensional security context: (Tenant / Company) → Project → Agent/NPC → Package → Record. Every request must explicitly include the project or agent context (no default).

• Principle of Least Privilege: Credentials (API keys, tokens) are scoped as narrowly as possible. For example, an agent’s token only allows reads/writes to its own packages and persona records. There is no “god mode” for normal operations.
• Hidden Authority: System-level authority (master keys) is held only by trusted service components and never embedded in persona data. E.g. the bootstrap process may use a “master-authority file” to enroll exactly one project (see import), but runtime API calls must use only the narrow agent credentials thereafter.
• Explicit Project Selection: All endpoints that involve packages or personas require an explicit project or agent identifier. The system never guesses or defaults to a project.
• Agent/NPC Isolation: An agent (or NPC) can read only records bound to it (via AgentBindings). It cannot list or access packages of other agents/players. Similarly, packages cannot be enumerated globally without project or agent scope.
• Disclosure Boundaries: Each record has a disclosure level (public/private). The API filters out private fields unless the caller has explicit consent or review permission. This is enforced both in file output and database queries.
• Runtime Leases: Only one active persona can be “in use” per agent session, but multiple persona packages can exist. A lease mechanism ensures an agent cannot secretly apply two personas at once.
• Credential Rotation/Revocation: All tokens have limited lifetimes. The system tracks revocation (e.g. user logs out or deletes persona = revoke associated credentials). On revocation, any read attempts (including ID/project verification) must be rejected.
• Authenticated Readback: The service provides an API for an agent to query its own identity binding, active package, and scopes. The agent cannot read these fields for other agents.
• Fail-Closed: If any required binding is missing (e.g. project not found) or inconsistent (e.g. packageId not in project), the service throws an authorization error rather than proceeding.

Automated Bootstrap: To streamline adoption, the service can accept a protected master-authority persona file (signed/verified by an out-of-band key). This file, once, authorizes one specific project ID. The service generates an enrollment credential for that project. After this, normal API usage only sees the narrower scoped credentials. The master authority is not used again or stored in session.

8. Startup Assembly and Retrieval

When an agent requests to “load” a persona package at startup, the service (or local client) performs a strict assembly:

1. Validate Package & Binding: Verify the package’s manifest packageId and declared agentId match the caller’s identity. Check the declaredScopes includes the expected profile (e.g. advanced_persona_profile).
2. Validate Manifest: Check manifestUaix.json against schema. Verify declaredScopes and required file set closure (closed-set rule). Ensure no undeclared or out-of-scope files.
3. Collect Required Records: Gather all universal required files (memory-maintenance.uai, identity.uai, … progress.uai) and all files required by the persona profile (persona.uai plus the 18 dimensions). Also include any content-triggered files (e.g. sub-styles) if indicated. No duplicates.
4. Verify Predecessors: For each required record, load the latest revision. Verify its parentRevisionId chain is consistent (no gaps). Also verify any references (e.g. adaptation-rules may point to certain other dims) are present.
5. Check Hashes & Order: For each record, check the stored sizeBytes and sha256 against the actual content in the storage. Compute semantic hash and compare if used. Ensure the sorted startup order matches the manifest’s entrypoints.
6. Check Review Status: Ensure all required records are not in review_required or pending state. If any are unreviewed, startup fails (requires human review first).
7. Privacy/Disclosure: Remove or blank out any fields that are marked private or sensitive. Verify no out-of-bound fields got in.
8. Tombstones/Conflicts: If any required record is Tombstoned or flagged in conflict, fail. Conflicted candidates must be resolved beforehand.
9. Index Freshness: Optionally verify search indexes: if relying on search for any enhancement (like augmenting context), ensure the index watermark covers this package. (This is implementation detail; the persona must be considered complete without external search.)
10. Return Assembly: The result is a deterministic list of persona records (file contents) in the exact specified order. The startup-packet.uai file or API output contains the list of paths to load (mirroring the manifest’s entrypoints).

If any check fails, startup returns an error (no partial persona is allowed). Semantic search MAY be used to pull related background (e.g. from a knowledge base), but it must not substitute any required persona field. The returned package contains exactly the UX expected – identical to what a .uaix file would contain.

Reference: UAIX manifest rules; persona package requirements; file set closure.

9. Short-term, Long-term, and Persona Memory Separation

We classify memory data into tiers, as in UAIX:

• Persona Source (immutable): The advanced persona files (.uai/personality/*.uai and related). Once created and reviewed, they remain fixed until explicitly updated (with a review). They should never be overwritten by runtime activity. (They are locked by persona.preserve=true.)
• Active Persona Projection: The in-memory persona state as used by the agent (including maybe dynamic “persona.seed” or embeddings). This is not durably stored except as needed. It’s reconstructed each session from the source.
• Short-term Memory: Transient conversation state (short-term-memory.uai). This contains working context, recent user inputs, and pending decisions. It is saved and updated with every turn but never pushed into long-term without review.
• Long-term Memory (LTM): A pointer-only ledger (long-term-memory.uai) that references durable knowledge (wiki articles, logs). It does not store raw chat logs – only reviewed facts or validated notes with strong provenance. The LTM package is only for highly curated or public memory.
• Raw Conversation Log: The raw chat should not be auto-promoted. It may be stored separately (for debugging) but is not considered trustworthy persona data. Any LTM entry from it must go through a “memory proposal” and review process (see UAIX Agent Comm model).
• Search/Index Projections: If used, these are ephemeral index documents (with indexes watermarks) to speed retrieval. They are not authoritative and can be rebuilt from source or logs.
• Runtime Variance: Records which note how the persona’s behavior diverged at runtime (e.g. “voice changed due to policy”). These are kept separate (e.g. in progress.uai or logs) and not in the source. The persona source remains untouched.
• Safety/Platform Notes: Any modifications due to platform policies (e.g. “banned content filtered”) must be recorded outside the persona. For example, persona.preserve=true forbids altering persona text. Instead, record a no-op or err at runtime.

The system strictly enforces: No automatic promotion of conversation text into long-term memory, and no silent mutation of persona source by safety filters. Reviews (NLP or human) are needed before any memory moves tiers.

10. Import, Export, Backup, and Reconciliation

We define robust algorithms and receipts for all I/O cases:

• First Import: Read a .uaix ZIP, validate (as in section 7), then create a new Package. Generate new IDs (except if identity-preserving mode). Store all records, revisions, manifest as above. Record an ImportReceipt with success or detailed failure reasons.
• Exact Retry: Use idempotency keys. If the same package (same packageId) is imported twice with same key, second attempt should detect duplicates (e.g. existing packageId) and return the original receipt.
• Duplicate Detection: If a different .uaix with the same persona.name etc arrives, or same packageId, decide by policy: either reject as duplicate or treat as update. Provide explicit error or merge instructions.
• Identity-Preserving Restore: The service can import a persona package into the same agent (e.g. moving between devices). In this case, we keep packageId, and maybe same stableIds, and link to existing persona identity records. In the database, this is a no-op or metadata re-link. Record the receipt as “restored existing persona.”
• Install-as-New: Alternatively, the user may clone a persona as a template for a new one. In that case, we generate new packageId, new stableIds, but copy content. We record lineage in provenance tables. The original remains unchanged.
• Database-to-File Export: Gather all current head records for a package in the required order, and write .uai files (UTF-8 text) to match exactly the file spec. Create a manifest JSON from DB fields (sha256 from DB or recalc). Zip into .uaix. The export receipt includes the raw hashes and semantic hashes.
• File-to-DB Re-import: Verify that importing the just-exported file yields identical records and hashes (round-trip). If any discrepancy, log error. This is part of conformance.
• Offline Changes: If a user edits files offline (e.g. modifies persona text and increments version) and later uploads, we treat that as a new revision (expectedRevision must match the old one). The importer will validate hashes and create new DB records under the same package. If expectedRevision doesn’t match, it’s a conflict.
• Outbox Replay: If a client made changes offline (in files or DB) and queued API calls, the service’s idempotency and revision checks ensure that replayed calls apply once (or error if state changed).
• Divergent Heads: If the file-based archive is updated independently of the DB (e.g. a parallel branch), detect it by comparing semantic hashes. The receiver should refuse inconsistent merges. On re-import of a different head, require a new branch (new package).
• Partial Upload/Download: If network fails mid-zip, the importer will notice missing manifest or files and abort (fail-closed). Partial commits must be rolled back by transactions.
• Crash Recovery: Use transactional DB commits. If crash before commit, revert. If after, the import receipt shows success. The client can retry safely by idempotency key.
• Validation Failure: If any file fails schema or manifest validation, abort the import with a detailed error. The export algorithm should similarly catch any violation.
• Rollback: There is no partial persona. If any required step fails, the entire operation is rolled back and the state remains as before.

Round-trip Test: As conformance, one must import a complete package, then immediately export it, and compare:

• Byte-level: The re-exported package must have the same raw bytes for each file as the original (except manifest createdUtc or software versions; those differences must be documented).
• Semantic: The semantic hash of each dimension must match. If any whitespace or JSON ordering differences exist, explain (e.g. normalized JSON). Unexplained drift fails.

Every allowed difference (timestamps, IDs for new persona, minor encoding normalizations) must be explicitly noted. Otherwise it is a bug.

11. Documentation Strategy

We will structure UAIX docs so the file-based workflow is front-and-center, with the database approach as an advanced supplement. Proposed outline:

1. Quick Start: Files-First Workflow – Guide for beginners. Titles like “Getting Started with .uai Files” and “Building a Persona Package”. Explain the wizard, editing .uai files in a ZIP, using source control, offline review. Example: show a simple persona.uai with a few fields (e.g. name and values) and how the wizard exports it.
2. One Semantic Model – A conceptual page explaining that “files and database are just two ways of storing the same thing.” Illustrate with a diagram: box representing a package, arrows to a ZIP and to a database. Emphasize shared fields (use same terms). Include cross-links between the file spec and data spec sections.
3. Advanced: Durable Data-Driven Storage – Under AI Memory docs, pages for “Database Schema and API”. Document the tables/collections (like section 5 here), transactions, and revisioning. Include the SQL DDL example and a sample C# model class. Title suggestions: “Storage and Transactions” or “Database Model for Persona Storage”.
4. Import/Export & Round-Trip Guarantees – A dedicated guide “Importing and Exporting UAIX Packages” that step-by-step describes the algorithms above. Include diagrams of the round-trip (file → DB → file). Possibly a flowchart. Include sample pseudo-code or sequence diagram of import operation.
5. Security and Entitlements – “Identity, Access, and Trust” section. Explain how binding, least-privilege, and review keys work. Provide the data model for credentials, explain authority vs agent tokens. Emphasize fail-closed and audit.
6. Storage Provider Conformance – A page listing “Requirements for a UAIX Storage Implementation”. Include keys to test: package import test fixtures, expected receipts, etc. Possibly provide JSON fixtures (minimally valid, missing required file example, etc.) for implementers to verify against their service.

Diagrams:

• A package archive layout diagram (tree of .uai files and manifest) labeled with sha256.
• A UML or ER diagram of the relational schema (tables and FKs).
• A flowchart of import→validate→store.
• A page example screenshot for beginner.

Beginner File Example: Show a minimal persona.uai, e.g.:

name: Alice
title: Dr. Alice Wong
identitySummary: Research scientist in AI
birthYear: 1985
romanticOrientation: straight
values: "Truth-seeking, curiosity, empathy"

Next to it, show an “Advanced API Example”: e.g. a JSON GET response from /api/persona/Alice:

{
  "packageId": "...",
  "profileId": "advanced_persona_profile",
  "persona": {
     "identity": { "name": "Alice", "title": "Dr. Alice Wong", "shortSummary": "Research scientist in AI", "depthLevel": 7 },
     "birthAgeGeneration": { "birthYear": 1985, "exactAge": 41 },
     "romanticOrientation": {"label": "straight"},
     "values": {"description": "Truth-seeking, curiosity, empathy"}
  },
  "metadata": {
     "createdUtc": "2026-07-15T12:34:56Z",
     "revisionId": "...",
     "sourceAuthority": "JaneDoe"
  }
}

This illustrates the same content in a JSON service format.

12. API Reference (Runtime-Neutral Table)

We outline core service methods (HTTP+REST example):

(*) All reads require a valid auth token proving the caller is that agent or has admin role. Each method’s scope is exactly the indicated package or record; no broader authority. The API returns both semantic fields and raw metadata to allow verifying content (e.g. client sees hashes and lengths). Idempotency keys apply to any POST/PUT that writes state; keys must be unique per action to avoid duplicates.

(Actual implementation details (URI, JSON vs YAML) can vary; this table conveys the canonical interface contract.)

13. Threat Model

Key threats and mitigations:

• Cross-Agent Leakage: Package isolation ensures one agent can’t access another’s persona. All endpoints require a project/agent context, and checks enforce it.
• Confused Deputy: API must drop any ability to specify someone else’s project/tenant. Auth tokens include project scope; service ignores any mismatched ID in request.
• Source Mutation: Persona source files are immutable after review. No runtime operation can alter them (no “write” to persona/uai paths). Only explicit import or authorized update can change them (and that requires review).
• Path/Archive Attacks: The ZIP validator forbids path traversal, absolute paths, duplicates, or missing manifest. Also forbids hidden files or OS links.
• Stale Indexing: If search augmentation is used, ensure the index watermark is up-to-date for all persona records. Never rely on stale search data to fill a required field. (Otherwise attacker could cause stale persona data to be presented.)
• Rollback/Replay: Use tokens and expectedRevision to avoid unintended repeats. Rollbacks are all-or-nothing, so partial package cannot corrupt state.
• Credential Exposure: All credentials (master or agent) are encrypted at rest and rotated. No credential leaks in persona files. If an attacker obtains a token, revocation ensures it’s no longer valid.
• Forgery of Persona: Only signed/verified .uaix uploads or user-initiated file copies are allowed. The service rejects unsigned persona changes.

In short, the system treats persona packages as evidence, not code. It quarantines and requires human review for anything out-of-norm (secrets, exec instructions). Failure is safe: if any authority check fails, the persona is not loaded (fail-closed).

14. JSON/File Fixtures (Valid/Invalid Examples)

• Valid Minimal Persona Package: A ZIP with manifest.uaix.json, persona.uai, and one dimension (e.g. personality/identity.uai) all matching rules.
• Invalid (Missing Required): A package missing persona.uai or one of the 18 personality files for an advanced profile – must trigger no-op or error.
• Invalid (Duplicate Path): Two files named personality/identity.uai (even differing case) – must be rejected.
• Invalid (Wrong Order): A manifest claiming an entrypoint order that doesn’t match actual startup order – error.
• Invalid (Content-triggered): personality/super-fashion-style.uai present but sense-of-style.uai missing – error (trigger without declared source).
• Privacy Leakage: A file contains a private field (e.g. psychic detail) but is marked public – importer flags it.
• Cross-Agent: A GET request for a package by a different agent – yields authorization error.
• Stale Revision: A PUT with wrong expectedRevision – API returns conflict error.
• Simultaneous Writers: Two concurrent updates with same expectedRevision – one succeeds, the other fails.
• Same Persona in Two Agents: The same persona copy used in two active sessions (allowed only if explicitly cloned or restored, otherwise conflict in identity binding).
• Revoked Credential: Using an old token yields auth error.
• Stale Index: If we rely on search augmentation, ensure index watermark prevents using stale results (e.g. an index entry older than last update is not used for required fields).
• Tombstoned Required: If persona.uai is accidentally tombstoned, package load fails.
• Partial Startup: Missing a dimension file -> load fails, not partially complete persona.
• Idempotency: Retry an import/upload exactly succeeds only once; second attempt returns same receipt.
• Bootstrap: A special master-authority package import auto-creates the first project credential, then only narrow creds are used after.

(In practice, each of these scenarios should be covered by automated tests using both file and API interfaces.)

15. Conformance Test Scenarios

• Advanced Persona Import and Sorting: Import a full .uaix with all 18 dimensions out of order; verify the DB stores them and exports them sorted exactly as required.
• Export Reconstructs Paths: Export a persona package and confirm all .uai files appear at the correct paths in ZIP as before.
• Byte & Semantic Round-Trip: Import a package, then export it and diff every file with original. Byte-for-byte equality (allowing timestamp/manifest differences). Compute SHA-256 on both; must match for each.
• Missing Dimension: Remove one dimension from the DB, attempt startup – should fail with a clear “missing required file” error.
• Duplicate Path Error: Inject duplicate path in the manifest or DB; validator must reject.
• Wrong Manifest Hash: Tweak a file after computing manifest hash; import must reject due to hash mismatch.
• Undeclared File: Add an extra file (e.g. extra.uai) not in manifest – package lint should fail.
• Private Leakage: Mark a persona field private; attempt a cross-agent request – it must not appear.
• Cross-Project Read: A user from project A tries to list package of project B – should be denied.
• Wrong Binding: Supply a package that declares agentId ≠ caller’s agent – startup fails auth.
• Expected Revision Mismatch: Update a record with outdated expectedRevision – should get a 409/Conflict.
• Concurrent Writers: Two updates to different records simultaneously should both commit (different PKs). Two to same record lead to one conflict.
• Same Persona in Two Sessions: If not explicitly cloned, the system should forbid re-using the same identity in two active sessions or should mark it as conflict.
• Branch New Identity: Perform an “export as new” of an existing persona. The system assigns new packageId and notes lineage; old package remains unchanged.
• Stale Index Use: Simulate that search index is behind: the startup should not include data only found via index if not yet indexed (must fail or warn).
• Tombstone as Required: Mark persona.uai as tombstoned, then attempt load – fails.
• Lost Mutation & Retry: Send an import, drop response, then retry. It should detect the package exists and not double-create.
• Bootstrap Flow: Use a special signed “master persona file” to enroll a first package; confirm that after bootstrap, only a narrow scoped token is used.

Each scenario should be a scripted test, verifying file vs database outputs match expectations, and that security rules hold.

16. Documentation Plan

As in section 11, the documentation will have a clear “Files First” narrative, with advanced topics clearly labeled Advanced. For example:

• Quick Start: Files-First – include example .uai snippet and steps to bundle/export.
• The One Semantic Model – conceptual overview (diagram).
• Advanced: Durable Data-Driven Storage – label this advanced and link to it from Files First for more detail.
• Import/Export – cookbook with exact I/O steps.
• Security – separate section emphasizing identity and entitlements.
• Conformance – a checklist for implementers.

Navigation: These pages will sit under the UAIX AI Memory guide section. Cross-link heavily between the corresponding topics (files vs DB views). Use callouts (“Advanced topic” or “File-first note”) to guide readers. Provide code blocks for examples. Include warnings (e.g. ⚠️ avoid mixing persona and project data).

17. Adoption Plan

We propose a UAIX-1 Pre-Release with a single semantic contract (no deprecated fields). All references use the UAI-1 current schema IDs. The beginner workflows (wizard, .uai format) remain unchanged. The new advanced profile and memory tables are an extension, but they do not alter any existing files or formats.

• Starter Kit: Release a new conformance pack including fixtures for advanced persona packages.
• Transition: Educate implementers that the same .uaix format is still valid, but now they can optionally use a backing database that supports the same semantics.
• No Deprecation: Do not introduce legacy aliases or silent compatibility rules. If something changes (e.g. a field name), it must be a new version. We avoid any ambiguity by versioning (e.g. UAIX-3.x with advanced persona).

Acceptance: The criteria is that a beginner can ignore all database details and use the existing UAIX toolchain (wizard, ZIP, validator) as before, while an advanced implementer can build a DB-backed service that passes all tests above.