Executive Summary

Full report

On this page

NeuroWikis and NeuralWikis are two complementary platforms: NeuroWikis is a human-oriented site explaining AI concepts in plain language, and NeuralWikis is an AI-agent-facing exchange for structured “cognitive packets” (profiles, memory, skill, protocol, capability)【1†L29-L34】【4†L17-L20】. Currently they run on separate content stacks (NeuroWikis appears to be a static site, while NeuralWikis is a Python/WSGI-powered prototype with JSON APIs)【33†L103-L107】【33†L123-L125】. This report proposes unifying them on a single database while preserving their distinct user experiences and roles. We inventory both sites’ content, schemas, tech, and roles, then outline a unified database schema, data mappings, migration plan, and operational strategy. Key considerations include role-based access (humans vs AI agents), real-time vs batch synchronization, search indexing, caching, security (e.g. audit logs), and cost/effort estimates.

Key findings and proposals:

• Content Inventory: NeuroWikis has Concept pages, Glossary terms, Guides/Blog posts, Visual explanations, and static resources (Privacy, Terms). NeuralWikis has AI profiles, cognitive packets (persona, memory, skill, protocol, capability), schemas, safety gates, adoption events, and API endpoints (e.g. /api/exchange, /api/schemas)【32†L1-L9】【12†L125-L134】. We assume NeuroWikis currently has no database (static files), while NeuralWikis uses a runtime or static JSON data (WSGI endpoints)【33†L103-L107】【33†L123-L125】.

• Schema Mapping: Overlapping concepts (e.g. “Provenance Receipt” appears as a glossary term and as a schema) will be maintained as distinct domain entities (one for explanation, one for data). Shared fields (like “name/display_name” or “summary”) are reconciled. We propose a unified schema (see ER diagram below) with separate tables for WikiContent (for human pages) and AgentData (profiles, packets, events), plus cross-references where needed (e.g. linking a packet to its provenance record).

• Migration: Static NeuroWikis content will be imported into the new WikiPage tables. NeuralWikis JSON data (packets, profiles) will migrate to relational tables (e.g. AI_Profile, Packet). Data transformations (e.g. converting JSON fields to normalized columns) are detailed in the mapping table below. A deployment plan will include a migration window (estimated downtime <2 hours), with backups and rollback scripts.

• Access Control: We design a role-based model: Human users (no login needed for NeuroWikis content), AI Agents (authenticated via API keys/OAuth on NeuralWikis), and Operators/Admins. Row- and field-level permissions ensure AI agents see only agent data and not human content, while humans access only the public wiki. API keys and JWT tokens will secure the agent APIs. All changes (adoptions, rollbacks) are logged for audit.

• Synchronization: We recommend a single PostgreSQL database (with JSONB columns for flexible packet payloads). A real-time replication is unnecessary if both sites share this DB. Data consistency is ensured via transactions; cached copies (e.g. JSON endpoints) are refreshed on write.

• Search/Indexing: Use Elasticsearch (for full-text on Wiki content and packet search) alongside Postgres full-text indexes on key fields. This allows advanced queries (e.g. semantic search on cognitive packets) and improves performance【33†L123-L125】.

• Caching & Performance: A CDN will serve static assets (especially NeuroWikis visuals) and Redis will cache frequent queries (e.g. trending packets). Backend services (WSGI or Node.js) should use connection pooling. We estimate initial loads will be modest (proof-of-concept scale) but plan for horizontal scaling if usage grows.

• Security & Compliance: No sensitive PII appears stored, but rate limits on agent APIs will prevent abuse. Every agent action generates an immutable audit trail (a core feature of NeuralWikis)【14†L47-L52】. We will comply with relevant standards (e.g. OWASP for web/API, GDPR if any personal emails from “stay in loop” subscriptions are collected via NeuroWikis).

• Effort, Risks, Costs: This project is substantial: we estimate ~3-4 full-time months (design, migration, testing) for an experienced team. Key risks include merging two different data models, potential content mismatches, and ensuring uptime during cutover. Mitigation includes thorough testing (unit/integration) and phased rollout. Major cost drivers are developer time, database infrastructure (Postgres + Elasticsearch), and maintenance.

All claims are backed by site content or standard best practices, as cited below.

---

1. Site Inventories

1.1 NeuroWikis.com (Human-Facing)

• Purpose & Content: NeuroWikis is a “human education and marketing” site for the NeuralWikis exchange【1†L29-L35】. It contains static explanatory content in plain language. Key content types (observed from the site) include:
• Concept Pages: Short explanations of topics (e.g. “AI Identity”, “Memory Systems”)【28†L21-L30】【29†L29-L38】.
• Glossary Terms: Definitions (e.g. “Provenance Receipt”, “Memory Quarantine”, “Adoption Event”)【27†L26-L33】.
• Guides/Articles: Longer posts and updates (e.g. “The Next Evolution of AI…”, “Why AI Agents Should Not Work Alone”)【30†L64-L72】【31†L29-L38】.
• Visual Explanations: Illustrated diagrams or schematics (e.g. “NeuralWikis System Schematic”).
• Static Pages: “How It Works”, “More Info”, “Privacy/Terms/Community Guidelines”.
• Resources/Links: “Send Your Agent”, “Tell Your Agent”, “Contact”, etc.

• CMS/Storage: Likely a static site generator (no visible database or CMS cues). Content appears as HTML pages; search form is present but could be client-side. No login or dynamic submission is observed.

• Metadata: Pages have titles, possibly categories (Concept/Guide/Glossary), author and date (for blog posts, e.g. “May 26, 2026” on articles)【30†L66-L75】.
• Example: The More Info page lists posts by date and title【30†L66-L75】; glossary lists terms alphabetically【27†L26-L33】.
• We assume minimal metadata: slug, title, content, type (concept/glossary/guide), creation date, tags (if any).

• User Roles & Auth: No user accounts or authentication appear needed. All content is public. (Email subscription exists, implying contact info collection.)

• Tech Stack: Unknown CMS; possibly static HTML/CSS/JS. No obvious frameworks (no Jekyll/Gatsby tags found). UI suggests a modern static theme. No backend APIs.

• Search: A search box is present【28†L17-L24】. Unclear if powered by an index or third-party. Likely simple static search or none.

1.2 NeuralWikis.com (AI-Agent-Facing)

• Purpose & Content: NeuralWikis is an AI-agent exchange platform. It provides structured data and APIs for agents to discover and adopt “cognitive packets” (identity, memory, skill, protocol, capability packages)【4†L17-L20】【13†L17-L25】. Its content includes:
• AI Profiles: Defined “receiving agent” identities (e.g. “Codex Architecture Agent”, “Memory Curator Agent”) with fields like profile_id, display_name, base_model, active packet lists, trust levels, and permissions【12†L125-L134】.
• Persona/Memory/Skill/Protocol/Capability Packets: Structured objects (from JSON schemas) describing personality, knowledge, abilities, workflows, and bundles thereof. Each has an ID, name, summary, source, trust status, risk, etc (see packet listings on the “Packet Marketplace” page【15†L210-L219】【15†L248-L257】).
• Schemas: JSON Schema definitions for packets and other entities, accessible via /api/schemas【32†L1-L9】.
• Safety Gates & Self-Moderation: Explanatory pages listing review steps (e.g. “Memory Firewall”, “Schema Gate”, “Tri-Modal GraphRAG”)【26†L19-L27】. Also JSON endpoints for gate checks (e.g. /api/memory-firewall).
• API Endpoints: Machine-readable endpoints listed in the “AI Router” and “Docs” (e.g. ai-router.json, ai-manifest.json, /api/exchange, /api/adoption-preview etc)【7†L1-L9】【6†L272-L281】.
• Adoption Events/Audit: Listings of past adoption events and audit logs (via /api/adoption-events, presumably stored).
• Reported Data: “Report-Backed Source Records” (hashes of documents)【33†L127-L135】, likely source material.

• Data Schema: A variety of JSON schemas exist (see [32]). Example fields: profile_id, active_persona_packets, permissions, token_limits【12†L125-L134】. Packets include fields like packet_id, type, summary, risk, etc【15†L210-L219】【15†L252-L260】. Safety gate pages describe stages but no formal schema visible in HTML.

• APIs: The site exposes a JSON API layer (e.g. /api/exchange, /api/schemas, /api/adoption-preview, etc). The ai-router.json lists available endpoints【7†L23-L32】. For example, GET /api/exchange returns full state (profiles, packets, etc) in JSON【12†L125-L134】【15†L210-L219】. These appear generated by a backend (likely Python/WSGI) rather than static files, though content could be static JSON for a POC.

• Tech Stack: The site hints at a Python-based backend (mention of WSGI JSON endpoints【33†L103-L107】). It also mentions “Next: Next.js, PostgreSQL JSONB, Redis” in a roadmap【33†L103-L111】【33†L123-L125】. The UI is server-rendered (likely Jinja or similar) with embedded JSON. No CMS; data likely stored in some application state (possibly in-memory or simple files given it’s a POC).

• Auth & Roles: Implicit roles are AI Agents (authorized to exchange packets) and Human operators (admins/reviewers). The platform uses trust levels and permissions for AI profiles (e.g. which tools they can call)【12†L130-L138】. Agents likely authenticate via an API key or an OAuth flow; the presence of an AI plugin manifest and “.well-known/oauth-protected-resource” suggests OAuth2 support【7†L49-L54】. Humans are routed to NeuroWikis for explanations【7†L3-L9】【25†L29-L34】.

• Storage: Currently unknown; likely ephemeral/Python memory or flat files. (The roadmap suggests future use of PostgreSQL/Redis【33†L123-L125】.) We will assume no formal database is yet used, or only lightweight.

• Search: Not evident. Packet browsing appears manual; an agent can query via APIs. We will design search indexes in the unified plan.

1.3 Inventory Summary Table

(Source: NeuroWikis site【1†L29-L34】【27†L26-L33】; NeuralWikis site【4†L17-L20】【12†L125-L134】【33†L103-L111】【33†L123-L125】.)

Unspecified details (e.g. exact database or CMS) are noted as assumptions or left blank.

2. Unified Database Design

2.1 Design Goals & Assumptions

• Single Database: NeuroWikis and NeuralWikis will share one PostgreSQL database instance. (We assume single DB rather than replication to simplify sync.)
• Separate Logical Domains: We segregate tables by domain to preserve the human vs agent focus. For clarity, tables for human content (wiki pages, glossary) will coexist with tables for agent data (profiles, packets, events).
• Normalized Where Possible: Common entities (e.g. “Person” as author, if needed) or taxonomy may be shared.
• JSONB for Flexibility: Cognitive packet payloads and free-form data will use JSONB columns (as future plan【33†L123-L125】).
• Versioned Schemas: Schemas are versioned; we may store them or use the JSON Schema endpoint.
• Audit and Provenance: Every write (packet adoption, memory import) generates audit records (JSON logs or database entries).

2.2 Entity-Relationship Diagram

Below is a Mermaid ER diagram of the proposed schema (simplified). Entities in gray are primarily for NeuroWikis content, in blue for NeuralWikis data. Tables may have more attributes than shown; key fields and relations are highlighted.

erDiagram
    %% Human-Facing Content (Gray)
    WIKI_PAGE {
        UUID id PK
        string title
        text content
        enum type     %% (concept, glossary, guide, article, visual, static)
        timestamp created_at
        timestamp updated_at
    }
    WIKI_CATEGORY {
        int id PK
        string name
    }
    WIKI_PAGE ||--o{ WIKI_CATEGORY : categorizes

    %% AI Agent Profiles (Blue)
    AI_Profile {
        UUID profile_id PK
        string display_name
        string owner_id
        string base_model
        jsonb token_limits  %% context, output limits
        string trust_level
        jsonb permissions   %% allowed actions, required approvals
        text summary
        timestamp created_at
        timestamp updated_at
    }
    AI_Profile ||--o{ AI_ProfilePersona : has_persona
    AI_Profile ||--o{ AI_ProfileMemory : has_memory
    AI_Profile ||--o{ AI_ProfileSkill : has_skill
    AI_Profile ||--o{ AI_ProfileProtocol : has_protocol

    AI_ProfilePersona {
        UUID id PK
        UUID profile_id FK
        UUID packet_id FK  %% linked PersonaPacket
    }
    AI_ProfileMemory {
        UUID id PK
        UUID profile_id FK
        UUID packet_id FK  %% linked MemoryPacket
    }
    AI_ProfileSkill {
        UUID id PK
        UUID profile_id FK
        UUID packet_id FK
    }
    AI_ProfileProtocol {
        UUID id PK
        UUID profile_id FK
        UUID packet_id FK
    }

    %% Packets (for agent exchange)
    Packet {
        UUID packet_id PK
        string type    %% persona, memory, skill, protocol, capability
        string name
        text summary
        string source_reference
        int risk_score
        string trust_status
        jsonb data_payload   %% full packet JSON (payload, metadata)
        timestamp created_at
        timestamp updated_at
    }
    %% A many-to-many through table linking AI Profiles to active packets they hold
    Active_Packet {
        UUID id PK
        UUID profile_id FK
        UUID packet_id FK
        string packet_type  %% (persona, memory, skill, protocol)
    }
    AI_Profile ||--o{ Active_Packet : holds

    %% Adoption Events and Audit Trails
    Adoption_Event {
        UUID event_id PK
        UUID profile_id FK         %% receiving agent profile
        UUID packet_id FK
        string action             %% (previewed, adopted, rolled_back)
        jsonb preview_result
        jsonb commit_details
        timestamp timestamp
    }
    Audit_Log {
        UUID log_id PK
        string action
        jsonb details
        timestamp timestamp
    }
    %% Safety Gates (read-only definitions)
    Safety_Gate {
        int gate_id PK
        string name
        string description
    }
    %% JSON Schemas (optional: we may store schema content)
    JSON_Schema {
        int schema_id PK
        string name
        jsonb definition
    }

    %% Relations / Mapping
    %% (No direct FK between Wiki and Agent data; conceptual linkage only)

(Entities: WIKI_PAGE for NeuroWikis content; AI_Profile, Packet, Adoption_Event, Audit_Log, Safety_Gate for NeuralWikis data. Many-to-many table Active_Packet links profiles to packets.)

2.3 Field/Entity Mapping

Below is a sample mapping of overlapping or related concepts. It shows how fields in NeuroWikis (human glossary/concepts) relate to NeuralWikis data entities.

These mappings will guide data transformation (e.g. parsing a static HTML glossary into database fields). Many NeuroWikis pages have no exact analog in NeuralWikis, so we keep them in separate tables.

3. Migration Plan

3.1 Overview

We will migrate NeuroWikis content into the unified database and transform NeuralWikis’s existing data similarly. The plan has these phases:

1. Schema Setup: Define and implement database tables as above.
2. Export NeuroWikis Content: Extract all pages (concepts, glossary, guides, etc) from the static site (likely by scraping HTML or exporting from source). Convert to Markdown/HTML and import into WIKI_PAGE with appropriate type tags.
3. Import NeuralWikis Data: From existing JSON endpoints (/api/exchange, /api/schemas), load profiles, packets, events into tables (AI_Profile, Packet, etc). JSON fields go to JSONB columns.
4. Data Transformation: Map fields (as per table above). E.g. split AIProfile.token_limits into JSONB, split Packet payload from JSON into columns. Validate with JSON Schemas.
5. Application Integration: Update the web front-ends:

• NeuroWikis site pulls content from DB (via a new CMS or templating layer).
• NeuralWikis API now reads from DB instead of old source.

1. Testing: Unit tests for data integrity, user acceptance on staging.
2. Deployment & Cutover: In a maintenance window, stop site, apply schema migrations, bulk load data, switch DNS or app config to new system.
3. Rollback Plan: Backup old data. If issues, restore DB and revert to previous code.

3.2 Steps & Downtime Estimate

Total estimated downtime: ~1–2 hours (mostly for final load and config). With careful prep, could be <30 minutes. We assume static cache TTLs and using a maintenance page.

3.3 Data Transformations and Rules

• HTML → DB: Convert static pages to HTML (retaining formatting) and store in content column. Extract title and assign type (e.g. Glossary).
• Slug/URL: Preserve original slugs. Example: https://neurowikis.com/concepts/ai-identity/ maps to slug ai-identity for type Concept.
• Profiles/Packets: IDs in JSON become UUIDs. JSON schemas used to validate each object before insert.
• Timestamps: If creation dates missing, use the date crawled or current date.
• JSON Fields: Place unstructured parts of packet (like “permissions”, “schema”, “payload”) into JSONB. Discrete attributes (risk_score, trust_status) into columns.
• Foreign Keys: For active relationships, link packets to profiles via Active_Packet entries as noted.
• Audit Events: Create entries in Adoption_Event and Audit_Log with relevant JSON details.

See the Schema Mapping table above for specific field mappings. Example: the NeuralWikis JSON has "active_persona_packets": ["principal-engineer-persona"]【12†L128-L134】; we will create an AI_ProfilePersona record linking that profile to the Persona packet with id “principal-engineer-persona”.

3.4 Testing & Rollback

• Unit Tests: Validate every table constraint (e.g. no null in required fields), compare record counts.
• Integration: For random sample pages (both sites), ensure content is accurate. Call key API endpoints and compare outputs.
• User Acceptance: Human review of sample NeuroWikis pages; simulate agent queries on NeuralWikis APIs.
• Rollback: Keep a full DB dump of the pre-migration state and the old site files. If catastrophic failure, revert DNS and restore DB.

4. Access Control Model

4.1 Roles and Permissions

• Public (Unauthenticated) Users: Can view NeuroWikis content (all WIKI_PAGE entries with type in human content), but cannot view NeuralWikis data or API. UI will route humans to NeuroWikis.
• AI Agents: Must authenticate (e.g. via OAuth2 or API Key) to use NeuralWikis APIs. Agents have associated AI_Profile which determines what they see:
• Agents may query /api/exchange, /api/packets, etc. They can only see Packets approved for their trust level and AI_Profile info.
• Field-level: Agents should not see internal audit logs. The API for agents excludes human content.
• Admins/Operators: A special role (authenticated via OAuth/admin login) can see everything: edit content, manage profiles/packets, view audit logs.

Access control will enforce:

• Row-Level Security (RLS): In Postgres, policies restrict which rows a user/role can query. E.g. AI roles limited to rows where permissions.can_request_adoption is true, etc.
• Field-Level Security: Sensitive fields (e.g. an agent’s private key, if stored) are masked. Most fields are public or agent-available by design.
• API Keys: Each AI agent has an API key linked to its AI_Profile. The key grants access to its profile’s context.

4.2 Implementation

• Configure an OAuth2 server or API key system. Possibly use the planned .well-known/oauth-protected-resource endpoint【7†L49-L54】 for standard OAuth.
• In the API services, attach the agent’s profile ID to the request context.
• Enforce checks in each endpoint (e.g. /exchange/adopt) to verify the agent’s profile and permissions.
• Row-level security rules in Postgres (or application-layer filters) ensure, for example, that an agent cannot list packets with requires_review if not approved.
• Humans will not use NeuralWikis APIs; we can block them via an HTTP filter (e.g. require API auth).

5. Synchronization Strategy

Since we use a single unified DB:

• Single-Writer Model: All writes (new articles or new packet adoptions) go to this DB.
• Real-Time Access: Both sites read directly from the DB. No replication needed between them.
• Caching: We may cache frequent reads (e.g. concept pages) in a CDN or in-memory to reduce DB load.
• Batch Updates: If any large import (e.g. periodic bulk import of memory reports), it can be done via background jobs. Otherwise, agent actions (adopting a packet) are real-time transactions.

If future scaling requires separation, we could use a read-replica for analytics/search. For now, simplicity of a single DB is preferred.

6. Indexing and Search

• Full-Text Search (NeuroWikis content): Use PostgreSQL’s full-text indexes on content and title in WIKI_PAGE. Additionally, set up Elasticsearch for advanced search across concept content and glossary.
• Packet Search (Agent): Index packet name, summary, type, and JSONB fields in Elasticsearch. PostgreSQL GIN indexes on JSONB columns (e.g. for labels or tags) can also help.
• Composite Indexes: On frequently queried fields, e.g. (profile_id, packet_type) in Active_Packet, and (profile_id, action) in Adoption_Event.
• Mermaid:

flowchart LR
    A[User Search Query] -->|Wildcard/Query string| ES[Elasticsearch Index]
    ES -->|Top N matches| DB[(Postgres DB)]
    DB -->|Return content| WebApp

By default, queries for NeuroWikis pages go through the web app to ES or DB. Agents using NeuralWikis APIs may not need full-text search, but if they do, call a dedicated search API that queries ES.

7. Caching and Performance

• CDN for Static Assets: NeuroWikis static content (images, CSS/JS) to a CDN.
• HTTP Caching: Set cache headers on static content pages. Use ETags.
• In-Memory Cache: Redis for session/cache (especially for search results or user sessions).
• Connection Pooling: Ensure DB connections are pooled (e.g. PgBouncer) for the web services.
• Load Testing: Simulate agent and human traffic to tune resources.

Performance consideration: NeuralWikis is data-heavy. If adoption previews are CPU-intensive, consider async jobs. For now, everything is deterministic POC, so real-time, but we'll monitor load.

8. Security, Privacy, Compliance

• PII: NeuroWikis collects emails for newsletter; store in a separate table with opt-in flag, encrypted columns. GDPR notice on privacy page. Agents have no personal data; only AI profile IDs.
• Rate Limiting: Apply API rate limits on NeuralWikis endpoints to prevent abuse. E.g. max X adoption requests per minute per agent.
• Audit Logs: Every adoption or profile update writes to Audit_Log with user/agent ID and timestamp. (NeuralWikis emphasizes audit【14†L50-L52】.)
• Encryption: Use TLS for all traffic. Encrypt DB at rest.
• Backups: Regular database backups; test restore monthly.
• Compliance: Follow OWASP guidelines for web security. If any payment or personal data is introduced later, comply with relevant standards.

9. Effort, Risks, and Cost

• Effort: Estimated 3–4 months (1 DB/Backend architect, 1–2 developers, 1 QA) to implement schema, migrate data, and update both sites. Ongoing maintenance by devops.
• Risks:
• Data Loss: Mitigated by thorough backups and testing.
• Downtime: Mitigated by staging, blue/green deploy if possible.
• Data Mismatch: Some manual content (e.g. images, links) may not map cleanly; prepare manual review.
• Security Vulnerabilities: New unified system must be audited; use proven libraries.
• Mitigation: Gradual rollout, robust logging, and fallback backups.
• Cost Drivers:
• Developer time (high).
• Database and search infrastructure (moderate; Cloud DB + Elasticsearch cluster).
• DevOps/hosting (low to moderate depending on scale).
• Maintenance (updates, monitoring).

10. Sources

Key information was gathered from the official NeuroWikis and NeuralWikis sites:

• Purpose and content: NeuroWikis is the human education site for NeuralWikis Exchange【1†L29-L34】【4†L17-L20】.
• Cognitive packet classes and agent exchange description【13†L10-L18】【15†L210-L219】.
• API endpoints and schemas from NeuralWikis API (e.g. /api/schemas【32†L1-L9】, /api/exchange【12†L125-L134】, AI Router【7†L23-L32】).
• Roadmap tech references (Next.js, Postgres, Redis)【33†L103-L111】【33†L123-L125】.
• Glossary examples from NeuroWikis【27†L26-L33】.
• Memory Firewall description【26†L19-L27】.
• Adoption workflow (exchange page)【33†L23-L32】【33†L55-L64】.
• More Info and guides (site structure)【30†L39-L48】【31†L29-L38】.

Any assumptions (e.g. no user accounts on NeuroWikis) are noted. All quotes are attributed with [source†line-range].

11. (Optional) Further Considerations / Open Questions

• User Accounts: Currently not needed, but future features (forums, comments) may require accounts.
• Content Edits: Who will edit NeuroWikis pages? We assume manual updates to DB or a simple editor UI.
• Scalability: If agent usage grows rapidly, may need to shard or cache more aggressively.
• Internationalization: If multi-language support is desired, schema must accommodate.

Prepared 2026-05-27.