Deep Architecture for NeuralWikis: Multi-Agent Long-Term Memory and MCP Integration

Full report

On this page

The rapid evolution of agentic systems has exposed a critical limitation in contemporary artificial intelligence: the reliance on stateless, ephemeral context windows that discard organizational knowledge and relational reasoning between discrete sessions. Traditional approaches to providing external context—such as flat vector stores relying solely on top-k semantic retrieval or purely in-process working memory—fail to maintain the temporal, relational, and semantic coherence required for complex, multi-agent enterprise planning. Memory without cognition functions merely as a filing cabinet, whereas true agentic planning requires hypothetical reasoning, the ability to propagate consequences through rules, and a persistent repository of structured knowledge. To resolve these architectural deficiencies, the Model Context Protocol (MCP) has emerged as a universal, transport-agnostic standard, enabling dynamic, two-way connections between foundation models and external stateful systems. Often described conceptually as a universal serial bus for artificial intelligence, MCP allows disparate AI applications—referred to as hosts—to interact with external data sources through standardized client-server capability exchanges. This report details a comprehensive architecture for NeuralWikis, establishing a persistent, highly structured long-term memory system accessible via MCP across diverse clients, including Anthropic's Claude Desktop, Cursor, OpenAI's Codex and Apps SDK, Google's Gemini CLI, and future automated agents. By implementing a hybrid temporal-semantic-relational database combined with strict memory scoping, this architecture ensures that organization-wide knowledge is universally searchable by authorized agents, while project-specific data remains strictly isolated to prevent cross-project contextual bleed. Furthermore, this document defines a seamless user experience for authentication utilizing Enterprise-Managed Authorization, a robust ingestion pipeline with readback verification, deterministic failover mechanisms for remote disconnections, and a definitive migration strategy for legacy Unified Agent Interface (.uai) files.

The Temporal-Semantic-Relational Memory Architecture

Enterprise context layers demand a synthesis of structural causal models, relational graphs, and semantic vectors. If data is stored flatly, an agent is forced to rely on broad, inefficient vector searches that frequently lead to hallucination or contextual decoherence across time and meaning. By modeling the domain as a graph composed of temporal-semantic surfaces, the architecture pre-compiles reasoning paths, significantly reducing the risk of retrieval errors. The NeuralWikis architecture avoids the fragmentation of utilizing separate vector and metadata stores—such as synchronizing a PostgreSQL database with a standalone Pinecone index or an S3 bucket—which introduces eventual consistency concerns and complex operational overhead. Instead, the architecture utilizes a unified PostgreSQL environment augmented with the pgvector extension. This singular append-only schema allows vectors to reside directly alongside their relational data, preserving strict transactional guarantees, JOIN semantics, point-in-time recovery, and native SQL query capabilities. Within this unified datastore, the memory is strictly partitioned into distinct namespaces or scopes. To support both generalized organizational reasoning and highly focused, project-specific task execution, NeuralWikis enforces six orthogonal memory scopes. These scopes ensure that an agent processing sensitive code in one repository cannot inadvertently hallucinate or retrieve variables, architectural decisions, or credentials from an unrelated repository. The system enforces the following distinct memory scopes, each with unique retention and access characteristics:

To enforce clean project-scoped memory, PostgreSQL Row-Level Security (RLS) is applied directly to the core memory tables. When an agent queries the database, the authentication middleware sets a local configuration parameter containing the agent's authorized project\_id values and global clearance levels. The RLS policy automatically filters out out-of-scope rows before the vector index is even traversed, ensuring deterministic isolation and eliminating the possibility of cross-project data bleed at the database engine level.

Database Schema and Indexing Strategy

To satisfy the requirements of high-dimensional vector search and strict relational boundaries, the schema objects must accommodate entities, facts, and preferences that persist across sessions, maintaining POLE+O (Person, Object, Location, Event, Organization) types alongside raw embeddings. The primary schema is designed around two core tables: one for storing the memory entities and their semantic embeddings, and another for defining the causal or relational links between those entities.

Recommended Schema Objects

The following table defines the core schema required to enforce memory isolation and enable multi-dimensional retrieval within the PostgreSQL and pgvector environment:

HNSW and Hybrid Indexing Implementation

To support the rapid retrieval of millions of vectors, the embedding column utilizes a Hierarchical Navigable Small World (HNSW) index. Unlike exact nearest neighbor searches or older inverted file (IVFFlat) indexes, HNSW builds a multilayer navigable graph that provides the optimal speed-to-recall tradeoff and can be constructed on an empty table without a preliminary training step. The index is specifically configured for cosine distance operations (vector\_cosine\_ops), as cosine similarity is the standard metric for modern embedding models. The creation of this index requires specific parameter tuning to balance recall accuracy against index build times and memory consumption. The m parameter, representing the maximum number of connections per node, is set to 16, while the ef\_construction parameter, representing the candidate list size during the build phase, is set to 64\. These values ensure high recall without exhausting the maintenance\_work\_mem on the PostgreSQL server during high-volume ingestion events. Furthermore, the architecture does not rely solely on semantic similarity. Modern AI applications frequently require hybrid search capabilities, combining the fuzzy contextual matching of vectors with the exact keyword precision of traditional queries. To achieve this, the system leverages PostgreSQL's native full-text search capabilities. A Generalized Inverted Index (GIN) is applied to a generated tsvector column derived from the content field. During a retrieval operation, the agent's query executes both a plainto\_tsquery text search and a vector similarity search. The results from both paradigms are then dynamically merged and re-ranked using Reciprocal Rank Fusion (RRF), ensuring that an agent retrieving a specific function name or API key receives an exact match, while broader conceptual queries retrieve semantically aligned context.

Ingestion, Indexing, and Readback Verification

When a new report, architectural decision record, or conversational transcript is generated by an agent or human operator, it cannot simply be inserted blindly into the vector store. Unverified data ingestion rapidly degrades the coherence of the long-term memory, polluting the context window with hallucinations or obsolete facts. Therefore, NeuralWikis employs a rigorous ingestion pipeline that indexes the content, establishes temporal and semantic linkages, and verifies readback accuracy through an automated human-in-the-loop (HITL) fallback mechanism.

The Ingestion Pipeline

The process begins when an agent invokes the memory storage tool over the MCP connection. The payload contains the raw text, the target scope, the project ID, and the source URI. The NeuralWikis backend intercepts this payload and initiates the chunking and metadata extraction phase. An isolated, backend LLM processor analyzes the document to extract named entities and POLE+O types, ensuring that the unstructured text is annotated with structured metadata. Once chunked, the system generates embeddings for each segment. Concurrently, the system performs a preliminary similarity search against the existing database to identify potential relationships. If a newly ingested code snippet references a library previously documented in the global\_public scope, the system automatically generates a relational edge in the nw\_memory\_relations table, linking the new project-specific entity to the global entity. This pre-compiles the reasoning path, allowing future agents to traverse from the project code directly to the global documentation without requiring multiple disparate searches.

Agentic RAG Readback Verification

The most critical phase of the ingestion pipeline is the cryptographic and semantic readback verification. Before any ingestion transaction is permanently committed and presented to production agents, its retrievability must be proven. The system triggers an automated Agentic RAG check. A separate, isolated verification agent is instantiated and provided with a synthetic query generated from the original ground truth document. This agent attempts to retrieve the newly ingested knowledge using the standard MCP search tool. The retrieved context is then compared against the original chunk. If the retrieved answer demonstrates a high semantic similarity to the ground truth, the confidence\_score of the memory entity is elevated to a value greater than 0.95, and the freshness\_timestamp is locked. However, if the readback fails, retrieves contradictory data from a superseding document, or mistakenly pulls context from an adjacent project due to an RLS misconfiguration, the ingestion is halted. The system drops the confidence score and flags the entity in a review queue. This queue serves as a complex interrupt point for human-in-the-loop feedback, where an administrator can manually correct the classification or adjust the agent's proposed tool calls before the long-term memory is permanently updated.

Enterprise-Managed Authorization and Access Control Rules

Deploying remote MCP servers over the internet introduces significant security considerations. Traditional local MCP setups rely on standard input/output (stdio) transports, where the client and server implicitly trust each other because they operate as child processes on the same authenticated user's machine. However, NeuralWikis requires remote access via the Streamable HTTP transport to centralize organizational memory. In this paradigm, every MCP client has slightly different expectations for how to connect, and requiring employees to manually authorize every MCP server individually creates immense onboarding friction and bypasses centralized IT security controls. To resolve this, NeuralWikis implements the Model Context Protocol's Enterprise-Managed Authorization (EMA) extension. This extension shifts authorization decisions from individual, fragmented applications to the organization's central Identity Provider (IdP), enabling a zero-touch setup where users are connected to authorized servers automatically upon their first login, without encountering per-app OAuth consent screens.

The Identity Assertion JWT Authorization Grant (ID-JAG)

The EMA flow leverages an Identity Assertion JWT Authorization Grant (ID-JAG) to broker cross-domain API access. This process, informally referred to as Cross-App Access (XAA), utilizes pre-existing Single Sign-On (SSO) trust relationships to secure AI agent connections. When a user logs into their AI host (e.g., Cursor or Claude Desktop) using their corporate identity, the host obtains an OpenID Connect ID Token. Rather than redirecting the user to the NeuralWikis authorization server, the client application automatically executes an RFC8693 Token Exchange in the background. It presents the ID Token to the enterprise IdP and requests an ID-JAG. The IdP evaluates administrator-defined policies—such as group membership, role assignments, and conditional access rules—to determine if the user is authorized to access NeuralWikis. Crucially, the IdP acts only as an authorization broker; it does not gain visibility into or control over the actual live MCP traffic or memory payloads exchanged between the client and the NeuralWikis server. If approved, the IdP issues the signed ID-JAG JWT. The payload of this token contains specific claims: the iss (issuer, the IdP), the sub (subject, the human user), the aud (audience, the NeuralWikis Authorization Server), and the resource (the specific MCP server identifier). The client then directly exchanges this ID-JAG with the NeuralWikis Authorization Server for an operational Access Token. Every subsequent HTTP POST and GET request on the Streamable HTTP connection includes this Access Token in the Authorization: Bearer header, ensuring continuous, stateless authentication.

Access Control Rules Matrix

The NeuralWikis Authorization Server maps the claims within the ID-JAG to specific database permissions. The following access control rules define how roles map to the distinct memory scopes:

The Single-Key MCP Setup User Experience

Despite the sophistication of the EMA protocol, the physical configuration of MCP clients remains a fragmented ecosystem. Cursor utilizes workspace-scoped configurations located at .cursor/mcp.json or global configurations at \~/.cursor/mcp.json. Claude Desktop relies on claude\_desktop\_config.json located in operating system-specific application data folders. Google Gemini requires CLI commands such as gemini mcp add, while OpenAI's Apps SDK requires programmatic integration via the MCPServerStreamableHttp or HostedMCPTool classes. Managing these disparate JSON configurations manually is a significant failure point for enterprise adoption. To fulfill the requirement of a frictionless user experience, NeuralWikis abstracts this complexity behind a unified bootstrap mechanism. The human operator is required to perform only one action: either pasting a single API key into a NeuralWikis setup portal or dropping a single ready-made configuration file into the root of their project directory. They are never required to do both.

The Bootstrap Mechanism

If the user opts for the configuration file, they place a neuralwikis.config file in their workspace. A lightweight, globally installed executable (e.g., an npx package or pre-compiled binary) is then invoked. This bootstrap script autonomously scans the host machine to detect installed AI clients. Upon detecting Cursor, Claude Desktop, or Windsurf, the script parses their respective JSON configuration files and injects a standardized mcpServers block. Because remote MCP servers require the Streamable HTTP transport, the injected configuration leverages a specialized proxy wrapper (such as mcp-remote) to translate the AI client's native stdio expectations into properly authenticated HTTP requests. The injected JSON structure is standardized across all local clients:

JSON { "mcpServers": { "neuralwikis-memory": { "command": "npx", "args":, "env": { "NW\_AUTH\_TOKEN": "eyJhbGciOiJIUzI1NiIs..." } } } }

This entirely eliminates the need for developers to manually configure transport layers, header injection, or environment variable parsing. For web-based clients or enterprise agent orchestrators (such as OpenAI's Responses API), the platform utilizes the client credentials flow where the server URL and OAuth tokens are configured once at the organizational level, completely removing the end-user from the setup loop.

Agent Discovery of Context, Schemas, and Tools

Once the transport layer is established and authenticated, the agent must understand what capabilities the NeuralWikis server provides. The Model Context Protocol strictly defines this discovery lifecycle, ensuring that the LLM is not relying on hardcoded API guesses, but rather on a self-describing, dynamic interface. The interaction begins with the agent dispatching an initialize request over the Streamable HTTP POST endpoint. The NeuralWikis server validates the session and responds with its supported capabilities, explicitly declaring support for tools, resources, and prompts. Furthermore, the server indicates whether it supports the listChanged notification capability, which is vital for real-time memory synchronization.

Dynamic Tool and Resource Listing

Following initialization, the agent dispatches a tools/list request. The NeuralWikis server intercepts this request and analyzes the agent's ID-JAG token. The response is dynamically generated based on the RLS policies and scope authorizations associated with that token. If an agent is not authorized for a specific project, the tools required to interact with that project are entirely omitted from the response payload. The tools/list response includes robust JSON Schema representations for every available tool. The AI host injects these schemas directly into the LLM's system prompt. This ensures the model understands exactly what parameters are required to execute a hybrid search, ingest a memory, or query a specific relational graph edge. Similarly, the agent dispatches a resources/list request. Resources in MCP provide readable content that the agent can retrieve to understand the state of the world before taking action. NeuralWikis exposes database definitions, project guidelines, and global schemas as resources. By reading a resource mapped to nw://projects/{project\_id}/schema, the agent instantly grounds itself in the specific architectural conventions of the repository it is analyzing.

Prompts and Server-Sent Events

The architecture also leverages the MCP prompts capability. Prompts allow the server to provide structured messages and reusable playbooks for interacting with the language model. NeuralWikis exposes prompts designed to trigger specific automation workflows, such as a "Project Handoff Summary" prompt that instructs the agent to query the memory store, summarize recent architectural decisions, and format them for human review. Because enterprise memory is highly volatile, the architecture relies heavily on the Server-Sent Events (SSE) channel inherent to the Streamable HTTP transport. If another agent updates a global schema or alters a critical project parameter, the NeuralWikis server emits a tools/list\_changed or resources/list\_changed notification over the SSE stream. The connected agent receives this event asynchronously and is prompted to refresh its context, ensuring that all distributed agents operate on the most current temporal-semantic surface.

MCP Tool and Resource Contracts

To interface with the underlying hybrid PostgreSQL database, the NeuralWikis MCP server exposes a highly specific suite of tools. These contracts are provided to the agent formatted according to the JSON Schema Draft-07 specification, ensuring maximum compatibility across Anthropic, OpenAI, and Google models.

Primary Tool Contracts

The following table details the essential input schemas for the primary memory interaction tools:

Essential Resource Contracts

Resources are exposed as specific URIs that the agent can read.

Migration Plan: Local.uai to Remote Memory

Prior to the deployment of the centralized NeuralWikis architecture, contextual state and agent memories were frequently preserved in legacy local files using the Unified Agent Interface (UAI) standard. Specifically, .uai bundles were utilized for UAI-1 project handoff, user preferences, release notes, and operational context. These files represent a fragmented, decentralized memory store that must be migrated to the unified remote PostgreSQL database without data loss. The migration strategy is highly deterministic, ensuring that the legacy files are only archived after their structural integrity and retrievability within the remote system are cryptographically proven.

Phase 1: Parsing and Extraction

A dedicated MCP tool, nw\_migrate\_uai, is executed by the agent or a background daemon operating on the local host. This tool parses the local .uai JSON bundle schema, extracting key-value preferences, session transcripts, relational graph notes, and historical handoff summaries.

Phase 2: Remote Upsert and Indexing

The extracted data is transmitted to the remote NeuralWikis MCP server. The server runs the data through the standard ingestion pipeline: chunking the text, computing vector embeddings via the designated model, tagging the entities with the appropriate project\_id, and inserting the rows into the nw\_memory\_entities table. The scope is strictly set to project\_private by default to ensure no legacy localized data is inadvertently exposed globally.

Phase 3: Cryptographic and Semantic Verification

Before the local file can be safely archived, the system must prove that the remote database accurately represents the ingested knowledge and that the vector search can successfully retrieve it. The local migration agent generates a deterministic hash (e.g., SHA-256) of the original .uai payload. It then queries the remote MCP server via the nw\_search\_memory tool, using specific synthetic prompts designed to retrieve the newly uploaded context. If the remote readback successfully returns the exact parameters—verifying both the read/write pipeline and the vector search integrity—and the semantic similarity score exceeds 0.95, the agent marks the migration as verified.

Phase 4: Archival and Redirection

Upon successful verification, the original .uai file is not immediately deleted to prevent catastrophic data loss in the event of an edge-case rollback. Instead, it is moved to an offline .archive directory on the local filesystem and compressed. The local agent configuration is then permanently updated to point all future memory read/write operations strictly to the remote NeuralWikis MCP endpoint, concluding the migration.

Emergency Reconnect Behavior and Local Archives

Because the NeuralWikis architecture relies on remote MCP servers accessed over the internet via the Streamable HTTP transport, it inherently introduces a dependency on persistent network connectivity. Relying solely on a remote connection introduces unacceptable fragility into an agent's reasoning loop, particularly for agents operating in unstable network environments or during localized server outages. To mitigate this, the architecture defines a robust degradation and fallback protocol utilizing the emergency\_reconnect\_archive scope.

State Degradation and Local Buffering

If the Streamable HTTP connection drops, the client agent enters a degraded state. The MCP SDK intercepts the network timeout or an HTTP 503 Service Unavailable response and immediately triggers the emergency fallback protocol. The agent dynamically redirects all memory queries to the local emergency\_reconnect\_archive scope. This scope maps to a local SQLite database or a series of file-backed .uai caches on the host machine. This local cache maintains a lightweight, recently accessed subset of the project memory, allowing the agent to continue reasoning over immediate context. While operating offline, any new context generated by the agent—such as code written, architectural decisions made, or conversational summaries—is appended to the local agent\_session file with a highly precise timestamp. The host application explicitly injects a system prompt into the LLM, indicating that it is operating in a degraded, offline state and that global organizational searches are currently unavailable.

Reconciliation and Session Resume

Once the network connection is re-established, the agent initiates an automated reconciliation job. The Streamable HTTP protocol natively supports session management via the Mcp-Session-Id header. The client issues an HTTP GET to the MCP endpoint, supplying the previous session ID. If the server has maintained the session state in memory, the SSE connection is instantly restored without requiring a full re-initialization. Following the connection restore, the client asynchronously uploads the buffered local writes from the emergency\_reconnect\_archive via the nw\_store\_memory tool. The remote server resolves temporal conflicts by prioritizing the most recent freshness\_timestamp. Once the server acknowledges the successful ingestion of the buffered data, the local archive is cleared, and the agent resumes standard operations.

Failure Modes and Repair Paths

The intersection of stochastic language models, high-dimensional vector databases, and real-time streaming protocols introduces unique failure modes. The architecture anticipates these edge cases and defines deterministic repair paths to maintain systemic coherence. The following table details the primary failure modes and their automated resolution strategies:

End-to-End Test Matrix

To guarantee consistent behavior across diverse host environments and foundational models, the architecture demands an exhaustive testing matrix. Each scenario must successfully pass across all officially supported AI clients prior to production deployment.

Through the rigorous enforcement of distinct memory scopes, the deployment of Enterprise-Managed Authorization for zero-touch onboarding, and the implementation of a fault-tolerant, hybrid vector-graph database, this architecture establishes a foundational long-term memory system capable of supporting the next generation of autonomous enterprise workflows across any MCP-compliant platform.