Runtime
Shared Brain Memory Systems Architecture for Enterprise Python Applications
Report summary
A “shared brain” for an enterprise Python application is best understood as a governed, multi-tier memory architecture that lets application components, background workers, and optionally LLM/agent subsystems read from and write to common context across requests, sessions, and services. The phrase i
Key topics
- Runtime
- AI
- AI Memory
- Agentic Web
- SQL
- Python
- Privacy
- Semantic Systems
Research provenance
For citation, use the report title and canonical URL. Archival presence does not establish authorship or promote report statements into portfolio evidence.
Source availability: 80 citation markers in the source export have no recoverable source links. Those markers are omitted from this reader; any supplied bibliography and ordinary links remain. Check the original sources before relying on the cited claims.
This page renders the archived Markdown as safe, formatted HTML. It is background research and does not become a portfolio claim without evidence review.
Full report
On this page
Executive summary
A “shared brain” for an enterprise Python application is best understood as a governed, multi-tier memory architecture that lets application components, background workers, and optionally LLM/agent subsystems read from and write to common context across requests, sessions, and services. The phrase itself is not a settled academic term; the literature instead speaks in terms of modular memory components, agent memory, external non-parametric memory, and long-term memory. In practice, the closest technical meaning is a shared memory layer that combines short-term state, durable factual memory, event-like episodic memory, and retrieval mechanisms under explicit application control.
For an enterprise Python stack, the strongest default architecture is tiered and hybrid: request- or thread-scoped working memory in application state or checkpoints; a transactional system of record in PostgreSQL for identities, ACLs, provenance, and memory metadata; Redis for low-latency caches and coordination; a vector index for semantic retrieval; and object storage for raw artifacts and cold history. This split reflects the different guarantees these systems provide: Python task-level synchronization is local to a process, multiprocessing.shared_memory is explicitly not distributed shared memory, PostgreSQL provides ACID transactions and multiple isolation levels, Redis optimizes low-latency access, and vector databases expose ANN tradeoffs between recall, latency, and consistency.
When LLMs are involved, the most useful design principle is to treat memory as a hierarchy rather than a single store. RAG formalized the distinction between parametric memory in model weights and non-parametric memory in an external index. Later work such as Generative Agents, MemGPT, MemoryBank, CoALA, and recent surveys extends that idea into explicit short-term, long-term, episodic, and semantic layers, plus reflection or consolidation loops. For enterprise systems, that means the “brain” should not merely store transcripts; it should retain facts, event histories, provenance, salience, retention policies, and embeddings, and it should support consolidation from hot-path interactions into durable, queryable memory.
The biggest architectural mistakes are usually mundane rather than exotic: sharing mutable SQLAlchemy sessions across concurrent tasks, assuming task-local locks protect multi-process state, using insecure serialization such as pickle across trust boundaries, treating vector-store acknowledgment as immediate read visibility when the system is eventually consistent, or coupling every memory write directly to user-facing latency. Official docs are explicit that asyncio locks are not thread-safe, SQLAlchemy sessions are not safe to share across concurrent tasks and are not a second-level cache, pickle is insecure for untrusted data, and some vector stores expose tunable or eventual visibility semantics.
If constraints are not specified, a sensible enterprise default is a moderate-scale, cloud-hosted, multi-tenant service with a budget that favors open-source infrastructure or managed services with low operational overhead; target p95 interactive API latency under a few hundred milliseconds excluding model inference; total LLM-assisted response latency in the low-single-digit seconds; one primary region; auditability and tenant isolation; and regulatory readiness for at least GDPR-style retention and deletion requirements. Under those assumptions, PostgreSQL plus pgvector is the most attractive default for teams that want strong transactions, joins, filters, and fewer moving parts, while a dedicated vector database becomes attractive once filtered ANN retrieval, replica scaling, or large-scale ingestion/search dominates.
Assumptions and decision frame
The report assumes the following because the prompt left them unspecified:
| Assumption | Working value |
|---|---|
| Deployment model | Cloud-hosted enterprise web application, Python-first backend |
| Multi-tenancy | Yes, logical tenant isolation required |
| Traffic | Moderate to high, with interactive APIs and background processing |
| LLM usage | Optional but important; memory should work with and without LLMs |
| Data types | Primarily text plus metadata, some documents and artifacts |
| Regions | Single primary region, disaster recovery design encouraged |
| Compliance posture | Security-conscious, audit-ready, GDPR-like retention/deletion expectations |
| Cost posture | Prefer open-source defaults or managed services with strong operational justification |
Under those assumptions, the design should optimize for five goals: reliable low-latency reads for hot memory, durable and auditable writes for important memory, isolation by tenant/user/namespace, asynchronous consolidation to keep the hot path fast, and clear demarcation between authoritative state and derived state such as embeddings, caches, summaries, and precomputed retrieval structures. This is a synthesis of storage, concurrency, and LLM-memory guidance rather than a requirement stated verbatim in any one source.
Definitions and taxonomy
The most useful way to define a shared brain in enterprise software is: a shared, governed memory substrate that stores and retrieves application-relevant context across scopes. “Shared” means more than one component can access it. “Brain” implies more than cache; it implies memory types, retrieval policy, provenance, consolidation, and actionability. In the language-agent literature, the nearest formal counterparts are CoALA’s modular memory components and recent surveys that organize AI memory by object, form, and time. In LangGraph’s terminology, short-term thread state is handled by checkpointers, while long-term cross-thread data is handled by stores.
Human-memory taxonomy remains useful because it maps surprisingly well to software architecture. UCSF’s clinical overview distinguishes episodic memory as event- and experience-oriented, semantic memory as factual knowledge, and working memory as the short-term holding and manipulation of current information. A recent survey of AI memory similarly maps short-term versus long-term memory and explicit versus implicit memory into LLM systems, and explicitly relates non-parametric long-term memory to episodic recall and model-parameter knowledge to semantic memory.
A practical enterprise taxonomy is therefore:
| Memory type | Enterprise meaning | Typical Python realization | Primary storage choice | Sources |
|---|---|---|---|---|
| Working memory | Request/session/thread-local context used to complete the current task | FastAPI request state, LangGraph checkpoints, in-process objects, bounded prompt context | In-process state, checkpoint store | |
| Episodic memory | Timestamped experiences with provenance, actor, and context | Event objects, message transcripts, tool-call logs, workflow history | PostgreSQL + object store + optional vector index | |
| Semantic memory | Durable facts, preferences, entities, normalized knowledge | Fact tables, key-value records, summarized profiles, canonical entities | PostgreSQL, Redis cache, vector index for semantic lookup | |
| Long-term memory | Memory retained across sessions and conversations | Namespaced stores, durable DB rows, archive artifacts, embeddings | PostgreSQL + pgvector or dedicated vector DB + object store | |
| Procedural memory | Learned rules or policies for how the system behaves | Prompt/tool policies, workflow code, model weights, routing rules | Code repo, config store, model artifacts |
Two distinctions matter a great deal in enterprise design. First, parametric vs non-parametric memory: RAG’s original formulation explicitly combines parametric model memory with non-parametric external memory. Second, thread-scoped vs cross-thread memory: LangGraph’s persistence docs make this distinction operationally explicit through checkpointers versus stores. A shared brain normally relies much more on non-parametric and cross-thread memory than on parametric memory, because enterprises need provenance, updateability, and deletion.
A good rule of thumb is this: if the data must be correct, attributable, erasable, and access-controlled, it should live in non-parametric durable storage even if the model also “knows” something similar. Parametric memory helps fluency and generalization, but external memory provides provenance and updatability, which RAG explicitly identifies as open problems for model-only knowledge.
Architecture patterns
The baseline enterprise architecture should separate the hot path from consolidation. The hot path receives a request, gathers working memory and a small amount of relevant long-term memory, performs business logic and optional retrieval, and responds. Consolidation happens asynchronously: transcripts, tool calls, search traces, and validated facts are normalized, deduplicated, embedded, summarized, and stored into their durable tiers. That split is directly aligned with LangGraph’s distinction between short-term and long-term memory and with its discussion of hot-path versus background updates, and it also matches the reflection patterns from Generative Agents and the hierarchical memory strategy in MemGPT.
flowchart LR
Client --> API[FastAPI / service layer]
API --> WM[Working memory\nrequest state / checkpoints]
API --> Cache[Redis cache / coordination]
API --> RDB[(PostgreSQL\nmetadata + facts + ACLs)]
API --> VDB[(pgvector / Qdrant / Milvus)]
API --> LLM[LLM / retrieval / tool orchestration]
API --> Bus[Kafka / RabbitMQ / NATS]
Bus --> Workers[Consolidation workers]
Workers --> RDB
Workers --> VDB
Workers --> Obj[(Object store\nraw artifacts / snapshots)]
LLM --> API
The diagram above is a synthesis of Python concurrency boundaries, LangGraph persistence, transactional data stores, ANN stores, and message-bus durability models. It is deliberately hybrid because no single substrate simultaneously provides request-local speed, ACID updates, semantic retrieval, archival durability, and replayable asynchronous processing.
Single-process deployments
In a single process, the shared brain is easiest to reason about: asyncio provides task concurrency, asyncio.Lock / Event / Condition protect local shared state, and in-memory caches or checkpoint savers can be very effective for short-term memory. This is the right place for prototypes, internal tools, single-instance workers, and edge services. However, Python’s docs are explicit that asyncio synchronization primitives are not thread-safe, so they only protect access within the event loop, not across threads or processes.
Within a single process, a common pattern is to keep working memory in request or graph state and use PostgreSQL/Redis for durable memory. LangGraph’s InMemorySaver is appropriate for local short-term memory, while production guidance points to database-backed checkpointers such as PostgresSaver. FastAPI’s dependency injection and background-task mechanisms fit naturally here, but anything important enough to survive process restarts should leave process memory quickly.
Multi-process deployments
The moment a Python web app runs multiple workers—such as multiple Uvicorn/Gunicorn processes—the architecture changes. In-process dicts and asyncio locks stop being a source of truth because each worker has its own memory and event loop. Python’s multiprocessing.shared_memory can share volatile memory across local processes and can reduce serialization/copy overhead, but the docs are explicit that this is for local shared memory and not distributed shared memory. It is most appropriate for local numeric arrays, precomputed embeddings, or large immutable-ish buffers on one host—not for authoritative business memory.
A subtle but important operational detail is lifecycle management. Shared memory blocks can outlive the creating process; Python documents the need to close() and eventually unlink() them for cleanup, and warns that resource tracking behavior differs depending on how processes are created. That makes local shared memory a performance optimization, not the center of a robust enterprise shared brain. For authoritative state in a multi-process deployment, use PostgreSQL, Redis, or a durable external memory store.
Distributed and microservices deployments
In distributed systems, the shared brain becomes a memory fabric rather than a memory object. Services publish and consume memory events, write to specialized stores, and reconstruct context on demand. Kafka is strongest when replay, log compaction, partition ordering, idempotent producers, and transactional streams matter. RabbitMQ quorum queues are stronger for work-queue semantics with publisher confirms and manual acknowledgements, at the cost of higher latency than the lowest-latency queueing patterns. NATS JetStream offers built-in persistence and replay with at-least-once delivery, and its docs note that JetStream consumers track acknowledgements and redelivery state.
For ML-heavy or agent-heavy distributed workloads, Ray provides a useful additional pattern: a distributed shared-memory object store with immutable remote objects per node. That is valuable for intermediate tensors, embeddings, or batched retrieval results, especially where zero-copy access to NumPy arrays matters. But because Ray remote objects are immutable and primarily optimized for compute workflows, Ray is best viewed as a distributed working-memory accelerator rather than the durable long-term shared brain.
When long-running workflows matter more than raw queue control, durable execution systems become attractive. Temporal’s model, as documented by Pydantic AI’s integration, separates deterministic workflow code from non-deterministic activities and uses replay to recover progress after crashes. That is especially useful when memory updates, tool calls, and human-in-the-loop steps span minutes to days.
Data models, storage, consistency, and scaling
A well-designed shared brain needs a canonical data model. The most useful abstraction is a MemoryRecord with at least: stable ID, tenant/user/namespace, memory type, source event ID, actor, textual content, normalized fact fields, provenance, timestamps (observed_at, created_at, last_accessed_at), salience/importance/confidence, retention metadata, ACL tags, embedding version, vector-store reference, and consolidation state. Without those fields, a system can retrieve “memories” but cannot reliably govern them, delete them, reconcile them, or explain why they are present. This schema recommendation is an architectural synthesis driven by RAG provenance needs, LangGraph store/checkpoint separation, and enterprise compliance constraints.
For many teams, the highest-leverage data-model split is: facts, policies, and provenance in PostgreSQL; fast ephemeral derivatives in Redis; semantic representations in a vector index; raw documents and artifacts in object storage. PostgreSQL’s JSONB support and GIN indexing make it effective for semi-structured metadata, while row-level security and transaction isolation make it suitable for tenancy and correctness. pgvector extends that same system with HNSW or IVFFlat ANN indexing, letting one database handle both transactional and semantic memory if scale and recall/latency requirements permit.
Storage comparison
| Storage option | Best use in shared brain | Consistency and durability | Indicative planning envelope | Key tradeoffs | Sources |
|---|---|---|---|---|---|
| In-process memory | Working memory, per-request caches, graph state | Process-local only; lost on restart | Extremely fast, but no cross-worker guarantees; benchmark locally | Simplest and cheapest, but unsafe as source of truth in multi-worker deployments | |
multiprocessing.shared_memory | Local multi-process sharing of large arrays/buffers | Host-local volatile memory; explicit cleanup required | Faster than socket/disk exchange; no distributed guarantees | Good for local performance optimization, bad as enterprise authority | |
| Redis | Hot cache, coordination, dedup keys, token/state cache, light semantic cache | In-memory with optional RDB/AOF persistence; ACL/TLS available | Sub-ms to low-ms planning target on healthy local/networked deployments; higher with persistence or noisy virtualization. Pipelining can raise throughput dramatically, initially nearly linearly and up to roughly 10× baseline in Redis’s docs | Excellent latency; weaker query model and transaction semantics than PostgreSQL | |
| PostgreSQL | System of record for memory metadata, facts, ACLs, provenance | ACID transactions, MVCC, multiple isolation levels, row-level security | Usually the main correctness tier; validate latency under your schema and indexes | Strong governance and joins; heavier than cache for hottest paths | |
PostgreSQL + pgvector | Unified transactional + vector memory | Same transactionality for metadata; ANN tradeoffs via HNSW/IVFFlat | Good default when filters/joins matter more than extreme ANN scale | Fewer systems, but vector throughput/replica features are not as specialized as dedicated stores | |
| Qdrant | Dedicated vector retrieval with filtered ANN and production replication features | Eventual visibility by default unless wait=true; replication/sharding supported | Use when filtered ANN and dedicated retrieval operations dominate. Official docs emphasize comparing performance only at the same precision threshold | Strong retrieval focus; separate system to govern and operate | |
| Milvus | High-scale distributed vector retrieval | Strong, bounded staleness, session, or eventual consistency | Official benchmark examples show thousands to tens of thousands of QPS with TP50/TP99 ranging from tens to low hundreds of ms depending on topology; some million-scale ANN benchmarks in other docs are far lower latency | Very scalable, but operationally heavier than pgvector | |
| Faiss | Embedded/local vector search library | Library only; HA/CRUD/replication are your responsibility | Very strong local ANN performance; not a full DB | Ideal for embedded or batch systems; not a production memory service by itself | |
| Object store | Raw artifacts, transcripts, snapshots, archive, cold memory | Strong read-after-write and list consistency in S3; high durability | Excellent durability; use for archive and reindexing, not tight prompt-time loops | Cheap and durable, but the wrong place for low-latency semantic lookup |
Consistency and concurrency strategy
Concurrency rules differ by layer, and the architecture should respect those boundaries rather than hide them. At the Python task level, asyncio.Lock gives exclusive access only within one event loop. At the ORM level, SQLAlchemy’s Session and AsyncSession represent a single mutable transaction and are not safe to share among concurrent threads or asyncio tasks; SQLAlchemy also says the session is not a global cache. Those two facts alone imply that any meaningful shared brain must externalize authoritative memory rather than hide it inside request-scoped Python or ORM objects.
For durability-sensitive memory writes, PostgreSQL should be treated as the authoritative store because it provides MVCC, explicit locking when needed, and serializable isolation when anomalies must be prevented. PostgreSQL documents that Read Committed is the default, that Serializable prevents the standard anomaly classes, and that explicit lock modes exist for application-controlled synchronization where MVCC alone is insufficient.
For vector memory, consistency must be explicit. Qdrant documents that write acknowledgements do not necessarily mean immediate search visibility unless wait=true is used. Milvus exposes four consistency levels, from strong to eventual, with explicit latency/visibility tradeoffs and a default of bounded staleness. That means retrieval systems should treat embeddings and vector visibility as derived state and avoid assuming immediate read-after-write unless the store and request explicitly require it.
For asynchronous memory pipelines, delivery semantics must be accounted for at design time. Kafka documents at-most-once, at-least-once, and exactly-once patterns with idempotent producers and transactions. RabbitMQ quorum queues recommend publisher confirms and manual acknowledgements, and note that the underlying consensus design has inherently higher latency. Celery and Dramatiq both lean toward retries and redelivery, which means application-level idempotency is not optional.
Scaling patterns
The most robust scaling pattern is scope partitioning: partition memory by tenant, then by namespace or user, and often by time bucket. Episodic memory grows fastest, so it benefits from append-friendly partitioning and archive offload. Semantic memory grows more slowly and benefits from normalization, deduplication, and profile aggregation. Vector indexes scale best when their filtered metadata aligns with these same partition keys. Qdrant’s docs discuss sharding and replication-factor choices directly; Milvus benchmark reports show largely linear scale-up with more cores and scale-out with more query-node replicas.
At the query layer, the biggest wins often come from reducing coordination cost rather than buying bigger hardware. Redis pipelining exists specifically to reduce RTT and socket overhead, and Redis’s own docs say throughput can initially increase almost linearly and approach 10× the non-pipelined baseline. On the vector side, pgvector makes the speed/recall knobs explicit: HNSW improves speed-recall tradeoffs relative to IVFFlat, while IVFFlat builds faster and uses less memory; query-time probes and index parameters are first-order latency controls.
ML and LLM integration and memory lifecycle
The canonical mental model for LLM integration remains RAG: model weights provide parametric memory, while the application retrieves non-parametric memory from an external index at inference time. Lewis et al. made the case explicitly: external memory helps updateability, factual grounding, and provenance, all of which are indispensable in enterprise systems. In other words, a shared brain should usually be the non-parametric side of the system, with the model acting as an adaptive reasoning layer rather than the authority on enterprise truth.
For working memory assembly, modern frameworks now expose memory scopes more explicitly than early chatbot stacks did. LangGraph’s persistence docs distinguish thread-scoped state from cross-thread stores, and its memory overview notes that long conversations can exceed context windows or degrade performance even before the hard limit is hit. That strongly supports a design where the prompt context is treated as assembled working memory, not the memory store itself.
A useful enterprise memory lifecycle has five stages: capture, normalize, consolidate, embed, and retrieve. Capture stores raw interaction artifacts. Normalize extracts structured facts, entities, and provenance. Consolidate merges duplicates, updates profiles, and maybe summarizes. Embed computes vector representations for semantically searchable fields. Retrieve assembles a prompt or downstream business decision context from the right mix of episodic, semantic, and current state. Generative Agents, MemoryBank, and MemGPT all point in broadly this direction: store experiences, synthesize higher-level abstractions over time, and move information between tiers instead of keeping everything in one flat transcript.
The strongest recent argument for enterprise agent memory is that episodic memory should be explicit. The 2025 position paper on episodic memory for long-term LLM agents argues that continual long-term behavior depends on retaining instance-specific contexts rather than only generic facts. That aligns well with enterprise needs such as case histories, ticket histories, negotiation trails, prior model/tool decisions, and user-specific exceptions. Semantic memory alone is too lossy for those cases.
The write path below shows a common operational sequence for hot-path interaction plus asynchronous consolidation:
sequenceDiagram
participant U as User/API caller
participant A as FastAPI service
participant C as Cache/working memory
participant DB as PostgreSQL
participant V as Vector store
participant B as Bus/worker
participant M as LLM
U->>A: Request
A->>C: Read short-term state
A->>DB: Read durable facts / ACLs / provenance
A->>V: Retrieve semantically relevant memories
A->>M: Build working context and infer
M-->>A: Response / tool decisions
A->>DB: Commit authoritative memory event
A->>B: Publish consolidation job
A-->>U: Response
B->>DB: Normalize / summarize / deduplicate
B->>V: Upsert embeddings / retrieval records
This sequence is consistent with LangGraph’s checkpoint/store split, PostgreSQL as authoritative transaction boundary, and vector stores as derived retrieval tiers. It also matches queue semantics that favor idempotent asynchronous workers and delayed consolidation rather than synchronously embedding everything on the request path.
Framework selection for LLM memory layers is best driven by control needs:
| Framework or tool | Strength in a shared-brain design | When it fits best | Sources |
|---|---|---|---|
| LangGraph | Explicit short-term checkpoints and long-term stores; agent persistence | Stateful agent workflows with clear memory scopes | |
| Haystack | Transparent control over retrieval, routing, memory, and generation | Production RAG pipelines that need modular orchestration | |
| LlamaIndex | Strong data-ingestion and context-augmentation abstractions; storage context separates doc/index/vector layers | Data-heavy knowledge assistants and context-augmented workflows | |
| Pydantic AI + Temporal/Prefect/etc. | Type-safe agents plus durable execution integrations | Long-running or failure-sensitive LLM workflows | |
| Ray | Distributed working set and parallel compute for embeddings/retrieval | Large-scale compute-heavy pipelines, not core durable memory | |
| Faiss | Embedded local ANN building block | Single-node, embedded, or custom retrieval infrastructure |
Security, privacy, compliance, and governance
Security for a shared brain begins with the observation that it often contains more concentrated context than the underlying transactional system: user preferences, conversation logs, inferred facts, access patterns, and potentially regulated data. NIST SP 800-53 frames security and privacy controls as organization-wide safeguards, while GDPR Article 5 requires lawfulness, transparency, data minimization, and storage limitation. In practical terms, memory systems need purpose limitation, retention policies, and deletion workflows from the first design pass—not after deployment.
Encryption should be layered. PostgreSQL supports native TLS for client/server communications. Redis supports TLS, but its docs explicitly note that authentication alone does not protect credentials against eavesdropping on an unencrypted network. Qdrant’s security docs recommend enabling API-key authentication and TLS, and its configuration docs warn that sending API keys over an unencrypted channel is insecure. For object storage, S3 applies server-side encryption by default and supports KMS-backed key management; AWS KMS documents envelope encryption as the standard key-protection pattern.
Access control should be enforced in the data tier where possible, not only in Python service code. PostgreSQL row-level security is designed to restrict which rows a user can see or modify. Redis ACLs can scope commands and key access by user. Milvus exposes RBAC for users/roles, and Qdrant Cloud supports API keys with granular access control. In a multi-tenant memory system, that means every memory record should carry tenant and namespace labels, and at least one authoritative store should be able to enforce them independently of the application process.
Auditability should include both who changed memory and why the system retrieved it. Compliance frameworks differ in wording, but both NIST and HIPAA emphasize security controls and auditing; HHS’s audit-program materials make clear that audit protocols review compliance with Privacy, Security, and Breach Notification rules. At the application level, that translates to immutable event IDs, provenance fields, versioned summaries, access logs, retrieval trace IDs, and deletion logs. OpenTelemetry and Prometheus help with technical observability, but they do not replace domain-level audit trails.
Privacy design should assume that memory is especially susceptible to silent over-retention. GDPR’s data minimization principle means you should keep only what is necessary, and storage limitation means you should keep it only as long as necessary. That pushes enterprise shared-brain systems toward classification tags, retention TTLs by memory type, explicit “do not remember” and legal-hold flags, and delete propagation to caches, vector stores, and archives. This is one of the strongest reasons to keep the memory system non-parametric: it is much easier to delete rows, vectors, and objects than to “delete” knowledge from model parameters.
Implementation guidance, tradeoffs, and operations
A practical Python implementation usually starts with a conventional web stack and then adds memory adapters rather than embedding storage logic everywhere. FastAPI supplies dependency injection and request handling; Pydantic Settings externalizes configuration; SQLAlchemy handles transactional persistence; redis-py supports synchronous and asyncio access; dogpile.cache gives a formal second-level caching abstraction; and LangGraph/Haystack/LlamaIndex/Pydantic AI layer on top depending on how agentic or retrieval-heavy the application is.
A maintainable package layout looks something like this:
app/
api/
routes/
schemas/
memory/
domain.py # MemoryRecord, MemoryType, policies
service.py # orchestration
retrieval.py # ranking / assembly
consolidation.py # summarization / dedupe / re-embedding
governance.py # retention / deletion / ACL checks
adapters/
postgres/
redis/
vector/
object_store/
message_bus/
workers/
consolidation_worker.py
indexing_worker.py
observability/
tracing.py
metrics.py
settings.py
That organization is not mandated by a framework, but it follows the boundaries implied by FastAPI DI, Pydantic settings, SQLAlchemy session scope, and the fact that checkpointing, long-term memory, and retrieval are separate concerns in modern agent frameworks.
APIs, message buses, and serialization
A shared-brain API should usually expose at least four application-level operations: record_event, upsert_fact, search_memory, and forget/delete. Separating events from facts is important because events are append-oriented and often immutable, while facts are consolidated and replaceable. Retrieval should return not just content but provenance, confidence, and authorization-filtered visibility. This recommendation is a synthesis of episodic/semantic memory taxonomy, RAG provenance needs, and enterprise governance requirements.
For internal payload formats, the tradeoff is straightforward:
| Format | Best fit | Strengths | Main cautions | Sources |
|---|---|---|---|---|
| JSON | External APIs, debugging, audit readability | Ubiquitous, simple, human-readable | Verbose; weaker type-evolution discipline | Common engineering use; supported by API ecosystems |
| MessagePack | Cache payloads and compact internal messages | “Fast and small” binary format | Less self-describing than JSON | |
| Protocol Buffers | Strongly typed internal RPC/messages | Smaller and faster than JSON; codegen across languages | Schema compilation overhead; not ideal for ad hoc inspection | |
| Avro | Event streams with explicit schema evolution | Authoritative specification; very strong schema-evolution story | More ecosystem ceremony than JSON | |
pickle | Trusted local-only snapshots, if ever | Can serialize arbitrary Python objects | Officially insecure for untrusted input; may execute arbitrary code on unpickling |
For buses, choose by semantics rather than fashion. Kafka is best when you want a replayable append log and exactly-once stream processing patterns. RabbitMQ is best when you want robust queues and task distribution, with quorum queues where data safety matters. NATS JetStream is attractive when you want built-in persistence and simpler operational shape than Kafka, especially for request/reply plus replay hybrids. Celery and Dramatiq are application-level worker frameworks above these transport concerns, and both assume retry/redelivery is normal.
Testing strategy
The testing surface of a shared brain is broader than ordinary CRUD because correctness depends on scope, timing, and reprocessing behavior. pytest remains the base framework; pytest-asyncio is the obvious extension for async services; Hypothesis is well suited to invariants such as “consolidation is idempotent” or “redelivery does not duplicate facts”; Schemathesis can generate property-based tests from OpenAPI; and Locust is a straightforward choice for Python-native load tests.
The most valuable tests are usually these: concurrency tests for duplicate writes and lock contention; retry/redelivery tests for worker idempotency; retrieval tests that verify authorization filtering; deletion tests verifying erasure across PostgreSQL, cache, vector store, and object store; and migration tests that validate schema/index upgrades under live traffic. Because SQLAlchemy sessions are not safe to share concurrently, tests should also verify one-session-per-task behavior and reject global mutable sessions by convention.
Performance planning and tradeoffs
Published storage and vector-search benchmarks are useful only when compared under similar recall and durability settings. Qdrant’s benchmark page says that vector engines should be compared only at the same precision threshold. Weaviate’s official ANN benchmark examples show mean latencies on some million-scale datasets around 1.44 ms, 2.15 ms, and 4.49 ms, with p99 values around 3.13 ms, 3.69 ms, and 7.73 ms on a 30-core benchmark machine. Milvus’s published 2.2 benchmark report shows search performance from roughly 7k to 30k QPS depending on cores and replicas, with TP50 and TP99 latencies ranging from the tens into low hundreds of milliseconds depending on topology. Those figures are useful for planning, but not a substitute for environment-specific tests.
Redis should typically be budgeted as the lowest-latency external tier. Redis’s latency guide shows intrinsic baseline latency can be as low as about 115 microseconds on a low-noise bare-metal example and can rise to multiple milliseconds in noisier virtualized environments; its pipelining docs say throughput can initially improve almost linearly and approach about 10× baseline versus non-pipelined access. That makes Redis ideal for semantic caches, dedup windows, and coordination, but not for the authoritative system of record. Persistence settings also materially change the durability/latency tradeoff.
Object stores are throughput-rich and durable, but architecturally different. S3 provides strong read-after-write consistency and at least 3,500 PUT/s and 5,500 GET/s per prefix according to AWS, making it excellent for artifacts, archives, snapshots, and replay sources. The engineering implication is simple: use S3-class storage for raw memory artifacts and reindex pipelines, not as the hot synchronous retrieval plane for a user request.
Failure modes and operational runbook
The highest-probability failure modes are:
- State divergence across workers because in-process memory is mistaken for shared memory.
- Duplicate or reordered background effects because worker retries are not idempotent.
- Stale retrieval because vector writes were acknowledged but not yet visible, or because bounded-staleness settings were misunderstood.
- Memory leaks or orphaned host-local segments from improper
shared_memorylifecycle handling. - Security failures from TLS-disabled internal traffic, overly broad ACLs, or untrusted deserialization.
- Governance failures when delete/retention policies are applied to PostgreSQL but not to caches, vectors, or object snapshots.
The operational runbook should include, at minimum: end-to-end tracing with OpenTelemetry for FastAPI, SQLAlchemy, and Redis; Prometheus histograms for request, retrieval, embedding, and consolidation latency; durable backup/restore procedures by tier; migration playbooks; and regular load tests that validate p95/p99 behavior under realistic recall settings. Official tooling exists for most of this: OpenTelemetry instrumentation for FastAPI, SQLAlchemy, and Redis; Prometheus histograms; Alembic autogenerate and offline SQL mode; Redis RDB/AOF persistence; PostgreSQL dump/backup tooling; Qdrant snapshots; and Milvus Backup.
A defensible migration strategy is tier-specific. For PostgreSQL schemas, use Alembic revisions and prefer expand-contract changes where possible. For vector indexes, prefer shadow collections or shadow indexes, backfill embeddings, verify recall and authorization behavior, then cut over. For Redis caches, treat data as disposable unless persistence is intentionally enabled. For object storage, version artifacts and keep index manifests so the vector plane can be rebuilt from authoritative documents. That final point matters because Qdrant Cloud’s own backup guidance notes that some teams can regenerate vectors from a regular database, but that this takes time—hence the need for both replication and backups depending on the recovery objective.
The most robust default recommendation, therefore, is:
| Design choice | Recommended default | Why | Sources |
|---|---|---|---|
| Authoritative memory store | PostgreSQL | ACID transactions, RLS, JSONB, TLS, mature migrations | |
| Hot low-latency tier | Redis | Very low latency, pipelining, ACL/TLS, flexible structures | |
| Vector retrieval default | pgvector first, dedicated vector DB when scale demands it | Operational simplicity first; specialize later | |
| Raw artifact tier | S3-class object store | Strong consistency, durable archive, good rebuild source | |
| Async pipeline | Kafka for replay/log-centric systems; RabbitMQ or Celery for work-queue-centric systems; NATS JetStream for lighter persistent messaging | Semantics should drive the choice | |
| Agent memory orchestration | LangGraph or Haystack, with Pydantic AI + durable execution if workflows are long-running | Explicit memory scopes and production orchestration |
The central tradeoff is simple but profound: the more your shared brain behaves like a database, the more governable and compliant it becomes; the more it behaves like a cache, the faster it becomes; and the more it behaves like an LLM-attached semantic layer, the more flexible and powerful it becomes. Enterprise architecture succeeds when those roles are separated cleanly, connected deliberately, and measured continuously.