Concurrent Agents in One Codebase

Full report

On this page

Architecture recommendation

The strongest design for simultaneous AI agents in one repository is a two-plane system: a local execution plane where each agent works in its own Git worktree and topic branch, and a hosted coordination plane that stores only coordination metadata, leases, overlap negotiations, evidence, and canonical coordination events. It should not store every local memory file. Git worktrees are a good local isolation primitive because one repository can support multiple working trees at once, sharing repository data while keeping per-worktree files such as HEAD and index separate; Git also refuses by default to create a new worktree if the same branch is already checked out elsewhere, which is a useful guardrail against accidental same-branch concurrency. That isolation is necessary, but not sufficient, because two agents on different branches can still make incompatible changes to the same files or contracts.

The coordination plane should be built on a strongly consistent metadata service with leases, watch notifications, revisions, and conditional transactions. etcd is the cleanest reference model here because its KV operations are atomic and strictly serializable, its watch API supports change subscriptions, its lease API binds TTLs to keys and deletes attached keys on expiry, and its transaction model uses revisions that naturally support compare-and-swap style updates. Under the hood, this is the same family of design as replicated-state-machine coordination systems such as Raft, Chubby, and ZooKeeper: an append-only log of commands drives a deterministic state machine, while clients use leases and notifications as coordination hints rather than as proof of overwrite authority.

The key architectural rule is this: a lease proves liveness, not authorship and not overwrite permission. Overwrite safety must come from revision-checked mutation, in the same spirit as Git’s own reference-update APIs: git update-ref can set a ref only after verifying the expected old OID, and git push --force-with-lease updates a remote ref only if its current value matches an expected value. Those are exactly the semantics you want for claim updates, shared-memory section edits, completion records, and deployments. The hosted service should therefore require every mutating operation to carry an expected revision, expected Git base commit, or expected section revision.

My recommendation is therefore a Claim, Lease, Revision, and Evidence protocol:

• Claim before editing.
• Lease to show the agent is still active.
• Revision checks on every contested mutation.
• Evidence required before completion or deployment.
• Append-only events as the audit log and rebuild source.
• Generated or section-owned active memory, rather than free-form concurrent edits, for the files that steer future agent behavior.

This gives local autonomy, outage tolerance, and deterministic integration behavior without turning the whole repository into a remote collaborative editor. It also avoids the weakest patterns in this problem class: ad hoc lock files, informal warning notes, or “the lease holder can overwrite whatever they want.” Chubby’s own experience is instructive here: advisory locking and low-volume metadata storage are useful for coarse-grained coordination, but the lock service is not where application data should live, and it is not a substitute for application-specific conflict rules.

Where each coordination technique fits

The techniques you listed are not peers. Some belong in the baseline design, some should be optional escape hatches, and some are overkill unless you expect offline multi-master replicas.

The practical outcome is a hybrid protocol: worktrees and branches for local isolation; claim records, leases, revision checks, and events in the hosted service; static ownership declarations only for defaults and escalation; patch queues for handoff or deferred integration; human arbitration only when semantic disagreement or risk crosses a threshold. Vector clocks and CRDTs are not the right first choice unless your coordination plane must work as a disconnected, multi-master replicated system.

Protocol and state machine

Before touching contested code or active-memory files, an agent must create a claim announcement. That claim must include: the objective, the working path, intended files and modules, expected contract or schema changes, test responsibility, expected rollout or deployment responsibility, lease duration, current base commit, and intended active-memory impacts. This is the minimum data required to make overlap discoverable and to let other agents reason about risk. The claim is then indexed in two ways: symbolically by path/module/symbol and semantically by an embedding of the objective, contract notes, and test description. Symbolic indexing catches direct overlap; semantic indexing catches “different file, same subsystem” collisions. This semantic layer is a design recommendation, but the evented notification and revision model should follow the same watch and replicated-log patterns used by coordination systems such as ZooKeeper and etcd.

A lease begins only after the claim is accepted. The hosted service assigns the claim an initial status and a lease with TTL. The agent must send heartbeats before TTL expiry. If it changes scope materially, it cannot simply keep the old lease alive and continue; scope expansion must create a claim-change event that reruns overlap analysis. That distinction matters because keepalive semantics are about liveness, not about expanding authority. etcd’s lease semantics are a good model: the lease expires if keepalives stop, and attached keys are removed; the system also emits delete events that other clients can observe.

When overlap is detected, the agents should move into a structured meeting room, not an informal chat. A meeting room is a first-class record with participants, contested targets, negotiation proposals, deadlines, and a required resolution type. Valid resolutions are: split ownership by file or section, serialize work so one claim waits on another, handoff one claim to the other, joint claim for intentionally coupled work, or human arbitration. The meeting room should publish notifications through watch-backed SSE or WebSocket streams, but clients must refresh the canonical record before acting because watch delivery can be delayed under unhealthy conditions; watch events are hints, not ground truth.

The state machine should look like this:

stateDiagram-v2
    [*] --> Draft
    Draft --> Announced: create claim
    Announced --> Negotiating: overlap detected
    Announced --> Active: no blocking overlap
    Negotiating --> Active: negotiated resolution
    Negotiating --> Blocked: unresolved / waiting human
    Blocked --> Active: arbitration resolved
    Active --> ReadyForReview: evidence attached
    Active --> Abandoned: voluntary abandon
    Active --> Expired: lease timeout
    Active --> Suspended: dependency / CI / review wait
    Suspended --> Active: resumed
    ReadyForReview --> Integrated: merge or patch accepted
    ReadyForReview --> Active: stale base / more work
    Integrated --> Completed: memory + deployment evidence done
    Expired --> TakenOver: emergency takeover approved
    Expired --> Abandoned: explicitly discarded
    Abandoned --> [*]
    Completed --> [*]
    TakenOver --> Active

The required semantics for lifecycle transitions should be strict. Renewal extends only the lease, unless the scope changed, in which case the claim must reenter negotiation. Expiration freezes the claim’s authority; it does not delete history and it does not give others the right to overwrite the abandoned work silently. Abandonment is a voluntary, explicit release that records the current branch head, diff summary, and memory state so the next agent can pick up responsibly. Handoff is a signed transfer between two live claims that preserves audit history. Split ownership creates formal sub-target assignments, such as src/api/** to one claim and docs/contracts/** to another, or section-level ownership within a shared memory file. Emergency takeover requires the predecessor claim to be expired or abandoned, a grace period to have elapsed, and a takeover event referencing the predecessor’s last known branch head and active-memory revisions.

The most important invariant is the one you explicitly requested: a lease must never silently grant authority to overwrite work. This is where many designs get weak. The overwrite rule should be: “you may integrate or mutate a contested object only if you hold an active claim covering that object and your mutation presents the expected current revision or base commit and any required meeting-room resolution or human approval is already recorded.” That is the coordination equivalent of Git’s protected ref update rules and CAS ref updates.

Database model and endpoint examples

The cleanest logical data model is event-sourced with materialized views. The append-only event log is authoritative for audit and rebuild. Current-state tables are projections for fast lookup and semantic discovery. That pattern maps naturally onto replicated-log coordination systems such as Raft and etcd, which already model the world as ordered commands applied to state.

A practical logical schema looks like this:

The authoritative write rule should be: every mutable current-state row also has a monotonic revision, and every update must include expected_revision. If the expectation does not match, the service returns 409 Conflict with the canonical latest row and the related event IDs. That is the database-level equivalent of Git CAS and --force-with-lease, and it is the right default for claims, meetings, memory sections, and deployment pointers.

The coordination API should look like this:

POST /v1/claims
Content-Type: application/json

{
  "repo": "acme/payments",
  "objective": "Add idempotency handling to settlement retries",
  "baseCommit": "4f2d6b1",
  "branch": "claim/settlement-idempotency",
  "worktree": "../wt/claim-482",
  "paths": ["src/settlement/**", "tests/settlement/**"],
  "modules": ["SettlementService", "RetryPolicy"],
  "expectedContractChanges": [
    {"type": "api", "area": "SettlementResult", "change": "add field retryKey"},
    {"type": "schema", "area": "settlement_attempts", "change": "new nullable column retry_key"}
  ],
  "testResponsibility": {
    "unit": ["SettlementServiceTests", "RetryPolicyTests"],
    "integration": ["SettlementFlowTests"],
    "migration": ["retry_key backfill verification"]
  },
  "leaseTtlSeconds": 1800,
  "memoryImpacts": [
    {"path": "memory/current-risks.md", "mode": "append_only"},
    {"path": "memory/system-overview.md", "sections": ["settlement-retry-flow"]}
  ]
}

POST /v1/claims/{claimId}/renew
Content-Type: application/json

{
  "expectedLeaseRevision": 12,
  "expectedClaimRevision": 7,
  "ttlSeconds": 1800
}

POST /v1/claims/{claimId}/scope-change
Content-Type: application/json

{
  "expectedClaimRevision": 7,
  "addPaths": ["src/api/contracts/**"],
  "addExpectedContractChanges": [
    {"type": "api", "area": "SettlementResult", "change": "public field retryKey"}
  ],
  "reason": "Implementation reached public API boundary"
}

POST /v1/meetings/{meetingId}/resolve
Content-Type: application/json

{
  "expectedMeetingRevision": 4,
  "resolution": {
    "type": "split_ownership",
    "assignments": [
      {"claimId": "482", "targets": ["src/settlement/**", "memory/current-risks.md#settlement"]},
      {"claimId": "491", "targets": ["src/api/contracts/**", "docs/api/**"]}
    ]
  }
}

POST /v1/claims/{claimId}/complete
Content-Type: application/json

{
  "expectedClaimRevision": 14,
  "expectedBaseRef": {
    "ref": "refs/heads/main",
    "oid": "4f2d6b1"
  },
  "evidence": {
    "headCommit": "a38b92e",
    "changedFiles": [
      "src/settlement/SettlementService.cs",
      "src/settlement/RetryPolicy.cs",
      "tests/settlement/SettlementServiceTests.cs",
      "memory/current-risks.md"
    ],
    "tests": [
      {"name": "SettlementServiceTests", "status": "passed"},
      {"name": "SettlementFlowTests", "status": "passed"}
    ],
    "apiChanges": [
      {"area": "SettlementResult", "compatibility": "backward-compatible additive"}
    ],
    "migrations": [
      {"id": "20260712_add_retry_key", "status": "applied-dev"}
    ],
    "deployment": {"environment": "staging", "status": "not_deployed"},
    "unresolvedRisks": ["retry-key uniqueness under dual-writer replay still needs prod telemetry"],
    "memoryUpdates": [
      {"path": "memory/current-risks.md", "revision": 22}
    ]
  }
}

Agents should subscribe through a watch-style stream:

GET /v1/events/stream?repo=acme/payments&since=9282
Accept: text/event-stream

That stream should carry claim changes, meeting-room openings, lease expirations, takeover attempts, and completion events. But clients must always treat stream notifications as refresh triggers, not final authority, because watch delivery can lag or fail while the durable KV state remains correct.

Git workflow and active-memory rules

The Git workflow should be deliberately boring. Each claim gets its own worktree and topic branch. The agent announces the claim, creates or attaches its worktree, edits locally, commits normally, and only then tries to integrate. The protected integration ref, whether main or a merge queue ref, must be advanced only by a service that uses an expected old OID check equivalent to update-ref or --force-with-lease, and server-side hooks should reject pushes that do not carry a valid claim reference, a live or correctly finalized claim state, and any required arbitration or ownership approval. Git’s update, pre-receive, and reference-transaction hooks are the right enforcement points on the server side.

Patch queues should exist as a fallback completion mode, not the default. A claim can complete by publishing a commit OID, a commit range, or a git format-patch artifact. Another agent can then apply it via git am, and git am --3way can fall back to a 3-way merge when the patch does not apply cleanly but the underlying blob identities are available. That makes patch queues valuable for handoff, air-gapped review, or service outage recovery, but they should not replace live claims, watches, and lease-aware overlap negotiation.

The active-memory policy should be class-based, not universal. Different memory documents need different coordination rules:

This is the most important recommendation in the report. Files that directly steer later agent behavior should not be free-form concurrently edited. They should either be:

• generated from canonical coordination events, if they are status/index/progress artifacts;
• append-only, if they are logs of decisions, risks, or observations;
• section-owned and revision-checked, if they are shared explanatory documents humans and agents both read;
• or private and non-authoritative, if they are just an agent’s scratchpad.

For files in the section-owned class, the service should know the document and section revisions, but not necessarily the full content. A section patch request should include the document path, section key, expected section revision, expected file hash, and patch payload. If either revision or file hash is stale, the patch is rejected and the agent must re-read. This lets the service coordinate high-risk memory files without moving all of them into the service. The local file remains local and authoritative for content, while the hosted service carries only metadata, revision state, and perhaps a minimal snapshot for diff previews. That is the right middle ground between “everything remote” and “everything unmanaged.”

To prevent contradictory instructions while preserving user edits, generated documents should use generated/manual boundaries. For example, a file can contain a generated region that is fully overwritten from canonical events and one or more manual regions that are never overwritten automatically. If a user edits a manual region, the service records a new section revision and future agent writes must CAS against it. If a user edits a generated region, the agent must not silently replace it; instead it should open a reconciliation event and ask whether the change belongs in the upstream coordination events or in a manual overlay section. This rule is how you prevent “the coordinator said X” and “the local file says Y” from drifting into contradictory, silently preserved instruction sets.

The rule for reconciling changes without discarding user edits should be explicit. If a user-edited file differs from the claim’s expected base, the agent may auto-merge only non-overlapping hunks. If the same section, symbol, or semantic contract moved, the user’s current content remains the working copy, the agent’s proposed edit is stored as a patch artifact, and the claim reenters negotiation or human review. The coordinator should never instruct an agent to overwrite unreviewed user edits purely because the agent still holds a lease. That is exactly the failure mode that CAS-style revision checks are designed to stop.

Security model and user experience

Identity should be strong and explicit. Every agent and human actor should authenticate with short-lived tokens issued by OIDC or equivalent, and the service should bind every claim, heartbeat, patch, and completion record to that authenticated principal. If you support autonomous agents outside your control plane, require signed requests and store a public key fingerprint for each agent identity. The event log should be immutable to ordinary actors and should be at least tamper-evident, ideally with per-event hashes chained to the prior committed event. This is not mandated by Git or etcd themselves; it is a best-practice layer on top of the append-only log model that Raft-style systems encourage.

Authorization should separate claim ownership, integration authority, and deployment authority. An agent should be able to create, renew, narrow, abandon, and complete its own claims, but not mutate someone else’s active claim except through a recorded meeting-room resolution or takeover protocol. Integration of protected refs should require server-side validation via hooks. High-risk paths should map to human owners by CODEOWNERS-style rules so the right reviewers or arbitrators are selected automatically, and organizations that want stronger controls can require those owners’ approvals before merge.

For agents, the UX should feel like a small Git-adjacent command set rather than a separate application. A good CLI would expose commands like claim open, claim renew, claim scope add, claim status, claim complete, and claim abandon. Each command should print the live overlap assessment, meeting requests, and required evidence. If an overlap appears, the CLI should not just say “conflict”; it should show the contested paths, the semantic reason, and the structured options: split, serialize, handoff, join, or escalate. This mirrors ZooKeeper’s lesson that coordination services are most useful when the service provides a minimal kernel and the client builds higher-level flows around watches and sessions.

For humans, the dashboard should center on live claims by subsystem, active lease timers, meeting rooms, unintegrated patch artifacts, memory docs at risk, and deployment readiness. Humans should be able to search semantically — for example, “retry behavior,” “auth contract,” or “stale deployment risk” — and see claims even when the exact path differs. But the UI should always present the symbolic overlaps too, because semantic similarity alone should not drive hard blocking decisions. The dashboard should also surface ownership defaults from CODEOWNERS to make arbitration routing predictable.

Failure scenarios and deterministic integration tests

The protocol needs hard answers for the obvious races. If two agents race to integrate, both are allowed to commit locally, but only one may advance the protected integration ref. The merge or queue service must compare the expected current ref OID with the actual one. The winner integrates; the loser gets stale_base, fetches the new tip, rebases or merges in its own worktree, and re-runs evidence collection. This is the exact same safety shape as Git’s CAS-style ref updates and --force-with-lease.

If one agent tries to deploy stale code, deployment should also be a compare-and-swap operation. The deployment request must include expected_prev_commit_oid for the target environment. If staging or production has already moved, the request fails. That prevents “claim completed against main@A, deployed after main moved to B” accidents. The deployment record should also capture whether required migrations were already applied and whether unresolved risks exceeded the environment policy. This is a protocol recommendation, but it is directly justified by the same revision-check discipline used by Git and etcd.

If generated files appear dirty, the system should distinguish between committed generated artifacts and incidental local build noise. Incidental generator drift should not extend a claim or create false overlap; it should be ignored or cleaned locally. Committed generated artifacts, by contrast, must name their generator inputs in evidence so the service can decide whether a path-level conflict is real or only derived. In practice, the clean rule is: coordinate on the source inputs, not on every generated output, unless the output is itself reviewed and committed as a first-class artifact. That keeps generated dirt from creating spurious claim contention while still preserving reproducibility. This is a design recommendation based on the separation between authoritative data and derived projections in replicated-log systems.

If an agent disappears mid-change, the lease will expire and the claim will move to Expired, but the branch, commits, patch artifacts, and memory revisions remain as evidence. The successor cannot simply overwrite the predecessor’s paths. Instead it must open a takeover claim that references the expired claim, imports the predecessor head or patch artifact, and declares whether it is continuing, superseding, or discarding each piece of unfinished work. If the predecessor’s work touched contracts, migrations, or active-memory instructions, emergency takeover should require either human approval or a policy-defined quorum of owners. Expiry is a liveness signal; it is not a garbage collector for semantics.

The deterministic integration test suite should be part of the coordination service itself. Use a fake clock, fake Git ref store, scripted watch stream, and in-memory event log so every race can be replayed exactly. The most important tests are these:

Those tests are what make the protocol trustworthy. Without them, the design will look good in documentation and then fail on exactly the races you called out. With them, you can verify that leases do not grant overwrite power, events replay cleanly, takeovers preserve history, and user edits are never silently discarded. The distributed-systems literature is very clear on the underlying reason: correctness comes from ordered, validated state transitions, not from informal coordination.

In short, the best architecture is not “two agents edit files and hope Git sorts it out,” and it is not “put every local memory file in a lock server.” It is a hosted, evented coordination control plane over a local Git worktree execution plane, with claims announced before editing, leases for liveness, revision-checked writes for safety, meeting rooms for overlap negotiation, human arbitration only for semantic risk, and a class-based active-memory policy that makes the most dangerous files either generated, append-only, or section-owned. That combination is the smallest design that is genuinely strong.