Production Assurance Program for the InternationalIntelligence.org Daily Brief

Full report

On this page

Executive SRE recommendation and research basis

InternationalIntelligence.org should treat the Daily Brief as a reader-facing publication service, not as a successful PHP process, scheduler invocation, provider request, or validator run. The central production question is:

Can an independent observer prove that the correct English and Spanish edition for the intended date was generated once, validated against a defined evidence policy, atomically published, consistently served by every public host and cache layer, and recoverable without repeating paid or externally visible work?

The project context describes a system intended to maintain one active process, an immutable target date, a monotonic revision sequence, one current provider operation, a single transition writer, shared bilingual state, atomic publication, and safe stale-worker recovery. It also specifies strict privacy constraints and the absence of direct access to code, infrastructure, databases, credentials, or private telemetry.

The recommended architecture has five mutually reinforcing controls:

1. A machine-verifiable edition manifest becomes the publication proof. It binds the target date, publication event, English and Spanish artifact hashes, evidence-policy result, revision, release, and site instance.
2. Compare-and-swap state transitions with leases prove that only the current revision and operation may advance the workflow. Old workers may finish computation but cannot commit state or publish.
3. An append-only, hash-chained event ledger explains every reservation, provider interaction, retry, reconciliation, validation result, publication preparation, publication commit, correction, quarantine, and operator action without retaining prompts or provider bodies.
4. Read-only production synthetics test the same public paths readers use while being structurally incapable of starting research, composition, publication, or paid provider work.
5. A release-proof bundle records deterministic tests, concurrency races, crash matrices, package checksums, origin and browser smoke tests, scheduler proof, synthetic results, canary observations, and rollback readiness for every deployed release.

These controls close the component-test/production-assurance gap because they test relationships that syntax checks and isolated validators cannot establish: scheduler-to-supervisor continuity, provider completion-to-reconciliation latency, state-to-artifact agreement, bilingual identity, release uniformity, host uniformity, cache freshness, and crash-safe idempotency.

Selected research topic. Because the uploaded project brief specifies a concrete topic, no cross-domain topic-ranking exercise is necessary. The selected topic is production observability and release verification for a bilingual, scheduled, AI-assisted publication pipeline.

Source basis. The architecture is grounded primarily in current official standards and first-party engineering guidance:

The standards progression relevant to this design is:

timeline
    title Standards and implementation timeline
    2021 : W3C Trace Context Recommendation
         : Portable traceparent and tracestate propagation
    2022 : RFC 9111 published
         : Current HTTP caching and freshness rules
    2025 : OpenTelemetry semantic conventions expanded
         : Stable service instance and service version identity
    2026 : Daily Brief production-assurance program
         : Publication manifests, state CAS, synthetics, chaos proof

W3C Trace Context became a Recommendation in November 2021, RFC 9111 standardized the current HTTP caching model in June 2022, and OpenTelemetry now defines stable service identity attributes such as service.instance.id and service.version.

Explicit assumptions and unspecified preferences.

The largest evidence gap is that no production measurements are available. Therefore, stage thresholds below are initial engineering limits, not claims about the system’s current latency. The first 30 days should measure normal distributions, after which thresholds should be recalibrated without weakening hard safety invariants.

Service definition, failure taxonomy, SLIs, SLOs, and error budgets

Reader-perspective service definition. A Daily Brief edition is successful only when all of the following are true for the target date:

Availability, freshness, and correctness are independent. A page can be available but stale when a CDN, PHP cache, browser cache, or service worker returns a valid 200 response for an earlier revision. It can be fresh but incorrect when the target date is current but evidence validation was bypassed, regional requirements were missed, English and Spanish diverged, or unsupported claims passed through composition. RFC 9111 explicitly distinguishes response freshness from origin validity and recognizes that application-level mechanisms can still display old representations; therefore, HTTP success cannot substitute for edition identity and age checks.

Failure taxonomy.

State and publication invariants. These are not error-budget objectives; they are correctness properties that every accepted transition and publication must satisfy.

SLI definitions. Each SLI must be computed from authoritative machine events, edition manifests, or independent synthetics—not from a progress bar.

Deadline models. The owner should choose one model and publish it as an editorial contract.

Recommended initial SLO tiers. Targets should be reviewed after the baseline period. Google’s SRE guidance treats the error budget as 1 − SLO, recommends documenting measurement caveats, and uses budget exhaustion to control release velocity rather than to punish teams.

A 99% deadline SLO over 90 expected editions permits only 0.9 late editions. One late edition therefore consumes approximately 111% of that period’s numerical budget:

xychart-beta
    title "Ninety-edition freshness budget consumption at a 99% SLO"
    x-axis "Late editions" [0, 1, 2, 3]
    y-axis "Error budget consumed (%)" 0 --> 350
    bar [0, 111, 222, 333]

This arithmetic shows why deadline-based services should combine a rolling SLO with a practical incident policy. A single late edition may exhaust the formal 90-edition budget, but the response should be a temporary reliability/change focus, not indiscriminate operational paralysis. Burn-rate alerts should be tuned to the objective and event frequency; Google’s guidance cautions that extremely high or low goals can make generic burn thresholds ineffective.

Initial stage limits. Before baseline measurements exist, configure stage-specific limits rather than one process timeout:

The rationale is operational separation: a provider stage may legitimately be much longer than a local state transition, whereas a two-minute publication commit almost certainly indicates a lock, storage, or partial-write problem.

Metrics, structured events, logs, and trace correlation

Metric design rules. Use prefix daily_brief_, SI/base units, counters for cumulative events, gauges for current state, and histograms for distributions. Prometheus recommends base units such as seconds and warns that every unique label combination creates a separate time series; unbounded values such as user identifiers must not be labels.

Counters answer “how many happened?” and never decrease except on process restart. Gauges answer “what is true now?” and may rise or fall. Histograms answer “how is duration or size distributed?” and permit SLO queries such as the fraction below a stage limit. Prometheus currently recommends native histograms when supported, while noting that library and exposition support varies; classic histograms are a sound initial choice for constrained PHP deployments.

Low-cardinality namespace.

Cardinality budget. A practical initial budget is fewer than 2,000 active series for the service, fewer than 12 values for any label, and no label whose value is created per process, provider request, source, article, trace, browser, operator, or reader. process_id, operation_id, event_id, publication hashes, provider request IDs, URLs, source domains, exception messages, and worker IDs belong in structured events or trace attributes, not metric labels.

Individual incidents are linked through:

• a low-cardinality alert label such as stage="publication";
• an exemplar containing a trace ID where the metrics backend supports exemplars;
• a dashboard link that queries the event store by incident time and pseudonymous process reference;
• a trace or event correlation ID stored outside the metric label set.

Structured state-transition event schema.

Sample redacted event.

{
  "schema_version": 1,
  "event_id": "evt_01K1EXAMPLE8R7Z",
  "previous_event_hash": "sha256:91a4…f208",
  "process_id": "proc_internal_opaque",
  "target_date": "2026-08-01",
  "from_state": "research_reconciling",
  "to_state": "evidence_ready",
  "prior_revision": 18,
  "new_revision": 19,
  "operation_id": "op_pseudonym_f3a9",
  "site_instance_id": "site_7c2f",
  "release": "2026.08.01+8ae73a",
  "trigger_type": "reconciler",
  "worker_id": "worker_ephemeral_12",
  "expected_revision": 18,
  "result_code": "committed",
  "error_class": null,
  "started_at": "2026-08-01T11:14:03.120Z",
  "ended_at": "2026-08-01T11:14:04.774Z",
  "duration_ms": 1654,
  "external_effect": false,
  "reconciliation_required": false,
  "evidence_before": {
    "retained_candidates": 9,
    "selectable_candidates": 5,
    "corroborated_candidates": 3,
    "represented_regions": 3
  },
  "evidence_after": {
    "retained_candidates": 14,
    "selectable_candidates": 9,
    "corroborated_candidates": 7,
    "represented_regions": 5
  },
  "publication_reference": null,
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "span_id": "00f067aa0ba902b7",
  "event_integrity_hash": "hmac-sha256:05d7…b0ce"
}

Append-only privacy-preserving ledger. Canonicalize the event JSON, omit the integrity field, and compute:

event_integrity_hash =
    HMAC-SHA256(audit_key,
        previous_event_hash || canonical_event_bytes)

The ledger stores only classifications, timestamps, counts, hashes, and opaque identifiers. It must never contain prompts, article text, source excerpts, provider response bodies, credentials, full private paths, raw account identifiers, reader information, cookies, IP addresses, or unredacted exceptions. Store a provider object identifier only when operationally necessary, encrypt it at rest, and expose only a pseudonymous reference to normal dashboards.

The ledger should be append-only at the application level and periodically checkpoint its current head hash to a separately protected location. A snapshot is a performance optimization; the ledger and immutable publication manifests provide reconstruction evidence. Because ledger and snapshot writes may fail independently, every transition follows a recoverable protocol:

1. calculate the proposed transition against expected revision;
2. append a transition_intent or transactional event;
3. commit the new snapshot using CAS;
4. append transition_committed;
5. on restart, reconcile any unmatched intent or snapshot revision;
6. never infer that external work must be repeated merely because the final audit event is missing.

Where a database supports an atomic transaction, the snapshot and event row should commit together. Where it does not, the protocol above makes ambiguity explicit and recoverable.

OpenTelemetry trace design. Use one trace per scheduler invocation, operator recovery action, or synthetic session, with linked traces for provider operations that outlive the initiating process. Recommended spans are:

daily_brief.scheduler.tick
  daily_brief.supervisor.reserve
    daily_brief.provider.create
    daily_brief.provider.poll
    daily_brief.provider.reconcile
    daily_brief.evidence.validate
    daily_brief.research.supplement
    daily_brief.compose
    daily_brief.final_validate
    daily_brief.publish.prepare
    daily_brief.publish.commit
    daily_brief.cleanup
  daily_brief.status.render

Set OTel resource attributes including:

service.namespace = "internationalintelligence"
service.name = "daily-brief"
service.instance.id = "<opaque per-instance ID>"
service.version = "<immutable release>"
deployment.environment.name = "production"

OpenTelemetry defines service.instance.id as unique for each concurrent instance of the same logical service and service.version as the exact version identifying the artifact. These attributes directly support multiple-instance and deployment-drift detection.

Use W3C traceparent for HTTP propagation. Do not place process IDs, source information, account details, or personal information in tracestate; the W3C specification expressly warns against personal or sensitive trace data. Treat incoming trace headers as untrusted, regenerate or sanitize context at the public trust boundary, and preserve only approved internal correlation.

For non-OTLP JSON logs, use top-level trace_id, span_id, and trace_flags; this matches OpenTelemetry’s stable compatibility guidance.

Constrained-hosting alternative. If a collector and tracing backend are not feasible, emit one structured JSON line per event to an append-only daily file or remote log sink:

{
  "ts": "2026-08-01T11:14:04.774Z",
  "severity": "INFO",
  "event_name": "daily_brief.transition",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "span_id": "00f067aa0ba902b7",
  "process_ref": "p_6C9A1D",
  "target_date": "2026-08-01",
  "revision": 19,
  "stage": "evidence_ready",
  "result": "committed",
  "release": "2026.08.01+8ae73a",
  "site_instance": "site_7c2f",
  "duration_ms": 1654,
  "error_class": null
}

A periodic exporter can derive counters and gauges from these events. The essential requirement is stable semantics and correlation, not a particular observability vendor.

Causal model.

flowchart TD
    A[Scheduler heartbeat] --> B{Current target reserved?}
    B -- No --> C[Create one process and immutable target]
    B -- Yes --> D[Load current revision and operation]
    C --> E[Reserve stage using CAS and lease]
    D --> E
    E --> F{External operation already exists?}
    F -- Yes --> G[Poll or reconcile same operation]
    F -- No --> H[Create operation and persist identity]
    H --> G
    G --> I[Validate and retain evidence]
    I --> J{Hard evidence floor met?}
    J -- No, repair allowed --> K[Supplement only deficits]
    K --> I
    J -- No, limits reached --> L[Quarantine or editorial exception]
    J -- Yes --> M[Compose from retained evidence]
    M --> N[Final bilingual validation]
    N --> O[Prepare immutable artifacts and manifest]
    O --> P[Atomically switch canonical publication pointer]
    P --> Q[Verify public EN and ES hashes]
    Q --> R[Close process and retain audit proof]

    S[Stale worker] --> T[Expected revision mismatch]
    T --> U[Reject transition; no external retry]
    V[Lost response] --> W[Mark reconciliation required]
    W --> G
    X[Stale cache] --> Y[Synthetic observes old event or Age]
    Y --> Z[Purge or revalidate cache; never republish]

Every control in the diagram follows the same contract:

Dashboards and alert catalog

Dashboard: Editorial Freshness. The first screen should answer whether readers have the right edition, not whether the worker is busy.

Color must not be driven by HTTP 200. “Green” requires current target, valid manifest, both languages, expected hashes, and bounded public age.

Dashboard: Workflow and Provider.

Dashboard: Scheduler, Origin, Cache, and Release.

Alert catalog. Threshold variables are configurable:

S = expected scheduler interval
D = editorial deadline
G = publication grace
M = maximum active-process age
Tstage = critical stage threshold
Cprovider = configured provider cost ceiling

Alert design principle. Alerts should page on reader impact, imminent deadline impact, violated hard invariants, or inability to recover—not on every retry or provider pending status. Multi-window burn alerts suit high-volume probe SLIs; direct state alerts are necessary for once-per-day publication events because a single miss can consume an entire formal budget.

Deterministic, integration, concurrency, and chaos test architecture

Test-layer matrix.

Deterministic no-network end-to-end proof. Implement this as seven phases covering the twelve required lifecycle events.

Global assertions.

count(distinct process_id for target T) = 1
target_date(process) remains T for all events
new_revision = prior_revision + 1 for every committed transition
provider_primary_create_count = 1
winning_reservation_count per stage/revision = 1
retained_evidence_hash before composition failure =
    retained_evidence_hash after composition retry
stale_transition_commit_count = 0
stale_transition_rejection_count >= 1
publication_manifest_hash before duplicate =
    publication_manifest_hash after duplicate
english.publication_event_id = spanish.publication_event_id
network_socket_attempt_count = 0

Enforce the final condition at the operating-system or test-runtime boundary, not merely with a mock: disable DNS and outbound sockets, make the provider adapter throw on any network attempt, and assert the adapter’s call transcript contains only deterministic fixture operations.

Concurrency race suite.

Chaos matrix.

Safe-ZIP and package verification. A release package test must:

1. extract into a new empty directory;
2. reject absolute paths, .. traversal, symlinks, devices, duplicate normalized names, and case-colliding names;
3. enforce file-count and uncompressed-size ceilings;
4. verify an allowlisted inventory and manifest before execution;
5. run syntax and schema checks against extracted bytes;
6. compare package hashes byte-for-byte with the approved build;
7. start a loopback web server with provider networking disabled;
8. verify that no runtime-generated secret, cache, log, private source, or credential file is present.

Production synthetic monitoring and incident response

Read-only synthetic contract. Synthetic monitoring must be structurally incapable of becoming a trigger source:

• Synthetic agents receive no scheduler, operator, provider, publication, cache-purge, or database credentials.
• They use only GET, HEAD, and a special edge-only mutation-gate probe.
• Public handlers do not interpret user agents, query strings, headers, or probe paths as workflow triggers.
• Scheduler execution is allowed only through CLI or an authenticated internal channel, never ordinary public HTTP.
• The monitor’s X-Synthetic-Read-Only: 1 header is informational, not an authorization control.
• A dedicated route such as /_assurance/mutation-gate tests host/method policy at the webserver or edge and terminates before application bootstrap. It never calls a real mutation endpoint.
• Synthetic state is stored in the monitoring system only to compare successive revisions and event IDs.
• Rate and response-size limits protect origin and CDN resources.
• Synthetic requests use no reader identifiers, cookies, or personal data.

Synthetic suite.

RFC 9111 requires cache behavior to preserve HTTP semantics but also allows cached responses to remain reusable according to freshness and validation rules. The synthetic therefore validates both headers and edition identity; a plausible cache header alone is insufficient.

Controlled provider probe. This is optional and separate from the normal synthetic suite. It must use:

• a separate provider project or key with the minimum possible permissions;
• a separate state namespace that cannot reference or write a Daily Brief process;
• an explicit per-call and monthly budget;
• a deterministic, non-sensitive test input;
• no publication, evidence, composition, or scheduler integration;
• a low frequency, such as once per day or after provider-configuration change;
• an immediate circuit breaker after one failure;
• retention settings reviewed against the exact provider mode.

OpenAI documents that some API modes retain application state and that background mode requires temporary stored state for polling, so a controlled probe must be reviewed against the account’s privacy requirements rather than assumed to be retention-free.

A provider probe proves connectivity and authentication only. It does not prove that the production workflow preserves operation identifiers, reconciles completions, validates evidence, or publishes correctly.

Incident severity.

Operator action policy.

Incident decision tree.

flowchart TD
    A[Alert received] --> B{Reader-visible identity or integrity wrong?}
    B -- Yes --> C[Declare SEV-1]
    C --> D[Disable unsafe mutations]
    D --> E{Prior complete shared manifest available?}
    E -- Yes --> F[Atomically restore prior manifest]
    E -- No --> G[Quarantine and serve bounded unavailable/stale notice]
    F --> H[Verify EN/ES, hashes, hosts, cache]
    G --> H

    B -- No --> I{Target overdue or deadline at risk?}
    I -- Yes --> J[Declare SEV-2]
    I -- No --> K[SEV-3 investigation]

    J --> L{Durable external operation exists?}
    K --> L
    L -- Yes --> M{Terminal provider/publication outcome known?}
    M -- No --> N[Wait or poll only if eligible]
    M -- Yes --> O[Reconcile same operation or manifest]
    L -- No --> P{External effect may have occurred?}
    P -- Yes or unknown --> Q[Quarantine; investigate ambiguous create]
    P -- No --> R{Valid retained evidence exists?}
    R -- Yes --> S[Retry downstream stage only]
    R -- No --> T[Start bounded research only with approval and budget]

    O --> U{Current revision and lease match?}
    U -- No --> V[Reject stale action and reload]
    U -- Yes --> W[Commit one transition]
    W --> X[Run public synthetic verification]
    X --> Y{All acceptance probes pass?}
    Y -- Yes --> Z[Resolve and schedule review]
    Y -- No --> C

First ten minutes.

Operator process report, release proof, rollback, and postmortem

Bounded operator process report. This report is authenticated and richer than public status but still contains no prompts, source text, provider bodies, credentials, private paths, account IDs, reader data, or raw exception strings.

{
  "report_schema": 1,
  "generated_at": "2026-08-01T11:25:00Z",
  "overall_health": "degraded",
  "reader_impact": "edition_not_yet_overdue",
  "expected_target": "2026-08-01",
  "last_published_target": "2026-07-31",
  "deadline": {
    "model": "fixed_local",
    "timezone": "REDACTED_CONFIG_VALUE",
    "deadline_local": "CONFIGURED",
    "minutes_remaining": 42
  },
  "process": {
    "reference": "p_6C9A1D",
    "target_date": "2026-08-01",
    "state": "research_reconciling",
    "revision": 18,
    "age_seconds": 1840,
    "stage_age_seconds": 620,
    "quarantined": false
  },
  "lease": {
    "status": "valid",
    "expires_in_seconds": 73,
    "renewal_count": 2
  },
  "operation": {
    "class": "primary_research",
    "reference": "op_F3A9",
    "identity_present": true,
    "provider_state": "completed",
    "age_seconds": 1210,
    "last_poll_age_seconds": 540,
    "next_poll_eligible_in_seconds": 0,
    "reconciliation_required": true
  },
  "evidence": {
    "retained_sources": 18,
    "retained_candidates": 9,
    "selectable_candidates": 5,
    "corroborated_candidates": 3,
    "represented_regions": 3,
    "hard_floor_met": false,
    "deficits": ["corroboration", "regional_representation"]
  },
  "publication": {
    "state": "not_prepared",
    "current_public_event": "pub_20260731_7B1C",
    "english_spanish_parity": true,
    "artifact_integrity": "verified"
  },
  "deployment": {
    "web_release": "2026.08.01+8ae73a",
    "cli_release": "2026.08.01+8ae73a",
    "site_instance_count_observed": 1,
    "unexpected_releases": 0
  },
  "scheduler": {
    "heartbeat_age_seconds": 112,
    "expected_interval_seconds": 300,
    "last_result": "supervisor_tick_completed"
  },
  "recommended_actions": [
    {
      "action": "reconcile",
      "reason": "provider_completed_not_consumed",
      "safe": true,
      "expected_revision": 18
    }
  ],
  "prohibited_actions": [
    "start_new_primary_research",
    "force_publish",
    "delete_operation_reference"
  ],
  "report_integrity_hash": "sha256:410d…98ac"
}

The public status endpoint should expose only:

generated_at
service_state: healthy | delayed | unavailable | correcting
expected_target
published_target
publication_event_id or short public edition ID
revision
language_parity: true | false
artifact_integrity: verified | unknown
scheduler_freshness_band: current | delayed | unknown
release_channel: current | previous

It should not expose worker IDs, provider names or IDs, attempt internals, detailed evidence sources, private topology, error text, file locations, account information, or trace identifiers.

Release gate.

Canary strategy. A web canary may serve read-only traffic first. Only one release may hold workflow mutation authority at a time. Mutation authority should be a lease or configuration generation held by the approved release, not inferred from deployment order. The new release should first prove:

1. it reads current state and manifests;
2. its public status agrees with the current release;
3. its metrics and events have expected schema;
4. it rejects stale expected revisions;
5. its scheduler doctor is healthy;
6. it can process a deterministic isolated canary namespace;
7. mutation authority can transfer atomically;
8. the prior release is drained before new live work begins.

Rollback strategy. Rollback consists of three independent decisions:

A rollback is complete only when external synthetics see one release, one site identity set, one shared English/Spanish event, correct hashes, fresh bounded status, and a valid scheduler heartbeat.

Release-proof bundle.

release-proof/
  release-manifest.json
  source-package.sha256
  extracted-inventory.json
  php-lint-results.json
  schema-compatibility.json
  deterministic-test-results.json
  property-test-seeds.json
  concurrency-race-results.json
  crash-matrix-results.json
  package-safety-results.json
  loopback-http-transcript.json
  browser-proof/
  scheduler-doctor.json
  telemetry-schema-validation.json
  cardinality-estimate.json
  rollback-rehearsal.json
  canary-observations.json
  post-deploy-synthetics.json
  proof-bundle.sha256
  approval-signatures.json

Postmortem template.

# Daily Brief Incident Postmortem

## Impact
Affected targets, languages, hosts, reader-visible duration, and edition identity.

## Reader-visible symptoms
What readers actually saw: missing, stale, inconsistent, invalid, or corrected content.

## Detection
First signal, detection time, expected signal, and why earlier controls did or did not fire.

## Timeline
UTC and editorial-local timestamps for scheduler, reservations, provider events,
state transitions, publication events, cache observations, operator actions, and recovery.

## Violated invariant or SLO
Exact invariant, SLI, SLO target, and error-budget consumption.

## Root cause
The specific technical or procedural mechanism that directly produced the incident.

## Contributing conditions
Deployment, topology, permissions, cache, lease, test coverage, provider, and operational factors.

## Why tests passed while production failed
Identify the missing relationship, environment assumption, race, cache layer,
scheduler path, release path, or observation gap.

## Immediate mitigation
Actions taken, why they were safe, and evidence that duplicate work did not occur.

## Permanent correction
Code, infrastructure, test, telemetry, runbook, and editorial-policy changes.

## Regression proof
Deterministic test, race test, crash test, synthetic, or release gate that now catches the defect.

## Data and privacy review
Confirm that diagnosis and escalation did not expose prompts, sources, bodies,
credentials, private paths, account IDs, or reader data.

## Follow-up actions
Owner, deadline, priority, verification method, and closure evidence for every item.

A postmortem is mandatory for every SEV-1, every freshness-budget exhaustion, and any single incident consuming more than 20% of a period’s budget. That 20% policy follows the pattern in Google’s example error-budget policy, adapted here as an explicit owner decision rather than an industry mandate.

Reliability roadmap, research directions, and final production acceptance

Thirty-day foundation.

Sixty-day resilience.

Ninety-day operationalization.

Five concrete research and policy directions.

Unresolved evidence and conflicting design pressures.

Final production acceptance criteria. The Daily Brief is accepted as production-ready only when all conditions below have reproducible evidence:

One-page operator checklist.

• [ ] Confirm the expected target date, editorial timezone, deadline, and grace period.
• [ ] Capture canonical English and Spanish publication IDs, revisions, hashes, releases, instances, and cache ages.
• [ ] Determine whether the issue is reader integrity, freshness, workflow progress, scheduler, provider reconciliation, publication, cache, or release topology.
• [ ] Freeze nonessential deployments and mutating operator actions during SEV-1 or SEV-2 response.
• [ ] Read the bounded process report; record process reference, immutable target, current revision, state, stage age, lease, and operation reference.
• [ ] Never start a new provider operation while an existing or ambiguous operation may exist.
• [ ] Poll only when eligible; otherwise wait within the stage/deadline bound.
• [ ] Reconcile completed or ambiguous provider work before retrying.
• [ ] Reuse retained evidence for composition, validation, and publication retries.
• [ ] Reject or ignore every worker result whose process, target, operation, lease, or expected revision is stale.
• [ ] Quarantine on hash mismatch, bilingual divergence, repeated lease expiry, unknown external outcome, or concurrent mutating releases.
• [ ] For stale public data, compare origin and cache identities; purge or revalidate the exact cache object rather than republishing.
• [ ] For publication failure, verify the manifest and both artifacts before committing or restoring a canonical pointer.
• [ ] Never repair only one language; restore or commit a shared bilingual generation.
• [ ] After any action, run external checks for canonical host, target, revision, event, parity, hashes, release, instance, cache age, and heartbeat.
• [ ] Record the operator action with expected revision, result, reason, and integrity hash.
• [ ] Escalate with opaque operation references, timestamps, statuses, and error classes only—never prompts, raw sources, response bodies, credentials, private paths, account identifiers, or reader data.
• [ ] Resolve the incident only after reader-facing synthetics pass and recovery evidence is attached.
• [ ] Open a postmortem for SEV-1, freshness-budget exhaustion, unsafe duplicate-work risk, or a major detection gap.
• [ ] Convert the discovered failure into a deterministic, race, crash, browser, cache, synthetic, or release-gate regression test.