Public Reliability and Recovery Radar

Full report

On this page

Executive summary

This bounded review looked only at resolved public incidents and public postmortem-style artifacts from four organizations that provide enough official English-language material to support a comparative read: OpenAI, GitHub, Cloudflare, and Google Cloud. The status-page hosts used were OpenAI Status, GitHub Status, Cloudflare Status, and Google Cloud Service Health; the analysis was supplemented only with each provider’s own incident write-ups, availability reports, or engineering blog postmortems.

The strongest pattern was not “random outage.” It was internal change risk. Across the reviewed incidents, the initiating event was usually a routine deployment, maintenance action, configuration change, feature-flag rollout, dependency update, or operator action: OpenAI’s ingress and telemetry changes, GitHub’s header rollout and dependency auto-consumption, Cloudflare’s abuse-remediation and routing/configuration errors, and Google Cloud’s feature-flag, maintenance, and safety-service configuration changes. The clearest non-change exceptions in this sample were Cloudflare’s June 2025 third-party-backed Workers KV outage and Google Cloud’s March 2025 us-east5-c utility/UPS failure.

The second dominant pattern was dependency concentration. Small changes repeatedly turned into multi-product incidents when they touched a load-bearing shared system: Kubernetes control plane and DNS at OpenAI, authentication/user-management and messaging systems at GitHub, Workers KV and Addressing/routing systems at Cloudflare, and GCE control paths, network control-plane state, or Gemini safety filtering at Google Cloud. Publicly, Cloudflare and GitHub now expose the clearest cross-incident remediation programs; Google Cloud publishes strong incident reports but less often shows a long-running public recovery program after the first report; OpenAI’s write-ups are often detailed for the incident at hand, but the public program is still more incident-scoped than programmatic.

For a bounded reliability review or a narrowly scoped ADR, Cloudflare and Google Cloud provide the strongest official artifacts in this sample, GitHub is close behind when the scope includes growth-related coupling and service isolation, and OpenAI is usable for incident-specific review but still light for broader release-readiness work. The limiting factor is not root-cause opacity alone; it is whether the public material shows a repeatable loop of incident → dependency analysis → concrete mitigations → follow-up evidence. Cloudflare shows that loop most clearly, GitHub increasingly does, Google Cloud does it well at the incident level, and OpenAI does it unevenly.

Scope and method

This review is intentionally bounded. I included only incidents that were publicly marked resolved and had enough official detail to extract a usable reliability signal. I excluded unresolved events, social-media summaries, and third-party writeups. Where a provider published both a status entry and a postmortem or availability report, I favored the richer official artifact.

A few timing notes matter. “Mitigation” means the first published milestone showing material recovery or successful rollback; “full recovery” means the provider’s stated end of customer impact. When a provider gave only impact windows rather than a formal mitigation timestamp, I marked the mitigation time as inferred or NR when public materials did not support a credible estimate. This matters because public transparency quality is itself part of the reliability signal.

One comparability caveat: public status metrics are not fully normalized across providers. OpenAI explicitly says its published availability metrics are aggregated across tiers, models, and error types, and that individual customer availability may vary; Google Cloud’s public Service Health also distinguishes it from project-specific Personalized Service Health. Those caveats make this a reliability-and-recovery radar, not a vendor SLO ranking.

flowchart LR
    A[Routine deployment, maintenance, or config change] --> B{Touches a shared dependency?}
    B -->|Yes| C[Control plane, identity store, storage, routing, policy, or safety service]
    C --> D[Multi-product blast radius]
    D --> E[Customer-visible outage or degradation]
    E --> F[Partial mitigation]
    F --> G[Full recovery delayed by backlog, stale state, reindexing, or manual restoration]

    H[Third-party outage or facility failure] --> C
    I[Observability blind spot] --> D
    I --> G
    J[Circular dependency in admin tooling or rollback path] --> G

That dependency shape is the common thread in the official reports: change events become broadly visible when they hit shared control paths, and recovery slows when monitoring, rollback access, or restoration tooling depends on the same degraded substrate.

Organization findings

OpenAI

\* inferred from the published two-wave impact windows rather than an explicit “mitigated” milestone.

The recurring OpenAI pattern is internal infrastructure change hitting control-plane assumptions rather than a steady stream of unrelated one-offs. In August 2024 the initiating issue was ingress/control-plane regression; in December 2024 it was telemetry load against Kubernetes API servers; in June 2025 it was a routine OS update that broke node networking. The surface symptoms differ, but the structural pattern repeats: shared substrate problems, incomplete rollout safety, and recovery paths that were not fast enough when foundational systems were unhealthy.

OpenAI’s public artifacts are genuinely useful for incident-scoped review because they explain root cause, the main dependency chain, and concrete preventive actions. What is thinner, compared with Cloudflare or GitHub, is the public follow-through layer: there is less recurring publication that shows the status of long-running reliability programs, completion evidence, or cross-incident trend accounting. That does not make the write-ups poor; it makes them episodic rather than a fully visible public recovery program.

GitHub

GitHub’s reviewed incidents show two overlapping patterns. The first is the same one seen elsewhere: release and change safety failures. The second is more platform-specific: GitHub publicly attributes its recent turbulence to rapid growth, architectural coupling, and insufficient load shedding. In March 2026, GitHub said recent instability was driven by rapid load growth, coupling that let localized issues cascade, and inability to shed load from misbehaving clients; in April 2026 it said the platform needed to be designed for 30x present-day scale rather than the 10x plan started in October 2025.

That matters for radar purposes because it distinguishes one-off service flaws from a broader operating pattern. The May 2025 CORS/header issue looks like an isolated feature rollout problem. The April and June 2026 incidents, together with GitHub’s own platform-wide explanation, look more systemic: a fast-growing platform reworking service isolation, gateway rate limiting, critical-path offloading, and blast-radius control in public view. GitHub is also unusually explicit that it publishes both status-page recaps and monthly availability reports, which materially improves trend visibility.

Cloudflare

Cloudflare shows the clearest recurring operating pattern in this sample, and the strongest public learning loop. The recurring pattern is not just “configuration error.” It is configuration or operator error plus large blast radius created by shared internal platforms or legacy control paths. February 2025 and March 2025 were both R2 human-error/configuration incidents. June 2025 exposed a major single-dependency hazard in Workers KV. July 2025 and February 2026 showed that routing/addressing systems and legacy deployment methods still carried unacceptable blast radius.

What separates Cloudflare is the public remediation program that followed. After the Workers KV outage, Cloudflare published a dedicated redesign update saying KV data was now stored on Cloudflare’s own infrastructure and served from Cloudflare infrastructure with third-party providers used only for redundancy. After the BYOIP outage, it tied the incident into the broader Code Orange: Fail Small initiative and published specific design changes for schema validation, rollback snapshots, and circuit breakers. That is unusually strong evidence for a bounded review because it provides both postmortem detail and follow-up architecture movement.

Google Cloud

Google Cloud’s sample has a split personality in a useful way. One event is clearly environmental and facility-linked: the March 2025 power/UPS incident. The other three are classic change-safety and control-path incidents: feature-flag rollout failure, maintenance/procedure failure interacting with network topology, and configuration change to a shared safety service. So the operating pattern is not “Google Cloud has power issues”; it is that even in a very large platform, shared control systems and rollout safety remain a recurring public risk, with a separate but real physical-infrastructure tail.

Google Cloud’s public incident reports are among the strongest in this sample for incident-level detail: summary, root cause, remediation, prevention, and impact notes are all usually present. The limitation is that the public material is often strongest at the incident-report layer rather than as an explicit long-running program narrative. That is still enough for a bounded reliability review, but it is a little thinner when you want public proof that a remediation campaign was completed across the relevant platform area.

Cross-organization trends

The sample’s recovery data clusters around a familiar operational shape: partial mitigation comes much sooner than full recovery, and the delta between them often reflects stale state, backlog drain, progressive safe rollout, manual restoration, or index rebuild rather than the original trigger. That is very visible in OpenAI’s June 2025 incident, GitHub’s April 2026 code-search event, Cloudflare’s July 2025 routing outage and February 2026 BYOIP outage, and Google Cloud’s March and May 2025 incidents.

xychart-beta
    title "Time to Full Recovery Across the Reviewed Sample"
    x-axis ["<1h","1-2h","2-4h","4-8h",">8h"]
    y-axis "Incident count" 0 --> 6
    bar [1, 6, 2, 4, 3]

xychart-beta
    title "Median Time to Full Recovery by Organization"
    x-axis ["Cloudflare","GitHub","Google Cloud","OpenAI"]
    y-axis "Hours" 0 --> 5
    bar [1.12, 4.03, 4.07, 4.37]

Several cross-provider trends stand out.

First, change-management gaps dominate. The most common failure class was an internal change that either lacked staged validation, bypassed safety checks, or touched a hidden shared dependency. OpenAI explicitly said it lacked enough Kubernetes API server load checks and robust phased rollout safeguards. GitHub described aggressive or insufficiently validated upgrades and dependency consumption. Cloudflare repeatedly described missing tooling guardrails, incorrect environment targeting, and risky or legacy deployment methods. Google Cloud explicitly documented failed rollout safety checks and insufficient validation checkpoints.

Second, observability limits and blind spots recur in ways that matter operationally. OpenAI’s December 2024 write-up said DNS caching delayed visible failures and that rollout monitoring lacked sufficient cluster-health protocols. Cloudflare’s March 2025 R2 rotation incident lasted longer because the team lacked clear visibility into which credential production was actually using, and its February 2026 BYOIP remediation work includes much stronger change-rate monitoring and circuit-breaker logic. GitHub said it lacked sufficiently granular alarms and later moved to add more detail to the status page. Google Cloud’s June 12, 2025 incident report said even Cloud Service Health infrastructure was impaired by the outage, and that some customers also lost monitoring signal because their monitoring lived on Google Cloud.

Third, data-integrity concerns were usually contained, but correctness and freshness still mattered. Cloudflare explicitly said there was no data loss in the February 2025 R2 outage and June 2025 Workers KV outage. GitHub explicitly said no repository data was lost during the April 2026 code-search outage because search is a secondary index, but users still saw stale results until reindexing completed. OpenAI’s reviewed incidents were availability and capacity events rather than data-corruption events. Google Cloud’s March 2025 zonal outage included unavailable disks and manual failover for some services, which is not broad public evidence of corruption, but it is a correctness and durability caution for zonal assumptions.

Fourth, one-off versus recurring pattern is easier to call when the same risk shows up in multiple write-ups. The best “likely one-off” incidents in this sample are Google Cloud’s March 2025 utility/UPS failure and GitHub’s narrow July 2025 GEI incident. The strongest recurring patterns are OpenAI’s control-plane/change-safety issues, GitHub’s growth-plus-coupling and release-safety issues, Cloudflare’s operator/configuration/legacy-control-path blast-radius problems, and Google Cloud’s production change-safety problems affecting shared control systems.

Suitability for bounded reviews and engagement types

The key question is not whether a provider had outages. All four did. The question is whether, after a cooling-off period, the public record is good enough to support a disciplined external exercise without private access.

The most important distinction is this: bounded reviews are realistic when the public record exposes the dependency chain and the mitigation program. Broad “platform is ready” claims are not. Cloudflare’s and GitHub’s materials are especially suitable for a bounded review of design choices around shared storage, addressing systems, service isolation, dependency pinning, gateway rate-limiting, or failure containment. Google Cloud’s public artifacts are good enough for bounded review of regional architecture, maintenance controls, and shared-service protection. OpenAI’s are best for narrower incident-family ADRs rather than broad release-readiness claims.

Recommendations and checklist

The most useful public-facing recommendation is not “publish more RCAs.” It is to publish the specific artifacts that let an outsider test whether the organization has actually reduced recurrence risk. Across the sample, the following public signals mattered most: whether the provider named the shared dependency, whether it separated mitigation from full recovery, whether it disclosed observability blind spots, whether it stated data-integrity/correctness impact explicitly, and whether it later reported architectural follow-through rather than leaving remediation as promise-only. Cloudflare’s follow-up Workers KV redesign post is the clearest positive example of that last step; GitHub’s capacity/isolation posts are another.

A practical checklist for bounded reliability reviews, ADRs, and release-readiness engagements should ask for the public evidence below before treating the incident record as decision-grade.

If you want to use public artifacts for an ADR or release-readiness decision, the safest posture is to limit the scope to a specific failure family. Good examples from this sample would be: “Should this service keep a hard dependency on shared KV/configuration state?”, “Should we allow automatic dependency consumption on a critical path?”, “Can our control-plane failure be made non-load-bearing for request serving?”, or “Do our routing/addressing changes require health-mediated staged deployment?” Those are the decision shapes the public record can actually support.

A final radar reading: Cloudflare currently shows the most decision-useful public recovery program in this sample; GitHub shows the strongest publicly visible transition from incident explanation to platform-wide capacity and isolation work; Google Cloud is excellent for incident anatomy and good for bounded review, but weaker on visible long-tail follow-up; OpenAI provides meaningful incident write-ups, but a bounded review should stay narrow because broader public architecture and remediation continuity are still comparatively sparse.