How an agent diagnosed 730 phantom 503s from Cloudflare Speed Brain

Chrome reported hundreds of 503 errors from one of our own SaaS apps, and not one user complained. A coding agent worked out why through our MCP server: Cloudflare Speed Brain was answering link prefetches with 503. One setting later, the reports stopped.

The setting

  • The app: a multi-tenant Rails app with Hotwire Turbo. Each customer has its own subdomain, and almost every page sits behind a login.
  • The edge: Cloudflare on the Free plan, proxying every tenant host. Speed Brain is on by default there, and nobody had turned it on knowingly.
  • The reports: the app sends Network Error Logging reports to Reporting API App. Chromium browsers file one whenever a request fails.

In the week of 5 October 2026, tenant traffic moved to a new host behind the Cloudflare proxy. Weekly 503 reports jumped from about 2 to between 550 and 1,600. By mid-November the largest of these issues, one route on one tenant subdomain, held 730 reports. Users reported nothing.

The original session

On 14 November we asked a coding agent to find out where the 503s came from. It had the Reporting API App MCP server, read access to Cloudflare's API, and the app's code.

It walked list_issues → list_reports → get_report, and the raw report bodies held everything it needed. Getting to them was slow:

  • One call per report. list_reports did not show status_code, phase or server_ip. Telling a 503 from a 404 took one get_report call per row.
  • Hand arithmetic. Browsers queue these reports: the age field was up to 90 seconds. To match a report against Cloudflare's log, the agent had to work out by hand when each error happened.
  • One failure, dozens of issues. Issues grouped by URL, so the same failure split across /publications/<id>, file downloads and paginated lists on several tenant subdomains.
  • Whole days only. Date filters took days in Europe/Berlin. Matching against CDN logs needed minutes in UTC.

Despite the friction, every sampled report showed the same signature:

  • http.error in the application phase, GET, status 503
  • elapsed_time of 10 to 50 ms
  • server_ip in Cloudflare's published IP ranges
  • Chromium browsers only: Chrome and Edge
  • The referrer was always a page that links to the reported URL

The agent's hypothesis: a 503 that fast, from a Cloudflare address, came from Cloudflare itself, not from the Rails app behind it.

From there it switched to Cloudflare's side:

  1. Cloudflare's HTTP request analytics had no 503 for the tenant hosts. The agent queried the GraphQL dataset httpRequestsAdaptive for 24 hours. Its only 503s were real origin errors, from an uptime monitor during a reboot.
  2. The zone setting speed_brain was on. That is the Free plan default.

Cloudflare's documentation closed the case. Speed Brain adds a Speculation-Rules header to your HTML. Its rules tell Chromium to prefetch same-site links with conservative eagerness: when a visitor presses down on a link. Cloudflare answers those prefetches from its cache only and never forwards them to the origin, and "prefetches that are not successful will respond with a 503". Every page behind a login is a cache miss. So the prefetch got a 503, Chrome filed a report, and the click that followed reached the origin and worked as normal.

A second check on 16 November matched the reports for one paginated list against Cloudflare's request analytics. For each 503 report, the only request Cloudflare had logged with the same URL, client and second was a successful one: 200 at the edge and from the origin. The 503 the browser saw appears nowhere in Cloudflare's data. The same clients had fetched /cdn-cgi/speculation, Speed Brain's prefetch rules.

Where the evidence stops: A network error report carries no request headers, so the sec-purpose: prefetch header never shows in the report itself. The proof is the signature, Cloudflare's log and the reports stopping after the fix.

What we built from it

The agent ended its session by filing its friction as feedback through the MCP server. Most points became features:

  • When the error happened: every report carries occurred_at, arrival time minus age, and filters and counts use it. Date filters accept UTC date-times to the second.
  • Network error fields in lists: list_reports returns status_code, phase, protocol, elapsed_time and server_ip, and filters by them.
  • Distributions on the issue: get_issue and the issue page show the status codes, server IPs and elapsed times of an issue's reports.
  • Counts in one call: count_reports answers "is it new, and is it growing?" by day, hour, issue, host or status code.
  • The diagnosis in the docs: a help article on Speed Brain 503s, which agents read through get_documentation.
  • One issue across subdomains: reports from tenant subdomains that match a wildcard origin join one issue.

The same diagnosis today: a replay

On 18 November we replayed the diagnosis against the same production data with today's tools. These are real, read-only calls, not the original session. We trimmed the output and replaced every ID, host and IP. The window is the last working week before the fix, 9 to 14 November. Issues now group tenant subdomains, so the IDs and counts differ from the original session: the top issue below counts 1,810 reports since June, including an earlier, unrelated failure on the previous host.

The Cloudflare Speed Brain noise rule now filters most of these reports as known noise. To replay the calls on your own data, add include_filtered: true to list_issues and count_reports.

1. Which issues are active?

Call: list_issues(type: "network-error", start_date: "2026-11-09", end_date: "2026-11-14")

{
  "total_count": 50,
  "issues": [
    { "id": "issue-1", "summary": "https://tenant-a.example.com/publications/<id>",
      "detail": "http.error", "reports_count": 1810, "window_reports_count": 219 },
    { "id": "issue-2", "summary": "https://tenant-a.example.com/publications/<id>/edit",
      "detail": "http.error", "reports_count": 761, "window_reports_count": 73 },
    { "id": "issue-3", "summary": "https://tenant-a.example.com/publications/files/<uuid>",
      "detail": "http.error", "reports_count": 515, "window_reports_count": 50 },
    …
  ]
}

Agent: Fifty network-error issues, nearly all http.error. The busiest has 219 reports this week. Look at what is inside it.

2. What do its reports have in common?

Call: get_issue(issue_id: "issue-1", start_date: "2026-11-09", end_date: "2026-11-14")

{
  "window_reports_count": 219,
  "distributions": {
    "browsers": [{ "value": "Chrome", "count": 219 }],
    "status_codes": [{ "value": 503, "count": 219 }],
    "elapsed_times": [
      { "value": "under_100ms", "count": 207 },
      { "value": "100ms_to_1s", "count": 12 }
    ],
    "server_ips": [
      { "value": "<cloudflare-edge-1>", "count": 125 },
      { "value": "<cloudflare-edge-2>", "count": 60 },
      …
    ]
  },
  "sample_report": {
    "occurred_at": "2026-11-14T16:32:16Z",
    "received_at": "2026-11-14T16:33:46Z",
    "age_ms": 90418,
    "network_error": {
      "status_code": 503, "phase": "application", "protocol": "h3",
      "elapsed_time": 44, "server_ip": "<cloudflare-edge-1>"
    },
    "body": { "method": "GET",
      "referrer": "https://tenant-a.example.com/publications?query=…" }
  },
  "daily_counts": [
    { "date": "2026-11-09", "count": 99 }, { "date": "2026-11-10", "count": 47 },
    { "date": "2026-11-11", "count": 44 }, { "date": "2026-11-12", "count": 13 },
    { "date": "2026-11-13", "count": 15 }, { "date": "2026-11-14", "count": 1 }
  ]
}

Agent: Every report is a 503, nearly all under 100 ms, the server IPs Cloudflare edge addresses. The referrer is a search results page that links to the reported publication. A fast 5xx from a CDN edge means the CDN answered. Is this one route, or the whole app?

3. How far does it reach?

Call: count_reports(type: "network-error", status_code: 503, max_elapsed_time: 100, start_date: "2026-11-09", end_date: "2026-11-14", group_by: ["issue"])

{
  "total_count": 581,
  "groups": [
    { "issue": "issue-1", "count": 207 },
    { "issue": "issue-4", "count": 86 },
    { "issue": "issue-2", "count": 73 },
    { "issue": "issue-3", "count": 50 },
    { "issue": "issue-5", "count": 40 },
    …
  ],
  "truncated": true
}

Agent: 581 fast 503s in one week, spread over more than ten routes: publications, edits, file downloads, lists. One cause across the app, not one broken page. What do the docs say about this signature?

4. What does the signature mean?

Call: get_documentation(path: "<the issue's docs_url>")

# Network Errors (NEL)
…
### Fast 503s from Your CDN

An `http.error` in the `application` phase with a tiny `elapsed_time` and a
`server_ip` that belongs to your CDN means the CDN answered, not your origin.
Behind Cloudflare, `503`s under about 100 ms usually come from failed Speed
Brain prefetches that visitors never see. See [Fast 503 Reports from
Cloudflare Speed Brain](/en/docs/report-types/network-errors/cloudflare-speed-brain).

Agent: The signature matches. Next: check the zone's speed_brain setting in Cloudflare and match a report's occurred_at against Cloudflare's request log.

Four calls get to the likely cause. The time arithmetic is done, and the fields that made the diagnosis sit on the issue. The help article in step 4 exists because of this case; without it, steps 1 to 3 still say "the CDN answered". Confirming the cause on Cloudflare's side remains the agent's job.

The fix

We turned Speed Brain off on all our Cloudflare zones through OpenTofu. The change went live on 15 November 2026 at 09:41 UTC.

We did not measure Web Vitals before and after. We expected no loss, for three reasons:

  • Speed Brain only helps HTML that Cloudflare can cache. Pages behind a login never are, so every prefetch missed.
  • Turbo Drive turns link clicks into its own fetch() requests, which a speculation-rules prefetch does not speed up. The same holds for any client-side router. Turbo 8 already prefetches links on hover itself, against the origin, which can answer pages behind a login.
  • On the app's marketing site the gain was small, and one setting for every zone is simpler than exceptions.

A server-rendered site with cacheable public HTML is where Speed Brain can pay off: measure before you turn it off there. To keep it, send your own Speculation-Rules header from the origin, which Cloudflare does not override, or make public HTML cacheable. The help article covers every option.

The result

Fast 503s (status 503, under 100 ms) per day, next to all network error reports. Counted with count_reports on 18 November, by when the error happened:

Day (2026) Fast 503s All network errors
Mon, 9 Nov 254 270
Tue, 10 Nov 101 124
Wed, 11 Nov 128 145
Thu, 12 Nov 49 70
Fri, 13 Nov 47 54
Sat, 14 Nov 2 3
Sun, 15 Nov (fix at 09:41 UTC) 0 0
Mon, 16 Nov 0 17
Tue, 17 Nov 0 10

The first two working days after the fix had no fast 503s, against 47 to 254 on each weekday before. Other network errors held steady: 7 to 23 a weekday before the fix, 17 and 10 after. They were 406s, 422s and 403s from the origin.

Takeaways

  • Behind Cloudflare's Free plan, check for Speed Brain. If your HTML sits behind a login, your browsers may be filing the same reports right now.
  • A fast error from a CDN address is the CDN's answer. Read elapsed_time and server_ip before you look at your origin.
  • Agents need aggregates, not pages of rows. The original session spent most of its calls fetching reports one by one. The replay needed four.

Connect your coding agent through the MCP server, included from the Pro plan up.