My analytics pipeline pulled App Store install data by matching a report named "App Store Downloads". Apple renamed that report to "App Downloads" with no changelog. The substring match stopped matching, the code fell back to a different report that has zero install rows, and zero rows got read as a real, legitimate zero. On its own that makes a number too low. What made numbers too high was a separate aggregation quirk that landed on top of the broken match, and for a stretch of days several apps reported install counts up to roughly 70x their real values. Wrong in both directions, and everything looked fine.

Reported installs versus real installs, drawn to scaleTwo bars at a true seventy to one ratio. The reported bar is full height; the real bar is a sliver barely thicker than the baseline. Nothing in the pipeline compared them.reportedfabricatedrealsame scale70×
Both bars are at the same scale. Reported installs ran up to ~70× the real number, and nothing in the pipeline was comparing the two.

How I found it

The numbers looked too good. One app's dashboard showed a burst of installs on a single day that did not line up with anything else I knew about it. I opened App Store Connect's own UI and compared, and the real number for that day was a fraction of what my pipeline reported. A different order of magnitude, not noise.

The bug

Apple's Analytics Reports API exposes several named reports. My code picked the installs report by matching its name:

SOURCE_REPORT_HINTS = ("app store downloads",)   # substring match

report = next(
    (r for r in available_reports
     if any(h in r.name.lower() for h in SOURCE_REPORT_HINTS)),
    None,
)
if report is None:
    report = fallback_report   # a different report, no install rows

Two things stacked up. First, Apple renamed the report. "App Store Downloads" became "App Downloads", the word "Store" dropped out, my substring stopped matching anything, and report came back None. Second, the fallback. When the name match failed, the code fell through to a different report that has no install rows, and its downstream logic saw zero matching rows and treated that as a real, genuine zero. A failed lookup and a true zero produced the same value, and nothing told them apart.

So the pipeline confidently emitted wrong numbers, and not in a single direction. Where the fallback lined up, it published a false zero. On other days a separate aggregation quirk stacked on top of the broken match and pushed a day's count far past reality, which is where the 70x came from. No alert fired either way, because from the code's point of view it had pulled a report and found some rows.

A fail-silent cascade from a one-word renameSix sequential steps: a substring match on a renamed report fails silently, a fallback returns zero rows, and zero is read as a real value, so the dashboard publishes false zeros. A separate aggregation quirk on top of the same broken match inflated other days.match report named "App Store Downloads"Apple renames it → "App Downloads"no changelogsubstring match failsno error raisedfallback query returns 0 rows0 rows read as a clean, real zerodashboard reports wrong numbers, both ways
Six reasonable steps, one silent failure, and a dashboard that lied in both directions.

A fail-silent fallback turns a missing signal into confident, wrong data.

The fix

The immediate fix was to match both names:

SOURCE_REPORT_HINTS = ("app store downloads", "app downloads")  # + newer Apple naming

That is a patch. The habit worth keeping is to fail loud when a name match that should find exactly one report comes back with zero, instead of falling through to something plausible. Anywhere you string-match a vendor's field or report name, a one-line assertion that the match count did not drop to zero would have caught this the day Apple shipped the rename. The only reason I caught it at all was that a number looked wrong and I opened the vendor's own dashboard, which should not be the detection system. The same shape shows up whenever a passing check hides wrong data, like the monitoring guard that froze a dashboard for three days. Vendors rename fields without a changelog. Their internal report names were never a contract I signed, and code that matches on them instead of a stable ID breaks the next time they do.