Fixing Our DNS Leak Rate: 39% to 17.4%, and What the Correction Teaches

Last week's intelligence weekly #3 carried an ugly number: the DNS mismatch rate jumped from 17% to 39% overnight. We couldn't identify the cause, so we published all three plausible explanations — more real leaks, false positives from public DNS, sampling noise — and promised to separate public anycast resolvers from the leak statistic in the next version.

That's done. New window: confirmed leak rate 17.4%, public DNS usage 5.8%.

What was wrong with the old measurement

The original rule was one line: if the resolver IP's country differs from the visitor's HTTP exit country, count a leak.

The flaw is the assumption that "resolver IP location = where resolution happened." True for a local ISP resolver. Completely false for public anycast DNS.

Anycast means 8.8.8.8 has hundreds of nodes worldwide. When you're in Shanghai, Google's Shanghai or Beijing node answers — but the IP database only knows the address is registered in the United States. So "Shanghai user + US resolver IP" got recorded as a leak, when the actual lookup happened a few kilometres away.

The more people use public encrypted DNS, the more of these misclassifications appear. The old metric couldn't separate how much of 39% was real and how much was noise.

A counterintuitive detail: 8.8.8.8 never queries you directly

The obvious first implementation was an IP allowlist of public resolvers (8.8.8.8, 1.1.1.1 and so on), skipping any match.

Then we ran an end-to-end test and it demolished that plan: the resolver IP we captured was 172.217.42.28, org "Google LLC" — not 8.8.8.8 itself.

The reason: when a public resolver queries an authoritative server, it comes from its own infrastructure egress, not from the address it advertises to users. You ask 8.8.8.8; Google asks our authoritative DNS from 172.217.x.x. A pure IP allowlist would miss every major public resolver.

The final approach is two-track: an IP allowlist plus organization-name matching (Google, Cloudflare, Quad9, Alibaba, Tencent, Baidu, Yandex and others). Org name is the stronger signal because it reflects who owns the resolver, not which address it happened to use.

The new classification

Every check now lands in one of three buckets:

Verdict	Condition	Counted as leak
Consistent	Resolver country = exit country	No
Confirmed leak	Mismatch, resolver is not public DNS	Yes
Indeterminate	Mismatch, resolver is public anycast DNS	No (reported separately as public DNS usage)

The third bucket is the point: it is neither "leak" nor "safe." It's "this signal is uninformative here."

Why "indeterminate" must be reported separately

This is the methodology point worth dwelling on.

Faced with 39%, there are two ways to handle it:

Option one (wrong): drop indeterminate samples from the denominator. It looks like "cleaning" the data, but it inflates the leak rate — the dropped samples skew toward well-configured users, leaving a population with a higher leak share. Textbook survivorship bias.

Option two (what we did): leave numerator and denominator untouched, publish an extra metric. The leak rate keeps its full denominator; only false positives are removed from the numerator; and the share of indeterminate samples (5.8%) is reported alongside, so readers can judge whether it undermines the conclusion.

The benefit: when public DNS usage reaches 40%, you immediately understand that the leak rate has limited meaning for that population — instead of being misled by a number that looks precise.

It's the same principle across all our metrics: better to publish one more number than to dress a mixed-meaning figure up as a precise conclusion.

Reading the new numbers

• Confirmed leak rate 17.4%: one in six checks has DNS egress and HTTP egress genuinely in different countries. That sits in the same range as the earlier 14% and 17% — far more plausible than 39%.
• Public DNS usage 5.8%: about one in seventeen visitors uses public anycast DNS. Their DNS geography is uninformative, but that doesn't imply risk — many are privacy-conscious DoH users.
• One limitation we must state: this week's window (November 9–15) straddles the switchover — the fix shipped on the afternoon of the 12th, so the first three days still use the old counting. The 17.4% therefore retains some historical false positives; next week's window is the first fully clean dataset.

What this means for you

The DNS leak test now tells you which of the three verdicts applies, and which resolver answered:

• "Consistent" — your DNS and HTTP exits agree; nothing to fix
• "Leak" — a local ISP resolver is answering directly; that's a real issue, see the DNS leak explainer
• "Public anycast DNS, location not comparable" — you're on public DoH; this dimension is not a risk signal for you

Detection runs server-side under the same logic; statistics remain anonymous counters with no individual IP or fingerprint stored — see the privacy policy.

What's next

Next week's intelligence weekly #4 will re-run the trend on the first fully clean dataset. If the confirmed leak rate holds in the mid-teens, the 39% episode can be formally filed as a corrected measurement error rather than an unresolved anomaly.

Numbers going wrong isn't the problem. Publishing them, then fixing the method and documenting the process, is worth more than a pretty figure from day one.