I was writing a deliberately paranoid RDAP client. Before it sent a domain query, it downloaded the Public Suffix List and the official IANA RDAP bootstrap file, computed their hashes, and validated every provider it might contact. A completed run would save both snapshots and their hashes.

The first live run sent exactly zero domain queries:

validate pinned IANA bootstrap: bootstrap service 0 contains invalid URL

It had made two requests: one for the suffix list, one for IANA's JSON. Then my validator killed the run.

The URL was valid. My policy did not like it. Those are not the same thing.

The invalid URL was valid

RDAP is the JSON replacement for WHOIS. Before querying a domain's registration data, a client must find its authoritative service using the longest matching entry in IANA's bootstrap registry. RFC 9224 defines the format, and IANA publishes the live DNS registry at data.iana.org/rdap/dns.json.

The first service in the snapshot I downloaded was this:

[
  ["kg"],
  ["http://rdap.cctld.kg/"]
]

My collector is HTTPS-only. Port 443, no downgrade, no exceptions. The parser reused that execution rule while validating the bootstrap document, saw http://, and declared the entry malformed.

Except RFC 9224 explicitly allows service arrays to contain HTTP and HTTPS URLs. It tells clients to prefer the secure version when one exists. The IANA file was representing its published registry data; my validator was grading it against the world I wanted.

The snapshot had one more HTTP-only entry, for .mg. Rejecting the whole document because of those two services also discarded 1,198 unrelated mappings with at least one provider that passed the collector's HTTPS URL checks.

The correct fix was not to allow HTTP. It was to separate parsing from dialing:

if strings.EqualFold(u.Scheme, "http") {
    sawInsecure = true
    continue
}
if validateTargetURL(u) != nil {
    sawInvalid = true
    continue
}
urls = append(urls, rawURL)

// Later, preserve the mapping even when this policy can dial no URL.
b.services[key] = append([]string(nil), urls...)
if len(urls) == 0 && sawInsecure && !sawInvalid {
    b.unavailable[key] = "insecure_bootstrap_service_only"
}

The parser now retains the TLD mapping but gives it no dialable provider under this policy. If every advertised URL is HTTP, it records insecure_bootstrap_service_only. The rest of the registry remains usable.

HTTP-only mappings became representable as unavailable; HTTP providers remained unreachable.

My generous limit wasn't generous

I fixed that, ran the probe again, and reached the next assumption:

validate pinned IANA bootstrap: bootstrap service 113 has invalid shape

“Invalid shape” sounded structural. The service was a two-element array containing an entry array and a URL array, exactly as the RFC specifies. The problem was a limit I had invented: no single service could contain more than 100 TLD keys.

That felt generous when I typed it.

In the IANA snapshot published on 23 July 2026, service 113 grouped 451 TLD labels under one endpoint:

https://rdap.identitydigital.services/rdap/

The group started with abb, abbott, abc, and academy, and ended with yahoo, zara, zero, and zone. RFC 9224 groups every entry sharing the same set of base URLs. It specifies no 100-entry maximum.

The bootstrap file had apparently not consulted me before grouping 451 labels under one Identity Digital endpoint.

The index 113 is only its position in that dated snapshot, not a stable service identifier. The useful fact was the distribution: one legitimate provider group was more than four times larger than my “defensive” maximum.

Keep the bounds, lose the guess

The answer was not to remove every limit and hope IANA stayed friendly forever. This is still downloaded JSON, and a parser still needs a finite resource budget.

The client already limited the document to 2 MiB. I kept five guards and raised the per-service key budget from 100 to 2,000:

  • at most 2 MiB of bootstrap JSON;
  • at most 2,000 service rows;
  • at most 2,000 keys and 10 provider URLs in one service;
  • at most 10,000 total mappings; and
  • at least one usable HTTPS mapping before the document can drive a run.

Those numbers protect memory and work. They are local processing limits, not claims about the RDAP format. The regression test uses 101 synthetic keys: just enough to cross the old boundary without freezing IANA's changing 451-key production group into a fixture.

Validation and policy are different jobs

I had collapsed three questions into one error:

  1. Is the document structurally valid?
  2. Is this entry allowed by my execution policy?
  3. Is the document within my resource budget?

They need different answers.

A malformed service entry can invalidate the source. A mapping whose only advertised provider is syntactically valid HTTP can remain represented as unavailable, while that URL stays outside the dialable provider list. A document beyond the local processing budget can fail closed as a local limit rather than pretending the RFC forbids its size.

External data often contains values an application must understand but must not execute. If “I refuse to use this” becomes “this cannot exist,” one unusual entry can turn a safe policy into a global outage.

The more precise state was useful too. An HTTP-only mapping is not the same as no IANA mapping. Keeping insecure_bootstrap_service_only means the report can explain what happened instead of collapsing both cases into “unsupported.”

The run that finally reached a domain

On the third attempt, the IANA snapshot published on 23 July 2026 parsed as:

  • 590 service records;
  • 1,200 suffix mappings;
  • 1,198 mappings with at least one usable HTTPS provider;
  • two mappings unavailable under the HTTPS-only policy; and
  • one largest provider group containing 451 suffixes.

Then the probe finally did its actual job. It made 52 bounded HTTP transactions: two registry downloads and 50 authoritative domain queries. No HTTP-only provider was contacted.

I changed what the parser could describe. I did not change what the collector was allowed to do.

Takeaway

Strict validation is only useful when it is strict about the published contract. The moment it starts enforcing what you assume production data ought to look like, it becomes fragility with a security-sounding name.

Validate the document against its actual format, then enforce your execution policy where execution happens. A guessed production shape is not a security boundary. Run against the boring official dataset before congratulating your synthetic fixtures for passing.

Unexpected is not a synonym for invalid.

Did production data break a perfectly reasonable validator? Send me the error and I'll take a look.