[WIP] Match Mozilla Necko's DNS cache

[da2x]

Match Firefox's DNS cache of 400 entries for up to 2 minutes. Mozilla experimented in 2018 with doubling their DNS cache from 400 to 800, but saw negligible improvements in cache hit ratios.

12 hours is much too long. Serving stale content for up to half a day isn’t a great user experience. Browsers cache TTL naïvely for 15 sec to 2 min. (See my DNS TTL client survey for details.)

Note that Windows DNS Cache, macOS’ mDNSResponder, and so on may cache DNS responses for up to their TTL so this change shouldn’t be a massive performance reduction. Negative DNS responses (websites without DNSLink) mybe a different story, however as caching is usually limited to just a few seconds to a minute.

Optimization potential:

• Cache negative DNS responses (no DNSLink found) for longer than positive responses. Reduces unnecessary lookups for websites without IPFS.
• Implement DNS TTL-aware caching.
• Implement IPNS TTL-aware caching.
• Implement "IPNS lifetime - now()" aware caching.

[lidel]

Thank you for suggesting this change, the insight from Mozilla is quite interesting!

There are more moving pieces than it seems, and we may not see the same benefits as Firefox had with raw DNS. Some quick notes why:

1. We have to deal with an overhead of DNS over HTTP API: there is no WebExtension API to do DNS TXT queries^[1], so we use DNSLink lookup API^[2] exposed by go-ipfs or the public gateway.
   This means two things:
   • request to previously unknown domain triggers HTTP GET to API
   • we have no access to DNS TTL value as /api/v0/dns only includes Path (adding TTL is tracked in Add TTL to /api/v0/dns kubo#5884)
2. The way things work right now, the actual DNSLink value is effectively ignored at ipfs-companion level – an entry in the cache acts as a boolean flag only to tell if TXT record exists or not.
   Browser extension uses the presence of DNSLink in its cache as a hint that it should redirect request to /ipns/<fqdn> at the local gateway, and it is go-ipfs that takes care of actually resolving IPNS to the latest revision. This means even if old value is in cache, the latest version will be fetched by go-ipfs.
3. The current default DNSLink strategy^[3] is "best-effort": DNSLink redirect is enabled and happens before HTTP request if DNSLink is already in cache. If not in cache, DNS TXT lookup is executed and cached in the background without blocking the page load. Due to this we want to maximize the time the DNSLink result is cached for and that is one of reasons 12 hour lifetime was picked.

That being said, in the future we may start using the actual DNSLink Path and TTL value at the browser extension level (eg. if native protocol handler API lands), and at that point it will be worth revisiting the cache expiration times.

For now, "Optimization potential" lays mostly in go-ipfs, so I am thinking about parking this PR at least until we get insight into DNSLink's TTL. When ipfs/kubo#5884 lands we could switch DNSLink cache to real TTL value + set higher cache for negative DNS responses just like you suggested.

---

[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1449171
[2]: https://docs.ipfs.io/reference/api/http/#api-v0-dns
[3]: https://github.com/ipfs-shipyard/ipfs-companion/blob/v2.6.3/docs/dnslink.md