Yes. There's a lot of yak, and every crawling task reveals new ones.
As an example, all of blogger is behind a single load balancer, with a rate limit. If you don't crawl blogs, you'd never know. Or the top million, plenty of blogger blogs in the top million.
well, yes, but I guess I think of unbound in a different category from setting up (e.g.) bind. but, my experience configuring bind is probably more than 20 years out of date.
you're right to make that correction though, so thank you. :)
BIND is odd in that it combines a recursive resolver with an authoritative name server, and this has actually led to a number of security vulnerabilities over the years. Other alternatives, such as djb's dnscache/tinydns and NLNet Labs' Unbound/nsd separate the two to avoid this entirety.
Setting up Unbound as a recursive, caching resolver is pretty straightforward; a million times more straightforward than doing the same with BIND. You don't need to configure much in a recursor; it just has to accept requests, and recurse until it finds an answer or NXDOMAIN; and then respond.
An authoritative nameserver has a lot more going on; primaries/secondaries, permissions, zone transfers and so on. BIND was the devil to configure, and TBH I have never needed an authoritative nameserver. But I think anyone who can should run their own recursor.
> more than 20 years out of date
Hah! I reckon it's about 20 years since I touched BIND.
