This article covers the current Mozscape crawler, which is going through an overhaul. Some of the reasons (and our solutions) for the overhaul are covered at the end of this article. Note that SEOmoz uses several other domain-specific crawlers (such as Freshscape and Custom Crawl) that work differently.

6 minute read Continue Reading