You too can have a web crawler and index!

With all the debate raging over SEOmoz’s newest tool Linkscape, the most ridiculous is whether or not SEOmoz has their own crawler/spider. All the word play is confusing. If you haven’t yet, this explanation of Crawling vs. Indexing clarifies the terms involved.

I realized something interesting yesterday, and believe I have found a helpful way of explaining at least one of the communication issues involved. Forgive me for the metaphor, but I am both a huge fan of Law & Order and the son of a man who explained everything through stories.

Here’s the basic plot synopsis of quite a few Law & Order episodes:

John Doe hires a professional hit man to shoot Jane Doe.

Discussion questions:

  1. Who owns the gun?
  2. Who is responsible for Jane Doe’s murder?

The District Attorney doesn’t care who owned the gun or who pulled the trigger. They always go after John Doe, the person who initiated the murder.

Reading Rand’s comments on Sphinn, one thing is pretty clear: He makes no distinction between owning the gun and pulling the trigger himself or hiring a hit man to pull the trigger.

For him, they are one and the same.

What’s entertainingly flawed about this logic is that, if you pay SEOmoz to use Linkscape and you download some of those reports to your computer, you can claim the same accomplishment they have claimed. Because you have paid them for access to their data, and you store your own copy of what they sold you, you too can advertise that you have your own web crawler that has indexed 30 billion pages.

This may be hard to pull of if you do in-house SEO (unless your boss thinks a spare hard drive can fit 30B pages) but if you have SEO clients, it’s all in how you spin it. Just add a page to your site with a friendly picture of a robot:


And, you know, keep saying over and over again that you have your own crawler. Good luck! :)

2 responses

Comments are closed.