Value Links And Understand Search Like Matt Cutts With Submarine Crawling
Matt Cutts and this Webmaster Central post recently explained that “high quality” sites were being given special treatment - submarine crawling. Since we all know that links from high quality sites are more valuable than those from average/mediocre sites, Matt and Google have in effect given us a new measurement for the value of a link - submarine crawling.
Russian Submarine courtesy of Orpheus Grey.
What’s submarine crawling?
In essence, it’s self-motivated crawling with the purpose of discovering “deep” web content (hence me calling it “submarine crawling”). What I mean is that Google is proactively crawling your site in new and unique ways because it associates high quality material with your own domain, and not because it’s reacting to a link pointing at your site.
Some parts of the web are poorly linked to, though they may have high quality content. And since both humans and search engines discover new content based on links, these parts of the web can be invisible to the SEs.
(Aside: Hence another moniker for this part of the web: “The Invisible Web.”)
From a search engineer’s perspective, delivering relevant results featuring premium content is imperative. Therefore, if you know that a site has premium content, you can go back to it and try and mine it with submarine crawling to discover new premium content for your index.
It’s the 80/20 rule applied to search: 80% of your premium content comes from 20% of the web.
So knowing that, Google is self-motivated (i.e. does not need external motivation aka links) to crawl these sites. And specifically, to submarine crawl them for their deep content. Where SEOs are concerned, this is great news as far as attracting longtail traffic and evaluating the potential value of a link. (The text link brokers of the world had no - public - comment. 
)
Moral of the story 1: If you listen closely to what Matt and Google say, you can imply tips and ideas for your search marketing.
Moral of the story 2: Due to domain age factors and self-reinforcing rankings (where people google a topic, then link to what’s ranking), if you don’t achieve high quality status with Google asap, optimizing for Google is only going to get harder as things move forward. The 80/20 rule is a self-reinforcing trend in search; that’s why Wikipedia is in the top 10 results for 970 of the top 1000 most popular keywords.
Further reading on Crawling, Link Value and self-reinforcing rankings:
Wiep’s Link Value Factors is a good starting location. More recently, he’s also got this post on link building job interviews which touches on some of this stuff (his Internet Marketing company is hiring, fyi).
Bill Slawski has a new post (featuring more layman’s language yay!!) on MSN’s treatment of old domains in ranking sites, in the context of a patent of theirs. An excerpt:
“A new patent application from Microsoft adds another twist, by also ranking domains based upon the ages of domains which link to those domains.
“This patent application assumes that newer domains have a ‘higher likelihood of being spam and/or being a part of a web farm that attempts to artificially inflate domain rankings for domains in the web farm.’”
(Meaning get started building your site’s trustworthiness in the SEs’ eyes before it’s too late. I note in passing that this post is about Google, but these points from MSN are valid regardless of engine, and the 80/20 rule in search is just reinforcing itself with every passing day.)
Jeff Quipp claims not to be an SEO but an “authority builder.” Sounds like someone’s ahead of the curve… got insider information you’d like to share Jeff ? That post was followed by “What is authority?” and Authority Building Tools.
My How to write robots.txt & Sebastian’s whole robots.txt category, robots meta tag category, and his X Robots Tag category are also worth reading.
If you liked this, why don’t you subscribe to my RSS feed?
Subscribe to SEO ROI's RSS Feed
Curious why you should subscribe? Here's 20 reasons why. On a related note, if you need SEO services, here's where to get more information.
