Seo:How Search Engines Work
Tuesday, July 1st, 2008Search engines come in two varieties, human-edited and spider indexed. The two major human-edited search engines (or directories as they are properly called) are the Open Directory Project (dmoz.org) and Yahoo. Search engine optimization is lost on the human-edited directories. Because humans will write your title and description for your site when they place it in their directories, using both objective and subjective methods, you have no control in the SEO-world over how your website will rank in these directories.
In both DMOZ and Yahoo, you can suggest a title and description, but most often your words will be rewritten by an editor and placed in their directories according to criteria of which you will not be privy to knowing. One of the big differences between DMOZ and Yahoo is that DMOZ is free and Yahoo is a paid inclusion. With both search directories there is the possibility of not being included, but as stated before DMOZ is free and with Yahoo, if you are not included, no refund will be given.
The only types of search engine in which you have some control are the search engines, which use spiders (sometimes called robots) to read, index and rank your site. The robots (and this is plural because some search engines like Google have several) are basically software applications that travel the Internet by following links and finding websites to read and add to their databases. These robots will travel to your website if you submit your site directly to them (hand-submit) or indirectly submit (through another software submission service or application) or fail to submit (they will follow incoming links to your sites also).
Once the robot finds your site and indexes it, it will move along to other sites from your outbound links. When you site is indexed, then another process takes place and this is the process of determining your ranking for specific keyword or key-phrase searches performed upon the search engine. All of the major search engines that use robots to index websites also use complicated algorithms for determining the page rank of your site. The search engine algorithm software looks at the title and description of the website along with keyword density and prominence to name a few to see which keyword or phrase your site is optimized for (if any). Certain weight is given for each of these components and the page is ranked accordingly.
The advantages of the spider-driven search engines over the directories are that first, search engine robots will continue to index your site on a regular basis whereas the directories basically will stick with the same title and description for the life of the website. Second, a webmaster or SEO professional has control over the optimization process in relation to the search engines, whereas in the directories, optimization is mostly irrelevant. Third, the volume (and popularity) of people using the search engines far outweighs the volume using the search directories. This means that yes, the major search directories are important places to submit your website to, but the major search engines are an absolute necessity.