Posts Tagged ‘robots’

can I add a robots follow meta tag to my own pages?

Saturday, September 20th, 2008

My chiropractor said that I should add a  robots follow  tag on all the pages of our site in order for it to be spidered deeply by the search engines. She also said that I should put the “revisit after” tag on as well, to let the bots know how often they should stop by. What do you think..

The revisit after tag is not necessary, nor is it helpful. The search engine spiders work on their own timetable, and couldn’t care less what your revisit tag says. I’ve heard that the tag was actually created for one specific search engine a long time ago, but I’ve never bothered to confirm it. At any rate, there’s no need to clutter up your code with it as today’s engines ignore it.

Seo:Create Robots.txt file

Monday, July 14th, 2008

The robots.txt file will instruct search engine robots what pages and/or folders on your blog or website should be or should NOT be crawled and indexed. Most Content Management Software (Wordpress, Joomla, etc.) will have files and folders that are not relevant for search engines (like images or admin files) and you really don’t want them to crawl them because there is no relevant content on them so creating a simple robots.txt file can actually improve your website crawlability and therefore rankings and organic traffic.

Create Site Map and Google XML Map

Monday, July 14th, 2008

Every website needs a Sitemap big or small. Sitemap should be linked to from every page on your site. A site map will help users and search engine robots find all the important pages on your site with just two clicks. This is especially helpful if your site has a hard-to-crawl navigation menu like java script or images based navigation. For WordPress blogs I recommend downloading this free sitemap generator plugin. You can see example on my sitemap page.

Smaller sites can use second navigation bar in the footer that will act as a small mini site map. This is nothing more than all your important pages as links in the footer.

By submitting a XML Sitemap to a search engine, you are making easier for that engine’s crawlers to crawl and index pages of your site. As Google describes it in this article

“Sitemaps are particularly beneficial when users can’t reach all areas of a website through a browseable interface.”

For regular websites there are many free online xml sitemap generators, just Google it but for your WordPress blog you can use this free Google XML sitemap generator plugin as it will also create Robots.txt file which is the next Free SEO Tip.

Seo:Meta Tags Optimization

Monday, July 14th, 2008

META tags are hidden code of information in the head area of your web pages and is used to communicate with search engine spiders/crawlers. The most important META tags you need to utilize in your webpages are:

1. Description
2. Keywords
3. Robots

I personally don’t believe that sequencing of these tags in certain order is that important. Even though I see many websites put title tag after description and keywords tags lately but there is no proof that it is better for SEO.

Optimization of Description Meta tags

The effective META description tag consists of 25 to 30 words or less but using no more than 160 to 180 characters total (including spaces) or some Search Engines would not consider it Search Engine Friendly.
Read more and see meta tags examples in this Meta tags article.

Optimization of Keywords Meta tags

The meta keyword tag is also sometimes useful as a way to help your page come up for synonyms or unusual words that don’t appear on the page itself but don’t fret too much over your Keywords Meta tag, instead utilize keywords and keyword phrases from your Title Tag element, Description Meta Tag, heading tag and first one or two paragraphs of visible content. Optimal Keyword Meta Tag has 15 to 20 words max. SEO Tip: Don’t repeat your keywords more than 3 times or search engine will consider it as spam of their search results and don’t include keywords that are not in the main content section of your page.
(Read more and see keywords tag examples in this Meta tags article.)

Robots Meta tag will let search engine know if you want a particular page to be or NOT be indexed or to follow or not to follow links on given page. (Read more and see robots tag examples in this Robots txt article.)

How to Display an Email Address on a Web Page and Protect it From Spammers

Monday, July 14th, 2008

Would you like to display an email address on a web page, but need a way to protect it from spammers? This tip is for you.

Displaying your email address on your web site is an important part of good customer service. However, the fear of spam robots harvesting email addresses prevents many webmasters from doing so.

If you would like to display your email address within your web pages and not worry about it being harvested, you can protect yourself. Instead of displaying the usual mailto:you@yourdomain.com, use the following code within your HTML:

<A HREF=”mailto:you&#64;yourdomain.com”>Contact Us</A>

When clicked on, it will display your email address correctly.

SEO : Include a robots.txt File

Thursday, June 26th, 2008

By far the easiest top 10 SEO tips you will ever do as it relates to search engine optimization is include a robots.txt file at the root of your website. Open up a text editor, such as Notepad and type “User-agent: *”. Then save the file as robots.txt and upload it to your root directory on your domain. This one command will tell any spider that hits your website to “please feel free to crawl every page of my website”.

Here’s one of my best top 10 SEO tips: Because the search engine analyzes everything it indexes to determine what your website is all about, it might be a good idea to block folders and files that have nothing to do with the content we want to be analyzed. You can disallow unrelated files to be read by adding “Disallow: /folder_name/” or “Disallow: /filename.html”. Here is an example of the robots.txt file on this site:

SEO : Optimize Your META Tags

Thursday, June 26th, 2008

META tags are hidden code read only by search engine webcrawlers (also called spiders). They live within the HEAD section of a web page. There are actually 4 very important META tags you need to worry about. Meta tags specifying who the author is and what the site is about really isn’t important to the search engines that matter the most (i.e.: Google). The META tags you need to be the most concerned about are:

1. robots
2. content-type
3. description
4. keywords

Sequencing of these tags may be extremely important. I say “may” because SEO is mostly hypothesis due to the changing algorithms of the search engines. Even though the W3C states that tag attributes do not have to be in any particular sequence, I’ve noticed a significant difference when I have the tags and attributes in the order described here. The only deviation from the list above is that the Title tag should come after content-type and before description.

The robots META tag tells the various search engine spiders whether or not you’d like them to crawl through your web page as well as where to start in their crawling activity. Top 10 SEO Tips wouldn’t be worthless without META robots, so long as you use a Robots.txt file. It’s not too hard to see why this tag can still be important. Here is the syntax:

<meta name=”robots” content=”index, follow” />

You can change the “index” to “noindex” and the “follow” to “nofollow” if you do not want your website to be indexed. Though, I have no idea why you wouldn’t want to be indexed.

Content-type is important to complex search engines like Google. This tag tells the spider what type of page you are posting, which helps the search engine categorize the listing. It also shows that you are following the World Wide Web Consortium (W3C) guidelines, which could be an indication of a site being “optimized”. Here is the syntax used on this page:

<meta http-equiv=”Content-Type” content=”text/html; charset=UTF-8″ />

The description META tag is the text that will be displayed under your title on the results page. See the OC Internet Advertising example above. There’s also a lot of controversy about the number of characters you should have in this tag. I’ve seen sites with a paragraph in their description listed in the top results, so I don’t think this tag has very much weight.

TEN SEO Tips

Friday, June 20th, 2008

1. Insert keywords within the title tag so that search engine robots will know what your page is about. The title tag is located right at the top of your document within the head tags. Inserting a keyword or key phrase will greatly improve your chances of bringing targeted traffic to your site.

Make sure that the title tag contains text which a human can relate to. The text within the title tag is what shows up in a search result. Treat it like a headline.

2. Use the same keywords as anchor text to link to the page from different pages on your site. This is especially useful if your site contains many pages. The more keywords that link to a specific page the better.

3. Make sure that the text within the title tag is also within the body of the page. It is unwise to have keywords in the title tag which are not contained within the body of the page.

Adding the exact same text for your h1 tag will tell the reader who clicks on your page from a search engine result that they have clicked on the correct link and have arrived at the page where they intended to visit. Robots like this too because now there is a relation between the title of your page and the headline.

Also, sprinkle your keywords throughout your article. The most important keywords can be bolded or colored in red. A good place to do this is once or twice in the body at the top of your article and in the sub-headings.

4. Do not use the exact same title tag on every page on your website. Search engine robots might determine that all your pages are the same if all your title tags are the same. If this happens, your pages might not get indexed.

I always use the headline of my pages as the title tag to help the robots know exactly what my page is about. A good place to insert the headline is within the h1 tag. So the headline is the same as the title tag text.

5. Do not spam the description or keyword meta tag by stuffing meaningless keywords or even spend too much time on this tag. SEO pros all agree that these tags are not as important today as they once were. I just place my headline once within the keywords and description tags.

6. Do not link to link-farms or other search engine unfriendly neighborhoods.

7. Do not use doorway pages. Doorway pages are designed for robots only, not humans. Search engines like to index human friendly pages which contain content which is relevant to the search.

8. Title tags for text links. Insert the title tag within the HTML of your text link to add weight to the link and the page where the link resides. This is like the alt tag for images.

My site contains navigation menus on the left and right of the page. The menu consists of links not images. When you hover over the link with your mouse, the title of the link appears. View the source of this page to see how to add this tag to your links.

9. Describe your images with the use of the alt tag. This will help search engines that index images to find your pages and will also help readers who use text only web browsers.

10. Submit to the search engines yourself. Do not use a submission service or submission software. Doing so could get your site penalized or even banned.