If I Have A Robots.txt File Do I Need A Robots Meta Tag?

[Trentide]

Thanks for looking at my post! I recently did a test of my site and an error came back with "no robots meta tag found" or something like that. Anyway, do I need one?

 

Any and all help is greatly appreciated!

 

Thanks, Jason

[Jack_mcs]

No, you don't need them both.

 

Jack

[Trentide]

Thanks Jack!

 

What would the OSCommerce forum members do without you?

 

Your awesome! Thanks again for your help!

 

Jason

[Guest]

No, you don't need them both.

 

Jack

Hi Jack

 

Are you sure? I do value your input, but I thought that the robots metatag contolled how frequently the site was indexed, along with what parts of the site the robots crawled. Am I missing something?

[Jack_mcs]

Hi Jack

 

Are you sure? I do value your input, but I thought that the robots metatag contolled how frequently the site was indexed, along with what parts of the site the robots crawled. Am I missing something?

You may be thinking of the revisit meta tag. That tells the search engines when they should visit your site again. The robots meta tg just tells them what should be looked at. The robots.txt file will be used before the meta tag and it is more efficient so it should always be used. An exception might be with popups. Some have reported on the forums that their popup windows appear in the listings even though they are listed in the robots.txt file. I've never seen this but that doesn't mean it won't happen. In that case, you could use the robot meta tag with nofollow to prevent that from happening.

 

Jack

[Guest]

You may be thinking of the revisit meta tag. That tells the search engines when they should visit your site again. The robots meta tg just tells them what should be looked at. The robots.txt file will be used before the meta tag and it is more efficient so it should always be used. An exception might be with popups. Some have reported on the forums that their popup windows appear in the listings even though they are listed in the robots.txt file. I've never seen this but that doesn't mean it won't happen. In that case, you could use the robot meta tag with nofollow to prevent that from happening.

 

Jack

Thank you Jack. I have bookmarked this thread for reference.

[ingieboy]

Thank you Jack. I have bookmarked this thread for reference.

 

Hi,

 

My oscommerce download didn't contain any robots.txt file

 

Can someone please explain what it is and how to install it as I'm getting an error log on my server saying it can't find the file?

 

I can't find where in the code it is trying to include it!

 

Thanks,

 

Marv

[Jack_mcs]

It is just a txt file that you can create with any editor. There are certain rules that need to be followed with it though. If you look up any site on the web and it has a robots bile, you can view it by going to http://thatdomain.com/robots.txt. That might give you an idea. There are many posts here on the forums about how to setup one. When your is ready to go, it should be uploaded to the web root of your account - not the shop root. Many times they are the same but if they are not and you have yours in the wrong place, it can cause you problems with the search engines.

 

Jack