Guest Posted January 8, 2006 Posted January 8, 2006 I need to block a spider from crawling my site & have been told that robots.txt is the best way to do it. My question is how do create & impliment it? I don't know where to put the tags aswell. I am using cDynamic_Meta_Tags v1.4 & Ultimate_SEO_URLSv2x if that makes any difference. Thanks for helping.
tina_boots Posted January 8, 2006 Posted January 8, 2006 I need to block a spider from crawling my site & have been told that robots.txt is the best way to do it. My question is how do create & impliment it? I don't know where to put the tags aswell. I am using cDynamic_Meta_Tags v1.4 & Ultimate_SEO_URLSv2x if that makes any difference. Thanks for helping. This is some good information on the topic http://www.searchengineworld.com/robots/robots_tutorial.htm Be well, Tina If you're not having fun you're not doing it right Teach a person to fish rather than give them a loaf of bread or however that saying goes.
Guest Posted January 8, 2006 Posted January 8, 2006 This is some good information on the topic http://www.searchengineworld.com/robots/robots_tutorial.htm Be well, Tina Thanks Tina but that doesn't tell me where to add the tags into oscommerce.
tina_boots Posted January 8, 2006 Posted January 8, 2006 Thanks Tina but that doesn't tell me where to add the tags into oscommerce. Robots.txt does not go "in" osC its a text file which you place inside your site's folder, often in the root or in your case perhaps the /catalog/ folder. Be well, Tina If you're not having fun you're not doing it right Teach a person to fish rather than give them a loaf of bread or however that saying goes.
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.