Guest Posted June 7, 2003 Posted June 7, 2003 Is it good to have both robots.txt and robot meta tags on your site? Or will this just confuse the hell out of a robot? I easily added the <meta name=robots content=....tag to linda's header tag controller to show up on all my pages. But I do have directories on my site that I don't want search engines to crawl, and figured a robots.txt file would be best to keep spiders out of those directories. All help is appreciated.
Guest Posted June 7, 2003 Posted June 7, 2003 Lee, Maybe this site will answer your question. http://www.searchengineworld.com/robots/ro...ts_tutorial.htm
Recommended Posts
Archived
This topic is now archived and is closed to further replies.