where to move the spiders/robots.txt and others?

[askewz]

Hi all,

 

Ok i have a osC shop based in /catalog

 

I have a blog based in /service

 

and I have a linking manager based in /links

 

I am creating a sitemap and i was wondering about the robots.txt and spiders.txt ..etc if they need to be removed from the /catalog to the main page?

 

or add new ones to the root page or.......?

 

Thank you

[Guest]

Hi all,

 

Ok i have a osC shop based in /catalog

 

I have a blog based in /service

 

and I have a linking manager based in /links

 

I am creating a sitemap and i was wondering about the robots.txt and spiders.txt ..etc if they need to be removed from the /catalog to the main page?

 

or add new ones to the root page or.......?

 

Thank you

Leave them where they are.

[askewz]

oh ok then thanks:)

[Guest]

oh ok then thanks:)

Do use this http://www.oscommerce.com/community/contri.../search,spiders

[Jack_mcs]

Hi all,

 

Ok i have a osC shop based in /catalog

 

I have a blog based in /service

 

and I have a linking manager based in /links

 

I am creating a sitemap and i was wondering about the robots.txt and spiders.txt ..etc if they need to be removed from the /catalog to the main page?

 

or add new ones to the root page or.......?

 

Thank you

The spiders file should stay in the includes directory since it is part of the oscommerce code. But the robots file should be in the root of the site, not the root of the shop, so that needs to be moved. To be sure it is in the correct place, you should be able to reach it via http://www.yoursite.com/robots.com. If you have it there and in your shop's root, it can cause your pages to not be listed so this is not a trivial matter.

 

Jack

[askewz]

Thanks jack :)