Jump to content
  • Checkout
  • Login
  • Get in touch

osCommerce

The e-commerce.

Spider or Robot Text


ise

Recommended Posts

Hi,

Do I need Robot text? It says Robot text can not be found. I see there is a notepad file called Spidders, is this the same thing as Robot text ? Spider text below,

 

$Id: spiders.txt,v 1.2 2003/05/05 17:58:17 dgw_ Exp $

almaden.ibm.com

appie 1.1

architext

ask jeeves

asterias2.0

augurfind

baiduspider

bannana_bot

bdcindexer

crawler

crawler@fast

docomo

fast-webcrawler

fluffy the spider

frooglebot

geobot

googlebot

gulliver

henrythemiragorobot

ia_archiver

infoseek

kit_fireball

lachesis

lycos_spider

mantraagent

mercator

moget/1.0

muscatferret

nationaldirectory-webspider

naverrobot

ncsa beta

netresearchserver

ng/1.0

osis-project

polybot

pompos

scooter

seventwentyfour

sidewinder

sleek spider

slurp/si

[email protected]

steeler/1.3

szukacz

t-h-u-n-d-e-r-s-t-o-n-e

teoma

turnitinbot

ultraseek

vagabondo

voilabot

w3c_validator

zao/0

zyborg/1.0

Link to comment
Share on other sites

No, they are two different files. You need to create a robots.txt file and place it in the root of your server, not the root of your shop (unless they are the same). You can do a search in the forums for robots and you should find samples of them. Or search the web for robots.txt and many sites will turn up. They should be edited to suit your shop.

 

Jack

Support Links:

For Hire: Contact me for anything you need help with for your shop: upgrading, hosting, repairs, code written, etc.

All of My Addons

Get the latest versions of my addons

Recommended SEO Addons

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...