Jump to content
  • Checkout
  • Login
  • Get in touch


The e-commerce.

To spider or not to spider!!!


Recommended Posts

Hi Guys....


Ok, been going through hundreds of past posts, and havent been able to come to a solid conclusion!


Ive read how spiders work(well, tried to)! Now I saw some posts where people were saying they were bad and others where spiders were good!??


Now, ive installed that allproducts contrib and placed a link in my footer!


Have a look My Webpage


Now, does that mean i still need to work on my meta tags - as ive installed the dynamic metatags contrib into my product_info.php?! Or must i just work on the one's in default.php? How important are they???


What can i do to make listing better? Im wanting to be listed on a search engine called ANANZI - do they have spiders aswell? How would i know, i cant find anything like that on their site??? Must i also submit like this "www.ngrcomputers.co.za/allproducts.php"???? Or does it find it automatically?


Some people are saying "this spider came to me and did this and did that and this may requests and hasnt left etc????" - where do they get that from, how can i see if a spider has come or not???


Im about to install the IP logging???


Im I on the right track here - help would be greatly appreciated!!!???


Nick :ph34r:

Link to comment
Share on other sites



Does no-one know or have an opinion??? :blink:


Some help/advice would be greatly appreciated... B)




Nick :ph34r:

Link to comment
Share on other sites

Lots of questions here. I'll answer as much as I can, and leave the rest for others.


You want the spiders to visit your pages, but you don't want them to read SIDs. If you are using osCommerce 2.2MS2, make certain that the spiders are listed in spiders.txt. If you have an older version, install one of the SID killer contributions.


Spiders are used by search engines to collect web pages for the search engine to index. The indexing criteria differs for each search engine, plus all search engines change how they index from time to time. Furthermore, the indexing methods used are a carefully guarded secret, so you can't ever know for certain.


There are still a few things that are worth doing to help your site out with most of the search engines:

1. Make certain that the spider can find all of your pages. The All Products contribution is a great way to do this. Link this page from every page on your site (the footer seems to work well for this). You can submit this page if you want. Better is to get others to link to your site and have the spider find everything itself.

2. Have a good title on every page, with the keywords appropriate to that page in the title. Use a Meta Tags contribution for this.

3. The meta Description and Keywords tags are of little use, but since the meta tag contribution takes care of them anyway, you might as well put them in. Your description may actually be read by your customers, so you can use it as a sales pitch for your products.

4. Have your keywords in an H1 tag on the page.

5. Have your keywords in bold on the page.

6. Have your keywords in link text on the page.

7. Have your keywords in plain text on the page.

8. Have good content that your customers really want to see. Too many people concentrate so hard on fooling the search engines that they forget to make their site readable by humans.


There are several web sites dedicated to just this kind of information. Try a search for them. But beware of people who want you to pay them to optimize your site; not all of these companies are honest. Pulling shady tricks to get a better position on the search engines can get your site permanently banned.




See my profile for a list of my addons and ways to get support.

Link to comment
Share on other sites

:P Hey, thanks for that reply!!!!!!!


Right, so ive dont Dynamic Metatags - so thats that done....

Done the Tags in the header of my pages...so thats done aswell...

Done the part of the allproducts page in the footer - hope that helps and is right!!

And done the rest of what u suggested....


About to go find a SID killer....


One question remains...how do i see if the spider has visited?? Through Admin/Who's online??And what if i didnt see, how would i know it was there???


Thanks alot people - i HAVE looked for answers, its just alot of it is old and people's opinions conflict ALOT!!!


Just wondering if there were any updates??



Nick :ph34r:

Link to comment
Share on other sites

if you are on your own server ore heve access to the logs that is one way.

The other way is a contribution i saw in a post yesterday somthing about who is on lin enhancment! So serch th contribs for that!

Link to comment
Share on other sites


NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit.

If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.

Link to comment
Share on other sites

Is the All Products contrib very hard to install. I haven't installed a contrib yet and was hoping it was painless ;)


Thanks for any help :P



Link to comment
Share on other sites

I just haven't tried to install any contribs yet. They look a bit confusing but I guess if you take it slow and back everything up you will be o.k.


When I try this out later today or tomorrow, and If I have any problems, I will give you a buzz...thanks :)

Link to comment
Share on other sites


This topic is now archived and is closed to further replies.

  • Create New...