davej Posted February 10, 2003 Share Posted February 10, 2003 Hi every one I have a interesting question. I recently enabled the "search engin safe url feature" on my site. I also have installed a script that emails me when the google bot has hit my site. I placed it on 3 pages, default.php, all_prods.php, and product_info.php . curently I have about 50 products on the site. before I activated the search engine safe urls the spiders would only hit those 3 pages and leave. this time around it hit all the pages repeatedly which I was excpecting. but I was excpecting it to go through the site throughly once. I am guessing maybe a total of 130 pages in all. instead I got over 3000 alerts. it seemed like I traped the spider is some loop. I know that the spyders bail when they hit dynamic pages to avoid that normally when they see a ? in the url. is it possible that with the websafe urls I really traped the spider and would that hurt my ratings on the engine? dave Link to comment Share on other sites More sharing options...
Mark Russell Posted February 10, 2003 Share Posted February 10, 2003 Check out this thread. http://www.oscommerce.com/forums/viewtopic.php?t=25595 Link to comment Share on other sites More sharing options...
andrei_ro22 Posted February 11, 2003 Share Posted February 11, 2003 dave ..where did you get that script... i kind of need it too.. :D Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.