sam6 Posted March 18, 2004 Share Posted March 18, 2004 Dose any one know if there is a way to speed up the spidering of my site. I have done all of the optimization i think i need but i would like to help the spiders get threw my site a little faster. If there are tips i would like to here them thanks Link to comment Share on other sites More sharing options...
user99999999 Posted March 18, 2004 Share Posted March 18, 2004 Pay them money. Link to comment Share on other sites More sharing options...
sam6 Posted March 18, 2004 Author Share Posted March 18, 2004 you miss understand im not having trouble getting spidered. But I would like to speed up the proccess google is on my site for 2 weeks at a time every thing is wonderfull just wanted tips on speeding them up. Maybee a robot.txt or somthing i dont know if a robot.txt would help but maybee any ideas Link to comment Share on other sites More sharing options...
user99999999 Posted March 18, 2004 Share Posted March 18, 2004 Google will come to your site everyday and read a few pages 20 maybe, they only stayed there a few minutes and left. Link to comment Share on other sites More sharing options...
sam6 Posted March 18, 2004 Author Share Posted March 18, 2004 ok ill try this google comes to my site for hours of indexing all day every day for two weaks or so this is not about google i just whant to know if there are tips on speeding them up like not allowing them to go to serten places or whatever any one has to say about speeding up the indexing. dynamic site are indexed slower. there ar many good optimization tips. But not many about helping the spiders get threw a site a little faster so if any one has input about this thanks Link to comment Share on other sites More sharing options...
mpiscopo Posted March 18, 2004 Share Posted March 18, 2004 Just have to be patient. Google (and other SE's) only update their index about once a month. If it was possible to speed up the spiders visiting your sites, it wouldn't make any difference because you won't show up until the next database update. Best thing to do is to plan that it will take 1 to 2 months to have new items indexed and plan ahead. I try to make pages for products before they arrive, mostly the product categories (i.e. put in some general information and expected arrival date). Other than that the only other advise is to be patient and plan ahead as much as possible. Link to comment Share on other sites More sharing options...
sam6 Posted March 18, 2004 Author Share Posted March 18, 2004 no no no i just want to know if there is somthing i should do to help a spider traverse my site more efficently. like how do you stop them frome ending up on the cookie usage page every time they index visit a page? Or any thing that will help them get threw my site more efficently! i have thousands of pages listed and get indexed regularly so this is not about getting listed or waiting on a spider to arrive. the question is How to get them threw the site more efficently. thanks Link to comment Share on other sites More sharing options...
mpiscopo Posted March 18, 2004 Share Posted March 18, 2004 Are you referring to using the robots.txt file to keep unneeded pages and directories from being indexed? Link to comment Share on other sites More sharing options...
sam6 Posted March 18, 2004 Author Share Posted March 18, 2004 any tips on how to speed up the spiders because of the dynamic pages they ar pretty slow !! i am being spidered all is well just want tips on how to help them on there way threw my site Link to comment Share on other sites More sharing options...
Guest Posted March 18, 2004 Share Posted March 18, 2004 You can create a spider map of your products. You can get more external links pointing to your site. (the higher your PR on Google, the more frequently & deeper they spider your site) A robots.txt files is used to block spiders, don't use it blindly. Bob any tips on how to speed up the spiders because of the dynamic pages they ar pretty slow !! i am being spidered all is well just want tips on how to help them on there way threw my site Link to comment Share on other sites More sharing options...
wizardsandwars Posted March 18, 2004 Share Posted March 18, 2004 As long as there are no SIDs in the URLs that the spider is parsing, then there is nothing you can do to 'speed them up'. Google works at it's own pace. ------------------------------------------------------------------------------------------------------------------------- NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit. If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help. Link to comment Share on other sites More sharing options...
sam6 Posted March 18, 2004 Author Share Posted March 18, 2004 Thanks Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.