assembler Posted February 11, 2005 Share Posted February 11, 2005 crawl-66-249-66-144.googlebot.com has been on my site for the last 3 days non-stop, with over 20,000 page loads. It seems to be affecting my shared server's speed. Is it normal for Google to stay on one page for so long? I would block the ip address, but I imagine it would affect my page rank if pages stopped loading for the spider. Does anyone have any ideas / suggestions? My Contributions Link to comment Share on other sites More sharing options...
Rob123 Posted February 11, 2005 Share Posted February 11, 2005 Googlebot is a necessary evil that reminds me of that bad joke that children tell eachother. Q: Where does a 500 lb. Gorilla sit?A: Anywhere he wants. In other words, you really don't have a choice. You can selectively restrict access to certain pages of your site using the robots.txt file. In addition make sure that the spider sessions in your admin are set correctly. However, if googlebot is affecting your "shared server's speed", imagine if you had more than a few customers browsing. I would consider upgrading your hosting plan or simply changing hosts. HTH, Robert Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.