Guest Posted December 16, 2005 Share Posted December 16, 2005 i've already added the code to one of the include files (i forget which one) that stops bots from sorting things, but every now and then i spot yahoo on a page that not only doesn't need to be sorted (no links available), but has only one item!! let's say my configuration is set to show only 10 products per category (the rest of the products get dumped to page 2, page 3 and so on) the /index.php?page=1&sort=3a in the url should only show if there's 11 or more products, no? on the particular page i spotted yahoo on, there's only 4 products. how does it keep finding these url's and why is it indexing them? Link to comment Share on other sites More sharing options...
Guest Posted December 16, 2005 Share Posted December 16, 2005 You should be able to keep it by adding the page to robots.txt. Keep in mind, though, that if it's already indexed the page, it might keep visiting for a while - -not 100% sure on that one. -jared Link to comment Share on other sites More sharing options...
Guest Posted December 16, 2005 Share Posted December 16, 2005 how do i add that to robots.txt? it's a legit category page... but it shouldn't be sorted i have my url's rewritten, so it shows up like: /Category+Name.html?page=1&sort=3a Link to comment Share on other sites More sharing options...
Guest Posted December 20, 2005 Share Posted December 20, 2005 Ahh. Not sure how to keep the "sorted' link out, but really, what harm does it do? The spiders will try to follow _every_ link on your page, regardless of how appropriate it may be for that particular page. -jared Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.