Disallow Indexing of Manuf. pages

[jond]

Hi,

 

I have a website where many category pages resemble the content from the manuf. dropdown box. I want to avoid getting hit with duplicate content penalties from google.

 

How can i avoid getting the manufacturer pages indexed? i want to avoid listing all manuf. pages, and every ?page= result in my robots.txt file.....

 

also, if there was a way i could disallow spiders from following any filtered page, that would be great, such as price sorts, etc. because the content is basically the same from a spider perspective.

 

any suggestions GREATLY appreciated!

[Guest]

i am interested in this as well

if possible, i would like to block search engines from indexing search results too

[jond]

for the search results, you should be able to just exclude the search page from your robots.txt file. to be safe you can pop a noindex, nofollow into the meta tags of the search and search results page.

 

now, anyone have any clue about disallowing sorted pages, and manuf. pages..??????????

[Guest]

you could check if a spider visits before setting the box in php, or... the manufacturer url goes through the redirect code I think. You could mod that code if a spider is present to go to the index page or something.

[jond]

that would be great. any idea on how to form the code for this?

 

i would ideally want any url containing a ? to be redirected back to the same page without the ?* extension - i believe it can be done somehow in htaccess

[Guest]

i would ideally want any url containing a ? to be redirected back to the same page without the ?* extension - i believe it can be done somehow in htaccess

 

yea, but if you change the .htaccess will probably be for everyone. (and then you start filtering spider/ips in htaccess) not much fun.

 

So there is some code in application_top.php that checks the spiders.txt file. When it finds one it sets the $spider_flag to true whch is a global var. So now you have in catalog\redirect.php the manufacturer case where there is a tep_redirect that you could modify if you see that the spider_flag is asserted. All you have to do is add a break and that should go to the default index page (at the end of the redirect file)

 

You could also check the flag before displaying the drop-down list. of manufacturers

[invasi0n]

yea, but if you change the .htaccess will probably be for everyone. (and then you start filtering spider/ips in htaccess) not much fun.

 

So there is some code in application_top.php that checks the spiders.txt file. When it finds one it sets the $spider_flag to true whch is a global var. So now you have in catalog\redirect.php the manufacturer case where there is a tep_redirect that you could modify if you see that the spider_flag is asserted. All you have to do is add a break and that should go to the default index page (at the end of the redirect file)

 

You could also check the flag before displaying the drop-down list. of manufacturers

Hi,

 

Can you please be more explicit?

 

What exactly we have to modify in aplication_top.php?

 

Thanks

[invasi0n]

Nobody can help please?

[phpnewb]

Hi,

 

I have a website where many category pages resemble the content from the manuf. dropdown box. I want to avoid getting hit with duplicate content penalties from google.

 

How can i avoid getting the manufacturer pages indexed? i want to avoid listing all manuf. pages, and every ?page= result in my robots.txt file.....

 

also, if there was a way i could disallow spiders from following any filtered page, that would be great, such as price sorts, etc. because the content is basically the same from a spider perspective.

 

any suggestions GREATLY appreciated!

Hi,

 

I have the same problem.

 

How to disallow indexing of manufacturers infobox?

 

Thanks,