NYCPHP Meetup

Looking for ideas on how to allow spiders to crawl authenticated pages

DeWitt, Michael mjdewitt at alexcommgrp.com
Mon Feb 24 13:45:16 EST 2003


For some of our sites, many of the pages require registration and login in
order to view the page.  I would like to open up those pages to spiders in
an effort to improve our web rankings.  

I have some ideas, but I am curious what others have done to allow spiders
to crawl a page which normally would not be available to them (since they
would have been redirected to a login page for access).

TIA

Mike



More information about the talk mailing list