[nycphp-talk] [OT] is your firewall getting you dropped from the search engines?
csnyder
chsnyder at gmail.com
Wed May 4 12:58:02 EDT 2005
On 5/4/05, inforequest <1j0lkq002 at sneakemail.com> wrote:
> (because robots.txt holds info on where you do not want spiders to go,
> it may highlight areas of your web server that contain privileged
> information. Given that, the robots.txt may provide sensitive info, and
> that is why Sonicwall claims it is a security risk)
The argument that robots.txt can be used as a roadmap to private areas
is bogus -- private areas should be password protected. I suppose they
would consider rel="nofollow" tags to be a security risk as well, eh?
More information about the talk
mailing list