Printable Version of Topic

Click here to view this topic in its original format

Ventrino Forum for Walker Scripts _ Interesting Stuff _ Robots.txt

Posted by: Chris Withey Jul 11 2008, 12:04 AM

Hi, after noticing that a few search engines had indexed my /manager/ folder, i added a robots.txt like this to my root folder:

CODE
User-agent: *

Disallow: /manager/

Disallow: /cgi-bin/


I would suggest everyone do the same. Also maybe more folders should not be indexed, please let me know here.

Regards,
Chris - MySurf.Biz

Powered by Invision Power Board (http://www.invisionboard.com)
© Invision Power Services (http://www.invisionpower.com)