IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Robots.txt
Chris Withey
post Jul 11 2008, 12:04 AM
Post #1


WSF Regular
*

Group: PM Unlimited
Posts: 49
Joined: 11-June 08
From: England
Member No.: 1,541
Exchanges: www.mysurf.biz



Hi, after noticing that a few search engines had indexed my /manager/ folder, i added a robots.txt like this to my root folder:
CODE
User-agent: *

Disallow: /manager/

Disallow: /cgi-bin/


I would suggest everyone do the same. Also maybe more folders should not be indexed, please let me know here.

Regards,
Chris - MySurf.Biz


--------------------
Go to the top of the page
 
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 17th July 2019 - 08:22 AM