-
A
hide your robots.txt from visitors and show it only for validated robots
Thanks for your alert. I will stop implementing then.- Azar
- Post #12
- Forum: Programming and Scripts
-
A
hide your robots.txt from visitors and show it only for validated robots
Hi Bagi!! Awesome post. This is really helpful! Can you add a few more search bots (host address) in this line? if(!preg_match("/\.googlebot\.com$/", $hostname) &&!preg_match("/search\.live\.com$/", $hostname) &&!preg_match("/crawl\.yahoo\.net$/", $hostname)) { } I would like to check for...- Azar
- Post #11
- Forum: Programming and Scripts
