B
Bagi Zoltán
Guest
After some hours searching and hacking i have finally found everything to build a solution which makes possible to hide the content of your robots.txt file from visitors but make it display ONLY for validated user agents such as googlebot, Yahoo Slurp and msnbot.
You may find the whole thing very strange why would somebody hide that content? My answer for this question is the following:
That content (folder structure of the core script files) is a private information, and don't want to share it with every script kiddies to make the possible to hurt my site.
How to execute this hack? I will guide through..
1. As first step you need to add these lines to your .htaccess file, or if you don't have create one and upload it to the root domain folder.
I think i don't have to explain the first row, the second and the thirs says that if you are not one of the three big search engines and want to reach the robots.txt file you will be redirected to the main domain. It is very handy since a lot of people set their homepage as the landing page of 404 errors, so the cloacking won't be recognised. (will talk about the cloacking a bit later as well)
The fourth row make possible that your robots.txt file behave as a php script.
Now you are ready with the first step, lets see what else you need to do.
2. Open a text editor or your favourite web editor application and insert the code below into a new file save as reversedns.php and upload it to your root folder.
This script can be famaliar for many of you. This is a hacked version of the reversedns.php file which was presented some months ago. According to the hack if the robot can not be validated the script will redirect it to your main domain. So i return back for a min to the cloaking or not cloaking issue. I had to recognise that google are not capable to protect my rankings from exploits, so i have to defend myself, hence i belice it is not a bad cloacking only a protection solution. If somebody mask him/herself as googlebot he/she will fail during this robot valadiation so will be redirected to the main domain via php. No way to recognise the cloacking!
3. And as the last step
Open the robots.txt file you would like to protect and insert the code below to the first line.
You are done, and your robots.txt file is in safe!
Thanks!
You may find the whole thing very strange why would somebody hide that content? My answer for this question is the following:
That content (folder structure of the core script files) is a private information, and don't want to share it with every script kiddies to make the possible to hurt my site.
How to execute this hack? I will guide through..
1. As first step you need to add these lines to your .htaccess file, or if you don't have create one and upload it to the root domain folder.
Code:
RewriteEngine On
RewriteCond %{http_user_agent} !(googlebot|Msnbot|Slurp) [NC]
RewriteRule ^robots\.txt$ http://seo.i-connector.com/ [R,NE,L]
AddHandler application/x-httpd-php .txt
I think i don't have to explain the first row, the second and the thirs says that if you are not one of the three big search engines and want to reach the robots.txt file you will be redirected to the main domain. It is very handy since a lot of people set their homepage as the landing page of 404 errors, so the cloacking won't be recognised. (will talk about the cloacking a bit later as well)
The fourth row make possible that your robots.txt file behave as a php script.
Now you are ready with the first step, lets see what else you need to do.
2. Open a text editor or your favourite web editor application and insert the code below into a new file save as reversedns.php and upload it to your root folder.
PHP:
<?php
$ua = $_SERVER['HTTP_USER_AGENT'];
if(stristr($ua, 'msnbot') || stristr($ua, 'Googlebot') || stristr($ua, 'Yahoo Slurp')){
$ip = $_SERVER['REMOTE_ADDR'];
$hostname = gethostbyaddr($ip);
if(!preg_match("/\.googlebot\.com$/", $hostname) &&!preg_match("/search\.live\.com$/", $hostname) &&!preg_match("/crawl\.yahoo\.net$/", $hostname)) {
$block = TRUE;
$URL="/";
header ("Location: $URL");
exit;
} else {
$real_ip = gethostbyname($hostname);
if($ip!= $real_ip){
$block = TRUE;
$URL="/";
header ("Location: $URL");
exit;
} else {
$block = FALSE;
}
}
}
?>
This script can be famaliar for many of you. This is a hacked version of the reversedns.php file which was presented some months ago. According to the hack if the robot can not be validated the script will redirect it to your main domain. So i return back for a min to the cloaking or not cloaking issue. I had to recognise that google are not capable to protect my rankings from exploits, so i have to defend myself, hence i belice it is not a bad cloacking only a protection solution. If somebody mask him/herself as googlebot he/she will fail during this robot valadiation so will be redirected to the main domain via php. No way to recognise the cloacking!
3. And as the last step
Open the robots.txt file you would like to protect and insert the code below to the first line.
PHP:
<?php include("reversedns.php"); ?>
You are done, and your robots.txt file is in safe!
Thanks!
Last edited: