The Most Active and Friendliest
Affiliate Marketing Community Online!

“Adavice”/  “1Win

Search Engine Crawler

First they gets a list of URL's to visit and store. How? A file called "robots.txt." tells the search crawler which files/pages it can request, and which files/page or directories it's not allowed to visit.
 
This depends on how it got to your site.

If you submitted a site map than it will go to the main page first.
If you only put links on some sites and the crawler got to your site through those links it will go to the page where the link points.
 
well always when we start linking we target main page or index page first so it is so daffiness that crawler go to index page first as every one said and it tool complete data of your site. What will be the time period of crawling of your site no one can tell you the exact time.
 
banners
Back