Many talks about this little file... robots.txt. Is it realy usefull ? what do you have to put inside ?

Usefull ?

In fact robots.txt is an robot/spider exclusion protocol. Every spider/reboot should read it and apply the rules it contains. The robots.txt file directive can be used to not allow a spider/rebot to index a specific directory, file, or more generally every single ressource you have on your hosting space/account..

Popular Tags


Deprecated: Non-static method modJoesWordCloudHelper::getModuleContent() should not be called statically in /var/www/clients/client1/web11/web/modules/mod_joeswordcloud/mod_joeswordcloud.php on line 18

Notice: Only variables should be assigned by reference in /var/www/clients/client1/web11/web/modules/mod_joeswordcloud/helper.php on line 228

Search & Share