The best thing to do would be to use a robots.txt generator. Try Googling, there are several. This will make sure that bots can understand it.
Secondly, not every bot follows robots.txt, so if you want to block them you will want to use .htaccess.
Make sure that it is in the root of your site [e.g /public_html] and chmodded so that it's easy to view.
|
