While you are constructing your website you need to make a robots.txt to stop the website getting accessed by
search engines and then indexed! This is so you don't miss out on your google
honeymoon.
Do not jump the gun by posting links to your website, otherwise content will start getting indexed, unless you
use the noidex meta tag, but no need to worry about that, just dont post links to your website.
To block search engines from your website, please use
User-agent: * Disallow: /
Write the text above into a text document, then rename the text document. robots.txt and upload
it to your root folder.