Create Robots Txt to Allow All.
If you added a robots.txt earlier to block google from seeing
your unfinished website, you will need to delete it and add a new robots text to allow all.
A robots text file is put in the root folder to block bots from certain areas of the website, such as script
directories, or you may block search engines completely from your website.
I typically allow all access. But some people want their pages to be kept private, or you might want to keep
your pages off the waybackmachine.
To make your robots.txt with Google Webmaster Tools - click here
You can use this Robots text generator
I also include a link to my xml google sitemap for good measure.
User-agent: *
Disallow:
Sitemap: http://www.website-checklist.net/GoogleSitemap.xml
This is an alternative robots.txt generator, there are many around.
Robots Txt generator
Another Robots.txt How To Do Guide
|