How to write a robots.txt?
by hyy on 2007-08-01 12:27:31
Robots.txt is a plain text file. In this file, the website administrator can state the parts of the site they do not want to be visited by robots, or specify that search engines only index designated content.
