Robots.txt Generator

Allowed User Agent
Select user agent or use default for all agents
All AgentsAlexa AgentAltaVista AgentAOL Netfind/Excite AgentGoogle AgentHotBot AgentInfoSeek AgentLycos AgentMSN Search CrawlerNorthern Lights AgentWebcrawler Agent

Enable All Webpages
Yes
No

Exclude These URL's
Enter URLs you wish to exclude
/cgi-bin/ /images/