# robots.txt allows all pages to be spidered
User-agent: *