# Make changes for all web spiders
User-Agent: *
Allow: /catalog/
Disallow: /reports/
Disallow: /test/
Allow: /