# FULL access (All Spiders) User-agent: * Disallow: /bin Disallow: /logs Disallow: /images Disallow: /etc