To exclude all robots from part of the server
User-agent: *
Disallow: /cgi-bin/
Disallow: /database/
Disallow: /stats/
Disallow: /survey/
Disallow: /_private/