# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file User-agent: * Crawl-delay: 17 Request-rate: 1/500ms Disallow: /layouts/ Disallow: /holds/select_hold/ Disallow: /item/report_match/ Disallow: /info/select_library/ Allow: /* Visit-time: 0200-0800