#robots.txt preventing certain directories from being crawled. #per user request User-agent: * Disallow: */Administration/SCAO/Forms/Draft%20Forms/ Disallow: */Search/ Disallow: */_layouts/ Disallow: /ReusableContent/ Disallow: /Reports%20List/ Disallow: /WorkflowTasks/ Disallow: /SiteCollectionDocuments/ Disallow: /SiteCollectionImages/ Disallow: */Documents/Forms/ Disallow: */Pages/Forms/ Disallow: */images/ Disallow: */_vti_bin/ Disallow: /allitems.aspx Disallow: /Recent%20Communications/AllItems.aspx Disallow: /Courts/COA/Documents/COATemplates.zip Disallow: /courts/coa/pages/templates.aspx Disallow: /Courts/COA/Documents/COATemplateInstallationGuide.pdf Disallow: /template.dotx Disallow: /forms/allitems.aspx Disallow: /opinions_orders/businesscourtssearch/BusinessCourtDocuments/