User-agent: * Disallow: /function/ Disallow: /imgs/ Sitemap: http://neatmetacrawler.whsites.net/sitemap.xml