############################### # # sample robots.txt file for this website # # addresses all robots by using wild card * # User-agent: * # in this file robots are not allowed to index any content as "node" in the site to help to avoid duplicate content with "named" pages. Sitemap: http://newideas.net/sitemap.xml Sitemap: http://newideas.net/sitemap.htm Disallow: /admin Disallow: /admin/ Disallow: /node/add Disallow: /tracker/ Disallow: /user/ Disallow: /u/ Disallow: /privatemsg/ Disallow: /mail/ Disallow: /files/ Disallow: /search/ Disallow: /comment/ Disallow: /taxonomy/ Disallow: /archive/ Disallow: /node/ # End of robots.txt file # ###############################