# Core One Generated Robots.txt # Robot Exclusion File -- robots.txt # Author: Core One Team # Last Updated : July 27, 2010 User-Agent: * Disallow: /markup/ Disallow: /dogfoodie/ Disallow: /sociallyweb/ Disallow: /socially-web/ Disallow: /express/ Disallow: /curl/ Disallow: /cgi-bin/ Disallow: /shop-clone/ Disallow: /shop_bulletcable/ Disallow: /shop_corex2/ # Current Testing pages Disallow: /index.php Disallow: /indexnewcoreone.html Disallow: /core-equipment-gear-cases.php Disallow: /bullet-cable.php Disallow: /core-x2-pro-cable.php Disallow: /inner.html Disallow: /kepur-guitar-strap.php Disallow: /trench-music-accessories.php Disallow: /wirething-pick.php Disallow: /branding/ Disallow: /company-info/ Disallow: /core-one-notices/ Disallow: /dealer-locations/ Disallow: /includes/ Disallow: /press-release/ Disallow: /warrnty-info/ # IF YOU DO NOT WISH TO HAVE THE GOOGLE IMAGE BOT SCAN YOUR DOMAIN FOR IMAGES # THEN YOU CAN INCLUDE THE FOLLOWING IN YOUR ROBOTS FILE. # I FOUND THAT MY BANDWIDTH USAGE DROPPED BY A MASSIVE AMOUNT AFTER I GOT RID # OF THE GOOGLE IMAGE BOT. ALL I HAD WAS IMAGE HUNTERS STEALING PRODUCT SHOTS # AND NOT EVEN BROWSING THE SITE. #User-agent: Googlebot-Image #Disallow: /