# Disallow Java-based bots User-agent: Java/* Disallow: / # Disallow Java-based bots User-agent: Java/1.4.2_08 Disallow: / # Disallow Java-based bots User-agent: Java/1.4.2_04 Disallow: / # Disallow JetEye.com Spider User-agent: Jetbot/1.0 Disallow: / # Disallow Gigablast Spider User-agent: Gigabot Disallow: / # Disallow Liberate.com Spider User-agent: Mozilla/3.0 (Liberate DTV 1.2) Disallow: / User-agent: * Disallow: /cgi-bin Disallow: /account.php Disallow: /includes/ Disallow: /login.php Disallow: /checkout_success.php User-Agent: MJ12bot Crawl-Delay: 5 User-agent: HTTrack Disallow: /