#This is a file retrieved by webwalkers a.k.a. spiders that #conform to a defacto standard. # See # The webmaster for this site is: # freenet.de AG #
webmaster@freenet.de
# #Format is: # User-agent: # Disallow: | # #----------------------------------------------------------------------------- User-agent: Mediapartners-Google* Disallow: User-agent: * Disallow: /partner/ Disallow: /thread/ Disallow: /wwwtest/ Disallow: /src/ Disallow: /kasten_elemente/ Disallow: /*?pictureshowoffset= Disallow: /*?notfound= Disallow: /*;jsessionid= User-agent: * Crawl-delay: 2 User-agent: bingbot Crawl-delay: 1 User-agent: Yahoo! Slurp Crawl-delay: 2 User-agent: AhrefsBot Crawl-delay: 4