Saygılarımla..
( Dipnot : Yeterli aramayı yaptım.. Buradada bw , bandwith , google , spambot vs diye arattım ancak sorunumun çözümüne tam olarak ulasamadım.. Topici bu nedenle açtım.. )
# robots.txt for canver.net
# $Id: robots.txt, v1.0.6 2006/03/25 05:35 ALEXIS Exp $
# "Any empty [Disallow] value, indicates that all URLs can be retrieved.
# At least one Disallow field needs to be present in a record."
# archive.org'tan sakla
#User-agent: ia_archiver
#Disallow: /
User-agent: Mediapartners-Google*
Disallow:
#User-agent: Slurp
#Crawl-delay: 10
#Disallow:
#User-agent: msnbot
#Crawl-delay: 100
#Disallow:
User-agent: *
#Crawl-delay: 100
Disallow: /admin/
Disallow: /attach_mod/
Disallow: /cache/
Disallow: /ctracker/
Disallow: /db/
Disallow: /images/
Disallow: /includes/
Disallow: /language/
Disallow: /templates/
Disallow: /xs_mod/
Disallow: /attach_rules.php
Disallow: /config.php
Disallow: /common.php
Disallow: /faq.php
Disallow: /groupcp.php
Disallow: /login.php
Disallow: /login_security.php
Disallow: /modcp.php
Disallow: /profile.php
Disallow: /viewonline.php
Bu forumu görüntüleyenler: Kayıtlı kullanıcı yok ve 0 misafir