1

Contents of robots.txt for SEO implementation

view story
linux-howto

http://askubuntu.com – Below is a piece of content from robots.txt file which governs that which search engine should follow the site and in which manner. Basically the SEO implementation: User-agent: MSNPTC Crawl-delay: 1 Disallow: /norob/ Can somebody provide clear and exact explain for the last two lines in this fragment. P.S. i am not allowed to post questions on stackoverflow, that's why i posted it here. If you think that it's not relevant here please migrate it and do the needful. (HowTos)