Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

Custom Robots TXT Generator For Bloggers

Custom Robots TXT Generator For Blogger is a free SEO tool developed by BlueSeotools.com. Robot.txt generates a text file that allows Google and Yahoo search engine bots to crawl and index the pages. You may permit search engine bots by robot.txt file what to index and what not. The understanding of robots.txt is relatively easy. The robots.txt comprises two attributes: A Name and content. Both of them are required. So, We must learn how they work. Name: Attribute handles the crawler bots(user-agents) and should follow the tags as instructed in the robot's meta tags. Allow all Crawlers to follow instructions name=" robots." Content The content attribute gives instructions to the specified user agent.

How To Use Custom Robots TXT Generator?

  1. Select bot wants to allow. Click Refused or Allowed.
  2. Select the crawl Crawl-Delay: can select between (5-120 seconds).
  3. Sitemap: The user can upload a site map though it is optional.
  4. Search robot: Now, select robot options. Refused or allowed.
  5. Local Directories:  may restrict files want.