Robots.txt is an important part in Search Engine Optimization, the form robots.txt file is .txt file in hosting at the root. The function of the robots.txt is to provide a code to the robot webcrawl googlebot or others, when it crawls your site or blog, to view please go to www.yoursite.com / robots.txt.

By default to the joomla robots.txt would look like this.

Code:

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/

To change the robots.txt, a few steps to be done as follows:
- go to cpanel, hosting of your site.
- Then proceed to file manager and look at part public html
- After that, locate the file form. txt or a file named robots.txt and then edit.

Use code like below example:

Code:

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
Disallow: /xmlrpc/

User-agent: Mediapartners-Google
Allow: /

User-agent: Adsbot-Google
Allow: /

User-agent: Googlebot-Image
Allow: /

User-agent: Googlebot-Mobile
Allow: /

Sitemap: http://www.yoursite.com/sitemap.xml

If you as a google partner, like Google Adsense can use this code.

Sumber : joomlatemplate99.blogspot.com