First of all you will need to go to global configuration in your joomla administrator site.
After this browse to the SEO settings and set search engine friendly urls to yes and add suffix to urls to yes. If you want cleaner urls we recommend turning on the mod rewrite feature, this will enable better urls ending in the .html extension.
:o