Thought of trying out some thing new and I wrote the "Hello World" Firefox extension. Precise instructions and explanation are available at https://developer.mozilla.org/en/Building_an_Extension.
The profile folder mentioned in the instructions is ~/.mozilla/firefox/*.default/extensions
After creating the id file in extensions folder when you restart you firefox, the addon window notifies that a new add on has been added.
You can uninstall your extension like any other extension. Uninstalling removes the id file from the extensions folder.
EDIT: An excellent tutorial here - http://www.borngeek.com/firefox/toolbar-tutorial/