The Mother-Ship - http://joomla.org

Joomla! Extensions Directory - http://extensions.joomla.org

Free Templates - http://www.joomla24.com 

Template Clubs:

Joomla Bamboo - http://demo.joomlabamboo.com

Rocket Theme - http://demo.rockettheme.com

Yoo Theme - http://www.yootheme.com/demo/joomla

Joomla Shack - http://demotemplates.joomlashack.com/

Gavick - http://demo.gavick.com

(stay away from a lot of the 'free joomla template' sites - they can hard code spam links in them, you can get free templates safely at the big clubs)

Joomla Hosting:

Rochen Performance Hosting - http://www.rochenhost.com/

Really Clicking - Local Hosting - http://reallyclicking.com

Tips:

Brian Teeman, Hidden Joomla! Secrets -  (one of the founders of Joomla!)