You can change this in the Firefox options. Click the Firefox button or Tools in the classic menu, click Options, then select the General tab. Use the drop down menu labeled "When Firefox starts:" to select what you want.

If you do not use "Show my windows and tabs from last time" then make sure that Firefox closes properly to prevent session restore from reopening a possibly crashed session.

See "Hang at exit":

• http://kb.mozillazine.org/Firefox_hangs
• https://support.mozilla.com/kb/Firefox+hangs

Thanks Dave, that as really annoying.