Why doesn't the sound come through my USB headset unless I restart Firefox?

If I am already in Firefox, say checking my Gmail, and then plug in my USB headset to make a free call, the sound only comes through the iMac internal speakers. I cannot make the sound come through the headset. It's only when I Quit Firefox and reopen it that the sound comes through.