Is it possible that you actually have two Firefox windows open and Windows 7 is showing one Firefox on the System task bar, but if you were wave mouse over that, you might see two windows.

To Close Firefox use File > Exit (Alt+F > Exit) and not the X in the upper right corner which only closes the Window and may or may not close Firefox if it is the last window. The reason it does not always close Firefox is that there may be other tasks running so you don't see Firefox anywhere but in the System Task Manager on the Processes Tab which is where you can see if firefox.exe is running or not. Shortcut to Task Manager on Windows 7 is Ctrl+Shift+Esc (for other Windows systems Ctrl+Alt+Del , select Task Manager,...)