Hello Zipcode99.

You are having an uncommon usability problem, which will hopefully disappear one you are more familiarized with your computer's interface. The cross you are referring to is used to close the window you are seeing at the moment. In other words, it will close the Firefox window you are seeing. If you want to exit the website you are seeing, simply click the cross in the tab you are currently using in Firefox. If there is only one tab (or you're not seeing the tab bar), navigating away from the site will effectively "close" it.

If you have any specific question, feel free to ask here.

You can also middle-click a tab on the tab bar to close that tab.

See also Use tabs to organize lots of websites in a single window