Firefox responding VERY slowly when downloading

This is kind of weird, I hope I'm posting in the right section. My Firefox does not completely crash but nearly does to the point where I am clicking bookmarks on my toolbar or even trying to close it by clicking the X and it takes forever for the click to register. When I actually do get the click to register, it takes about 3 minutes to load one page. This ONLY happens when I'm downloading something with the Firefox downloader, after I close the browser, and then after I open another browser up. So if I start downloading something and I keep surfing on the same browser, there is no problem with Firefox responding to any commands. The only problem is when I close the browser with only the downloads tab open left, then open another browser. Just an FYI, I know this is a Firefox issue, not an internet issue. When I try to re-open a browser, it takes a long time for any commands to register and has nothing to do with the speed of my internet. Every other program I'm running is unaffected during this as I can run my other applications with no noticable changes.