You're not forced to use an updated version, but older versions expose your system to exploits. For example, see Java SE 6 Update 24 fixes floating point bug

However, you can remove 1.6.0_25 if you wish since it isn't a security release. Oracle released it to speed up apps written in Java to work with Firefox 4.0 See: Java SE 6 update 25 brings faster server startup

Yes I could remove it but then I wasn't able to manage my switches. If I want to use a previous version how can this be done?

It would appear that the file is no longer available for download from Oracle. You can find it from third party sites by doing a google search for jre-6u24-windows-i586-iftw.exe.

Other than that though, the only other option appears to be a support call costing $50 See: http://www.liveperson.com/lp/java-help/?BanID=22489

http://www.mozilla.com/en-US/blocklist/
Mozilla blocks the Java Deployment Toolkit, versions 6.0.200.0 and older by disabling it, but that shouldn't affect Java itself. If you need that "Toolkit" for your app, try 6.0.021 and see if that works - Mozilla didn't block the versions newer than 020.

http://www.oldapps.com/java.php?old_java=58

@ the-edmeister, I was told not to post links to external sources where older apps could be downloaded. I posted just such a link to oldapps like you've done yourself, but it was removed by one of the mods.

Has that policy changed now?

AFAIK, that "policy" is specifically for older versions of Firefox, all of which are all still available from Mozilla. Considering that OldApps is one of the secondary download mirrors for Mozilla, IMHO they should be trusted as a source for other apps, too.

http://openaccess.oldapps.com/mozilla.org/

OK, understood, thanks.