It seem to bypass my chosen setting of looking for updates and let me choose to install them. Next time I start FF, it is automatically updated. Is there a registry setting maybe I can try to set to hinder this, or any other way?

I would preferrably use latest version, but have services I use where FF screws up the login page. I know you would suggest to contact the supplier of the service to tell them to update at their end, but it will not be possible in this case.