Try Firefox Safe Mode to see if the problem goes away. Firefox Safe Mode is a troubleshooting mode that temporarily turns off hardware acceleration, resets some settings, and disables add-ons (extensions and themes).

If Firefox is open, you can restart in Firefox Safe Mode from the Help menu:

• Click the menu button , click Help and select Restart with Add-ons Disabled.

If Firefox is not running, you can start Firefox in Safe Mode as follows:

• On Windows: Hold the Shift key when you open the Firefox desktop or Start menu shortcut.
• On Mac: Hold the option key while starting Firefox.
• On Linux: Quit Firefox, go to your Terminal and run firefox -safe-mode
  (you may need to specify the Firefox installation path e.g. /usr/lib/firefox)

When the Firefox Safe Mode window appears, select "Start in Safe Mode".

If the issue is not present in Firefox Safe Mode, your problem is probably caused by an extension, theme, or hardware acceleration. Please follow the steps in the Troubleshoot extensions, themes and hardware acceleration issues to solve common Firefox problems article to find the cause.

To exit Firefox Safe Mode, just close Firefox and wait a few seconds before opening Firefox for normal use again.

When you figure out what's causing your issues, please let us know. It might help others with the same problem.

I did figure it out, it's a Firefox Extension called Firefox Color that is doing this. I just migrated to a new computer and this came with Firefox, not something I had before, so I disabled it and now the colors are stable.

That was very good work. Well Done.
Please flag your last post as Solved Problem as this can help others with similar problems.