Also when the player does work it does not seem to load the MP3 as aggressively as in the past and frequently will give up and stop playing the MP3 even when there is a solid connection.

As a work around use Open with App and use the built-in Sound player

I usually recommend the work around you already mentioned. I tested on a Note 4

• https://archive.org/details/testmp3testfile

There is an add on that may help OpenH264 video codec provided by cisco

• http://www.openh264.org/

This is what it looks like when the MP3 player gets stuck and doesn't seem to initialize at all:

What codec are you using in the link you are testing initialization?

Not sure what codec is being used but this is the link:

https://studiovisitshow.files.wordpress.com/2014/11/marisa.mp3

I guess it's a standard MP3 but I don't know

This behaviour is inconsistent BTW

On first visit the MP3 player loads fine. On second visit it gets stuck.

This is a known issue that Mozilla is trying to resolve. In the meantime, try downloading the MP3 file and play it on the MP3 player that comes with your device instead.