Private Browser can breaks sites as some sites require cookies to work. So it's opposite of what your asking here.

https://support.mozilla.org/en-US/kb/private-browsing-use-firefox-without-history

In what way are sites not working in non-private windows?

The main differences between private windows and regular windows are:

• Private windows do not re-used cached files from regular windows. Workaround: You can clear Firefox's web content cache from time to time. See: How to clear the Firefox cache.

• Private windows use temporary cookies instead of cookies saved on previous visits to the site. Workaround: When a site is acting up, you can clear the previously set cookies for the site. Steps below the line.

• Private windows may not run all of your add-ons -- each extension needs to be enabled to run in private windows. Workaround: If an extension is slowing your regular window browsing, you can disable or remove it. Steps below the line.

Removing the Current Site's Cookies

While viewing a page on the site, click the lock icon at the left end of the address bar. After a moment, a "Clear Cookies and Site Data" button should appear at the bottom. Go ahead and click that. Firefox will ask you to confirm; go ahead and confirm.

Then try reloading the page bypassing the cache (Shift+click the Reload button on the toolbar, or press Ctrl+Shift+R). Does that help?

Extension Review

You can view, disable, and often remove unwanted or unknown extensions on the Add-ons page. Either:

• Ctrl+Shift+a (Mac: Command+Shift+a)
• "3-bar" menu button (or Tools menu) > Add-ons
• type or paste about:addons in the address bar and press Enter/Return

In the left column of the Add-ons page, click Extensions. On the right side, find the "Manage Your Extensions" heading.

If there is at least one extension before the next heading -- "Recommended Extensions" -- please continue:

Then cast a critical eye over the list below that heading. Any extensions Firefox installs for built-in features are hidden from this page, so everything listed here is your choice (and your responsibility) to manage. Anything here that could block content in pages -- such as ad blockers and privacy tools -- or that you just do not remember installing or why? If in doubt, disable (or remove). For your privacy and security, don't let mystery programs linger here.

Any improvement with minimal blocking/interference?