There is Clear Recent History (ctl+shft+del) which you can run before exiting. The custom setting (which I have never used) is, evidently, problematic.

See also "Clear history when Firefox closes" Note that you lose pinned tabs if you use this feature to clear the browsing history.

• Settings -> Privacy & Security
  Firefox will: "Use custom settings for history":
  [X] "Clear history when Firefox closes" -> Settings
• https://support.mozilla.org/en-US/kb/delete-browsing-search-download-history-firefox

• clearing "Site Preferences" clears exceptions for cookies, images, pop-up windows, and software installation and exceptions for passwords and other website specific data
• clearing "Cookies" will remove all cookies including cookies with an allow exception you may want to keep