Most of the tings you mention would indicate you have an anti virus (or other resident file system monitoring like cloud sync or streaming backups) that is interfering with file access.

Regular appearances of NSTMP file is the first clue. You really only get them when there is a system shutdown mid compact, or something else is messing with the Profile folder while Thunderbird is trying to update itself.

Basically try life without whatever it is (probably antivirus) that is scanning and monitoring the profile folder while it is in use. We generally recommend a general exclusion in antivirus product for the Thunderbird profile folder. Other symptoms as the duplication of standard files with the number appended. Thunderbird expects it's files to be available, if they are not it generally creates anew file. I am not sure of the wisdom of that, but it is how it works. This does cause other issues for folk, like having multiple prefs.js files. So changes get saved, but never accessed because only the original prefs.js is read on startup. Or for pop accounts the same mail gets downloaded again later because the last update went into a numbers popstate.dat file, so is not subsequently accessed.

The maximum number of connections is generally defaulted to 5 in account settings > server settings > advanced reducing this to three is often required when working with Yahoo as they are just special I guess.

I added the folder to the exclusions, but I think it's more likely that because my TB hangs so often, and after many minutes of not knowing what it's doing I control-alt-delete and restart it. When it's re-indexing a folder it does not do a very good job of managing resources so you can keep working.

But the question remains, how should I clean up the years worth of stuff in the profile that is doing nothing?

Yet another symptom if anti virus scanning.

POP mail accounts need to be compacted. he inbox especially so as mail that is deleted is only marked as such. It is not actually removed until a successful compact occurs. If you have pop accounts use the file menu to compact them. This can have a huge impact on not only file size but general usability.

You might also want to try https://support.mozilla.org/en-US/kb/rebuilding-global-database in your scenario is may well be in a bad way.

Do you have more that 4Gb of Ram? Do you use saved search folders?

I have 12Gb Ram. I use IMAP not POP, but I compact constantly. I rebuilt the global database this morning and at least the folders re-synced. Now it's just been taking all day to re-index a couple of them. I have one saved search folder of large files which I would be happy to delete.

What I find with the search folders is the hold open all the folders that they search. This in my case at least consumed some 2gb of ram in Thunderbird process. All I know is hangs went way down once I did not have it.

I deleted that folder and things sped up a little so that was a good tip to know. My email still keeps re-indexing and downloading one folder over and over, and I have another folder showing in the tree with a blank name, and it constantly gets a sync error because it's not on the server. Maybe because it is blank it doesn't let me delete it. I did rebuilt the global index again and it didn't fix these issues.

I deleted that folder and things sped up a little so that was a good tip to know. My email still keeps re-indexing and downloading one folder over and over, and I have another folder showing in the tree with a blank name, and it constantly gets a sync error because it's not on the server.

Who is your mail provider?