Is this a pop account or an imap account ? Are you saying the Trash folder became empty ?

Compacting does not delete good wanted emails. So, good emails should not have been deleted. Compacting cleans up the files that have hidden 'marked as deleted' emails, so those emails would have already been deleted and not visible. From your point of view they had been deleted some time ago.

If files were corrupted and were in fact displaying emails previously deleted then their sudden disappearance is explainable. If files were corrupted and Thunderbird was unable to read markers, it might also explain why. Did anything interupt the compacting process eg: you exited Thunderbird or computer started scanning etc? Files can also get corrupted by Anti-Virus products, do you allow your Anti-Virus product to scan Thunderbird profile files and folders? What AV are you using?

Thank you for your reply and thoughtful questions.

After tihs thing happened I was left with EVERY folder having had about ½ of the messages deleted. The Trash folder was NOT fully purged, but affected in just the same way as the other folders. At this time I cannot find any rationale for what was deleted and what was not, although more recent messages seem to have survived.

I have used Compact Folders many times with nothing like this happening; although the was the first time I used it in response to the message. "The messages you have deleted can be purged from disk. This operation will save about xxx of disk space." Obviously purging is not the same as compacting, but I thought it applied only to the Trash folder.

I am not aware of a "marked as deleted" function in Thunderbird -- I have always seen deleted messages go into the Trash folder. I have no idea whether messages have been corrupted in the past -- I have never seen an error message about that.

I drive several email accounts with Thunderbird, they are all pop3 The anti-virus is ESET, which I have been using for more than 5 years with no problem. I was not aware that it might be corrupting the files -- sounds like a good suggestion. It is possible that ESET was running a scan during the Compacting – I did not notice that because I was watching the purge taking place. I need to check whether ESET reviews Thunderbird files, but again, I have been using both products for more than 5 years.

Thanks again for reviewing my problem.

You say about 1/2 of the contents.... is there a cut off date where they just disappear before? or after?

Just to be sure it is not just an indexing problem on the compact, right click one of the folders and select properties and then repair. If on the off chance everything comes back do it on other folders.

Thunderbird stores email in a single file per folder. The start of the file is the earliest email in the folder and the last is the latest. Hence my question about a cut off date. What s generally the case with these type of issue that a revealed by compact is they are historical, sometime years old. Thunderbird maintains an index of mail that is used to populate the user interface and is also used to fetch email of display when you click on them in the lists.

A Compact involves a complete rewrite of the storage file into a new file, less the old previously deleted content, with each email being fetched using the old index This file is then renamed and a new index is generated from it. If the rename messes up NSTMP folders appear in the Thunderbird interface with what appears to be mail duplicated from the whatever folder was being compacted. This is actually the new file before the delete and rename process and is saved just in case the old file has already been deleted when the power went out or whatever.

The issues can be that anti virus in particular can delete the store without Thunderbird noticing. When new mail is added to the folder is is added using the indexed "next" location, so a new file is created the same size as the old and the new message is written. Except most of the content is blank. When the compact occurs a new index is generated with basically missing entries because they are not there to index.

I say anti virus in particular, but these days we are seeing more and more of folks trying to use cloud storage to sync profiles and network storage as the profiles location. Like with the anti virus they can and do cause file truncation etc that result in apparent dataloss at the time of compaction that s historical in nature.

BTW Thunderbird does not notice the file thing, because it is usually not running when the files get clobbered.

More BTW... I use ESET NOD32 and have never had a data loss issue. But I have heard of plenty involving their security suite. I do have email detection permanently disabled along with an exception for the Thunderbird profile folder. My only real issue was last December and I documented it on my blog https://thunderbirdtweaks.blogspot.com/2020/12/eset-stopped-me-getting-mail.html

Thanks for all the info – very helpful in understanding the internals. One clarification for me was that a Thunderbird FOLDER is actually a Windows FILE. In any case, I will outline my latest analysis of the problem, some of which will relate to your suggestions . . . For Local Folders, I had recently activated the Disk Space option “Delete messages more than [70] days old”. Don’t ask me why – I don’t know myself. I mistakenly believed that this was for permanently deleting messages only from the Trash folder. When I got a message asking for an OK for TB to execute this activity, I ignored that invitation, but just took the opportunity to do a global Compact, and that is when I saw things disappearing. It is still puzzling that while all deleted messages were more than 70 days old, there are many many surviving messages that are older than that. In any case, I am pretty sure my arming of that function is what made this Compact different than the others. Until now, had always believed that Compacting had only to do with fragmentation – it never occured to me that it would trigger permanent deletion of valid messages. At this point, I really do not believe that ESET had anything to do with this. For one thing, TB is always running. My problem now is that my backups to Carbonite include folders with deleted messages, at his point those messages may be totally irretrievable.

re :For Local Folders, I had recently activated the Disk Space option “Delete messages more than [70] days old”.

That would explain why you lost a load of emails. You had set up to delete any email stored in the 'Local Folders' mail account that was older than 70 days. Those emails would have been deleted not long after you made that setting. But at that point you would have not realised any change in folder / file size because the compacting process would not have been run.

re :It is still puzzling that while all deleted messages were more than 70 days old, there are many many surviving messages that are older than that.

In the 'Disk Space' setting - Perhaps you selected the checkbox: 'always keep starred messages', so emails stored in the 'Local Folders' mail account that are still visible might be 'starred' emails.

It also would have only effected any emails in 'Local Folders' mail account.

If you have POP mail account(s) and set them up as deferred accounts; commonly known as 'Global Inbox'; you would typically not see separate POP mal account in Folder Pane and all of your pop accounts emails would auto appear in folders under the 'Local Folders' mail account.

Compacting - when you delete or move an email, it is hidden and marked as deleted in that folder (which is actually a text file in the stored profile). The deleted/moved email would then display in another folder. The point is this, the original deleted copy is still in the original folder, so compacting removes that copy which is no longer required. This tidies the file and saves space.