One common problem is Thunderbird mail file corruption caused by anti-virus software.

I was able to recover the missing e-mails from a backup file, but I feel obliged to comment on the "it's the anti-virus software" as I have heard this before, and I think it is patently nonsensical. Here are my observations:

1) the only anti-virus software is the Microsoft Defender and - from what research I have done - upon pattern detection it removes (or quarantines) files - it does not update them. 2) the corrupted "main e-mail file" (that is the one without a filename extension) seemingly got truncated (my backup file was 50% larger than the corrupted one) and I could find no appearance of the missing e-mails in the corrupted file. 3) looking at the corrupted main e-mail file, the data seems to be in reverse chronological order (older elements appearing subsequent to new ones) - I have no idea if (a) they are added in this order (seems unusual/difficult to implement) or (b) file compaction rewrites the main file in reverse chronological order. 4) I am not seeing any other indicia of file truncation on the system.

Jonathan

Thanks for offering so much detail. I will attempt to add some more...

I was able to recover the missing e-mails from a backup file, but I feel obliged to comment on the "it's the anti-virus software" as I have heard this before, and I think it is patently nonsensical.

https://wiki.mozilla.org/Thunderbird:Testing:Antivirus_Related_Performance_Issues suggests otherwise.

Here are my observations: 1) the only anti-virus software is the Microsoft Defender and - from what research I have done - upon pattern detection it removes (or quarantines) files - it does not update them.

There are two types of email scanning: 1) intercepting incoming messages before they get the mail client, 2) scanning email that has been saved by the mail client on disk.

In the case of #1, the AV can't affect your mail files.

In the case of #2, the AV certainly can affect your mail files. Unless you have either a) you have enabled enabled "Allow ... quarantine" in Thunderbird Settings > Privacy & Security, or b) changed your Thunderbird account message storage to maildir

2) the corrupted "main e-mail file" (that is the one without a filename extension) seemingly got truncated (my backup file was 50% larger than the corrupted one) and I could find no appearance of the missing e-mails in the corrupted file.

noted.

3) looking at the corrupted main e-mail file, the data seems to be in reverse chronological order (older elements appearing subsequent to new ones) - I have no idea if (a) they are added in this order (seems unusual/difficult to implement) or (b) file compaction rewrites the main file in reverse chronological order.

Knowing what is posted in support, I've never seen anyone report messages stored in reverse order.

Knowing the code ... this is quite impossible. Messages are copied in their original order, which would be chronological. Even if it was possible, we'd have other reports of this. And, it is many, many months since the compact code was rewritten to be more robust.

4) I am not seeing any other indicia of file truncation on the system.

Such observations are not sufficient disprove AV involvement.

This is not to say it is impossible for the problem you are reporting to have been caused partly or completely by Thunderbird. But there are other ways.

Wayne – Thanks for your attention to this. A few more data points and speculations for you. I am going to date myself by admitting that it has been decades since programming Windows in ASM86 and C, Windows message loop programming, et.al. But I do remember a bunch of that stuff …. 1. I looked a the corrupted no-extension file (using Notepad, which is not a good tool for this), and while I saw some date inversions, they were likely e-mail forwards or replies. Not that I thought Thunderbrid had some way of prepending new e-mails to that file (that is not a thing that Windows’ Kernel32.dll supports). 2. But that fact that the missing e-mails would have be the first to have been written to that file suggests that it is Thunderbird that is messing up during some compaction process (I cannot imagine an “operator error” that would delete messages without updating the .msf file and the .msf file still referenced the missing e-mails - let alone several hundred e-mails). I certainly did not run any other piece of software that should have opened the file. 3. I would also note that 2-6 months ago (on whatever version of Thunderbird I thought current at that time – I do update regularly) I was seeing some bad behavior. Specifically, a. Thunderbird would silently fail to complete an operation b. Task Manager would report small CPU utilization (and other applications would be completely responsive, so I think the Task Manager data was accurate) c. Clicking on the red “x” at the top-right corner would get the “application not responding” message, which I believe Windows issues after some time interval of no interrogation of an application’s Windows event queue. The only way I know of to get into this condition is for Thunderbird to be looping over a call to “WaitSingleObject” system call. I imagine that Thunderbird has all sorts of functionality running in separate threads, and I further speculate that this can get “messed up”.

Thanks for the added info.

It is certainly possible for missing messages in the file to be cause by Thunderbird, especially if the file is already corrupted prior to whatever operation - move, compact, etc. I have a few corrupted folders - not missing messages, but contains partial messages - mostly caused a few years ago. But I do have partial recent message in the past few months, likely related to some message filtering. I do see that we do have a new pending for properly gating folder updates, so may that will help in some areas.

There def are many threads but mostly from imap. A simplified description is pretty much everything Thunderbird-specific runs on main thread (including pop, IO, UI etc), iirc import runs it's single thread. imap spins up multiple threads.

In the case of Thunderbird, waitsingleobject is often IO, sound, or some other SW, including AV.

Also, I should have included "but can affect Thunderbird performance and stability." to my earlier statement "In the case of #1, the AV can't affect your mail files."