DUPLICATE EMAILS RECEIVED
There is a similar thread on this here: https://support.mozilla.org/en-US/questions/1005565
Like that user, I have POP3. The main symptom is that emails that are one month old are downloaded a second time from the server each day (i.e. today is May 12, so it downloaded emails from April 12). I have had problems with duplicate emails for the last 10 years I have used Thunderbird. I can't believe this problem has not yet been solved. I hope there are better workarounds than those below.
With POP3, I keep my email on the server rather than deleting it, because I want my cell phone to be able to access as well. And additionally, I want it to stay on the server as a backup. Although IMAP would likely not have this problem, I don't like the requirement than the server folder structure matches IDENTICALLY to that of Thunderbird. I just want it to download new messages and then let me do with them what I want without my actions to the email message having any effect on the message on the server. That's what POP3 does.
The solutions offered in the other thread were:
(1) First check this and set Thunderbird to delete messages from the server: go to "Tools -> Account Settings -> Server settings", and uncheck "Leave messages on server."
UNACCEPTABLE: I want my messages on the server.
(2) If not you could also try Compacting the Mail folders or Compact folders, exit Thunderbird, and go to your mail folders (in your profile folder). Delete the index files, named "*.msf" (such as "Inbox.msf"). These will be rebuilt when your launch the application again.
I HAVE DONE THIS BEFORE. The problem has come back anyway, so I am not convinced it would help. If this does *temporarily* actually fix something, I would appreciate an explanation about why it helps and why the problem comes back. It might be a solution. I just want to know what is broken.
(3) There is a file in your profile (Mail subfolder, then the subfolder for this mail account) called popstate.dat. It contains the information about which messages have been downloaded from the server. Possibly your popstate.dat file has been corrupted. Exit ThunderBird, find and delete (or rename) the popstate.dat file for that mail account, restart ThunderBird. It will download all messages residing on the server one more time, but should download only new messages after that. Where to find your profile: http://kb.mozillazine.org/Profile_folder
NO WAY. There are thousands of messages. I do not a duplicate of every one of those messages.
Is the problem that sometimes Thunderbird is improperly shut down, and that the popstate.dat is corrupted? Is there a way for popstate.dat to be corrected.
I am also going to try the add-on mentioned in the above 'remove duplicate email'. I am not sure how it works. It may also be a solution.
Mostly I would just like an explanation here what the problem is that has gone on for some many years, which I keep hoping will fix itself with a new version of Thunderbird and never goes away. :-(
-David
All Replies (8)
Looking at this you say mail is downloaded a month latter. My guess with basically zero data is that your mail provider is doing something to age mail, perhaps some sort of auto archiving process that is changing the message ids. The record Thunderbird has has to match the server. If the server ids change then they get downloaded again as their is no match in the popstate.dat file.
I assume your provider is small scale, as I have been floating around Thunderbird forums for a number of years and have never heard of such an issue as you describe before. I doubt the problem will be solved as no one knows there is a problem.
Thanks for the reply. I am using Hotmail. More than one hotmail account does this. I haven't noticed this problem on Gmail accounts, so you might be right that it is something funky about hotmail settings related to an archive bit, etc.
Is there a way to compare the duplicate message(s) with the original to see if there is a difference that would explain what you think might be going on? I am familiar with unix "diff.exe", "cmp.exe" and "od.exe", as well as a similar utility for DOS which I think is comp.exe. I'm not sure what differences I would be looking for that would be substantive and whether there could be differences when I save them that are not substantive.
-DT
There will be no differences. The mails will be identical in every regard. What will be different is the internal server index number used to identify the emails.
My suggestion is; Set Thunderbird to delete messages from the Outlooks servers. Set Outlook.com to move mail into the special POP folder on the server when it is deleted by a mail client.
So your mail remains on the server in a different folder and Thunderbird or any POP mail client will not see if because it is not in the inbox.
Just a note on your original (2). Compacting writes a completely new mail file and reindexes it when it is done. So a new MSF is created
Further, it may not be Hotmail or Thunderbird there is a third player. Your anti virus program. When you suggest incorrect shut down of Thunderbird. It is more likely incorrect termination of the mail transfer because the anti virus burped. If you use Norton. Simply turn off mail scanning. It does not work on SSL secured connections anyway.
SIncerily........I have this problem by many years and I´m so tired to try solving. My problems are with gmail ND I don´t know to read/write english very well, so I don´t know what to do.
I rename all the MSF to MS2 to try to solve the problem, but apperas many new folders in Thunderbird with the name of MS2. The problem occurs with POP and IMAP accounts. I´m so crazy. My skype = robertoraimondojr
My e-mail robertoraimondojr@gmail.com If someone wants to have remote access to my notebook to try helping me, it´s OK for me.
Thanks, Matt. I will try your suggested solutions to this the next time this comes up. I'm not yet ready to hit "reply solved your problem", but I would like to think it has.
-DT
Here is a potential solution (which has worked for me) - an existing add-on for Thunderbird. Here is the link to the webpage: https://addons.mozilla.org/en-us/thunderbird/addon/remove-duplicate-messages/
It might be a little slow in execution, but still much better than *manually* removing all duplicates.
Regards, Eric J