Where did you install Firefox from? Help Mozilla uncover 3rd party websites that offer problematic Firefox installation by taking part in our campaign. There will be swag, and you'll be featured in our blog if you manage to report at least 10 valid reports!

Avatar for Username

Search Support

Avoid support scams. We will never ask you to call or text a phone number or share personal information. Please report suspicious activity using the “Report Abuse” option.

Learn More

This thread was archived. Please ask a new question if you need help.

Thunderbird with Dovecot, sent messages not getting removed from full text search backend

  • 1 reply
  • 1 has this problem
  • 7 views
  • Last reply by elyograg

elyograg
elyograg
more options

I am using dovecot for IMAP services, dovecot is configured with Solr as it's full text search backend. One day I was working with Thunderbird as my mail client, and watching Solr's log file at the same time. I was noticing that dovecot was not sending delete requests to Solr when messages were getting expunged. I noticed it when using shift-delete in Thunderbird, to delete a message immediately, bypassing the Trash folder. For that, I have a workaround -- don't use shift-delete.

Then I noticed the same thing happening when sending a message -- dovecot was not sending a delete request to Solr for the message that had been temporarily saved in the Drats folder. Test steps:

1) Begin email message with unique text in message body. 2) Wait a while so Thunderbird saves the unsent email to Drafts folder. 3) Query Solr for unique text in message, see one message in results, note ID value: 

 198/6a146e1262248061a9190f00b69bc403/elyograg@elyograg.org

4) Send message. Query Solr again, see two results, ID values: 

 198/6a146e1262248061a9190f00b69bc403/elyograg@elyograg.org
 10796/63146e1262248061a9190f00b69bc403/elyograg@elyograg.org

Second ID value is for the message saved to the Sent folder. Same test with TypeApp, an android mail client, behaves correctly. There is one search result at step 4. TypeApp does not have an expunge message option like Thunderbird does. All deleted messages go to the Trash folder, I could not find a way to bypass Trash.

When deleting messages "normally" (delete sends message to Trash, then I empty Trash), everything works fine with Thunderbird. I use shift-delete so I won't have to remember to empty the Trash folder. Behavior is the same with an older version of Thunderbird on Linux and the latest version on Windows.

I asked about this on the dovecot mailing list, because it seemed like the problem was there. They asked if I had tried any other mail clients. At the time I hadn't. I was going to try Windows Mail, but it could not connect to my mailbox, so it never gave me the option to visit advanced settings to change port numbers or change TLS to STARTTLS like Thunderbird does. The only other mail client I have access to is the one on my phone, and as noted, it works correctly.

I do have a way to fix this problem. My mail server, which handles mail for me and a few others, can do a full reindex in about 8 minutes to clean up expunged messages from Solr. But there are dovecot installations that house billions of email messages. A full reindex for those admins could take several days -- not practical.

Is this a bug, or have I just got something configured wrong? I checked the Config Editor for settings that contain "expunge" ... all those options are at default settings and I do not know what some of them will do, so I haven't changed anything.

I am using dovecot for IMAP services, dovecot is configured with Solr as it's full text search backend. One day I was working with Thunderbird as my mail client, and watching Solr's log file at the same time. I was noticing that dovecot was not sending delete requests to Solr when messages were getting expunged. I noticed it when using shift-delete in Thunderbird, to delete a message immediately, bypassing the Trash folder. For that, I have a workaround -- don't use shift-delete. Then I noticed the same thing happening when sending a message -- dovecot was not sending a delete request to Solr for the message that had been temporarily saved in the Drats folder. Test steps: 1) Begin email message with unique text in message body. 2) Wait a while so Thunderbird saves the unsent email to Drafts folder. 3) Query Solr for unique text in message, see one message in results, note ID value: 198/6a146e1262248061a9190f00b69bc403/elyograg@elyograg.org 4) Send message. Query Solr again, see two results, ID values: 198/6a146e1262248061a9190f00b69bc403/elyograg@elyograg.org 10796/63146e1262248061a9190f00b69bc403/elyograg@elyograg.org Second ID value is for the message saved to the Sent folder. Same test with TypeApp, an android mail client, behaves correctly. There is one search result at step 4. TypeApp does not have an expunge message option like Thunderbird does. All deleted messages go to the Trash folder, I could not find a way to bypass Trash. When deleting messages "normally" (delete sends message to Trash, then I empty Trash), everything works fine with Thunderbird. I use shift-delete so I won't have to remember to empty the Trash folder. Behavior is the same with an older version of Thunderbird on Linux and the latest version on Windows. I asked about this on the dovecot mailing list, because it seemed like the problem was there. They asked if I had tried any other mail clients. At the time I hadn't. I was going to try Windows Mail, but it could not connect to my mailbox, so it never gave me the option to visit advanced settings to change port numbers or change TLS to STARTTLS like Thunderbird does. The only other mail client I have access to is the one on my phone, and as noted, it works correctly. I do have a way to fix this problem. My mail server, which handles mail for me and a few others, can do a full reindex in about 8 minutes to clean up expunged messages from Solr. But there are dovecot installations that house billions of email messages. A full reindex for those admins could take several days -- not practical. Is this a bug, or have I just got something configured wrong? I checked the Config Editor for settings that contain "expunge" ... all those options are at default settings and I do not know what some of them will do, so I haven't changed anything.

All Replies (1)

elyograg
elyograg Question owner
more options

Something completely unexpected: A little over 15 minutes after my last "send" test, I quit Thunderbird so I could reboot my work laptop. Version is Thunderbird 78.14.0 from Ubuntu packages. When I did the Quit, I immediately saw a delete in the Solr log for the message that had been in Drafts.

Then I tried another test: I shift-deleted a message. Nothing in Solr's log, until I quit Thunderbird, at which point the delete request showed up.

Are there two parts to a full expunge operation, and Thunderbird is waiting before it does the second half? If so, is there a config option I can change that controls that behavior?