TBird Set Up - Getting it right the first time (Keep some, but not all email on server, while keeping all email locally)
Hello! I want to set up TBird to: 1) Get all my messages and attachments stored locally, and... 2) Remove *most* of the above from gmail's server to avoid the data limit, but... 3) Keep, perhaps some months or a year of activity on gmail's server so I can access recent items from wherever.
Questions: 1) I have 10s of thousands (more?) of msgs, many with attachments. Is it better to just set up TBIrd and let it grab all the messages, or start by using Google TakeOut? 2) I presume I use IMAP vs. POP? 3) To preserve messages on my harddrive, I could archive to a local folder. Then I could safely delete those same messages on the gmail web interface? 4) If this turns into a disaster, can I just remove the entire account from TBird without suffering a loss of my entire email history? 5) Any insurance maneuvers to prevent the disaster of accidentally losing my entire email history while attempting to set this up? (Obviously will need to be religious about backing up my own system, though keeping a year of email on the gmail server offers a bit of insurance once the set up is successful.)
I am really hoping to avoid creating a unholy mess of my email/spending a week trying to fix it. (Indeed, I have avoided this project for years for this reason, but I really, really want to do this.)
All Replies (6)
My suggestion:
- keep IMAP to ensure all remains accessible from wherever
- for archiving, I suggest this:
- set up local folder for each archive
- select and copy messages to be archived and removed from server to the new local folder
- check to verify that you have both content and header of the new archive in the new folder
- now, you can delete from server
- repeat the above four steps for each archive.
Specifically, do not use the thunderbird archive feature. use your own archiving setup, as suggested above. AND, be religious on backups.
To be clear, you are recommending AGAINST the archive process described in the link below, even if I configure it to archive to a local folder? https://support.mozilla.org/en-US/kb/archived-messages
Any thoughts on using Google Takeout or not?
Thank you!
That's right; although there may be better approaches than mine, not losing messages is my top priority, and by manually controlling it, you avoid abdicating it to Thunderbird. That works ok for POP, but IMAP is a different animal. I confess to total ignorance on google takeout. Also, my approach is my personal strategy; some may say I am wrong, but I see too many people losing messages from IMAP downloads where they end up with just the headers. ..
I'm in the midst of the same process. I have Google Takeout, and have fiddled with it some, but am not presently trying to use it for this process; mostly because when I succeed getting things properly set up using Thunderbird with IMAP everything will be synchronized (except for the old messages that still need to be deleted from the Gmail All Mail folder); by using takeout I could get everything where I want it in Local Folders, but then I'd still need to clean up Gmail. (To use Takeout you need to import mbox files.) (On my setup it only took something like 20 hours or so to start from scratch and fully synchronize with Gmail.)
I'm still trying to understand the differences in the Thunderbird archive setup choices- so far I have not found an explanation that (to my mind) sufficiently explains the difference between ["Archives" Folder on:] and [Other:]. (Perhaps the only difference is the presence of a top level folder named "Archives"?)
I am exploring whether the Import Export Tools NG Extension will prove useful.
One thing I have read is that the Thunderbird Archive process is manually driven - since I also have thousands of messages in my history I'm looking for a tool that is somewhat automated. (Here is where I agree with David - not losing messages remains the top priority.)
So far, my takeaway from this convo is that this isn't going to work, if for no other reason that I won't be able to reasonably expect to select tens of thousands of emails and both quickly (ish) and safely archive them. Is that right?
is there way to take a completely different tactic, and download all my messages, then somehow sever that connection to gmail (thus preserving all messages), then set up a second T-Bird instance of the same gmail account to use to delete messages/view any new messages from that point going forward?
Has anyone found a good method (perhaps with a different client?) to do what I'm trying to do? (ie - download all messages, then delete most from server but keep all messages locally, thus vastly reducing the amount of data on the gmail server?
Let me encourage you to do what I did (since I didn't keep notes of where I found what information it would be difficult for me to generate a list of links) and search this forum / read knowledgebase articles. I used a variety of search terms, but found enough information to make progress.
The general process I followed: (these steps are [mostly] all on gmail) (general process - not detailed step by step instructions) - Make sure that special gmail folders are not being synchronized- All Mail especially, then Purchases, Social, Updates, and etc. (These seem to be additional gmail labels, and would result in Tbird downloading them multiple times.)
-- Gmail | Settings | Labels | Show in IMAP -- Thunderbird | Account Settings | Synchronization & Storage --- CHECK Keep messages in all folders on this computer --- Advanced ... Tick or Clear each folder
- (I downloaded thousands of duplicates before I learned what was happening.) - Manually, on Gmail, verify that ALL messages are in some folder. That is, no unlabeled messages. This was a bit time consuming, because I didn't realize all the ways gmail was doing things under the covers that were breaking what I was trying to do. For instance: the search command [ has:nouserlabels ] was failing because of messages that gmail added special labels to.
Also, in conversation view (which I use), a labelled thread can have unlabelled messages. The simplest way I found to fix this was to create a dummy label (I used ,aaa) then: 1. select ALL messages in one label (myLabel) and add label ,aaa 2. select ALL messages in ,aaa and under the Labels button, CLEAR the ,aaa checkbox and SET the tri-state checkbox for (myLabel). This will add the label to any unlabeled messages. 3. Repeat for all your labels.
I ran into trouble because I got impatient and didn't wait for gmail to finish executing the command before starting the next step. With tens of thousands of messages in some labels the process took some time.
Because I'm not syncing All Mail it's necessary to assign a label to every message that I want to preserve. For has:nouserlabels to correctly find all the messages I wanted, I had to remove all the special gmail labels. (If you organize your mail by folders (gmail labels) All Mail will be everything in one big pile - no labels.)
4. Make certain that Tbird has downloaded ALL your messages, not just headings.
5. Close Tbird. Using your OS file manager, COPY (don't move) the imap.gmail.com [ default name for only one gmail account ] folder from Your profile's ImapMail folder to your profile's Mail/Local Folders folder. When you re-start Tbird it will find your duplicated structure under Local Folders.
6. After verifying that everything is in your local folders, you can use either Tbird or gmail to delete whatever you want.
Note: Even after compacting all folders, the IMAP created mbox files can be dramatically larger than a new mbox folder that you copy messages into. For instance: using ImportExportTools NG Search and Export Messages to export all messages from a large IMAP mbox to a new mbox file I saw a compacted mbox drop from 2.6GB to 1.3GB. I'm still in the process of verifying, but it appears that all the messages and attachments are in the copy.
So - moving messages from gmail (any IMAP provider, actually) to local storage is possible. Is there a magic tool available? Not that I have found.
The time consuming piece for me was learning all the ways I didn't fully understand what gmail and Tbird were doing. Once Tbird had fully downloaded all the messages, creating the local copies was fast - simply an OS file manager copy. [ Shrinking the final mbox sizes adds time but is an optional step. As far as I know, the only tradeoff is disk storage space (vs) the time to export messages one folder at a time. ]
Now, the housekeeping for freeing up space on gmail --- I used the gmail web interface; I believe using Tbird's search and selection and bulk deletion would have worked as well. Which way has more (network traffic) overhead? I have no idea. So: use the tools (gmail or Tbird) that are most comfortable for you.
(It took me about 2 1/2 days to figure this out and transfer something like 80,000 messages.)