excessive memory use in database imap and heap unclassified that gradually but continually increases, up to 30gb [Mac]
Thunderbird will continue using more and more memory the longer it is open. In a week or so, it can get as high as 30GB of memory. Closing and reopening fixes it. I have done the memory diagnostics, and my guess is that it is the fact that I have a large amount of shared imap folders that I am subscribed to. I have 18 shared IMAP folders, many of which contain years of email, equivalent of thousands of messages, so it's definitely a high scale use case but I have the server settings set to only sync the last 30 days of messages. Happy to provide any additional info that might be helpful, but you can replicate this yourself via https://trac.tools.ietf.org/group/tools/trac/wiki/Imap - that said, I am keeping a copy of the messages locally so that I have offline access.
Below is the basic troubleshooting info, and the memory report. I tried doing free GC and CC but this doesn't seem to have had any effect on the memory allocated according to the OS.
Application Basics
Name: Thunderbird Version: 52.7.0 User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 Profile Folder: Show in Finder Application Build ID: 20180322145026 Enabled Plugins: about:plugins Build Configuration: about:buildconfig Memory Use: about:memory Profiles: about:profiles
Mail and News Accounts
account1:
INCOMING: account1, , (imap) puck.nether.net:143, alwaysSTARTTLS, passwordCleartext
OUTGOING: , puck.nether.net:587, alwaysSTARTTLS, passwordCleartext, true
account2:
INCOMING: account2, , (none) Local Folders, plain, passwordCleartext
account3:
INCOMING: account3, , (imap) imap.ietf.org:143, alwaysSTARTTLS, passwordCleartext
OUTGOING: , puck.nether.net:587, alwaysSTARTTLS, passwordCleartext, true
Crash Reports
Extensions
Enigmail, 2.0.3, true, {847b3a00-7ab1-11d4-8f02-006008948af5}
Lightning, 5.4.7, true, {e2fda1a4-762b-4020-b5ad-a41df1933103}
Important Modified Preferences
Name: Value
browser.cache.disk.capacity: 358400
browser.cache.disk.filesystem_reported: 1
extensions.lastAppVersion: 52.7.0
font.internaluseonly.changed: false
layers.acceleration.disabled: false
mail.openMessageBehavior.version: 1
mail.spotlight.enable: true
mail.spotlight.firstRunDone: true
mail.spotlight.global_reindex_time: 1511276263
mailnews.database.global.datastore.id: abedc333-2b45-e049-9d13-22eb8801b7a
media.gmp.storage.version.observed: 1
network.cookie.prefsMigrated: true
network.predictor.cleaned-up: true
places.database.lastMaintenance: 1526071601
places.history.expiration.transient_current_max_pages: 122334
privacy.donottrackheader.enabled: true
security.sandbox.content.tempDirSuffix: {b72aaf77-b07d-d349-b940-b9107e1f537b}
Graphics
GPU #1
Vendor ID: 0x8086
Device ID: 0x0d26
Features
WebGL Renderer: Intel Inc. -- Intel Iris Pro OpenGL Engine -- 2.1 INTEL-10.25.19
AzureCanvasBackend: skia
AzureCanvasAccelerated: 1
AzureFallbackCanvasBackend: none
AzureContentBackend: skia
TileHeight: 1024
TileWidth: 1024
JavaScript
Incremental GC: 1
Accessibility
Activated: 0 Prevent Accessibility: 0
Library Versions
Expected minimum version
Version in use
NSPR
4.13.1
4.13.1
NSS
3.28.6
3.28.6
NSS Util
3.28.6
3.28.6
NSS SSL
3.28.6
3.28.6
NSS S/MIME
3.28.6
3.28.6
Geändert am von Wayne Mery
Alle Antworten (6)
The memory report exceeds the 10,000 character limit, so I'll break it up into two:
Part 1: Main Process (pid 58244)
WARNING: the following values are negative or unreasonably large.
explicit/maildb explicit/maildb/database(imap://wesley.george%40twcable.com@imap.ietf.org/Shared%20Folders/ietf) explicit/(26 tiny) explicit/(26 tiny)/heap-unclassified
This indicates a defect in one or more memory reporters. The invalid values are highlighted. Explicit Allocations
7,607.27 MB (100.0%) -- explicit ├──15,150.37 MB (199.16%) -- maildb [?!] │ ├──13,871.80 MB (182.35%) ── database(imap://wesley.george%40twcable.com@imap.ietf.org/Shared%20Folders/ietf) [?!] │ ├─────231.02 MB (03.04%) ── database(imap://wesley.george%40twcable.com@imap.ietf.org/Shared%20Folders/i2rs) │ ├─────204.42 MB (02.69%) ── database(imap://wesley.george%40twcable.com@imap.ietf.org/Shared%20Folders/grow) │ ├─────179.22 MB (02.36%) ── database(imap://wesgeorge@puck.nether.net/Trash) │ ├─────166.13 MB (02.18%) ── database(imap://wesgeorge@puck.nether.net/INBOX) │ ├─────147.97 MB (01.95%) ── database(imap://wesgeorge@puck.nether.net/lists/NANOG) │ ├─────133.92 MB (01.76%) ++ (7 tiny) │ ├─────110.84 MB (01.46%) ── database(imap://wesley.george%40twcable.com@imap.ietf.org/Shared%20Folders/dots) │ └─────105.04 MB (01.38%) ── database(imap://wesley.george%40twcable.com@imap.ietf.org/Shared%20Folders/i2nsf) └──-7,543.09 MB (-99.16%) -- (26 tiny) [?!]
├──────72.09 MB (00.95%) ++ js-non-window ├──────65.42 MB (00.86%) ++ storage ├──────46.59 MB (00.61%) ++ gfx ├──────45.76 MB (00.60%) ++ heap-overhead ├──────26.92 MB (00.35%) ++ workers/workers(chrome) ├──────20.59 MB (00.27%) ++ window-objects ├───────9.42 MB (00.12%) ++ add-ons ├───────8.16 MB (00.11%) ++ images ├───────6.18 MB (00.08%) ++ network ├───────4.76 MB (00.06%) ++ xpconnect ├───────3.47 MB (00.05%) ── spell-check ├───────2.28 MB (00.03%) ++ atom-tables ├───────2.22 MB (00.03%) ++ startup-cache ├───────1.78 MB (00.02%) ++ layout ├───────1.27 MB (00.02%) ── xpti-working-set ├───────0.70 MB (00.01%) ── preferences ├───────0.65 MB (00.01%) ── telemetry ├───────0.30 MB (00.00%) ++ xpcom ├───────0.21 MB (00.00%) ++ dom ├───────0.19 MB (00.00%) ── icu ├───────0.13 MB (00.00%) ++ cycle-collector ├───────0.01 MB (00.00%) ── script-namespace-manager ├───────0.00 MB (00.00%) ── cookie-service ├───────0.00 MB (00.00%) ── history-links-hashtable ├───────0.00 MB (00.00%) ── media/libogg └──-7,862.20 MB (-103.35%) ── heap-unclassified [?!]
Part 2: Other Measurements
67.01 MB (100.0%) -- decommitted ├──50.99 MB (76.10%) -- workers/workers(chrome) │ ├──47.04 MB (70.20%) ── worker(resource://enigmail/enigmailprocess_worker_unix.js, 0x119d90000)/gc-heap/decommitted-arenas │ ├───3.48 MB (05.19%) ── worker(resource://enigmail/enigmailprocess_worker_unix.js, 0x1293d1800)/gc-heap/decommitted-arenas │ └───0.48 MB (00.71%) ── worker(resource://gre/modules/osfile/osfile_async_worker.js, 0x125ccb800)/gc-heap/decommitted-arenas └──16.02 MB (23.90%) ── js-non-window/gc-heap/decommitted-arenas
5,324 (100.0%) -- event-counts └──5,324 (100.0%) -- window-objects
├──3,200 (60.11%) -- top(chrome://messenger/content/messenger.xul, id=3)/active │ ├──3,115 (58.51%) -- window(chrome://messenger/content/messenger.xul)/dom │ │ ├──3,091 (58.06%) ── event-listeners │ │ └─────24 (00.45%) ── event-targets │ ├─────72 (01.35%) -- window(chrome://messenger/content/msgAccountCentral.xul)/dom │ │ ├──71 (01.33%) ── event-listeners │ │ └───1 (00.02%) ── event-targets │ └─────13 (00.24%) ++ window(about:blank)/dom ├──1,611 (30.26%) -- top(none)/detached │ ├──1,610 (30.24%) ── window(chrome://messenger/content/messengercompose/messengercompose.xul)/dom/event-listeners │ └──────1 (00.02%) ── window(about:blank)/dom/event-listeners ├────471 (08.85%) -- top(chrome://messenger/content/hiddenWindow.xul, id=1)/active │ ├──468 (08.79%) -- window(chrome://messenger/content/hiddenWindow.xul)/dom │ │ ├──467 (08.77%) ── event-listeners │ │ └────1 (00.02%) ── event-targets │ └────3 (00.06%) ++ window(about:blank)/dom └─────42 (00.79%) ++ (6 tiny)
7,528.84 MB (100.0%) -- heap-committed ├──7,483.08 MB (99.39%) ── allocated └─────45.76 MB (00.61%) ── overhead
8.16 MB (100.0%) -- images ├──6.00 MB (73.51%) -- uncached │ ├──5.29 MB (64.92%) -- vector/used │ │ ├──5.09 MB (62.44%) ── source │ │ ├──0.14 MB (01.71%) ── decoded-heap │ │ └──0.06 MB (00.77%) ── decoded-nonheap │ └──0.70 MB (08.59%) -- raster/used │ ├──0.37 MB (04.50%) ── decoded-nonheap │ ├──0.20 MB (02.50%) ── source │ └──0.13 MB (01.59%) ── decoded-heap ├──2.16 MB (26.46%) -- chrome │ ├──1.70 MB (20.85%) -- vector/used │ │ ├──1.61 MB (19.73%) ── source │ │ └──0.09 MB (01.12%) ── decoded-heap │ └──0.46 MB (05.61%) -- raster/used │ ├──0.36 MB (04.41%) ── decoded-nonheap │ └──0.10 MB (01.20%) ++ (2 tiny) └──0.00 MB (00.03%) ++ content/raster/used
91.83 MB (100.0%) -- js-main-runtime ├──42.19 MB (45.95%) -- compartments │ ├──28.33 MB (30.85%) -- classes/objects │ │ ├──22.45 MB (24.45%) ── gc-heap │ │ └───5.87 MB (06.40%) -- malloc-heap │ │ ├──3.18 MB (03.46%) ── slots │ │ ├──2.68 MB (02.92%) ── elements/normal │ │ └──0.02 MB (00.02%) ── misc │ ├───6.42 MB (07.00%) -- scripts │ │ ├──4.06 MB (04.42%) ── gc-heap │ │ └──2.36 MB (02.57%) ── malloc-heap/data │ ├───2.57 MB (02.80%) ++ (8 tiny) │ ├───1.98 MB (02.15%) ── cross-compartment-wrapper-table │ ├───1.86 MB (02.02%) ++ type-inference │ └───1.04 MB (01.13%) ── private-data ├──33.04 MB (35.98%) -- zones │ ├──11.21 MB (12.21%) -- strings │ │ ├───6.79 MB (07.39%) -- gc-heap │ │ │ ├──5.79 MB (06.31%) ── latin1 │ │ │ └──1.00 MB (01.08%) ── two-byte │ │ └───4.42 MB (04.82%) -- malloc-heap │ │ ├──3.41 MB (03.72%) ── latin1 │ │ └──1.01 MB (01.10%) ── two-byte │ ├───9.23 MB (10.06%) -- shapes │ │ ├──7.50 MB (08.17%) -- gc-heap │ │ │ ├──5.86 MB (06.38%) ── tree │ │ │ ├──1.41 MB (01.53%) ── dict │ │ │ └──0.24 MB (00.26%) ── base │ │ └──1.73 MB (01.88%) -- malloc-heap │ │ ├──0.94 MB (01.02%) ── tree-kids │ │ └──0.79 MB (00.86%) ++ (2 tiny) │ ├───3.92 MB (04.27%) -- scopes │ │ ├──2.22 MB (02.42%) ── malloc-heap │ │ └──1.70 MB (01.85%) ── gc-heap │ ├───2.30 MB (02.51%) ── unused-gc-things │ ├───2.28 MB (02.48%) ++ (7 tiny) │ ├───2.19 MB (02.38%) -- object-groups │ │ ├──2.15 MB (02.34%) ── gc-heap │ │ └──0.04 MB (00.04%) ── malloc-heap │ └───1.91 MB (02.08%) ── type-pool ├──14.58 MB (15.87%) ── runtime └───2.02 MB (02.19%) -- gc-heap
├──1.02 MB (01.11%) ── chunk-admin ├──1.00 MB (01.09%) ── unused-chunks └──0.00 MB (00.00%) ── unused-arenas
413 (100.0%) -- js-main-runtime-compartments ├──406 (98.31%) -- system │ ├──391 (94.67%) ++ (377 tiny) │ ├────9 (02.18%) ── [System Principal], about:blank [9] │ └────6 (01.45%) ── [System Principal], inProcessTabChildGlobal?ownedBy=chrome://messenger/content/messenger.xul [6] └────7 (01.69%) ++ user
49.98 MB (100.0%) -- js-main-runtime-gc-heap-committed ├──46.68 MB (93.39%) -- used │ ├──44.78 MB (89.60%) -- gc-things │ │ ├──22.45 MB (44.92%) ── objects │ │ ├───7.27 MB (14.54%) ── shapes │ │ ├───6.79 MB (13.58%) ── strings │ │ ├───4.06 MB (08.13%) ── scripts │ │ ├───2.15 MB (04.29%) ── object-groups │ │ ├───1.70 MB (03.39%) ── scopes │ │ └───0.37 MB (00.74%) ++ (4 tiny) │ ├───1.02 MB (02.03%) ── chunk-admin │ └───0.88 MB (01.77%) ── arena-admin └───3.30 MB (06.61%) -- unused
├──2.30 MB (04.61%) -- gc-things │ ├──0.88 MB (01.76%) ── objects │ ├──0.77 MB (01.53%) ── strings │ └──0.66 MB (01.31%) ++ (8 tiny) ├──1.00 MB (02.00%) ── chunks └──0.00 MB (00.00%) ── arenas
87 (100.0%) -- message-manager └──87 (100.0%) -- referent
├──63 (72.41%) -- global-manager
│ ├──63 (72.41%) ── strong
│ └───0 (00.00%) ++ weak
├──17 (19.54%) -- parent-process-manager
│ ├──17 (19.54%) ── strong
│ └───0 (00.00%) ++ weak
└───7 (08.05%) -- child-process-manager
├──7 (08.05%) ── strong
└──0 (00.00%) ++ weak
1,530 (100.0%) -- observer-service └──1,530 (100.0%) -- referent
├──1,223 (79.93%) ── strong
└────307 (20.07%) -- weak
├──307 (20.07%) ── alive
└────0 (00.00%) ── dead
546 (100.0%) -- observer-service-suspect ├──278 (50.92%) ── referent(topic=xpcom-shutdown) └──268 (49.08%) ── referent(topic=memory-pressure)
1,164 (100.0%) -- preference-service └──1,164 (100.0%) -- referent
├──1,004 (86.25%) ── strong
└────160 (13.75%) -- weak
├──160 (13.75%) ── alive
└────0 (00.00%) ── dead
10.28 MB (100.0%) -- window-objects ├───5.14 MB (49.97%) -- layout │ ├──1.36 MB (13.18%) ── style-structs │ ├──1.03 MB (09.97%) ── pres-shell │ ├──0.79 MB (07.69%) ── frames │ ├──0.71 MB (06.87%) ── style-contexts │ ├──0.59 MB (05.72%) ── style-sets │ ├──0.43 MB (04.21%) ── rule-nodes │ ├──0.15 MB (01.43%) ── pres-contexts │ └──0.09 MB (00.91%) ++ (2 tiny) ├───3.25 MB (31.63%) -- dom │ ├──2.35 MB (22.83%) ── element-nodes │ ├──0.75 MB (07.29%) ── other │ └──0.16 MB (01.51%) ++ (5 tiny) ├───1.88 MB (18.30%) ── style-sheets └───0.01 MB (00.09%) ── property-tables
0.00 MB ── gfx-surface-quartz
183.68 MB ── gfx-textures
389.18 MB ── gfx-textures-peak
0.00 MB ── gfx-tiles-waste
0 ── ghost-windows
7,483.08 MB ── heap-allocated
1.00 MB ── heap-chunksize
7,810.00 MB ── heap-mapped
1.00 MB ── imagelib-surface-cache-estimated-locked
1.00 MB ── imagelib-surface-cache-estimated-total
0 ── imagelib-surface-cache-overflow-count
1.71 MB ── js-main-runtime-temporary-peak
17,497 ── page-faults-hard
11,812,698 ── page-faults-soft
4,104.04 MB ── resident
6,956.32 MB ── resident-peak
2,400.51 MB ── resident-unique
12,866.88 MB ── vsize
End of Main Process (pid 58244)
When you say you have done the diagnostic tests. Do you mean these? https://wiki.mozilla.org/Thunderbird:Testing:Memory_Usage_Problems#Diagnosis_Steps
If so, why did you not file a bug?
Thanks for the response. No, I meant what I posted above, using the help-> troubleshooting info - mainly about:memory. Sorry for being unclear.
I will start working through the applicable steps in the diagnosis section. FWIW, the links to software/extensions with known problems that might affect me are to bugs that are 8-9 years old, making me think this list is likely stale. In the memory leaks section, the link to Leak Monitor Detector looks like a firefox extension, no mention of support for Thunderbird, so I'm not clear on how to install that - thunderbird's add-on manager doesn't have a search or a way to feed it that URL that I could find. I'm not a developer, so the additional devtools referenced there will take some work for me to get something useful from, so if there is anything specific I should be focusing on based on the output I provided above, I'd appreciate the guidance.
It is daunting, but without information the developers get very dismissive. that is perhaps a reflection on how over worked thy are.
The Wiki page is maintained basically on an ad hoc basis. but it mostly gets things added. No one has the time to do a rewrite and check all the links. But the diagnostic steps are largely timeless and are where I suggest you concentrate your efforts. A lot of them are also fast.
I found on windows 10 after updating to it that I had to disabling windows indexing on the profile folder and turn of the disk defragmenter that was basically constantly running..
Others have had to create exception in their anti virus for their Thunderbird profile folders. File contentions grinds them to a stop in some cases.
In your case I would try a repair on the folders that apparently consume the most memory in your memory test. or more simply select compact from the file menu. That digs out corruption and fixes it as well as repairs the index like the repair option does. The down side for non IMAP accounts is mail in corrupt folders gets blitzed from the point of corruption on. So you end up with a loss of new items but retain old stuff in the folder.
You might also want to recreate the global-messages.sqlite file in you profile folder and see if that is a cause due to some corruption. you can move the file elsewhere and Thunderbird will rebuild it when it restarts. (step 9)
Does it also happen with beta 60? http://www.mozilla.org/en-US/thunderbird/channel/