It happend again.

Firstly, upon the restore from backupabove, Fx had actually upgraded from v41 to v45.

Then i updates all the addons, and backed up then I changed preference setting , to clear search history on close (only search history as i unchecked al the other boxes) closed down. all good

The restarted fx. didnt come up again. and Fx created 2 new places.sqlite files , in addition to 2 mentioned above. Plus the profile size almost doubled. places.sqlite-shm places.sqlite-wal

So then i restored from backup a second time.

So how do I fix the corruption i the place.sqlite? or better still, how to create a brand new place.sqlite and does it store anything more than my search history?

thankyou

found this prior support post by searching, the preference setting "Clear history when firefox closes"

Which is the setting I choose in second instance (my second post) that caused Fx to not open again.

this post is helpful https://support.mozilla.org/en-US/questions/1096434

and directs me to the addon called Places Maintenance, to help look at places.sqlite when corrupt. That is my situation. https://addons.mozilla.org/en-US/firefox/addon/places-maintenance/

post from above link is here:

You can check for problems with the places.sqlite database file (bookmarks, history) in the Firefox profile folder.

   Places Maintenance: https://addons.mozilla.org/firefox/addon/places-maintenance/ 

If the Places Maintenance extension can't repair the places.sqlite database file then remove all places.sqlite file(s) in the Firefox profile folder to make Firefox rebuild the places.sqlite database file from the most recent JSON backup in the bookmarkbackups folder.

   http://kb.mozillazine.org/Bookmarks_history_and_toolbar_buttons_not_working_-_Firefox
   https://support.mozilla.org/kb/Bookmarks+not+saved#w_fix-the-bookmarks-file 

Issue is replicated when I change this preference setting 1) Preferences 2) Clear History when Fx Closes 3) ONLY select Browsing and Download history see screenshot

questions 1) says in the post, his resolution was "There were just too many bookmarks, " I have a lot of bookmarks.

What is Firefox limit on number of bookmarks?

2) The post infers he started from scratch, but on reading it I am not very clear exactly what he did to do that. If someone could let me know, if i have to start from scratch, how do I do that? e.g. in yosemite, do i just delete the firefox profle and mozilla folder or is there an uninstall application for firefox?

3_ And is it better to do a clean install of firefox completly to ensure anything bad is gone, (it made a huge difference when i did fresh yosemite install versus upgrade from mavericks, so i wondered maybe firefox is the same. It prefers fresh install). So is it better to do a fresh install of firefox?

4) I rarely use my browsing history. and I never use my download history. So need to set new settings when this problem is fixed and I can use the prerference settings without causing fx to hang (as currently is so)

Support ARticle https://support.mozilla.org/en-US/kb/delete-browsing-search-download-history-firefox#w_what-things-are-included-in-my-history

5) In above support artcile, unders section "What things are included in my history?" Shows site preferences are available to be cleared. But these are not in the automatic preference setting "2) Clear History when Fx Closes". They appear to only be available manually as per this post (scroll down in article). Is this correct that no automatic option, only manual option for clearing site preferences?

6) to rebuild the Fx profile from scratch, I have backup of bookmarks (using the standard export), and addons settings (using OPIE addon), plub my FEBE backups. I need to work out how to export (so I may import) >Fx preference settings (i took screenshots) > site preferences e.g. zoom settings of pages > search plugins > About:New Tab preference settings and selected tiles (remember: you add a new tile only from bookmark manager) > manual changes advised in past fx support to about:config > list of my addons (I could not cut and paste from the screen in FEBE tools. OPIE exports only SOME addons with preference settings, so you need a list of your complete addons and check which ones are not exported) >and find a checklist to see if I have missed anything

found: password exports https://addons.mozilla.org/en-US/firefox/addon/password-exporter/?src=search

Thankyou for your time.

Plant maintenance using the addon was successful.

But at the end of the succcess message it does show I am exceeding the restore of browse history.

can restore 100664 unique pages I have 110017

Which is the very thing i am trying to delete. Please refer to screenshots

Just in case it helps. the statistics the addon gave about the places.sqlite

History can store a maximum of 100664 unique pages Table moz_places has 110009 records Table moz_historyvisits has 314830 records Table moz_inputhistory has 4 records Table moz_hosts has 5250 records Table moz_bookmarks has 37262 records Table moz_bookmarks_roots has 5 records Table moz_keywords has 4 records Table sqlite_sequence has 1 records Table moz_favicons has 3275 records Table moz_anno_attributes has 23 records Table moz_annos has 27413 records Table moz_items_annos has 21653 records Table sqlite_stat1 has 15 records Index sqlite_autoindex_moz_inputhistory_1 Index sqlite_autoindex_moz_hosts_1 Index sqlite_autoindex_moz_bookmarks_roots_1 Index sqlite_autoindex_moz_keywords_1 Index sqlite_autoindex_moz_favicons_1 Index sqlite_autoindex_moz_anno_attributes_1 Index moz_places_faviconindex Index moz_places_hostindex Index moz_places_visitcount Index moz_places_frecencyindex Index moz_places_lastvisitdateindex Index moz_historyvisits_placedateindex Index moz_historyvisits_fromindex Index moz_historyvisits_dateindex Index moz_bookmarks_itemindex Index moz_bookmarks_parentindex Index moz_bookmarks_itemlastmodifiedindex Index moz_places_url_uniqueindex Index moz_places_guid_uniqueindex Index moz_bookmarks_guid_uniqueindex Index moz_annos_placeattributeindex Index moz_items_annos_itemattributeindex Index moz_keywords_placepostdata_uniqueindex

Hi jendrew, Sorry you have not been getting any answers yet. This is a slightly vague answer but I will probably have covered most of your questions.

If you are posting from the problem machine, it appears you have succeeded already in updating to Firefox 45.

Firefox stores the history of sites visited, that may include sites where you do a search, and the URL may include search terms but it does not specifically store only search History.

I am not sure the maximum number of bookmark allowed, but IIRC the Places Maintenance addon you are already using does have an option to generate a short text log file so using that and pasting it into your reply will let us know the current status of your places database.

IIRC places uses some algorithm to set a maximum size, but if that is based on hdd size it may well be outdated given likely hdd sizes on modern PCs. As you discovered it also contains History. Again some algorithm comes in to play, because it gives precedence to bookmarks over History entries.

The starting from scratch probably refers to deleting the original places.sqlite database file. However

• That causes loss of all History. There is no Firefpx backup of History.
• It is safer and better if considering that route to rename the places.sqlite, not delete it. Just call it places.sqlite_old_01 or something.

There is probably no need to do a clean install at present.

You may not directly use the History, but note History is normally involved in location bar prompting.

• Remove websites from the address bar suggestions _clearing-all-items-in-the-location-bar-history
• Address bar autocomplete suggestions in Firefox_changing-your-location-bar-settings

You probably not even need to use a new profile. But if you do the article is

• Back up and restore information in Firefox profiles

I did not notice the log when I posted.

History can store a maximum of 100664 unique pages

 Table moz_'places has 110009 records

Table moz_bookmarks has 37262 records

So you have lots of free space for bookmarks you have only !! used 37,262 out of up to 110,009 the spare available space is used for History.

Thanks john99 for your time. I will try and find the log file you speak of and post from Plant Maintenance addon

C-orel says in the prior post on this issue

this: If the Places Maintenance extension can't repair the places.sqlite database file then remove all places.sqlite file(s) in the Firefox profile folder to make Firefox rebuild the places.sqlite database file from the most recent JSON backup in the bookmarkbackups folder.

from this post: https://support.mozilla.org/en-US/questions/1096434

When it says: recent JSON backup in the bookmarkbackups folder. Is that the automatic backup on boot down or the one I create when i go into bookmark manager and say export bookmarks?

Does that mean, if I do these steps 1) Backup my bookmarks agin, so we have up to minute latest 2) rename places.sqlite to OLDplaces.sqlite 3) Fx recreates the places.sqlite database automatically, based on the backup file of bookmarks i just created.

That means we new places.sqlite 1>bookmarks are essentially retained and we do not have 2>site preferences 3>cookies, active logins, 4>cache ( clear this using the addone cache status regularly. is this that cache or some other cache?) 4>Browse and Download history (which i am trying to delete) 5>Search and Form history (its ok i can do without this)

so 2) 3) is not desired but rest seems ok.

John99 said

I did not notice the log when I posted. So you have lots of free space for bookmarks you have only !! used 37,262 out of up to 110,009 the spare available space is used for History.

Oh.. we must of been working at same time.... sorry to confuse. --- This is great news. I have been fretting about my beloved bookmarks. These are critical to me as I dont write anything down anymore. I bookmark it!

John99 said

You may not directly use the History, but note History is normally involved in location bar prompting.

Nice point. Thankyou.

I rarely use the prompting, as typically i am coming from a bookmark or search google results page. I probably have not got to the stage of my internet surfing where i trust that suggestions... so perhaps still using the old way e.g. bookmarks.

Its only really nice for me, in the same day or maybe at worse case, same week e.g. going back a page to something quickly.

John99 said

.., but IIRC the Places Maintenance addon you are already using does have an option to generate a short text log file so using that and pasting it into your reply will let us know the current status of your places database.

I ran it again, and could not find any option for short text log file.

I dont know what IIRC is? What is this?

What it gave was a popup window, so I highlighted the text in that, then copy paste the results into early post - it was called statistics.

IIRC Sorry my own silly jargon: If I Recall Correctly  ;-)

Make sure you have a backup of your bookmarks, better still backup the whole profile. Firefox is now pretty good with its own internal backups.

JSON = Backup|Restore They overwrite. Maybe occasionally make a backup of the bookmarks from the bookmarks manager.

• Restore bookmarks from backup or move them to another computer

HTML = Export|Import They tack on or append. HTML bookmarks are not really used other than for transfer to other browsers. They do have one big advantage when importing they add to the existing bookmarks, rather than overwriting them. So if for instance you have some small number of bookmarks from another browser or another machine you can add them on to the new set.

• Import Bookmarks from an HTML file
  • Import Internet Explorer Favorites from another computer
• Export Firefox bookmarks to an HTML file to back up or transfer bookmarks

I checked the Fx profile directory since the 2nd time restore, to see if we have the 2 places.sqlite files or 4 have come back. I wanted to check as i have rebooted a few times to test.

4 are there now. But good news the Fx profile is still about the same size. When fx struggles to reboot it bloats out the 2new sqlite files and so my profile jumps from about 878mb to 2gb.

see screenshot

standard 2 always there places.sqlite, 104.8mb 24/52015 places.sqlite.corrupt, 83mb 24/5/2015

2 new ones, since rebooting a few times places.sqlite-shm places.sqlite-wal

Not sure when this one appeared, places.sqlite.corrupt. Looks like it was in my old Fx profile before i freshly created this one on new yosemite image, new fx etc.. on 24/5/2015.

Can I delete places.sqlite.corrupt this one? (its 83mb being backed up every 3 days.)

I ran it again, and could not find any option for short text log file.

The details in your #answer-855089 were what I was thinking about. it show you are in no danger of running short on space for bookmarks.

Note the internal backups shown from the bookmarks manager will show the date, number and size of file. Just remember if you try them out they will overwrite whatever you currently have.

cool.

so with your help, it appears the problem could now be described as a too big moz_places table within the places.sqlite database.

moz_places has 110009 records

History can store a maximum of 100664 unique pages aka, meaning per table in the database.

Correct?

Can i delete these records in moz_places table through the backend (e.g. at the database level) using sqlitebrowser app? or is it better to go with original thought, rename existing places.sqlite and let fx recreate?.

I have the sqlitebrowser app installed. I believe I can open the database, places.sqlite successfuly with it. I understand databases.

I only ask this as i have tried 2 ways to delete the browse and download history thru the front end (e.g. User Interface) . But in both cases, fx became broken. first case, hung, so had to force reboot. Second was a preference setting change to clear on close, and it crashed and would not to reopen on reboot.

With Firefox closed rename places.sqlite to something like say places.sqlite_001 Firefox will then recreate it. you will lose the last few bookmarks (Unless you preserve them all by doing a backup from the library). You will lose all your History.

As long as the places.sqlite file is renamed or moved somewhere else you stand a chance of getting the History back.

Export the bookmarks as HTML, those may come in handy trying to recover bookmarks later.

There is or was a recent bug that causes this issue or a similar one I will look that up and get back later.

Just a thought; and I can not remember whether it is involved with this sort of issue, but do you set Firefox to clear History of or Cookies ? (or effectively use permanent private browsing ? ) because there have been recent issues with that too.

I was trying to set the preference clear history of boot down and that caused the hang. my history setting had always been to save. Never ever thought of it until now.

cookies use to be set to prompt me, and id essentially select allow for session. Now about 4 days or maybe 6 days ago, I changed it to stop prompting me and clear on close.

I do believe the corruption occurred a while back as i have been having performance issues with Fx for a while. hence why i put off upgrade to v45 until today. I was on v41.

Before I do as requested, rename places.sqlite and have Fx automatically recrecreating the places.sqlite, I do need to understand about the other things I loose.

from above: That means we new places.sqlite 1>bookmarks are essentially retained and we do not have 2>site preferences 3>cookies, active logins, 4>cache ( clear this using the addone cache status regularly. is this that cache or some other cache?) 4>Browse and Download history (which i am trying to delete) 5>Search and Form history (its ok i can do without this)

so 2) 3) is not desired but rest seems ok.

Before I do as requested, rename places.sqlite and have Fx automatically recrecreating the places.sqlite, I do need to understand about the other things I loose.

places.sqlite is only history and bookmarks If you do remove it or rename it at the same time remove or rename any places.sqlite-shm places.sqlite-wal & places.sqlite.corrupt

The bookmarks include tags, did include keywords. If you save the original file or rename it you don't really loose anything anyway. I would not try hacking the database, it may be about 13 tables from what I remember, and have no idea whether it includes any cross validation or check sums.

If you rename, delete or move any of the files that must be while Firefox is closed.

cool. understand. Many thanks for your big help john99.

Just making some notes at the moment. As I said earlier I was aware there are some known issues. I am not sure there will be any quick fix for all of this lot.

By all means read or follow bugs, but unless you are directly answering requests for information you probably should not post in bug unless it is one you have filed yourself.

• etiquette
• Fixed in Fx45, and with some background on clear History issue
  Bug 1248489 - crash in mozalloc_abort | NS_DebugBreak | nsDebugImpl::Abort (with abort message chrome://browser/content/sanitize.js) and Firefox no longer responding on startup
• Currently open Bug 1248837 - shutdownhang in nsThread::Shutdown() | Firefox crashes, sometimes the restart fails
  & dependencies
• Bug 871908 - Clearing "browsing and download" history does excessive disc IO (due to huge wal file) & dependencies

P.S. A definitely outdated diagram of the database

Ok. I did as requested.

Result:

• Received 2 "Warning: Unresponsive Script" messages during bootup, clicked continued, and script continued. See screenshots for 2 messages
• Size of places.sqlite is much smaller 104.8mb now 31.4mb.
• Lost ALL icons of bookmarks. Which is a biggy for me as i use the roomy toolbar and only show icons. Using the addon for icons to be fetched is useless. I have tried that once before when an upgrade in fx lost all my bookmark icons. (note: I dont use bookmark tags or keywords). Solved by manually clicking on each of the bookmarks and the icon was grabbed again. That took about 2hrs.
• Crashed again, after I set new preference setting for Clear History on bootdown (Browser and Downlod history only checked) see screenshot. But Fx, did come after, after I choose restart from the crash report. (Before nothing worked, i had to full restore from profile backup)
• Since crash above, those 2 extra files are in the profile directory. Do i delete these? places.sqlite-wal and places.sqlite-shm
• Rebooted to test if Fx crashed again due to that preferences setting. No. Fx loaded successfully.
• Lost of elements unexpected -Appears cookies gone too? as Fx support asked me to login again. mm.. Also I just went to amazon and typically my username/passcode is there, they are not there. Something odd. I have tested this twice now by checking the setting, rebootting and doing some more work. Then seeing if the setting changed. I think this preference setting must of been reset when it crashed. As it has not occurred a 2nd time. So its watchout
• Failed to restore bookmarks from last backup json file. I tried to do this, hoping to get my icons back. Took a long time, then Fx, said "Unable to process backup file"