Here's a new add-on with that feature, but it needs a little work.
https://addons.mozilla.org/en-US/firefox/addon/disable-private-browsing/

See my comments posted here:
https://support.mozilla.com/en-US/questions/762104#answer-218969
And also see cor-el's comments in the posting following mine.

BTW, the Private Browsing feature isn't new for 4.0 or 5.0, it's been part of Firefox since the 3.5 version was released in June of 2009, over two years ago.

You really need Parental Control software to keep track of where your child has been surfing. Plugin the Portable Firefox version on a USB Flash stick, and Firefox loaded on the hard drive is by-passed. Parental controls will keep track of all web browsers used on that PC, "plugged in" or "installed".
https://support.mozilla.com/en-US/kb/Parental+controls

Firefox 3.5 already had private browsing.

You can't prevent your kid from deleting history and other private data even if you would disable PB mode.
There is also Clear Recent History and the Library or History sidebar to remove visited sites.
You need to use other means outside Firefox if you want a reliable record of visited sites.

See also:

• http://kb.mozillazine.org/Parental_controls
• http://en.wikipedia.org/wiki/Category:Content-control_software
• http://en.wikipedia.org/wiki/K9_Web_Protection