Sorry, the developers didn't provide a "switch" to turn that feature off, or limit which logon users have access to it. There is a Bug filed to have Private Browsing respect the Parental Control feature that is part of Vista and Win7, but since you're using WinXP that wouldn't affect you, even when (and if) that Bug is ever resolved.

Note that removing or disabling "Private Browsing" will not make it impossible to delete private data like visited websites (history) or cache or cookies.
There are a lot of ways to clear such data, either via Delete browsing, search and download history on Firefox or directly by deleting the History in the Library or sidebar.
Private Browsing makes it easier because all data is kept in memory and nothing is saved to disk, so nothing needs to be removed.

If you want to see which sites have been visited then you should look at others means outside Firefox like a router or firewall with a password protected logging feature that can't be bypassed or a program like Wireshark.

• Wireshark: http://www.wireshark.org/download.html

See also http://kb.mozillazine.org/Parental_controls