Restore bookmarks from backup or move them to another computer or Recovering important data from an old profile will help you transfer your Firefox profile to your new PC.

I have a USB flash drive to download, will this work

Yes, you can copy your Firefox profile to a USB Drive, then just copy that into your Firefox profile on your new PC.

Note that it is not recommended to copy a full profile folder from one computer to another, especially if both computers have a different OS like in your case from Windows XP to Windows7.
To make this work you would also need a copy of profiles.ini to make Firefox find such a profile.

It is best to only restore the files (SQLite database files) that you really need and reinstall extension and avoid copying files like localstore.rdf and prefs.js (sessionstore.js is OK).
For saved passwords you need the files key3.db and signons.sqlite.

See also:

• http://kb.mozillazine.org/Transferring_data_to_a_new_profile_-_Firefox
• http://kb.mozillazine.org/Profile_folder_-_Firefox