Starting in Firefox 20, the downloads are more integrated with history. You can use the option to remove "Download History" when you exit Firefox, as described in this article: Remove recent browsing, search and download history > How do I make Firefox clear my history automatically? However, I don't know if there's still a built-in way to clear them from the list immediately after they finish, or whether you would need an add-on for that.

Thanks - I do know about the integration, which, by the way, is a lousy idea. I don't want to clean my history, but I do want to remove my downloads from the list after they're done.

Just downloads, yes, we can do that. The clear at shutdown method has separate checkboxes for each element of history, including one for download history. You can uncheck the others.

Edit: This is wrong as documented below.

yes, but download history seems to be merged with browsing history.

Oh, what the... I'm sorry, I didn't notice that before. I guess we're back where we started, removing them individually after the fact.

Note that the Downloads folder in the Library is present since Firefox 8 (2011), so this is not a recent addition.
Only now that the new Download Manager that uses this folder has been enabled its presence is now more noticeable.

Currently both Download Managers are active (but you see only one of them as selected via browser.download.useToolkitUI) and they use different files to store their data (downloads.sqlite and places.sqlite), so it is complicated to have full overview.