The address of the downloads list is about:downloads so if you paste that into the address bar and bookmark it, you can have quicker access to it from the Bookmarks Menu or Bookmarks Toolbar.

After saving a bookmark, you can go into its properties and define a keyword, such as d for about:downloads. With that in place, if you type d in the address bar and press Enter, Firefox will open about:downloads.

But, unfortunately, the Ctrl+j keyboard shortcut opens the Library window (two-pane view) and it can't be reassigned.

Note that about:downloads is the default for Private Browsing mode windows.