Referring Url problems keep coming back
My library, Hennepin County, MN, hclib.org, uses ezproxy to validate users. For most things, no problem, but when I try to access JSTOR, an online Journal site, I get the message: "Sorry, we were not able to authenticate you for access to this resource. Please adjust your Internet Security software to allow referring URLs." I've tried several suggested fixes for this, and sometimes they work, but at the next session the problem invariably comes back.
So, PLEASE, don't type that easy answer you've got memorized, because I've probably already done it, and I really don't feel like clearing all cookies and browsing history every time I start a session, or resetting to default every day. Instead, try to figure out why the "solution" only works once.
All Replies (15)
By default, Firefox will provide the referring page URL. I have a test page up where you can see this in action:
http://dev.jeffersonscher.com/jstest.asp Whoops, not from this site.
Try this instead: https://jeffersonscher.com/res/jstest.php
If you do not get a referring page in the first box, "something" is blocking it. That could be external security software, or an add-on (e.g., privacy or security add-on), or you may have changed a setting in about:config.
All 3 of those potential culprits are invisible to the volunteers here. If a Reset solved the problem, we can rule out external software and focus in on the other two: add-on or setting.
(1) You could try Firefox's Safe Mode to bypass interference by extensions, the type of add-on most likely to be filtering out the referring URL. More info: Diagnose Firefox issues using Troubleshoot Mode.
You can restart Firefox in Safe Mode using
Help > Restart with Add-ons Disabled
In the dialog, click "Start in Safe Mode" (not Reset)
Any difference?
(2) To check or change the internal preferences, try this:
(A) In a new tab, type or paste about:config in the address bar and press Enter. Click the button promising to be careful.
(B) In the filter box, type or paste refer and pause while the list is filtered
(C) If either of the following preferences appears in bold type on your list and says "user set", then right-click it and choose Reset:
(i) network.http.sendRefererHeader
(ii) network.http.sendSecureXSiteReferrer
Were those modified?
Modified by jscher2000 - Support Volunteer
One other thought: at startup, Firefox will check for a user.js file and use the settings in that file to override the preferences used during your previous session. External software might be creating such a file. To check for one, see the steps in this article: How to fix preferences that won't save.
OK, it seems that the referral is being blocked. The first box on the test page shows:
Browser "User Agent" string: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0) Your public IP address: 184.97.249.40 Referring page (if any):
Since there's no referring page, I gather that means something is stripping the referring URL. Now I'll start working through your solutions.
Regardless of what happens next, thanks for the answer, and the link to the test page.
Btw, I currently have my security software turned off, but the URL still isn't being passed. So it seems Trend Micro was correct when they said it wasn't their software doing it.
Well, I restarted in safe mode, and that didn't fix it.
The two settings in about:config aren't modified, they read:
network.http.sendRefererHeader; default integer 2 network.http.sendSecureXSiteReferrer; default boolean true
The bolded ones are
browser.preferences.advanced.selectedTabIndex; userset integer 3
and
pdfjs.previousHandler.preferredAction; userset integer 4
No user.js file.
I just deleted and reinstalled Firefox. Still won't access the damned JSTOR website. I'm stumped.
What were the temporary fixes that worked temporarily? Perhaps that will provide a clue as to the problem.
Once, deleting the cookie for the library's website worked. But after the browser session ended, the problem was back, and deleting the cookie again didn't help.
Another time, resetting Firefox to default state did it, but again the problem returned, and again the fix failed to work in future attempts.
In case Firefox is using cached files in a dysfunctional manner, did you try clearing Firefox's disk cache? I know you said you don't want to keep clearing history (between sessions or when needed), but if it's a caching problem, the alternative of completely disabling the cache would give you much worse performance overall.
How do you engage with the ezproxy -- is it transparently incorporated into the library's network or do you enter settings manually into Firefox? Just wonder whether somehow Firefox starts to bypass the proxy at some point.
For quick reference, Firefox has its proxy settings here:
orange Firefox button (or Tools menu) > Options > Advanced > Network > "Settings" button
I just tried clearing the cache. No joy.
It's all part of the website. For example, at this page:
A library patron like myself clicks on either of two links below, both of which take have the address:
and both of which give the message:
Sorry, we were not able to authenticate you for access to this resource. Please adjust your Internet Security software to allow referring URLs.
Except that it doesn't seem to be my security software, because I went so far as to uninstall it, and I still couldn't get in.
Weirdly, Henn County Library uses ezproxy for WorldCat as well, and THAT I can access, no trouble.
Thanks for the links. I noticed it goes from HTTPS for the results results to HTTP for the login page.
(Light bulb goes on...)
Firefox does not pass the referring page information from secure pages to insecure pages. That would explain why the referring page was blank on my test page earlier. WHOOPS!
If you compare with HTTP and HTTPS you should see the referrer on the second link but not the first:
http://jeffersonscher.com/res/jstest.php
https://jeffersonscher.com/res/jstest.php
Ahem, sorry about that.
Back to your page...
You said it works one time when you delete your cookies, and my guess is that is because after deleting the library's cookies, you have to stop and enter your library card number, while on subsequent requests that step is bypassed because you have the cookie. I have a suspicion that the way it is being bypassed also prevent the page from sending the referrer information that JSTOR wants.
If that theory is correct, unfortunately I do not have a brilliant solution that both avoids entering the card number every time AND passes the correct information to JSTOR. (The easiest workaround of accessing the catalog using HTTP instead of HTTPS doesn't work because the site redirects back to HTTPS.)
Do you know whether anyone at the library has tried to replicate this issue? If the secure-to-insecure, cookie-bypassing-login theory is correct, all modern browsers should generate the same error.
Actually, when deleting cookies, sometimes it works the next time, and sometimes it doesn't. I haven't been able to find any reliable method of getting access, even for a one-off, 'It will definitely fail next time, but it will work this time, once' event.
On the occasions when it has worked, I've had to enter my library card number. And you are correct that the problem is not specific to Firefox, it also happens in IE. I'd try other browsers, but they all feature blinking insertion points, whether you want the insertion point to blink or not. Throwing up all over the keyboard doesn't do my computer or me any good.
I've tried getting support from the library, but they didn't much help. But I'll try again there.
I'd like to thank you for all your help. You've put a lot of effort into this, and I appreciate it.