
Update mess
Why can I suddenly NOT get a link to New York Times articles to open since the last update. It accuses me of being a bot!
All Replies (20)
Hi! This sounds like an issue with your browser’s User-Agent or a security filter on the New York Times site. After the latest update, Firefox might be getting flagged as suspicious.
Try this:
1. Open about:config in the Firefox address bar.
2. Search for: privacy.resistFingerprinting
3. If it's set to true, change it to false.
4. Restart Firefox and try again.
Also try:
Opening the link in a Private Window (Ctrl+Shift+P).
Temporarily disabling extensions like uBlock, Privacy Badger, etc.
Checking if you're using a VPN or proxy, which might trigger blocks.
Hope this helps!
Sebastian
It was already on False. I don't have those extensions. (I just have an adblocker that I've had forever and it has never been a problem.) And I do have a VPN. NYT has always worked with the VPN, and nothing changes if I turn it off. This is what I keep getting, attached.
One more question: what OS version do you have?
Thaks for your help. Its Win 10.
Hello again,
- Please check if you have Enhanced Tracking Protection (ETP) enabled. You can do that by clicking the shield
icon at the left of the address bar.
- If ETP is enabled, try disabling it. Does the problem persist?
- If the problem is not fixed, try clearing Firefox cache. Does it help?
- If it does not help, try also clearing cookies and data for the website you experience troubles with (warning: you will be logged out on this website). Does it help?
- If it also does not help, try running Firefox in Troubleshoot Mode. Does the problem persist?
- Finally, if the problem is still not fixed, try creating a new Firefox profile. Does the problem persist there?
Sorry for taking so long to respond. I was moving, and now I am in Israel! Running troubleshoot mode showed me that one of the three add-ons seems to be the problem, but I do not understand why it would suddenly be a problem when it has never been a problem before.......?
Hello,
Thank you for getting back! Could you please give the name of that add-on? It's hard to say why it started causing problems, but there are several possible reasons:
- the add-on's update;
- the add-on's incompatibility with the latest Firefox version;
- the NYT website code update (which made the code interfere with the add-on's work).
Either way, you'd want to report that to the add-on's developers (so that they can fix the issue). To do that, head to the add-on's AMO product page and look for the support/contact information.
I think I was mistaken. It is a paywall blocking add-on. When I took it off I stopped getting the "you are a bot" error, and I thought I was getting through to the article, but I was not. The article began to show, and then the paywall popped up. With the add-on back on it goes back to mostly showing the error message but occasionally going to the article. So, I do not understand it. This is what it is doing with the add-on..... on. (Attached). The first two links are examples of articles that will not open and I get the "bot" error message. The second two links allow you to go through to the aritcle with no problem. Maybe they will help you unravel this?
If the site loads as expected (and displaying a paywall seems to be normal for the NYT) with the add-on disabled, then this add-on (extension) is the cause. In that case, there's not really much we can do, since it's up to the extension's developer to update it and fix the bugs. You'd want to inform them about the issue; usually, you can find the contact information on the extension's homepage on addons.mozilla.org.
I do not think they are updating that program anymore, but I do not understand why some articles open and others do not, like in the page of examples I attached to my last message. Something must make a difference. And, all of this time the paywall program has worked as expected until your last update. So how did the update affect it?
In addition to my earler message today, I think we need to go back to the beginning. Attached is the error message I originally sent. This does not seem to be the paywall program. It was working. The New York Times suddenly thought I was "clicking too fast," or that I was a bot. That occurred at the time of the last update. I do not think it is the paywall program that is the problem. I was working just fine. I think it was something in your update.
New York Times uses bot-detection systems (Cloudflare-style fingerprinting) to prevent automated scraping. These systems can falsely tag real users as bots if there is any following conditions regarding you!
You’re using a browser extension that modifies the page. Your browser is blocking or altering trackers/scripts (like Ad/PPT blockers). Cookies or session data are corrupted. You’re routing traffic via a VPN, proxy, or network shared with bots.
try these steps and also clear your browser cache and then try again,
Modified
Hi TikvahCo, I have a answer to why you're seeing this issue only after recently updating. The NYTimes site has web developers that create code to detect adblockers or addons that try to bypass/skip the paywall. If they didn't do this, the news company would go bankrupt. You would not be able to read any NYT stories ever again.
Firefox realized this and removed those paywall blocking addons from their addons website because those kinds of addons allow people to read the articles for free which blocks too much of the profits that news websites make. They will lose thousands of dollars and eventually millions of dollars if everyone could read their stories for free.
Now there are many news websites which constantly ask you to pay for a subscription before you can even read 1 article. I do not respect those news sites at all. One example is the WSJ (Wall Street Journal). They aggressively show visitors a paywall & demand you pay first before you can read anything. There's a few news sites that are extremely aggressive with their paywalls. NYTimes did not used to be like that but now they are. They removed their 3 free articles option they had.
Read here: Are there just no free articles anymore on New York Times? https://www.reddit.com/r/firefox/comments/1djr47y/are_there_just_no_free_articles_anymore_on_new/
Multiple DMCA takedowns were taken on the websites that were sharing the addons to read news articles for free. They removed the addons after that.
Please consider buying a small subscription from NYTimes. I believe they have weekly and monthly options. Usually they want you to buy a yearly subscription but many people do not read the news that much. So you could be better off just buying the cheapest subscription then reading as many articles as you can before it expires. Current deal: $1/week for your first year. Billed as $4 every four weeks, then $25 thereafter.
Unfortunately free journalism is gone. That is something from the old days. When you could read articles & just have to watch the ads on the sides of the stories. Only very few news companies use the ad supported model like local news sites, CNN.com, Foxnews.com, MSNBC.com, abcnews.go.com, cbsnews.com, yahoo.com/news/ and so on. But for deeper journalism where more investigation is required, those articles are being charged for so the reporter can be paid.
My only issue with this is that years old articles need to be made free to the public. We should not be paying to read 1 year old, 2 year old, 5 year old and 10 year old articles. There has to be a time limit of when a article can be made available to the public for free.
So long story short, it is not the Firefox update that is the problem. You will always encounter blockers on all major news sites that need money to keep operating. The big news companies have a lot of money from TV deals, advertising deals, etc. The smaller news companies do not. And their web developers have been ordered to detect and block any addons that try to get to the articles for free. So even though you can see a few for free, that is a accident. They will keep working on their blocker code until it can block all ways of reading the articles for free. There is nothing Firefox can do against that.
Read this as well for a debate about news readers against the paywall & those who support it: Change My View: The New York Times paywall is actively doing harm https://www.reddit.com/r/changemyview/comments/1ie7pyp/cmv_the_new_york_times_paywall_is_actively_doing/
Modified
So, from what you are saying, and which no one else has said, Firefox DID do something in the last update that prevents people from using programs that were working properly until that point. Why would Firefox do such a thing? It does not affect your bottom line, but you have now prevented your clients from doing what they do in their daily lives. The more I use Firefox they more upset I become. The browser uses up way too much memory, and now this. (There have been other issues that I don't happen to recall right now.) Firefox is supposed to be on OUR side, not big business........ 🤦♂️
Hi Noah,
The OP would've got the same screen as this comment if this happened due to the paywall thing. And because of that, I don't believe this is something to do with paywall (though ofc, I highly encourage people to support media by paying for their content whenever possible). Also, we've heard several questions about the same issue since version 139.
It looks like it's caused by an add-on (because it can be solved by using the Troubleshooting Mode). Maybe the OP here can try to pin point which add-on may have caused this?
@Tikvah Can you try out to replicate this issue in the Troubleshoot Mode?
Interesting that others are having similar problems. Ive been saying its the update since the beginning. I did try troubleshoot mode. It seems to have something to do with the paywall blocker, but it still sometimes works and sometimes does nt work, whether I have it on or not. So, if that was the problem, it would not work at all. (Description above.) I think something happened in the update that made thngs go haywire, and I'd love for someone to figure out what it is so I can fix it. 😕
Hi Kiki, I based my anwser off his later post: /questions/1517927#answer-1749765 "I think I was mistaken. It is a paywall blocking add-on. When I took it off I stopped getting the "you are a bot" error, and I thought I was getting through to the article, but I was not. The article began to show, and then the paywall popped up. With the add-on back on it goes back to mostly showing the error message but occasionally going to the article."
I don't get that "...we suspect that you're a robot" message. Instead I get a huge banner that blocks half the screen saying I need to signup for a New York Times account. "Create a free account or login".
But my point to TikvahCo was that all the old paywall blocking addons are not supposed to be working. There is a coordinated effort to remove & kill those paywall blocking addons on any site they are hosted on like Github, etc. He still has a paywall blocking addon he installed years ago. It's working for him sometimes. I'm trying to explain to him that Firefox is not the only thing that gets updates. Websites get updates too. The New York Times website gets a lot of updates. Especially to its code that detects adblockers & paywall blockers. That paywall blocking addon that he has installed is old & outdated. It is not going to open every article for free like he wants.
I know you don't like the answer but what I'm saying is the NYTimes website has updated itself to protect its content. If you want to read articles there you have to make a account. And then you probably have to buy a subscription.
The "You have been blocked from The New York Times because we suspect that you're a robot." message is a totally different issue that will be fixed eventually. But it doesn't mean that you will get to read NYTimes articles for free again unless you disable that addon & create a account on NYT.
Is the name of your blocking addon Bypass Paywalls? https://www.ghacks.net/2023/04/20/mozilla-removes-bypass-paywalls-clean-extension-from-its-add-repository/
I am not sure who is talking to who at this point since there is so much cross conversation but that is the add-on I have. It works part of the time and not the rest of the time. Whatever the case, I really don't think I am going to be buying a subscription. News online, just like on TV, should be free to the public. They (online) should be getting their money from advertisers not by forcing the public to pay. And again, this started at the same time as the Firefox update so I still think it has something to do with that. It is too big a coincidence that the NYT would do some kind of update at the same time as the Firefox update. And, as I said before, as much as I like Firefox (usually) I am wondering if I should try a different browser. It uses WAY too much memory and something should be done about it. This browser is supposed to be put together by users for users and pulling that much memory is a problem that needs to be fixed. There are SO many complaints about it and it does not seem there there has been any kind of resolution. 😕
NoahSUMO said
Hi Kiki, I based my anwser off his later post: /questions/1517927#answer-1749765 "I think I was mistaken. It is a paywall blocking add-on. When I took it off I stopped getting the "you are a bot" error, and I thought I was getting through to the article, but I was not. The article began to show, and then the paywall popped up. With the add-on back on it goes back to mostly showing the error message but occasionally going to the article."Aha. You're actually right. I have too many extensions to realize that I also have the same addon the OP has.
@Tikvak I'd highly recommend to remove or at least disable the extension to see if it may help with this issue. The webcompat bug that I filed was also resolved as worksforme and suggest that this might be caused by an addon.