Does it still happen in Troubleshoot Mode?

I've been having the same trouble. Glad to know I'm not the only one.

Recent months have seen YouTube stepping up its fight against users who block the display of ads, contrary to the terms of service. Previously, people with some blockers installed could face a complete blocking of access to content, but now YouTube is experimenting with another method - deliberately slowing down the service to force removal of the ad blocker or purchase of YouTube Premium.

The slowed performance of YouTube can be seen mainly in the context of the video loading buffer. With the ad blocker enabled, the buffer seems to run much slower, or for a good few seconds almost not at all, when the user decides to spontaneously jump to another moment in a given video.

YouTube users who have noticed the problem report that it is very easy to recognize. The site acts as if there is a problem with the Internet connection. However, it is enough to disable the ad blocker in the browser so that the smooth operation of the buffer, and thus the loading of videos, is immediately restored. YouTube users have noticed such a method by Google before, but only now the measure seems to have been implemented on a large scale and may affect everyone.

yep turns out it was adblocker making it lag so hard, I heard about google doing this to people who use adblockers but I didn't think to check it since i already had youtube premium. I have since whitelisted youtube on my adblocker and have had no issues since.