{"video_id":"fp_4oDoLUGBEx","title":"Are Search Engines Worse Now?","channel":"Techquickie","show":"Techquickie","published_at":"2024-03-08T21:26:00.031Z","duration_s":467,"segments":[{"start_s":0.0,"end_s":4.32,"text":"Why do search engines suck now? Wait, before we get ahead of ourselves,","speaker":null,"is_sponsor":0},{"start_s":4.32,"end_s":7.72,"text":"do search engines suck now? Are they actually getting worse,","speaker":null,"is_sponsor":0},{"start_s":7.72,"end_s":13.6,"text":"or have they just changed in a way that I personally hate? Just by glancing at the first page of a Google or Bing search,","speaker":null,"is_sponsor":0},{"start_s":13.6,"end_s":16.76,"text":"it's easy to find a long list of potential complaints.","speaker":null,"is_sponsor":0},{"start_s":16.76,"end_s":21.32,"text":"Paid sponsors crowd out the first few search results. There's annoying little widgets everywhere,","speaker":null,"is_sponsor":0},{"start_s":21.32,"end_s":25.36,"text":"and it keeps giving you barely related suggestions for what to search next,","speaker":null,"is_sponsor":0},{"start_s":25.36,"end_s":29.96,"text":"like a lame choose your own adventure book. But why does this search page have so many ads?","speaker":null,"is_sponsor":0},{"start_s":30.08,"end_s":33.68,"text":"Isn't an interesting question. It's because money.","speaker":null,"is_sponsor":0},{"start_s":33.68,"end_s":38.88,"text":"The more interesting question is why it feels like it's gotten harder to find the information you want","speaker":null,"is_sponsor":0},{"start_s":38.88,"end_s":42.4,"text":"despite all those supposedly helpful widgets.","speaker":null,"is_sponsor":0},{"start_s":42.4,"end_s":47.56,"text":"The answer is also because money, but it's worth unpacking why.","speaker":null,"is_sponsor":0},{"start_s":47.56,"end_s":52.52,"text":"It's difficult to get a lot of rigorous data on this subject, but at least a few researchers have tried to answer","speaker":null,"is_sponsor":0},{"start_s":52.52,"end_s":57.12,"text":"whether or not search engines are, in fact, getting worse. One recent German study did a survey","speaker":null,"is_sponsor":0},{"start_s":57.12,"end_s":62.48,"text":"of over 7,000 product review searches on Bing, Google, and DuckDuckGo over the course of a year,","speaker":null,"is_sponsor":0},{"start_s":62.48,"end_s":65.88,"text":"and concluded that you could still find useful information,","speaker":null,"is_sponsor":0},{"start_s":65.88,"end_s":72.72,"text":"but it was being drowned out by a torrent of low quality content, especially SEO spam.","speaker":null,"is_sponsor":0},{"start_s":72.72,"end_s":75.96,"text":"Top ranked pages typically were heavily optimized","speaker":null,"is_sponsor":0},{"start_s":75.96,"end_s":82.32,"text":"and littered with affiliate marketing links, and they also showed clear markers of lower content quality.","speaker":null,"is_sponsor":0},{"start_s":82.32,"end_s":86.52,"text":"There's not a ton of academic papers on this issue, but there's plenty of data on the web","speaker":null,"is_sponsor":0},{"start_s":86.52,"end_s":92.52,"text":"showing how users have changed their behavior to sidestep low quality, highly optimized results.","speaker":null,"is_sponsor":0},{"start_s":92.52,"end_s":99.04,"text":"One possible indicator that search engines suck now is the growing use of Reddit as a de facto search engine.","speaker":null,"is_sponsor":0},{"start_s":99.04,"end_s":105.28,"text":"Sadly, Reddit's own internal search function has long been considered what the experts call absolute trash,","speaker":null,"is_sponsor":0},{"start_s":105.28,"end_s":108.92,"text":"but that hasn't stopped savvy users from just sticking the word Reddit","speaker":null,"is_sponsor":0},{"start_s":108.92,"end_s":114.08,"text":"on otherwise unrelated Google search queries. It's a well-known tact for cutting out SEO-ified garbage","speaker":null,"is_sponsor":0},{"start_s":114.08,"end_s":120.76,"text":"and vapid listicles because it effectively bypasses the weaknesses of both search engines and Reddit itself.","speaker":null,"is_sponsor":0},{"start_s":120.76,"end_s":126.32,"text":"Reddit isn't perfect, just ask any Redditor, but on the modern internet, it does a rare and special thing.","speaker":null,"is_sponsor":0},{"start_s":126.32,"end_s":131.92,"text":"It allows users to direct their question to a bunch of big old nerds who care more about being right","speaker":null,"is_sponsor":0},{"start_s":131.92,"end_s":138.56,"text":"than they care about making money off the interaction. If you search site colon reddit.com search engine bad,","speaker":null,"is_sponsor":0},{"start_s":138.56,"end_s":142.48,"text":"you'll find plenty of posts complaining about the decaying state of modern search engines","speaker":null,"is_sponsor":0},{"start_s":142.48,"end_s":146.24,"text":"going back over a decade. There has, however, been a pretty major uptick","speaker":null,"is_sponsor":0},{"start_s":146.24,"end_s":149.36,"text":"in such posts over the last two years.","speaker":null,"is_sponsor":0},{"start_s":149.36,"end_s":154.32,"text":"Separately, Google Trends data shows that Reddit has been steadily gaining popularity","speaker":null,"is_sponsor":0},{"start_s":154.32,"end_s":158.88,"text":"as a search term since 2010, when news aggregator Dig shot themselves in the foot","speaker":null,"is_sponsor":0},{"start_s":158.88,"end_s":162.4,"text":"with a controversial redesign and started bleeding users.","speaker":null,"is_sponsor":0},{"start_s":162.4,"end_s":165.8,"text":"That growth remained steady until December, 2021,","speaker":null,"is_sponsor":0},{"start_s":165.8,"end_s":170.28,"text":"when users started appending Reddit to their searches at an increased rate,","speaker":null,"is_sponsor":0},{"start_s":170.28,"end_s":173.6,"text":"over 40% of Reddit's growth as a Google search term","speaker":null,"is_sponsor":0},{"start_s":173.6,"end_s":177.68,"text":"since 2010 is from the end of 2021 onward,","speaker":null,"is_sponsor":0},{"start_s":177.68,"end_s":182.04,"text":"a bit over two years. Now, there are a lot of potential confounding factors here,","speaker":null,"is_sponsor":0},{"start_s":182.04,"end_s":187.08,"text":"but this could be, at least in part, a consequence of widespread dissatisfaction","speaker":null,"is_sponsor":0},{"start_s":187.08,"end_s":190.8,"text":"with search engines. An interesting contrast to Reddit's upward trend","speaker":null,"is_sponsor":0},{"start_s":190.8,"end_s":195.0,"text":"as a search term is Wikipedia, which long predated Reddit as the kind of word","speaker":null,"is_sponsor":0},{"start_s":195.0,"end_s":198.0,"text":"you add to the end of a search query in order to cut through the noise.","speaker":null,"is_sponsor":0},{"start_s":198.0,"end_s":201.04,"text":"Wikipedia is a far more popular website,","speaker":null,"is_sponsor":0},{"start_s":201.04,"end_s":204.64,"text":"currently ranked seventh for global traffic to Reddit's 16th,","speaker":null,"is_sponsor":0},{"start_s":204.64,"end_s":208.04,"text":"but it's been on the decline as a search term since 2010,","speaker":null,"is_sponsor":0},{"start_s":208.04,"end_s":211.64,"text":"in part because Google heavily prioritizes Wikipedia already,","speaker":null,"is_sponsor":0},{"start_s":211.64,"end_s":216.2,"text":"both in search results and as part of its knowledge panel widget.","speaker":null,"is_sponsor":0},{"start_s":216.2,"end_s":220.24,"text":"But this might also indicate that the decline in quality for search engine results","speaker":null,"is_sponsor":0},{"start_s":220.24,"end_s":223.32,"text":"isn't hitting every search subject equally.","speaker":null,"is_sponsor":0},{"start_s":223.32,"end_s":228.2,"text":"There's not a ton of money riding on a search query like, when was the war of 1812?","speaker":null,"is_sponsor":0},{"start_s":228.2,"end_s":232.32,"text":"So the top results are mostly authoritative for reliable history sources.","speaker":null,"is_sponsor":0},{"start_s":232.32,"end_s":236.84,"text":"But if the most important goal of search engines is to find useful results,","speaker":null,"is_sponsor":0},{"start_s":236.84,"end_s":241.08,"text":"why haven't they fixed the problem? It's not that search engine companies don't care","speaker":null,"is_sponsor":0},{"start_s":241.08,"end_s":244.28,"text":"that spam is cluttering up the first two pages of results.","speaker":null,"is_sponsor":0},{"start_s":244.28,"end_s":247.36,"text":"They've been in an arms race with spam since the very beginning.","speaker":null,"is_sponsor":0},{"start_s":247.36,"end_s":250.92,"text":"It's just that the spam is now apparently winning.","speaker":null,"is_sponsor":0},{"start_s":250.92,"end_s":256.04,"text":"According to the authors of the German study that we mentioned earlier, search engine companies banning spam sites","speaker":null,"is_sponsor":0},{"start_s":256.04,"end_s":261.0,"text":"and readjusting their parameters had a positive, but ultimately temporary impact.","speaker":null,"is_sponsor":0},{"start_s":261.0,"end_s":264.6,"text":"There was still a general downward trend in terms of text quality and relevance","speaker":null,"is_sponsor":0},{"start_s":264.6,"end_s":269.24,"text":"for all three search engines studied, which could imply that this isn't a problem","speaker":null,"is_sponsor":0},{"start_s":269.24,"end_s":273.56,"text":"with search engines, but a problem with the internet itself.","speaker":null,"is_sponsor":0},{"start_s":273.56,"end_s":277.28,"text":"Appearing on the first page of Google can be life or death for a company.","speaker":null,"is_sponsor":0},{"start_s":277.28,"end_s":280.58,"text":"So there's massive financial incentive to game that system.","speaker":null,"is_sponsor":0},{"start_s":280.58,"end_s":285.04,"text":"The same as how there's a massive financial incentive to game ratings on sites like Amazon","speaker":null,"is_sponsor":0},{"start_s":285.04,"end_s":289.48,"text":"where fake reviews are notoriously rampant. Companies both big and small have realized","speaker":null,"is_sponsor":0},{"start_s":289.48,"end_s":294.0,"text":"that word of mouth personal recommendations from a financially disinterested human being","speaker":null,"is_sponsor":0},{"start_s":294.0,"end_s":299.2,"text":"are far more effective than traditional advertising, which means that the shadier among them","speaker":null,"is_sponsor":0},{"start_s":299.2,"end_s":303.6,"text":"put a lot of effort and resources into infiltrating so-called organic,","speaker":null,"is_sponsor":0},{"start_s":303.6,"end_s":309.04,"text":"user-generated systems of validation, drowning out authentic user reviews.","speaker":null,"is_sponsor":0},{"start_s":309.04,"end_s":312.04,"text":"Not to get too metaphorical, but the only way to be heard in a room","speaker":null,"is_sponsor":0},{"start_s":312.04,"end_s":315.56,"text":"where everybody's already yelling is to scream even louder.","speaker":null,"is_sponsor":0},{"start_s":315.56,"end_s":318.88,"text":"Everyone's incentive is to make more and better garbage.","speaker":null,"is_sponsor":0},{"start_s":319.56,"end_s":324.44,"text":"Large, vertically integrated companies like Google don't really help this hyper-competitive,","speaker":null,"is_sponsor":0},{"start_s":324.44,"end_s":327.68,"text":"low-effort environment when they leverage their platform","speaker":null,"is_sponsor":0},{"start_s":327.68,"end_s":331.52,"text":"to prioritize their own products over competitors.","speaker":null,"is_sponsor":0},{"start_s":331.52,"end_s":334.8,"text":"Google has had long-standing beef with Yelp","speaker":null,"is_sponsor":0},{"start_s":334.8,"end_s":338.56,"text":"since at least 2011 when the FTC investigated allegations","speaker":null,"is_sponsor":0},{"start_s":338.56,"end_s":343.92,"text":"that Google's search algorithm consistently favored Google Places over Yelp.","speaker":null,"is_sponsor":0},{"start_s":343.92,"end_s":348.08,"text":"That allegation was serious enough that Google actually agreed to allow online services","speaker":null,"is_sponsor":0},{"start_s":348.08,"end_s":353.76,"text":"to opt out of data scraping. Yelp further contributed data to a 2015 academic paper","speaker":null,"is_sponsor":0},{"start_s":353.76,"end_s":357.36,"text":"claiming that Google manipulates search results to favor itself.","speaker":null,"is_sponsor":0},{"start_s":357.36,"end_s":362.88,"text":"Small companies perceive often accurately that the platform they essentially have to use","speaker":null,"is_sponsor":0},{"start_s":362.88,"end_s":366.56,"text":"is a potential competitor that can and will replace them","speaker":null,"is_sponsor":0},{"start_s":366.56,"end_s":372.12,"text":"with a store-brand version at any time. Those fancy widgets and rich snippets exist","speaker":null,"is_sponsor":0},{"start_s":372.12,"end_s":375.84,"text":"so that the engine that can take you anywhere you wanna go","speaker":null,"is_sponsor":0},{"start_s":375.84,"end_s":379.96,"text":"is now a place that you never have to leave. So what are you gonna do?","speaker":null,"is_sponsor":0},{"start_s":379.96,"end_s":383.72,"text":"Build a better product? Pay for your position at the top","speaker":null,"is_sponsor":0},{"start_s":383.72,"end_s":388.08,"text":"or find a louder way to scream. That's why even though we said earlier","speaker":null,"is_sponsor":0},{"start_s":388.08,"end_s":391.2,"text":"that it's not necessarily search engine companies fault","speaker":null,"is_sponsor":0},{"start_s":391.2,"end_s":394.36,"text":"that they've gotten worse, it also kinda is.","speaker":null,"is_sponsor":0},{"start_s":394.36,"end_s":399.0,"text":"You know how captures have gotten harder and harder over time? Well, that's in part because they were used","speaker":null,"is_sponsor":0},{"start_s":399.0,"end_s":403.08,"text":"to train machine learning, which then led to the bots becoming more sophisticated,","speaker":null,"is_sponsor":0},{"start_s":403.08,"end_s":406.8,"text":"which then led to the need for stronger and stronger captures and so on.","speaker":null,"is_sponsor":0},{"start_s":406.8,"end_s":411.84,"text":"What we're seeing here is likely something similar, only internet-wide, where search engines are struggling","speaker":null,"is_sponsor":0},{"start_s":411.84,"end_s":416.08,"text":"to distinguish between quality content and spam from AI systems trained","speaker":null,"is_sponsor":0},{"start_s":416.08,"end_s":419.12,"text":"on traditionally trusted sources like Wikipedia.","speaker":null,"is_sponsor":0},{"start_s":419.12,"end_s":422.64,"text":"The increasing cheapness of relatively sophisticated spam tools","speaker":null,"is_sponsor":0},{"start_s":422.64,"end_s":428.6,"text":"has resulted in numerous odd trends. Some funny, like when oodles of products on Amazon","speaker":null,"is_sponsor":0},{"start_s":428.6,"end_s":432.32,"text":"wound up being named, I'm sorry, I cannot fulfill that request.","speaker":null,"is_sponsor":0},{"start_s":432.32,"end_s":436.84,"text":"It goes against open AI use policy. And others, disturbing.","speaker":null,"is_sponsor":0},{"start_s":436.84,"end_s":440.36,"text":"Like the rise of procedurally generated clickbait obituaries,","speaker":null,"is_sponsor":0},{"start_s":440.36,"end_s":443.4,"text":"often for private citizens, many of whom aren't even dead.","speaker":null,"is_sponsor":0},{"start_s":443.4,"end_s":447.24,"text":"You used to have to be at least a D-list celebrity in order to get that kind of treatment.","speaker":null,"is_sponsor":0},{"start_s":447.24,"end_s":450.24,"text":"Search engines have not lost the fight against spam,","speaker":null,"is_sponsor":0},{"start_s":450.24,"end_s":453.92,"text":"at least not yet. But as machine generation continues to progress","speaker":null,"is_sponsor":0},{"start_s":453.92,"end_s":457.68,"text":"and proliferate, your search experience probably isn't going to get any better.","speaker":null,"is_sponsor":0},{"start_s":457.68,"end_s":464.76,"text":"Thanks for watching guys, if you liked this video, maybe you'd like another video we have about how streaming is basically becoming cable.","speaker":null,"is_sponsor":0},{"start_s":464.76,"end_s":466.96,"text":"You can click on it somewhere.","speaker":null,"is_sponsor":0}],"full_text":"Why do search engines suck now? Wait, before we get ahead of ourselves, do search engines suck now? Are they actually getting worse, or have they just changed in a way that I personally hate? Just by glancing at the first page of a Google or Bing search, it's easy to find a long list of potential complaints. Paid sponsors crowd out the first few search results. There's annoying little widgets everywhere, and it keeps giving you barely related suggestions for what to search next, like a lame choose your own adventure book. But why does this search page have so many ads? Isn't an interesting question. It's because money. The more interesting question is why it feels like it's gotten harder to find the information you want despite all those supposedly helpful widgets. The answer is also because money, but it's worth unpacking why. It's difficult to get a lot of rigorous data on this subject, but at least a few researchers have tried to answer whether or not search engines are, in fact, getting worse. One recent German study did a survey of over 7,000 product review searches on Bing, Google, and DuckDuckGo over the course of a year, and concluded that you could still find useful information, but it was being drowned out by a torrent of low quality content, especially SEO spam. Top ranked pages typically were heavily optimized and littered with affiliate marketing links, and they also showed clear markers of lower content quality. There's not a ton of academic papers on this issue, but there's plenty of data on the web showing how users have changed their behavior to sidestep low quality, highly optimized results. One possible indicator that search engines suck now is the growing use of Reddit as a de facto search engine. Sadly, Reddit's own internal search function has long been considered what the experts call absolute trash, but that hasn't stopped savvy users from just sticking the word Reddit on otherwise unrelated Google search queries. It's a well-known tact for cutting out SEO-ified garbage and vapid listicles because it effectively bypasses the weaknesses of both search engines and Reddit itself. Reddit isn't perfect, just ask any Redditor, but on the modern internet, it does a rare and special thing. It allows users to direct their question to a bunch of big old nerds who care more about being right than they care about making money off the interaction. If you search site colon reddit.com search engine bad, you'll find plenty of posts complaining about the decaying state of modern search engines going back over a decade. There has, however, been a pretty major uptick in such posts over the last two years. Separately, Google Trends data shows that Reddit has been steadily gaining popularity as a search term since 2010, when news aggregator Dig shot themselves in the foot with a controversial redesign and started bleeding users. That growth remained steady until December, 2021, when users started appending Reddit to their searches at an increased rate, over 40% of Reddit's growth as a Google search term since 2010 is from the end of 2021 onward, a bit over two years. Now, there are a lot of potential confounding factors here, but this could be, at least in part, a consequence of widespread dissatisfaction with search engines. An interesting contrast to Reddit's upward trend as a search term is Wikipedia, which long predated Reddit as the kind of word you add to the end of a search query in order to cut through the noise. Wikipedia is a far more popular website, currently ranked seventh for global traffic to Reddit's 16th, but it's been on the decline as a search term since 2010, in part because Google heavily prioritizes Wikipedia already, both in search results and as part of its knowledge panel widget. But this might also indicate that the decline in quality for search engine results isn't hitting every search subject equally. There's not a ton of money riding on a search query like, when was the war of 1812? So the top results are mostly authoritative for reliable history sources. But if the most important goal of search engines is to find useful results, why haven't they fixed the problem? It's not that search engine companies don't care that spam is cluttering up the first two pages of results. They've been in an arms race with spam since the very beginning. It's just that the spam is now apparently winning. According to the authors of the German study that we mentioned earlier, search engine companies banning spam sites and readjusting their parameters had a positive, but ultimately temporary impact. There was still a general downward trend in terms of text quality and relevance for all three search engines studied, which could imply that this isn't a problem with search engines, but a problem with the internet itself. Appearing on the first page of Google can be life or death for a company. So there's massive financial incentive to game that system. The same as how there's a massive financial incentive to game ratings on sites like Amazon where fake reviews are notoriously rampant. Companies both big and small have realized that word of mouth personal recommendations from a financially disinterested human being are far more effective than traditional advertising, which means that the shadier among them put a lot of effort and resources into infiltrating so-called organic, user-generated systems of validation, drowning out authentic user reviews. Not to get too metaphorical, but the only way to be heard in a room where everybody's already yelling is to scream even louder. Everyone's incentive is to make more and better garbage. Large, vertically integrated companies like Google don't really help this hyper-competitive, low-effort environment when they leverage their platform to prioritize their own products over competitors. Google has had long-standing beef with Yelp since at least 2011 when the FTC investigated allegations that Google's search algorithm consistently favored Google Places over Yelp. That allegation was serious enough that Google actually agreed to allow online services to opt out of data scraping. Yelp further contributed data to a 2015 academic paper claiming that Google manipulates search results to favor itself. Small companies perceive often accurately that the platform they essentially have to use is a potential competitor that can and will replace them with a store-brand version at any time. Those fancy widgets and rich snippets exist so that the engine that can take you anywhere you wanna go is now a place that you never have to leave. So what are you gonna do? Build a better product? Pay for your position at the top or find a louder way to scream. That's why even though we said earlier that it's not necessarily search engine companies fault that they've gotten worse, it also kinda is. You know how captures have gotten harder and harder over time? Well, that's in part because they were used to train machine learning, which then led to the bots becoming more sophisticated, which then led to the need for stronger and stronger captures and so on. What we're seeing here is likely something similar, only internet-wide, where search engines are struggling to distinguish between quality content and spam from AI systems trained on traditionally trusted sources like Wikipedia. The increasing cheapness of relatively sophisticated spam tools has resulted in numerous odd trends. Some funny, like when oodles of products on Amazon wound up being named, I'm sorry, I cannot fulfill that request. It goes against open AI use policy. And others, disturbing. Like the rise of procedurally generated clickbait obituaries, often for private citizens, many of whom aren't even dead. You used to have to be at least a D-list celebrity in order to get that kind of treatment. Search engines have not lost the fight against spam, at least not yet. But as machine generation continues to progress and proliferate, your search experience probably isn't going to get any better. Thanks for watching guys, if you liked this video, maybe you'd like another video we have about how streaming is basically becoming cable. You can click on it somewhere."}