Some AI papers have a finding so good you just want to quote it everywhere. But many turn out to be trash. They’ve got bad statistics, they’ve got bad methodology, they’re really sales pieces, or t…
  • UnderpantsWeevilEnglish
    4·
    10 months ago

    The most glaring problem here is that MOST PEOPLE USE AI ON APPS ON THEIR PHONES

    The sleight of hand in the industry, wherein businesses that used to provide users with search results to explicit queries and now repackage their search tool as “AI” with a more natural language interface, has produced a group of people who are outspokenly insisting AI is a success because users still need the web to search for data.

    And beyond that, Chrome doesnt cache browsing history in Incognito mode, which is likely where a lot of AI use is happening.

    No idea why you would presume the bulk of AI use was happening Incognito. This seems like you’re trying to argue in the negative - “you can’t prove they’re not using AI in secret and loving it”.

    And yet the researcher, this blogger, and everyone here simply ignore these gigantic problems with the study

    Seems like they’re collecting some basic popularity metrics and finding the results that are consistent with their audience’s own experiences. The tools are clumsy, inaccurate, cumbersome, and unprofitable for both the host and the end user. LLMs aren’t ready for prime time, no matter how many ways a business tries to shoehorn an AI interface into their application model.

    However, I am pro-science, and this is straight up not science.

    Hypothesis, data collection, analysis, results.

    That’s textbook scientific analysis.

    • blarghlyEnglish
      12·
      10 months ago

      The sleight of hand in the industry,

      What are you talking about? Most web traffic these days is mobile traffic. The most common way people use AI (other than in search summaries, which the study doesn’t seem to have counted) is by whipping out the chatgpt app on their phone. If you want to know “what tree is that?” or “what is the name of Elrond’s hair stylist?” you aren’t going to run back to your laptop to query chatgpt on Chrome.

      Other common uses of AI are for coding (independent applications) or chat bots (independent applications)

      No idea why you would presume the bulk of AI use was happening Incognito.

      Porn

      Seems like they’re collecting some basic popularity metrics and finding the results that are consistent with their audience’s own experiences.

      Sure, go collect data. But point out that you shouldn’t draw conclusions from the data since ut is woefully incomplete.

      Hypothesis, data collection, analysis, results. That’s textbook scientific analysis.

      It’s garbage scientific analysis. This crap shouldnt make it past peer review. It’s worse than p-hacking and the replication crisis since it isn’t even hard to find the flaws in their methodology. This is “science” for the sake of getting published - it is disingenuous and actively sets back the public’s faith in science.