Anyone who has been surfing the web for a while is probably used to clicking through a CAPTCHA grid of street images, identifying everyday objects to prove that they’re a human and not an automated bot. Now, though, new research claims that locally run bots using specially trained image-recognition models can match human-level performance in this style of CAPTCHA, achieving a 100 percent success rate despite being decidedly not human.

ETH Zurich PhD student Andreas Plesner and his colleagues’ new research, available as a pre-print paper, focuses on Google’s ReCAPTCHA v2, which challenges users to identify which street images in a grid contain items like bicycles, crosswalks, mountains, stairs, or traffic lights. Google began phasing that system out years ago in favor of an “invisible” reCAPTCHA v3 that analyzes user interactions rather than offering an explicit challenge.

Despite this, the older reCAPTCHA v2 is still used by millions of websites. And even sites that use the updated reCAPTCHA v3 will sometimes use reCAPTCHA v2 as a fallback when the updated system gives a user a low “human” confidence rating.

  • @[email protected]
    link
    fedilink
    English
    90
    edit-2
    3 months ago

    This is actually a good sign for self driving. Google was using this data as a training set for Waymo. If AI is accurately identifying vehicles and traffic markings, it should be able to process interactions with them easier.

    • @[email protected]
      link
      fedilink
      English
      713 months ago

      As I understand it, the point of those captchas was never really “bots can’t identify these things” (though you’re right on that it was used to train). They use cursor movement, clicks, and other behaviours while you’re solving it to detect if you are a bot or not.

      • @Grimy
        link
        English
        433 months ago

        The image choosing was always just to train their own bots

      • @Takumidesh
        link
        English
        11
        edit-2
        3 months ago

        It’s a combination.

        Most captchas goals generally aren’t 100% prevention, it’s to put a workload in front, this makes spamming the site cost money, a bankrolled attempt could just as easily outsource the captchas to real humans.

        • @[email protected]
          link
          fedilink
          English
          12 months ago

          a bankrolled attempt could just as easily outsource the captchas to real humans.

          Exactly. I’ve been using 2captcha for that for over a decade now

      • @[email protected]
        link
        fedilink
        English
        93 months ago

        Since I started getting good at yosu and that fishing mini game in farmrpg I’ve been failing more captchas. I wonder if they’re related knowing this

      • @nieceandtows
        link
        English
        52 months ago

        Is that why I’m asked to do this over and over for 14 million times when I’m on a VPN?

        • @[email protected]
          link
          fedilink
          English
          42 months ago

          It is probably part of it, yeah. But to be clear I’m not a captcha expert or anything, just a layman.

    • @grue
      link
      English
      323 months ago

      The annoying thing is that they held us hostage for our free labor, but the results are proprietary for Google’s benefit only.

      That training data ought to be forced to be made freely available to the public, since we’re the ones who actually created it.

    • @[email protected]
      link
      fedilink
      English
      33 months ago

      Afaik this is precisely what the captcha data was intended for - training AI models. Originally leveraged machine learning. LLMs are a slightly different paradigm but same purpose and results here.

      • @[email protected]
        link
        fedilink
        English
        13
        edit-2
        3 months ago

        Its never been confirmed by Google, so I may be wrong. It still tracks that the data harvesting company with a AI self driving car project would use free human labor to identify road hazards.

        • Arthur Besse
          link
          fedilink
          English
          -9
          edit-2
          3 months ago

          I was referring to the “This is actually a good sign for self driving” part of their comment.

          The captcha circumvention arms race has been going on for over two decades, and every new type of captcha has and will continue to be broken as soon as it’s widely deployed enough that someone is motivated to spend the time to.

          So, the notion that an academic paper about breaking the current generation of traffic-related captchas (something which the captcha solving industry has been doing for years with a pretty high success rate already) is “good news” for the autonomous vehicle industry (who has also been able to identify such objects well enough to continue existing and getting more regulatory approval for years now) is…

          fry not sure meme template, no text

          • @[email protected]
            link
            fedilink
            English
            16
            edit-2
            3 months ago

            Not really. I’m not even sure what you’re disagreeing with based on the above comment.

            My point is that if bog standard AI can accurately identify all of the road information from pictures, that is good news for self driving.

            What was once a nearly impossible task for computers is now mundane, and can be used to improve safety/utility for self driving, especially for FOSS projects like comma.ai

    • Draconic NEO
      link
      English
      262 months ago

      Captcha these days isn’t even really a CAPTCHA in the traditional sense since most of the work it does is based on filtering of IP and browser fingerprinting, with a certain level of gamification because the goal is not just to keep out the people they fight against, but to waste their time, would work great if it didn’t waste normal people’s time, while real bad actors have easy ways to get around it.

    • @[email protected]
      link
      fedilink
      English
      102 months ago

      I was going to say I’ve straight up just left whatever website I was trying to access because I was stuck in some endless loop of clicking on street crossings, buses, bikes, and street lights.

      • @[email protected]
        link
        fedilink
        English
        1
        edit-2
        2 months ago

        Fellow vpn user here, it’s been really bad lately. I’m definitely installing this.

  • @samus12345
    link
    English
    553 months ago

    So can we stop using those damn things? They’re super annoying!

    • ohellidk
      link
      fedilink
      English
      163 months ago

      I’m kind of hoping the AI permanently beats them. I hate them too.

    • @aidan
      link
      English
      32 months ago

      Just means they’ll get harder, but maybe not for people, just needs to be harder for a computer

  • @[email protected]
    link
    fedilink
    English
    432 months ago

    Aren’t these Captchas designed to get training data for AI models anyway?

    “System does what it was designed to do” doesn’t feel that surprising…

    • @aidan
      link
      English
      42 months ago

      Aren’t these Captchas designed to get training data for AI models anyway?

      Yes and no, the captchas are just meant to be hard for computers to solve but easier for humans. People saw that, and thought that “if we’re making people do this might as well have them do something useful” not meant to be malevolent- and the purpose is still stopping bots, training them is a side-effect.

      • @finitebanjo
        link
        English
        32 months ago

        No, you’re wrong, the Traffic Light examples ARE specifically to gather data to train models. Being a good Captcha was just a byproduct of that. If people just wanted a good captcha they wouldn’t need hundreds of millions of photos of street lights and bicycles.

        • @aidan
          link
          English
          -22 months ago

          No, you’re wrong, the Traffic Light examples ARE specifically to gather data to train models.

          No you’re wrong, because the sites that embed those captchas on their page are not doing that to help good.

          If people just wanted a good captcha they wouldn’t need hundreds of millions of photos of street lights and bicycles.

          Yes, they are getting something productive out of the human labor that would be done anyways. Trust me as a web developer, and web scraper, some kind of captcha is necessary for many free services to be useful/economically viable. The core of a good captcha is just making it marginally more expensive for the scraper/bot than it is for you.

  • @pixxelkick
    link
    English
    423 months ago

    Well yeah, I’d hope so, that’s the entire point.

    Catcha’s data collection always was with the intent for training ai on these skills. That’s “the point” of them.

    It’s reasonable to expect that the older version of captchas can now be beaten by modern ai, because they’re often literally trained on that exact data to beat it.

    Captcha effectively is free to use on websites as a tool because the data collection is the “payment”, they then license that data out to people like OpenAI to train with for stuff like image recognition.

    It’s why ai is progressing so fast, captchas are one of humanity’s long term collected data silos that are very full now.

    We are going to have to keep progressing the complexity of catches as it will be the only way to catch modern AIs, and in turn it will collect more data to improve it.

    • @MIDItheKID
      link
      English
      123 months ago

      Yeah, my understanding is that these capchas were made to harvest data to use for AI/Autopilot driven cars. That’s why they are always having you identify motorcycles, bycicles, crosswalks, stoplights, busses, etc. It’s all stuff that automatic driving cars have had a hard time identifying.

    • @[email protected]
      link
      fedilink
      English
      3
      edit-2
      2 months ago

      We are going to have to keep progressing the complexity of catches as it will be the only way to catch modern AIs, and in turn it will collect more data to improve it.

      I wanted to use 4chan alot before I came here, but FUCK that slider capcha. I bailed after the first time I didn’t pass.

      • @Riccosuave
        link
        English
        72 months ago

        I wanted to use 4chan

        I am relatively confident that you are one of the first people to ever type that sentence out.

          • @Riccosuave
            link
            English
            22 months ago

            I think I’m good on that, but you do you m8.

        • @[email protected]
          link
          fedilink
          English
          22 months ago

          I reread his comment three times because I was convinced I must have read it in error somehow.

  • @[email protected]
    link
    fedilink
    English
    39
    edit-2
    2 months ago

    I fucking hate these. I’ve seen old people that don’t know any better get stuck on these for at least 30 minutes.

    • @pyre
      link
      English
      242 months ago

      it’s super ableist. if someone has poor vision or colorblindness chances are they’re going to miss things.

      • @Dozzi92
        link
        English
        242 months ago

        I have regular everything and I still fuck them up. “click the ones with a fire hydrant”. But a tiny piece of fire hydrant is spilling into another box. Does it count? Does it not count? Good luck!!

        I had one the other day that was deep fried jpegs to the max. Like, what the fuck am I supposed to do.

        • @Lost_My_Mind
          link
          English
          42 months ago

          Sprinkle powdered sugar on them. Delicious deep fried jpegs.

        • @scottywh
          link
          English
          12 months ago

          Spillovers into other boxes definitely count…

          I don’t want to do this next part but I can’t resist…

          Just ask my girlfriend…

          Ba dum tiss

      • Dark Arc
        link
        fedilink
        English
        22 months ago

        FYI as someone that’s colorblind these captcha’s don’t seem to have anything specially relevant to being colorblind in them.

        Now if they start showing me a dozen traffic cones and asking me to pick the green one, we might have a problem.

        • @pyre
          link
          English
          72 months ago

          a hard to see option, aptly enough

  • Destide
    link
    fedilink
    English
    353 months ago

    When it’s asking for motorcycles but it’s clearly a scooter

    • @ripcord
      link
      English
      263 months ago

      Or, like, “there’s the bottom 10% of a traffic light in this one. Do I click that box? Ia that supposed to count?”

      • @[email protected]
        link
        fedilink
        English
        82 months ago

        What they are doing is comparing your answer and seeing if it is consistent with how it has been answered previously. They realize that not everyone is going to give the exact same answer, so as long as you answer it in a way that enough other people have answered it, it should let you in.

        I’ll usually go with the minimum number of clicks that I think will get me through, since I’m lazy and it’ll also at times slow down how fast you can click which is annoying.

        I’ll also answer them wrong if I think it’s a mistake that enough other people will make. “Yes… that RV over there is a bus…”

        • Echo Dot
          link
          fedilink
          English
          42 months ago

          They are also overly US centric.

          One of the questions asks you to click on only the school buses. I had to Google how you tell the difference between a school bus and not a school bus.

          Also is it a crosswalk if it’s at an intersection or is it only a crosswalk if it’s in the middle of a road somewhere?

          The questions either need to be not cultural or they need to be adapted for where they detect the user is coming from, the first option seems easier.

          • @AA5B
            link
            English
            22 months ago

            Interesting. Do you not have school buses, or are school buses not distinctly marked? How do kids get to school when it’s beyond walking distance?

            • Echo Dot
              link
              fedilink
              English
              12 months ago

              They are just buses.

              I guess the British government just assume that school children are smart enough to get on the right bus without them being individually distinct.

              I knew school buses are yellow but I did not realize that they are always yellow. I did not realize that the yellow color meant school. I just assumed that the yellow color was a color busses could be.

              • @AA5B
                link
                English
                12 months ago

                Maybe it’s lack of transit in the us, I don’t know. Almost every public school district I’m familiar with, uses standard yellow school buses to bring kids to and from school. However Boston city schools give the kids an MBTA pass - I don’t know if that differed by age - and I imagine that’s true of other downtown schools where there’s transit

              • @[email protected]
                link
                fedilink
                English
                0
                edit-2
                2 months ago

                The size of the UK verses the exponentially larger size of the US probably has a lot to do with it.

                And if you knew school busses where yellow… Where’s the problem?