Bots Are Better Than Humans At Cracking 'Are You a Robot?' Captcha Tests, Study Finds (independent.co.uk) 78
A recent comprehensive study reveals that automated bots are substantially more efficient than humans at cracking Captcha tests, a widely used security measure on over 100 popular websites. The Independent reports: In the study, scientists assessed 200 of the most popular websites and found 120 still used Captcha. They took the help of 1,000 participants online from diverse backgrounds -- varying in location, age, sex and educational level -- to take 10 captcha tests on these sites and gauge their difficulty levels. Researchers found many bots described in scientific journals could beat humans at these tests in both speed and accuracy.
Some Captcha tests took human participants between nine and 15 seconds to solve, with an accuracy of about 50 to 84 per cent, while it took the bots less than a second to crack them, with up to near perfection. "The bots' accuracy ranges from 85-100 per cent, with the majority above 96 per cent. This substantially exceeds the human accuracy range we observed (50-85 per cent)," scientists wrote in the study. They also found that the bots' solving times are "significantly lower" or nearly the same as humans in almost all cases.
Some Captcha tests took human participants between nine and 15 seconds to solve, with an accuracy of about 50 to 84 per cent, while it took the bots less than a second to crack them, with up to near perfection. "The bots' accuracy ranges from 85-100 per cent, with the majority above 96 per cent. This substantially exceeds the human accuracy range we observed (50-85 per cent)," scientists wrote in the study. They also found that the bots' solving times are "significantly lower" or nearly the same as humans in almost all cases.
AI lies better than humans (Score:2)
Ha! We've finally found a use case for LLM weak AI systems.
Not surprising (Score:5, Funny)
Have you seen the type of humans wandering around of late? I believe a chipmunk could do better at these tests than humans.
Re: (Score:2)
Everything changed with LLMs. Intelligence is more than language tests, but if tested with just language, an LLM scores above genius level IQ. It's in the top 0.1% and will get better.
https://www.scientificamerican... [scientificamerican.com]
You don't have to qualify with "type of humans". You and I and just about everyone here will lose to an LLM on these tests.
Re: (Score:2)
Well, captcha's aren't usually language tests, but yeah, it's not surprising that they can do better than people when trained on that kind of test. It *is* a bit surprising that publicly available tools can do that, however.
Re: (Score:2)
Oh man, you cracked me up on this one. LOL
Re: (Score:2)
I believe a chipmunk could do better at these tests than humans.
Jokes on them really, because these humans can hold more food in their mouths.
That was called hashcash (Score:4, Informative)
Addition or alternative to captcha make the client mine some crypto or something?
That was called hashcash [wikipedia.org], and use for email is what inspired the invention of cryptocurrency. Use of proof of work was found to unfairly advantage desktop users with a recent GPU over users of desktop and laptop computers with integrated graphics or (eventually) smartphones.
By design (Score:5, Insightful)
Considering so many captchas are being used to feed ML datasets, it's very not surprising AI trained on those datasets can now solve them effortlessly...
Re:By design (Score:5, Funny)
If you solve it quickly and correctly, you are a bot. If you struggle and fail, come right in!
Re: (Score:3)
You joke, but you know how this will go now... you'll *have* to spend at least 15 seconds looking at the damn thing before it'll ever consider you a human. Guess what'll happen next... yep, some ads will appear alongside - just to keep you entertained while you wait.
For this next step of enshitification, I for one can't wait - well, I can, because I'll have to ;-)
Re: By design (Score:4, Interesting)
Thatâ(TM)s literally how reCaptcha and hCaptcha work. The âoechoose all pictures containing traffic lightsâ is just you providing data for them to train their ML model. The actual captcha involves looking at how long it takes you to choose them, along with various other metrics.
Re: (Score:2)
You can program it to wait a few seconds before responding, and to give the wrong answers a certain proportion of the time.
Re:By design (Score:4, Funny)
Re: By design (Score:2)
Does javascript have sleep()?
Re: (Score:3)
Certainly. That's why it's so slow. You can even program how slow you want it to be, for when you need your clients to buy new computers, because current ones are too slow. :)) Just joking...
Can they get past The Cookies Check? (Score:5, Insightful)
Re: (Score:1)
Re: (Score:2)
Best I've got is I Don't Care About Cookies to kill the popup and Privacy Badger to castrate the cookies themselves. Gets almost all of them.
What we really need is the ROTW to catch up to the EU and UK with making cookie skulduggery illegal (a "G" GDPR). No, you can't ask me to give up my statutory rights with a popup. Also the irony of American sites doing this is palpable. The EU knows it's own laws don't apply to entities outside it's own b
Cost of article 27 compliance multiplies (Score:2)
What we really need is the ROTW to catch up to the EU and UK with making cookie skulduggery illegal (a "G" GDPR).
If different countries and free trade areas were to adopt their own counterparts to GDPR, in how many different countries would each website operator need to register with a designated local representative pursuant to article 27?
The EU knows it's own laws don't apply to entities outside it's own borders
Laws of the Union apply at the border. US-based online stores without a representative pursuant to article 27 could see their shipments turned away at member states' customs.
Re: (Score:2)
What we really need is the ROTW to catch up to the EU and UK with making cookie skulduggery illegal (a "G" GDPR).
If different countries and free trade areas were to adopt their own counterparts to GDPR, in how many different countries would each website operator need to register with a designated local representative pursuant to article 27?
Erm, maybe they can just stop trying to store people's data and sell it off to other parties. That seems the cheapest solution
The EU knows it's own laws don't apply to entities outside it's own borders
Laws of the Union apply at the border. US-based online stores without a representative pursuant to article 27 could see their shipments turned away at member states' customs.
Erm, a server in a completely different country, no physical presence in the EU... How would they enforce that. You clearly know nothing about the EU if you think that they'll block any site that doesn't register with them. Even in the worst case scenario... why am I still getting anti-EU annoyance popups when I'm in the Americas?
Also article 27 only applies if you're holding data
Physical goods shipping address is personal data (Score:2)
Erm, maybe they can just stop trying to store people's data and sell it off to other parties. That seems the cheapest solution
Say a company operates a web-based store through which it sells something goods, such as toys or physical copies of a work, to customers. I fail to imagine how the company might accomplish that without collecting two pieces of customers' personal data: the billing address and the shipping address. Even a website that doesn't ship physical goods is taking billing addresses once it sells a subscription.
Erm, a server in a completely different country, no physical presence in the EU... How would they enforce that.
If a website neither mentions any member state nor quotes prices in euros nor offers a translation of the we
Re: (Score:3)
Re:Can they get past The Cookies Check? (Score:4, Funny)
"The Internet doesn't work without cookies."
Aaand now I'm visualizing "The Internet" as having blue fur and googly eyes.
Re: (Score:3)
Re: (Score:2)
But if you don't accept the cookies, the website may or may not work correctly. You want to receive the best possible browsing experience, right? The Internet doesn't work without cookies.
No one is talking about rejecting all cookies. In fact that is not an option presented to the user by these annoying popups.
Re: (Score:2)
>The Internet doesn't work without cookies.
in fact, the reason m UID here is so *high* is that I refused to register for a while due to the use of cookies!
I finally caved, but that also meant having to stop using a folder named .cookies to blanket-block them.'
hawk
Re: (Score:2)
Great (Score:5, Insightful)
Re: (Score:1)
"Click on all the pictures of people wearing jumpers."
How big are the 'bots'? (Score:3)
Re: (Score:2)
The number of pictures is limited. I've seen the same bicycles hundreds of times by now.
Even if it took 10 seconds that's not a big deal. There aren't _that_ many sites you'd care to crawl that use them and once you've hit it, you're free to do whatever after that.
Your crawler will be multi threaded. Just send the captcha threads to go do their thing while the rest continue on other sites. Turning a 20 hour crawl session into 20 hour, 15 minute crawl session isn't a big deal.
Found the paper (Score:5, Informative)
Tracking how you SOLVE the puzzle (Score:5, Interesting)
Re: (Score:2)
Re: (Score:1)
I'm pretty sure they're not collecting mouse position data on my iPad.
I have seen times where I realized I missed one after I hit go but I suspect they simply have a "good enough" setting rather than some complex and clever algorithm.
Re: (Score:2)
Re: (Score:1)
Interesting idea, thanks. Like everyone, I find them frustrating and useless. Similar to the useless "check here if you're not a bot" but worse.
My kid had one for some game she was trying to sign up for. Click the animal in the upright position. 4 animals, each successively 90 degrees off. I couldn't do it either. So they stupidly lost some number of sales because they couldn't bother the most basic testing of their captcha clone.
Re: (Score:1)
My understanding of many current implementations is that they don't just look for a correct answer, but analyze how you solve it - by tracking things like cursor movements or the time between clicks.
Yes, but that is "part of the solution" regarding the bots solving them.
They also use things like browser fingerprinting. A human, for example, would not be able to click on 4 tiles with exactly 15ms between clicks.
A bot, while it would be able to do that, would intentionally not do that either.
The bot is going to be sending "cursor move" events along with "click" events, while Not moving in a straight line between two points, while also Not moving the cursor at a consistent speed between pixels.
A bot won't select the same exact point within a tile to click, and will vary the time between the "down button" and "up button" events.
They also will add
Re:Tracking how you SOLVE the puzzle (Score:5, Insightful)
Anything that these systems use to try to differentiate between humans and bots can be faked by a bot. Anything you think of, including human weaknesses, can be faked by a bot. There is absolutely no way around this.
Maybe current implementations don't do some of this faking. That's because they don't need to. The moment they need to, they will be made to.
The enterprise of making a website try to weed out bots is fundamentally misguided. It can't be done. That ship has sailed. Give it up.
I can hear the objections already...we just need a webcam so you can see the user's face, watch them type it! We just need a secure key based mechanism that gives the website some kind of superior access to the end-users hardware, to make sure a human is using it and not a program! We just need some more laws!
It's folly. None of these will work. Nothing you can think of will work. We are in the AI age now. We cannot put that genie back into the bottle, and we cannot make websites determine when they are being accessed by a bot instead of a human. We must adapt our designs such that it simply doesn't matter if a bot is doing the work. We must build a world that tolerates bots, and still functions well enough. That's the shape of things to come.
Re: (Score:2)
We are in the AI age now.
We're not in the "AI age". That's absurd. We're in the middle of a hype cycle, and it looks like we're already past the peak. We've been using ML methods to defeat captchas since they first appeared. It's always been an arms race and we've had bots that outperform humans on various types for almost as long as we've had captchas.
The study [arxiv.org] is comparing human performance (from Mechanical Turk) across a variety of captcha types to bot performance reported in other, sometimes much older, studies. Whatever
Re: (Score:2)
The solution is simple (Score:2)
I have this plugin which helps me fill in the captcha, even does it for me. It's only the start though, the next generation of captchas requires more advanced tools to answer them and guess who is making and selling those tools? The same guys who are making the captchas.
Captchas have become pretty terrible (Score:2)
Captchas have become so terrible and annoying, I am starting to question my humanity.
Re: (Score:2)
Captchas have become so terrible and annoying, I am starting to question my humanity.
Please don't. If you were a bot, you wouldn't mind captchas at all. Your anguish proves that you're a biologic intelligent unit.
Link? (Score:2)
The linked site contains no link to the study, nor does the /. summary.
I guess this is that faith based reporting we hear so much about. Which captchas were tested? Which bots? All the linked site has is links to their own page for keywords that I am sure boost search ranking.
So? (Score:1)
well... (Score:1)
Captchas force you to train their AIs for free. (Score:2)
Not surprised (Score:2)
Eventually I gave up and chose the audio option, which I solved first time.
Re: (Score:2)
Exactly! I've failed captcha's recently for the same reasons. They used to make the answer obvious. Some of them aren't as obvious now and are frustrating.
Personally, I hate Captchas (Score:2)
The makers of captcha are evil fucks. (Score:4, Insightful)
I wish them bodily harm.
A grid of cells with, "select all the cells containing motorcycles", when sometimes a sliver of a helmet is part of it and sometimes it isn't, is a dark pattern. Whomever set that up needs to be beaten with sticks.
They absolutely offer captchas that increase in ambiguity to block content distribution. It's shady as hell.
Re: (Score:2)
Re: (Score:2)
I wish them bodily harm.
Do you routinely attack the people trying to help rather than the cause of the problem? Do you hate an ambulance instead of a terrorist? A police officer instead of the robber?
Captchas exist to solve an underlying problem. Hating on them rather than those who cause the issue in the first place is one of the most dumbfucking stupid things I've read on the internet this week.
Re: (Score:2)
" one of the most dumbfucking stupid things I've read on the internet this week."
Glad to help. Hope you found it entertaining.
In case it wasn't clear, my complaint was not about trying to find a solution to the problem. It's that the solution doesn't confine itself to that. It's because what they've built is unworkable, and used for shady purposes - like making it difficult to access the thing you have - in some cases - paid for. This is a conscious choice, I guarantee it. You can ask them to make it exasperating to pierce the veil, and that's what they'll give you.
POW Captcha (Score:2)
Too Fast (Score:2)
So if they solution takes 3 seconds it's not a human.
That seems too obvious. I can't believe this work.
Maybe I’m an AI (Score:2)
Wait, WHAT? (Score:2)
How is this NOT an Onion post?!
Re: (Score:2)
Or an ig-nobel?
bleep blip (Score:1)
I'm thinking of making a T-shirt saying, "Yes, I'm a robot! Whaddya gonna do about it!?"
Re: (Score:1)
Re: (Score:1)
Perl one-liners can pack enough info to run air traffic control. Reading it is another story.
Re: (Score:1)
I make captcha mistakes on purpose (Score:2)
I make mistakes on purpose when solving captcha tests, to find out how many mistakes you can make for captcha to be accepted. Like, when you get 2 words "gimme shelter", I enter "giemm shetler". That can pass sometimes, but entering "gimme shetler" will always pass. Same with pictures - you don't have to click on all the pictures, just a couple and even some wrong ones, to pass the test. You should try it sometimes. It's fun. I guess this also makes machine learning harder, and us humans look more stupid th
This is getting pretty obvious (Score:2)
One of the major commercial applications for today's AIs is interpreting digital scans. AIs that are getting good at finding tumors in a noisy mammogram are going to be really good at identifying fuzzy letters and numbers in a CAPTCHA.
Meanwhile, you as puny human get shown endless grids of motorcycles and traffic signals, and you will fail every time because they CAPTCHAs are now beyond human pattern matching ability. Time to retire the entire idea.
Re: (Score:2)
It's always been an arms race. We've had bots capable of defeating captchas almost as long as we've had captchas. That doesn't mean we need to scrap the whole idea, only that we likely won't have a long-term solution. Remember that 'bots' can't just magically adapt to new challenges and that training takes a great deal of time and effort.
While the study compares human performance on a few different kinds of captchas (from Mechanican Turk) to bot performance reported in other, sometimes much older, studie
Slow speed (Score:2)
bots not affected by aging in solving captcha's (Score:2)
bots aren't affected by problems that aging eyesight causes in solving captcha's
What counts as success? (Score:2)
Is success getting though the stupid gate, or accurately performing the task? I still run into plenty of captchas were system has misinterpreted markings on the side of the road as as a crosswalk, misses half of the stop-lights in a scene, etc. And I therefore end up having to answer more of the stupid things for doing them right.
Test taken the wrong way (Score:2)
Have I misjudged Captchas? (Score:1)
I have always thought that speed and pace of clicking were being evaluated along with accuracy... such that solving too quickly would indicate a 'bot, and fail the test.
Internet Poll (Score:2)
This means they put it on Fiver and/or Mechanical Turk.