24

The CAPTCHA website says:

...humans can read distorted text as the one shown below, but current computer programs can't...

I realize this technique is widely used, and that this could indicate that they are indeed effective, however visual analysis is an extremely advanced field, and so when recently filling one of these challenges out, I became curious.

  • Anecdotally, at work our manufacturing plants have cameras/software that can scan items going by at thousands of parts per minute and correct for printing or packaging offsets down to extremely small discrepancies. These things are really accurate, and do what they do at ridiculous speeds.

  • There are sites like WhatTheFont where you can upload a picture of sample text, and some sort of algorithm will try and match what font it used.

  • The Zbar software can take an image file of a QR Code and output the decoded information. I used this just last week and was surprised that the command line took an image file (not an image as seen through a camera) as the input.

  • The iNote pen manufacturer claim to be able to convert handwriting to text as you write.

So... can no image recognition software, analysis algorithm, etc. actually detect, read and/or interpret a CAPTCHA?


EDIT: In response to @Christian, I guess the type/variant of CAPTCHA needs to be specified. I wasn't aware of this. Perhaps using the reCAPTCHA project would suffice?

Another way of approaching this would be to rephrase like so: "Are there any known instances of some type of system being able to interpret the primary CAPTCHAs mentioned on the official site?":

A free, secure and accessible CAPTCHA implementation is available from the reCAPTCHA project. Easy to install plugins and controls are available for WordPress, MediaWiki, PHP, ASP.NET, Perl, Python, Java, and many other environments.

7
  • +1 always asked myself if spammers actually develop letter recognizing algorithms just to be able to mass register on sites. These guys surely have too much time. Spamming on boards/forums doesnt seem to be big problem still. Afaik some handys offer via their camera objective recognizing of letters on e.g. real walls for instant translation with apps...*augmented reality* Commented Jul 9, 2011 at 21:28
  • 1
    Without speaking about a specific CAPTCHA software the question is meaningless. Different CAPTCHA have different difficulty for computers.
    – Christian
    Commented Jul 9, 2011 at 22:07
  • @Werner - handy ... oh, these crazy german pseudo anglicisms ;)
    – Oliver_C
    Commented Jul 9, 2011 at 22:40
  • reCAPTCHA used to be excellent, but a handful of months ago, I started getting multiple bots per day sign up on a few sites I used reCAPTCHA for, so I had to find another solution. So far, that one's working, but it's only a matter of time before hackers create bots that successfully answer its challenge, too. Commented Jul 10, 2011 at 23:30
  • It is comparable as locking your home door...It won't prevent thief from entering, but it help... an if your neighbor don't use lock, it is more probable that the thief decide to go there instead.
    – Zonata
    Commented Sep 20, 2012 at 2:11

1 Answer 1

29

The short answer is yes, computers can read CAPTCHAs.

But, as mentioned on the Wikipedia article, in most papers/articles discussing this issue, and in the comments above, it depends very much on the CAPTCHA and the algorithm used - some are easy to decode, and other are very hard ( no solutions exist so far ). Font detection is useful for simple CAPTCHAs, but characters can be deformed ( or the image altered ) in ways that render font irrelevant.

reCAPTCHA is discussed in this stackoverflow question, the accepted answer providing a link to site that claims to decode reCAPTCHAs - but it is offline now. The second answer is more informative, mentioning the manipulation of the third annual TIME 100 poll by the infamous Anonymous. Here's a full description of the operation; the salient points being that The Times employed reCAPTCHA after realizing a subversion attempt is under way, which forced Anonymous to vote in an almost brute-force manner:

Update – Just to be perfectly clear, anon didn’t hack reCAPTCHA. It did exactly what it was supposed to do. It shut down the auto voters instantly and effectively. The only option left after Time added reCAPTCHA to the poll was a brute force attack. Ben Maurer, (chief engineer on reCAPTCHA) comments on the hack: “reCAPTCHA put up a hard to break barrier that forced the attackers to spend hundreds of hours to obtain a relatively small number of votes. reCAPTCHA prevented numerous would-be attackers from engaging in an attack. In any high-profile system, it’s important to implement reCAPTCHA as part of a larger defense-in-depth strategy”.
As Dr. von Ahn points out “had Time used reCAPTCHA from the beginning, this would have never happened — anon submitted tens of millions of votes before Time added reCAPTCHA, but they were only able to submit ~200k afterwards. And to do this, they had to resort to typing the CAPTCHAs by hand!” One thing that Time inc. did that made it much easier for the anonymous hack was to allow leave the door open for cross-site request forgeries which allowed anon to create a streamlined poll that never had to fetch data from Time.com.

They did, however, manage to significantly optimize the process, but that is outside the scope of this question.

Another interesting aspect is mentioned in another answer:

Adaptive Security

reCAPTCHA is a Web service. That means that all the images are generated and graded by our servers. In addition to the convenience that this provides (you don't have to run costly image generation scripts on your own servers), this also provides an extra level of protection: our CAPTCHAs can be automatically updated whenever a security vulnerability is found. For example, if somebody writes a program that can read our distorted images, we can add more distortions in very little time, and without Web masters having to change anything on their side. This is significantly more secure (and convenient) than having to re-install a CAPTCHA every time a vulnerability is found.

This is probably true of most CAPTCHA web services - it is much easier to change the CAPTCHA generating algorithms, then it is for hackers to crack them - other methods would yield better results, such as crowdsourcing to China, or trying to hack the audio CAPTCHA, where available. But a good CAPTCHA has to be readable by humans - which limits the amount of obfuscation that can be employed; a researcher seems to have been able to break reCAPTCHA in 2010, with a success rate of 30%; here's an article, and a video; however, reCAPTCHA seems to have fixed the implementation since.

reCAPTCHA has also allegedly been cracked in February 2011, but no evidence was provided to back the claim.

It should be noted, however, that some popular web sites still use 'broken' CAPTCHAS; for example, PayPals CAPTCHAs can be read by PWNtcha, a program that hasn't been updated for three years.

5
  • 1
    Wow -- great digging. Fascinating read, and the most surprising thing I read was that some places use known-to-be broken implementations. I actually nearly asked that question in my edit, but figured it might be unanswerable (proving a negative). In the end, it got answered anyway! Thanks.
    – Hendy
    Commented Jul 10, 2011 at 3:52
  • February 2011 sounds roughly about right for when my reCAPTCHA-enabled sites started getting lots of bot accounts. It might've been March. I've since switched to another CAPTCHA tool, so I don't know if reCAPTCHA has changed its algorithm enough to defeat the bots that broke through it. Commented Jul 10, 2011 at 23:32
  • @Erik - many similar reports are found in the comment stream of this allspammedup article, as well as this reCAPTCHA Google Groups thread; however, I'm not sure that is enough proof that it's OCR has been broken. It might have been the audio component, or sweatshops. I also assumed Google would acknowledge the cracking if it were true, since many people rely on it for keeping their sites spam-free. Commented Jul 11, 2011 at 9:53
  • MSO has a post about recaptcha getting harder, likely in response to detection getting better.
    – Ryathal
    Commented Sep 19, 2012 at 20:12
  • Personally, I like XKCD's idea for CAPTCHA. xkcd.com/810
    – JAB
    Commented Nov 11, 2016 at 21:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .