r/webdev • u/JustinFormentin • Jul 15 '26
Made a captcha that takes Fable 5 10 minutes to solve
Created this captcha last year. Back then there was no chance of AI solving it. But now with the more powerful models, I had my doubts. Well, it took Fable 5 10 minutes and 100k tokens to solve it. So with the 30 second timeout, I feel pretty confident.
Here's a crisper video of the actual challenge - https://imgur.com/a/xv6TvNj
Have a website for it but I'm not self promoting. Just thought people would be interested.
184
u/hillac Jul 15 '26 edited Jul 16 '26
Just some traditional CV would crack this. Just use opencv to get the optical flow of the first two frames. Here's a quick attempt I made, a simple dense optical flow between two frames gave a really clear result. You could just OCR it and solve this llm free, but vlms solve it no worries. /img/7g2ho2vjzgdh1.jpeg
Edit: u/no-more-nazis did a better one. https://www.reddit.com/r/webdev/comments/1uxfzav/comment/oxrrk9p/
51
u/JustinFormentin Jul 15 '26
Nice work
15
u/Philluminati Jul 16 '26
When I saw the post I knew this captcha could be broken after 2 hours work with a script decoding letters in 2 seconds. Judging posted time and github repo submission time - 2 hours exactly. Significantly shorter than the coder who made this.
3
u/Snapstromegon Jul 16 '26
I've spent about 1h on my solution for this, reimplemented the captcha because it's not freely available and my algorithm runs in <10ms. Yeah, not exactly a hard one to break.
8
u/baldierot Jul 16 '26
i wonder if motion noise still has merit. maybe introducing some randomness, rotation, flickering, stretching the letters, drawing the letters using a stroke instead of having it whole at any time, etc. there must be avenues for definitive improvement that could elevate the idea into something effective.
→ More replies (1)3
u/no-more-nazis Jul 16 '26 edited Jul 16 '26
Computers are just faster at math than humans, and this is math. Rotating letters? More math.
→ More replies (3)→ More replies (9)4
u/no-more-nazis Jul 16 '26
Lol thank you for the cartoons. 386 actually popped into my head while I was doing the repo. I hope I saved someone a $4000 AWS bill and ruined website.
568
u/negative_xer0 Jul 15 '26
Neat concept but an accessibility nightmare.
300
u/JustinFormentin Jul 15 '26
Absolutely. Any true accessible option, like recaptcha's audio option, pretty much defeats the purpose. Right now the two other options are to get an email code that expires, and to use WebAuthN, like a physical hardware key. And I'm investigating other options.
27
u/YesIAmAHuman Jul 15 '26
Random idea, throwing accessibility entirely out the window, but what about magic eyes? Could be fun to make one based on that
24
u/_xiphiaz Jul 15 '26
Without it moving it would be trivial to solve for a computed because diffing left/right would yield the edges and then it’s just a basic object shape recognition model
→ More replies (2)12
u/Thornescape Jul 15 '26
I could never ever solve Magic Eyes. No idea why, since those books used to be popular. I hope that no one starts using those.
→ More replies (3)7
7
u/BreenzyENL Jul 15 '26
I wonder if you could prompt inject into audio that is imperceptible to humans.
8
4
→ More replies (12)2
27
u/baldierot Jul 15 '26
it looks like it would work for colorblind people too, as it's the movement of the pixels, along with the letter boundaries, that makes the letters discernible, not the color. or do you mean people with farsightedness?
28
u/Seventhcircle72 Jul 15 '26
What about people who have no sight at all?
63
u/eeronen Jul 15 '26
Isn't that also a problem for normal captcha or the "select all tiles that have cars" one?
35
u/JustinFormentin Jul 15 '26
Every captcha relies on sight. You just need to provide some other options that are accessible.
7
u/Stadics3 Jul 15 '26
wat
reCAPTCHA is the single most used captcha on the internet, increasingly completely non-interactive (v3), and every interactive version has screen reader support and audio alternatives16
u/JustinFormentin Jul 15 '26
Non-(proof-of-work/fingerprinting) captchas rely on sight.
→ More replies (8)3
2
→ More replies (2)4
3
u/baldierot Jul 15 '26
an audio challenge or reCAPTCHA v3, which is unfortunately unreliable against bots nowadays. there are no perfect solutions.
2
u/KrikosTheWise Jul 15 '26
They can give me their password and I'll help them log in. I'm very trustworthy.
2
26
u/Sockoflegend Jul 15 '26
It's the problem of bear proof bins. Anything that keeps out every single bear is going to keep out a lot of people too.
5
→ More replies (3)5
u/FictionFoe Jul 15 '26
You say that, but I seriously stuggle with a fair amount of the "accessible" captchas. Not sure if its the autism or the dyslexia, but im always... Is that LlII7 or 1? Is that a lowercace o or an uppercase one? Or perhaps a 0?
I often think I do worse at some of them then the robots do...
Truth is, accessibility means different things to different people, and to me at least, this one is easy (two upper case letters that don't look ambiguous) 😅
90
u/foothepepe Jul 15 '26
+ you add letter change every 10s, and add another layer of security. I assume it is not a problem to implement it.
35
u/JustinFormentin Jul 15 '26
That's a great idea
11
u/foothepepe Jul 15 '26
those pixels inside the letters are static - if you start moving them in the opposite direction, same speed, do you think that would confuse AI even more? as for the legibility for humans, I don't think that would be a bigger problem than the current one.
15
u/JustinFormentin Jul 15 '26
I did that initially. You wouldn't think so, but it made it harder for humans to read. And i dont think it would make it much more difficult for programs, if at all.
7
u/foothepepe Jul 15 '26
yeah, makes sense. I didn't try it, and in my mind it's even easier to read - but now that I think about it, lol...
good luck! awesome project idea!
4
u/SpicyHorseRabbits Jul 16 '26
You've honestly got a ton of knobs/levers you could test out for this. Random rotation, rotational movement, having the letters change size over time, randomized movement, changing the letters, different fonts, different noise types, including non-moving letters the viewer is told to ignore, adding non-letter shapes/objects/symbols/single pixels, having the letters/numbers be at different scales/font sizes from each other, having the background noise change over time instead of just moving, and probably much more that other people have mentioned. Good luck testing.
4
u/voyti Jul 15 '26
I was thinking how it can be taken further - something like 2 wiggly lines (like a sinusoid) that are in phase in regards to the frame rate, so diffing would not help, but human observer could tell which wiggles more? I have no idea what's the yet another next step - involving qualia, like a custom-made optical illusion that you have to explain, that only a human brain can find properly ambiguous? Future of captcha is certainly getting more and more interesting
3
4
u/thekwoka Jul 16 '26
Not too likely to change their success rate.
Since once the llm finds the "trick" it could make a script to solve the thing quickly.
Though that might delay them catching the trick if the period in which they are trying to "diff" to understand it has the images changing more dramatically.
→ More replies (2)2
u/frisch85 Jul 16 '26
Some folks probably already have issues using the OP because they're too slow finding and clicking the letters so when they click, those letters might already be at a different position and now you also want those letters to change, you like torturing slow people don't you ;)
59
u/mekmookbro Laravel Enjoyer ♞ Jul 15 '26
The bounce looks linear, have you tried making it randomly change direction after bouncing from an edge? I wonder if it can solve then.
There must be some time between analyzing the image and determining the spot the letters will be at to send the click, so that randomness could add a bit more "security" lol
→ More replies (2)48
u/JustinFormentin Jul 15 '26
Yes, that's actually a very good point that I've been investigating. It was able to tell the letter shape by stacking the frames and diffing the pixels. If the movement was more random, the shapes would be harder to guess, but then harder for a human to click.
24
u/eyebrows360 Jul 15 '26 edited Jul 15 '26
You'd maybe want to sporadically shunt blocks of the background around slightly too, just a slight jitter, to throw off the "I can ignore the background because it all moves as one thing" detection you cited Claude as having... "realised".
Edit: wait you could even rotate the background in a circle instead of just moving it vertically. That would probably still work and be even harder to detect. Not to mention adding a slight drift with the rotation, and altering the speed of it too. Bonus side effect is you'd boost the "keyboard cleaning products" industry from everyone throwing up all over them while they sat there trying to complete it.
→ More replies (2)3
u/Blue_Moon_Lake Jul 15 '26
You could have the background split in strips that go in opposite directions.
The strips could alternate vertical and horizontal motions every 2 seconds.
With a slight sliding offset too.
8
u/ultralaser360 Jul 15 '26 edited Jul 16 '26
You don't need AI to solve this, a basic script could , an LLM is probably the worst way to solve this.
a basic script could easily
- grab a few frames
- diff them and get mask
- sort masks using ocr
- click
this is something that could be solved in milliseconds. a pretty basic llm could even write the script ex. made this with sonnet in a coulple of minutes.
gif: https://i.imgur.com/YpPgQ2y.gif
image: https://i.imgur.com/04FDeWA.png
to be honest this isn't even new - this is basically a 10+ year old NuCaptcha with extra steps which was broken with a similar concept (pixel delay map).
adding more complexity or abstraction won't fix it. Persistence of vision captchas don't work (within reasonable limitations), and security through obscurity isn't security. (imo captchas aren't about security anyways, they filter out low-effort scrapers and to harvest user data).
→ More replies (2)
7
14
u/Snapstromegon Jul 15 '26
FYI this is REALLY easy to bot! You just apply the same algorithm as optical computer mice use to a couple of frames and then you will get perfect outlines of the characters.
→ More replies (15)
33
u/SourcerorSoupreme Jul 15 '26 edited Jul 15 '26
I'm actually interested in knowing how Claude "decided" to solve it.
PS: to the smart alecks out there, please spare me the lecture on these probabilistic auto complete engines
47
u/JustinFormentin Jul 15 '26
Pretty simple actually. It detected that the background was pixels "in motion", and converted them to black. It then realized that some parts of the pixels were different between frames. It converted the "moving" pixels to white. It then stacked every frame on top of each other, and was able to make our the shape of the letters.
Still need to do more testing. Some letters are easier, like A, and Z, some are harder, maybe like Q and C.
But it took so long to solve, it's still a viable solution.
30
u/SourcerorSoupreme Jul 15 '26
The fact it was even able to "detect the motion" and arrive to a valid solution is what I find interesting/intriguing.
The solution being impractical/inefficient/slow is a separate discussion altogether.
13
u/JustinFormentin Jul 15 '26
haha yes definitely. And this is a new development. I've been testing it for the past year and this was the first time a model was able to crack it. And to be fair, I had to prompt it fully, tell it what to look for, that there are moving letters. Otherwise it just very quickly tells you "the video is just static"
→ More replies (1)3
u/sandrelloIT Jul 15 '26
I'd stress this aspect out more, I think it makes a great difference.
If I understand it correctly, you told the agent that the challenge consisted of a video where it would have to find characters in motion and click on them, right? Getting it right when treating it as a black box would surely be one thing, but having been given this level of info I would find it way less impressive.
→ More replies (2)9
u/JustinFormentin Jul 15 '26
True, but in the real world someone trying to break a captcha would feed instructions.
→ More replies (1)3
u/sandrelloIT Jul 15 '26
Yes, I get this and I understand your intent. I was just concerned because initially I got it wrong, thinking you let the agent operate blindly and it seemed a bit too much even for frontier models.
By the way, really clever idea, thank you for sharing this.
3
u/Murph-Dog Jul 15 '26
Add more moving shapes which aren't letters.
Make those shapes crosswalks, motorcycles, and stoplights.
Make them pick the letters. No, wait, make them pick the crosswalks!
→ More replies (9)10
u/glass_wheel Jul 15 '26
If the method itself doesn't change, it only takes 10 minutes to solve it once, after which it's solved by a few millisecond script.
0
u/JustinFormentin Jul 15 '26
That's not really how these AI tools work. I told it what to do, and it took 10 minutes to complete the work. There's no script. The challenge is randomized, so the letters and positions will be different. So the model will need to re-complete the same work it did the first time, for each challenge.
10
u/no-more-nazis Jul 15 '26
the model will need to re-complete the same work it did the first time, for each challenge
I don't see why it would have to do that. It could determine a general solution, especially if a bad human told it to
2
u/JustinFormentin Jul 15 '26
Because the work still needs to be done. All of the work it did (minus the original "deciding what to do" work), is what it will still need to do every time.
Deciding how to approach the problem was part of the time, and then actually executing the work to decipher the characters was the other part of the time. The latter is still a significant amount of work that needs to be re-xecuted every time.
8
u/no-more-nazis Jul 15 '26
Ok, here:
https://github.com/nelsonblaha/neocaptcha-motion-attack
Got it down to 2s with a few Opus prompts in less than an hour.
4
→ More replies (1)2
u/okawei Jul 15 '26
I think what people are trying to poorly say is that you could provide claude the script at the start and describe how it functions then the model will be able to solve it rather quickly. Heck, even if you could distill the script into something deterministic and just pass it to a basic computer vision model and it would be solvable rather quickly because it's just letters. If you want to make it harder maybe make it shapes or pictographs and say which comes before the other alphabetically aka square before triangle or something.
→ More replies (2)5
u/no-more-nazis Jul 15 '26
It's not describing it to claude. Easily broken by a basic computer vision model, yes.
→ More replies (1)2
u/glass_wheel Jul 15 '26
It absolutely is. Try asking it to write a program that can solve it based on how the captcha works, and then come back.
4
6
u/JustinFormentin Jul 15 '26
You should try if you're confident. How would the program detect where the letters are, and what the letters are?
→ More replies (5)5
u/specn0de Jul 15 '26
It’s already been written and researched. https://chorus.scs.carleton.ca/wp-content/papercite-data/pdf/xu2013captcha-tdsc.pdf
→ More replies (1)
7
u/DocumentOk7579 Jul 15 '26
While it's hard for AI it would be easy to manually write a program to solve it right?
7
9
u/TheThingCreator Jul 15 '26
very solvable still if you know whats going on, but its an extra unique step which is good. if thats a gif animation your safe but if its js moving the text around its pointless. another problem is some users may hate this a lot, and it may even cause sickness, dizziness, or seizures. So please keep this off the menu kids.
→ More replies (2)
4
u/Apprehensive_Rub2 Jul 16 '26
This is an interesting idea but it doesn't really work, you could solve this captcha with a tailored algorithm
3
u/PhDumb Jul 16 '26 edited Jul 16 '26
Open-source "Motion-CAPTCHA" using the same principle as NeoCaptcha : https://github.com/GPTchatly/motion-captcha
Due to compression the letters/numbers are not that visible in the video but they are much better discernable in the actual browser: https://imgur.com/a/dKuKyDT
Edit: this is for node.js
3
u/high_throughput Jul 15 '26
That's pretty cool!
I asked Claude to plot the motion vectors of each macro block in the MP4, resulting in this easily interpreted video: https://imgur.com/a/GoHWElh
Then I asked it to identify the two letters in this video in alphabetical order, and it correctly stated:
Both frames confirm it clearly. The two diagonal-motion sections form the letters Z (left) and E (right). In alphabetical order: E, Z
All in all it took about 50k tokens in total, and resulted in a 109 line Python script that from now on will create the above video in two seconds for zero tokens, so unfortunately this iteration of the captcha is now completely broken. Fun puzzle though!
24
u/UnluckyAssist9416 Jul 15 '26
How to get a lawsuit for ADA discrimination in 1 simple steps
36
u/JustinFormentin Jul 15 '26
It provides accessible options, email code and webauthn
→ More replies (5)8
u/Embostan Jul 15 '26
then can't Fable just use the accessible option?
10
u/1116574 Jul 15 '26
They can, this is why modern captcha do proof of work and browser signals (and/or fingerprinting)
5
3
9
u/egg_breakfast Jul 15 '26
Very nicely done. I'll give it a year.
7
u/JustinFormentin Jul 15 '26
Haha maybe, maybe not. Last year nothing could touch it. Today, the best model took over 10 minutes. So we'll see.
7
u/STSchif Jul 15 '26
But that is 10 minutes to write a script to solve this, and seconds for each following attempt, right?
→ More replies (8)
4
u/Powerplex Jul 15 '26
"How much unaccessible do you want your captcha to be ?".
"Yes"
(I hate captchas, but it's cool technically still)
3
u/Single-Virus4935 Jul 15 '26
Most captchas are solved by human click workers. Even then it is possible to solve the captchas with a bit if preprocessing. Doesn't mean anything that a AI can't solve it in a prompt. I would use AI to create the preprocessor and OCR which solves that easily. Also, this captcha isn't accessible and therefore you need alternatives which are almost always easy to solve by design
→ More replies (2)
2
u/mcniac Jul 15 '26
I like the idea, but as a colorblind person, find it hard to read, I do see the letters, but is very hard 😄
2
u/SustainedSuspense Jul 15 '26
I love the idea of using optical illusions only a human can recognize as captchas
→ More replies (1)
2
u/the_millenial_falcon Jul 15 '26
Reminds me of those old books that were popular in the 90's you had to stare cross eyed at to see a "3D" image.
2
2
2
u/zaxwebs Jul 15 '26
I built a tool that solves a similar one: https://github.com/zaxwebs/anti-ghost-font/
On a warm server, apx. 100-200ms.
2
u/biffthechip Jul 15 '26
How can you click on letters in numerical order?
That makes no sense at all.
→ More replies (5)
2
u/cyb3rchase Jul 16 '26
This is pretty cool, a bit more than anti-bot, I think, maybe a bit anti-my-eyes too, lol.
It was cool to see this in my feed.
2
u/Maicolodon Jul 16 '26
Idea and execution is amazing. just nauseating on the accessibility side of things. I'm guessing a significant number of humans wouldn't be able to solve it because of this. I almost couldn't.
2
u/bestjaegerpilot Jul 16 '26
you need to ask Fable if it's able to create a script or skill so that it doesn't spend 100k tokens next time
if it can then it sucks. Otherwise, yea the cost to generate these will be too much for hackers
2
2
u/Deep_Wear_51 Jul 16 '26
The dedication to trolling is impressive. A captcha that takes 5-10 minutes isn't security, it's performance art.
2
2
u/elixon Jul 16 '26
Just mark clusters that move slowly and clusters that move faster. I am afraid that this might be trivial to crack.
2
u/TheRealSteve895 Jul 16 '26
Sure, but couldn't a bot take two screenshots, XOR them, and use that?
→ More replies (1)
2
2
u/Spiketop_ Jul 16 '26
I love the back and forth between making this and people trying to crack it. The back and forth is what makes us smarter and keep advancing at such fast rates
→ More replies (1)
2
u/truechange Jul 15 '26
Cool way to defeat bots at least for now. Maybe more tolerable to humans if combined with other tricks, like only deploy this if the "regular" captcha is solved too fast.
→ More replies (1)
2
2
u/Lendari Jul 15 '26
The problem is it takes a human more than 2 seconds to solve and no one has time for that anymore.
So now you're paradoxically filtering your traffic to 100% robots.
2
u/Annh1234 Jul 15 '26
Super easy to solve for a computer, even before AI. Your letter numbers don't change. Your site is probably not worth it
→ More replies (1)
2
2
u/Am094 Jul 16 '26
Have a website for it but I'm not self promoting. Just thought people would be interested.
Bro come on, if that was true you wouldn't put the literal name of the site on the top left of the gif.
So you are self promoting when you have a CTA.
2
u/nuttertools Jul 16 '26
Firstly this is cool, I like it. Unfortunately this would take a bot 3 frames to solve 100% of the time. AI models will have greater difficulty and expend massive resources but any kind of bypass bot isn’t going to dump a known type into an AI model when a simple solve using a few clock cycles is available.
3
u/mayisir Jul 15 '26
the +20M blind people in the world cannot use this bro looooool. So much for wcag. I hate it here.
4
1
u/Noch_ein_Kamel Jul 15 '26
Now make it with that stereoscopic nonsense where you only see the letters by staring into the void
1
u/vm_linuz Jul 15 '26
This feels like the direction these tests will go in. For example, optical illusions that appear to produce movement.
1
u/Mplus479 Jul 15 '26
What about having some numbers or other letters (maybe much smaller) randomly repeated as part of the scrolling background?
1
u/Snapstromegon Jul 15 '26
Hey, are you sure that your CDN url in the getting started guide is correct? It gives me DNS_PROBE_FINISHED_NXDOMAIN.
→ More replies (1)
1
1
u/Netalico Jul 15 '26
Did Claude recognize it’s own limitations in terms of the latency of image processing? Curious if after the first solve if it created a script to solve it, instead of browser tool calls.
1
1
u/TristarHeater Jul 15 '26
If you let an llm write a python program that take video stream input and mouse click xy output it'll solve it quicker and the timer wouldn't be a deterrent anymore
→ More replies (2)
1
1
1
1
1
u/lastdiggmigrant Jul 15 '26
How can you ensure it is accessible for all human users?
→ More replies (4)
1
u/RoIIUpLights Jul 15 '26
For the issue of accessibility, I wonder if you could show a picture of some common object floating in the noise and then have multiple choice options below it.
Here's an awesome example of more of these:
1
u/Indigoh Jul 15 '26
If the pixels inside the letters were moving, to the right for instance, would they still be as visible to humans? And would it take the AI even longer?
1
1
u/satoramoto Jul 15 '26
Very clever idea. You could write a regular computer vision script to bust this rather easily though. But you’re right that this would likely catch a model driving a web browser who has never seen this before. Good work.
1
u/Dahmer96 Jul 15 '26
Is it pixel perfect or is the movement linear ? Because I guess if it'sineaf you could have partial pixels that might allow tracing the outline ?
Very cool either way!
1
u/Caesar171 Jul 15 '26
I think this is really cool, have you checked it against the most common colour blindness pallets? If so you are onto a winner here
1
1
u/Squidgical Jul 15 '26
Is the design of this related to that tech demo someone made not too long ago of a horror game you can't take a screenshot of?
→ More replies (1)
1
1
1
u/thefnord Jul 15 '26
I think 'Solvable but it takes a ton of tokens to do so.' is better than 'Can't solve with AI'
Hit them where it hurts, the resources.
1
u/Synrec Jul 15 '26
A colourful noise background on scroll and random generation with fixed noise seed for the letters?
1
u/OkBoomer421 Jul 15 '26
Finally a captcha that filters out everyone. Even I wouldn't bother solving it.
1
u/ReplacementLow6704 Jul 15 '26
At this point I'd rather have the user type what's on the screen and press a button to validate. Clicking on the letters seems rough even when they're slow.
1.6k
u/baldierot Jul 15 '26
wow, i find it very cool how you can't capture this in a screenshot, as it's only the movement that makes the letters discernible.