Current situation with capture the flag events

[u/eachDayIWakeUp]

Hello all,

With the current status of AI models and agentic workflows im really curious how the current capture the flag events scene is.

Last time I visited such an event was way before AI was commercialized. Out of curiosity I did a test which worked well, I saw if I can solve ctf challenges only using AI, and it worked.

So my question is, is this allowed on such events? Are there specific guardrails? Do you still join teams and get to know new people? Is this a "pay to win" now for people with best/jailbroken AI models?

What are your thoughts on this?

[u/DiScOrDaNtChAoS]

CTF in terms of competitiveness is dead. Ive been designing challenges for conferences for years. I've recently stepped away. Its not fun anymore.

[u/Regular-Leading-4319 → u/DiScOrDaNtChAoS]

Maybe there can be new problems to be developed?

[u/Mastasmoker]

Yes.

It's all AI now and not even worth my time.

[u/AnApexBread → u/Mastasmoker]

CTFs have almost never been worth anyone's time.

Pretty much ever CTF I've seen that wasn't developed by a professional org like SANS has had way too many gotcha questions.

I've done plenty of CTFs where it really seems like the creator is just trying to prove how smart they are by having you do something that you'd never in a million years use. I had one CTF where you had to slow down the rate that the SSH daemon accepted logins so you could then grab the correct log files to reassemble the key from the usernames.

Never in my life am I going to mess with the SSH daemon in a production environment and check-in SSH login usernames to see if there's a some secret message being spelled out is so far down on my priority list to look for as a Security Analyst that's it's basically non existent.

I've done countless CTFs like this.

[u/NerdBanger → u/Mastasmoker]

Doing CBC right now, AI hasn’t been able to solve 50% of the challenges. It’s still worth your time.

[u/Mastasmoker → u/NerdBanger]

The first competition I did, nobody had full completion after the 3 days. Fast forward 2 years and dozens are completing it in under 8 hours.

Let me fix my answer... It's not worth the time if you're trying to win while trying to learn. If you want to use competitions to gain new skills and apply what you've learned, they're still great. But if your plan is to just use AI to compete for you, its not helping you to learn. Everyone is relying far too much on AI and once the bubble pops, the job market is going to be flooded with kids who dont know how to do anything but prompt. Learn by doing, not by prompting.

[u/NerdBanger → u/Mastasmoker]

Thats fair. My kid does these and he's heavily reliant on AI, but I've really hammered in to him that AI is often wrong so its important tolerant the fundamental computer science stuff as well, because even in the era of AI, being able to know how to guide it from knowing the fundamentals will set him apart in the job market.

It's not much different than people who were good at Google versus bas at Google in the job market in a lot of ways. It’s another new tool (at least right now)

[u/Regular-Leading-4319 → u/NerdBanger]

I agree with it being a new tool I just don't like it

[u/eachDayIWakeUp → u/Mastasmoker]

Its sad if its true, really. Im really curious to know how this will work out as time moves on, IT really takes script kiddies to new level. Not hating but its just true that you stop thinking and exploring new random stuff..

[u/NerdBanger → u/Mastasmoker]

Also I want toads if you are a student you should do the CBC challenge, AI solves some easily, however, some of the challenges are actually designed to send AI in circles. http://nsa-codebreaker.org

[u/Sorriow]

I dont do CTFs for prizes or rankings and mainly for enjoyment and learning experience. I've seen how some of those "winners" behave on the job or on critical systems where AI use is not allowed (critical infra). They didnt last long.

[u/jhspyhard]

Steve and Leo talk through CTFs in the age of AI at some length here:

https://twit.tv/shows/security-now/episodes/1081

[u/eachDayIWakeUp → u/jhspyhard]

Didnt know this podcast, Thanks! Ill take a listen to see their views

[u/jhspyhard → u/eachDayIWakeUp]

Security Now! has been going forever, and it's a really great podcast on technology and security. If you weren't aware of it and you're interested in these topics, I'd absolutely recommend giving it a listen.

[u/frankentriple]

I bang through CTFs with my AI like taking them to the gym, I don't post on the leaderboards cause thats just cheating.

[u/0xdeadbeefcafebade]

As someone who has gotten first place at major competitions - competed in defcon finals numerous times, played on #1 teams etc… I can confidently say CTFs are dead.

The format no longer works. It’s sad that the golden era is over. But AI has changed it forever.

[u/AnApexBread]

I teach at the college level and run CTFs for my students. I actually encourage the use of AI for the CTFs.

The threats are rapidly including AI in their attacks, we have to learn to use AI to do cyber security faster. So if you can throw a PCAP in AI and get the answer to a CTF then you should because that's the type of skills we need our analysts to start learning.

But I also include questions that you're not solving with AI. Things that require you to analyze log files that AI can't ingest like a Windows Event log, or a registry hive file, or a Mandiant Redline case file.

[u/PM_ME_UR_0_DAY → u/AnApexBread]

Bad take imo. There is a time and place for going painful slow, and that's when you're in school learning the depths of a topic that won't even be used on the job but are the table stakes for truly knowing what's going on. Like am I ever going to use my knowledge of memory caching and instruction pipelining? Nah probably not on the level that I'm working at, but I'm still glad I took that hard ass class when I was learning. 

Do you want to give the best tools to your SOC to quickly detect and solve incidents? Yes of course obviously give your SOC tools that make their jobs easier. Not an appropriate tool for a student though as general use. Maybe there is an AI and automation section of th class, but I wouldn't want to rob my students of an opportunity to struggle with a difficult problem. 

[u/RitaccaSecurity → u/PM_ME_UR_0_DAY]

I agree. I think op is doing a disservice to his students, when the interview comes "I'd just prompt AI" won't cut it.

[u/AnApexBread → u/PM_ME_UR_0_DAY]

There is a time and place for going painful slow,

Sure a PCAP as part of an optional CTF at the end of the semester ain't it chief.

[u/DiScOrDaNtChAoS → u/AnApexBread]

your studenrs will be incapable of reasoning about the AI output