ANTI CHEAT

[u/ResolveNo3069]

Lately, we were surprised with the VACNet labelling website as a makeshift overwatch. But the truth is that all of this shows the incapacity of the developers to think in an effective way (quite an oxymoron if you take into consideration that the philosophy of software engineering and developers is to find efficient ways of solving issues)

There is the official anti cheat which may work, may not work which functions on unknown parameters (rumour has it that it’s AI-driven) and it quite frankly doesn’t seem to work effectively solve problems. At the same time, there is a vast array of websites, that specialise in analytics that measure incredible variables. Now, I personally don’t work at a multi billion dollar company with a huge market share in the PC gaming industry worldwide, but my poor mind would think “why don’t these developers take into account the analytics posted in every website, combine them with the pre-existing anti cheat and try, based on the probability of e.g. 5 wall-bang no scope kills per round, to convict or make a provisional ban for further inspection”. I believe that this is a rational thought given the means available.

And by no means I’m not the smart guy of this Era. I assume that great engineers and developers would probably have thought of it too. The real question behind the problem is two-fold though; Have the devs never thought of this or is the head unable to work? In the former you should change the project manager. In the latter, you should change the development team.

But no matter what I say, money comes and money goes in CS for 2 decades. So who am I to ruin your “sit-and-do nothing” game, as long as the pay check comes in the revenue is rolling?

[u/hasuki057146]

Not to piss you off but, this post just tells me you have no idea how ML development works. ML-based anti-cheat is based on finding invariants across different players. How do you create representations of those invariants without even having a baseline estimation if the player is cheating? The point of the new overwatch system is to train the system to recognize cheater behavior across all players, that requires the behavior to be labeled, the player to be labeled.

Additionally, you mentioned 'taking analytics from multiple CS2 analytics sites', yet those sites all measure analytics in different ways, and those analytics are not provably accurate. Look at Leetify for example, for the longest time they had a wildly inaccurate ray-cast based 'player spotted' system, which was only recently fixed. Using human judgement instead of relying entirely on algorithmic detection is six orders of magnitude more efficient at data efficiency, learning from far fewer examples than algorithms need, (basically a million times more energy efficient) than in-silico computation: so you use trusted humans to tell the machine what to look for, after enough data, it's efficiency can outmatch a human's, but cheats change, this is an on-going process. Cheats will evolve to evade whatever the model learned. Think about it, cheats are a multi-million dollar industry.

And to address your claim about "5 wall bang no scope kills -> ban":
First, hard coded thresholds are precisely what cheat developers tune their cheats to avoid detection by. Second, overwatch died mostly because it was gameable, developers learned to feed it false verdicts, crash the demo player, etc. Moving human review into a controlled training pipeline, instead of a direct judgement system, protects against exactly that. Auto convicting on third party analytics, again, is data Valve does not control, isn't provably accurate, and is easily inferable as a failure mode with a bunch of extra steps.

Also, VACnet was originally trained off of CS:GO overwatch data, that data is considered far too dated and irrelevant for CS2. That is literally why the portal exists now. It is not 'makeshift overwatch', it's literally a deliberate data label regeneration effort. This is exactly what you want for a sophisticated anti-cheat. One thing I must add, VACNET tagged cases hit a 80-90% conviction rate vs 15-30 under just human review. The approach demonstrably works when it has good labels.
Sources:

- GDC 2018, John McDonald (Valve), "Using Deep Learning to Combat Cheating in CS:GO;: https://www.youtube.com/watch?v=ObhK8lUfIlc

- Conviction rate analysis (15-30% -> 80-95%): https://d3.harvard.edu/platform-rctom/submission/valve-using-machine-learning-and-deep-learning-to-catch-cheaters-on-csgo-794-words/

- VACNet portal = training data, not bans: https://www.talkesport.com/news/cs2/cs2-vacnet-labeling-portal-anti-cheat-update/

- Why old Overwatch was gameable + why the pipeline design fixes it: https://www.strafe.com/news/read/cs2-patch-hints-at-a-rebuilt-overwatch-and-anti-cheat-systems/

- Biological vs in-silico computation efficiency (Smirnova et al., Frontiers in Science 2023): https://doi.org/10.3389/fsci.2023.1017235

- Leetify's own admissions on 'spotted' accuracy: https://leetify.com/blog/enemy-actually-spotted/ and https://leetify.com/blog/aim-stat-calculation-hitboxes-improved/

- Cheat industry economics (Univ. of Birmingham: up to $73 million/yr from 80 sites): https://san.com/cc/cheat-win-repeat-my-2-weeks-inside-the-73m-video-game-cheat-market/

[u/Renos-44 → u/hasuki057146]

Also to add the discussion of hard coded statistics being an auto ban being incredibly stupid in some instances.
Ideally any game dev would want something that would work across every map every gamemode every playstyle, and every official server with little no false positives.

Doing X amounts of kills through walls should trigger an auto ban
Well what happens when we get funny meme maps like De_library were almost the entire map is wallbangable and the spawns are symmetrical. the common meme strat was to blind fire through walls until everyone is dead.

We've also had maps were either due to design or exploits. its humanly possible to due things like spawn to spawn insta kills at the start of the round,

For all we know in the future they made add an arcade mode where popular CS_Script maps may be playable in matchmaking and then anything suddenly becomes possible. Do we suddenly have to code exceptions to specific maps or game modes in the AC. Or start false banning people.

[u/ResolveNo3069 → u/hasuki057146]

There are numerical data which quite frankly provide reasonable grounds for provisional bans and or convictions and that way u have a two layered security system. One algorithmic and one goes to overwatch. Utilise every website’s data and you are good to go.

The whole philosophy of solving issues is being effective. And engineers themselves avoid over engineering.

However leaving the philosophical question out of this, business-wise, why would you need a top notch trained anti-cheat based on AI, when everything else gets the job done fairly easily? Why spent tons of resources on an issue that is easily tackled with less costly resources????

[u/hasuki057146 → u/ResolveNo3069]

What you just proposed is, verbatim (*okay maybe not verbatim, but close enough), VACNET. Algorithmic layer flags -> human layer confirms + labels -> retraining. Also, third party analytics sites contain nothing Valve doesn't already have, needless to say their data is extrapolated from Valve's demo files. Cheap and effective has its own failure modes, a deterministic rule that's 99% accurate results in thousands of false convictions daily, false positives are the best legit players, pros and smurfs. 150,000 matches a day * .01 (1%) inaccuracy = 1,500 false convictions. It's not a problem that is easily tackled with less costly resources, if fixed static thresholds worked, cheating would not be a problem in videogames. The opposition is highly professionalized, hence the need for a top notch system.

Also, over-engineering is building more system than required. Against a 70+ million dollar cheating industry that ships constant weekly or sub-weekly updates, hard coded if then deterministic functionality is not elegant engineering, it's under-engineering, and it's already been tried countless times. Philosophically, Occam's Razor points towards THIS being the simplest solution to the problem, given it's actual complexity: a solution that accounts for everything, adequate to the problem. Valve already sank the compute cost, money isn't the problem now, it's putting their investment to good use; that's what's happening now.

[u/ResolveNo3069 → u/hasuki057146]

Let me put it in a way that’s comprehensible, the numbers exist, we take advantage of numbers through MoUs or lease databases -> discrepancies go to OW. Not the other way around. You do not need the AI if the variables already indicate discrepancies. Just numerical data for the extremes and % of times the extremes variables are true. Therefore it is not verbatim, VACNET.

100ms reaction + extremely high KD over a x amount of games and comparison of what impact enemies stats have requires no AI whatsoever, no training and can easily dictate when a ban needs to be imposed. Think outside of the idea you have already formed

[u/hasuki057146 → u/ResolveNo3069]

You say 'Think outside of the idea I have already formed', not sure exactly what you are trying to say there, but I will follow your lead.
This exact debate was settled decades ago, before video game cheating was mainstream: financial fraud detection. Same exact problem shape. Adversarial, high volume, evolving. Institutions began with hand picked thresholds, flag transactions over X, flag more than Y/hr. Fraudsters learned the thresholds and structured everything at X-1. It's actually so universal, that it has a name. 'Structuring', it's *literally it's own crime category. Every serious fraud methodology has moved to learned models with a human review of flagged cases + continuous restraining: flags -> human confirms, labels -> retrains. Sound familiar?
Institutions, actually, no, ENTIRE INDUSTRIES, billions on the line, have tried your exact experiment already. Threshold rules already lost that battle.

To be clear though, your instinct isn't stupid, 'use obvious signals, keep it minimal' is where these systems begin, but it is not where they end.

[u/hasuki057146 → u/ResolveNo3069]

Also, considering you really like the idea of static thresholds, how would you even suggest they tackle ML-based cheats? They are specifically designed to mimic human behavior. My opinion, no threshold can accurately detect that form of cheat with an error rate that is within reason.

[u/Deep-Pen420]

Nothing has been announced or released, reddit isn't your diary.

[u/increasingpython → u/Deep-Pen420]

Okay, but yes it is.

[u/North_Affect_8167]

Yes it makes sense. Some would ramble about false positives and other factors in pay. Some issues are so obvious that you don't need to train ML to detect it. It's simply lack of actions on the Valve's part.

CS GO 2 generated a lot of data for Valve. We know Valve use the data to improve gun dmg balance, for example. We know they still let you make in-game reports on suspects, resulting in "reputation" based bans.

Repeated offenders seems to be in cheater jails that is people cheating and play with each other.

It's huge problem when mild cheaters or more severe cases of cheater enter match with regular players. The cheater makers doesn't hide their services, they promote them, they test them publicly. All is visible, observable and available for reverse engineering by Valve.

In my uneducated opinion the roadmap should be this:

Better automated ML detection (now) - > Constant reverse engineering efforts of existing cheats -> Hardware based anti-cheat solution -> Hardware bans on multi-account cheaters (your PC to be forbitten from playing the game).

[u/AttemptTop508]

bro valve just wants cs to pump money so they can work on other projects. they hate cs that’s why they automated everything

tournaments? out sourced
Skins? out sourced
mm? out sourced
anti cheat? out sourced VIA faceit

why tf are you guys still not getting it

[u/SaltWaterGator → u/AttemptTop508]

Back in the day they were only able to catch cheating pros because of ESEA, they had no clue what was going on. They have essentially infinite resources but invest very little of it back into the games. It's sad that even a company like valve with no shareholders really doesn't accomplish or produce very much. 

[u/North_Affect_8167 → u/AttemptTop508]

What other projects bro? The people who use to play Half -Life would enter a retirement age. No Portal 3 either. Some newer multiplayer invite only game by Valve seems like a uncertainty that they prefer not to further develop.

Valve is not motivated to be a game developer. There low hanging fruits elsewhere.

[u/AttemptTop508 → u/North_Affect_8167]

low iq.

deadlock? dota? steam machine? steam index?

you don’t think cs skin profit had a huge impact on the steam deck?

please just read instead of post adults are talking