Sponsored: 22 Do's and don'ts when fighting cheating in online games
This article is a very brief summary on the topic of cheating based on materials from an upcoming book “Development and Deployment of Multiplayer Online Games”. BTW, Vol. 1 of this book is currently on Kickstarter (what a coincidence ;-)), so you have a chance to get it at Kickstarter-discounted prices (KICKSTARTER SPECIAL PRICE: €15~=$17 for e-book and €25~=$28 for paperback).
It is next to impossible to find a successful multiplayer online game (one that goes beyond only playing with friends) that is free from cheaters. In other words, if your open-to-public MOG doesn’t have cheaters, then either it is not popular enough yet, or your cheater detection is not good enough . For the rest of you who do have to deal with cheaters – take a look at the following list of DO’s and DON’Ts (and for more discussion on the subject of cheating– make sure to get all three volumes of the book as it comes out).
#1. DO include your right to ban cheaters into your ToC
Not only you need to have this right to ban in your ToC, but also, as a rule of thumb, you should also say that all such decisions are “at your own discretion”. However, there are complications on the way:
- Disclaimer: I am not a lawyer and nothing in this article should be understood as legal advice. Ask your lawyer instead.
- This is a complicated legal issue that may easily depend on jurisdiction. DON’T make any judgments on this issue yourself, ask your lawyer (preferably from your respective jurisdiction) whether your ToC will be accepted as a valid contract in a court of law. FWIW: on GDC2016 I spoke with a guy from a Really Big Company and their main concern was about having legal problems with banning cheaters they identified.
- The idea here is as said on the tin — to be able to ban your players if they cheat. However, you should phrase your ToC in a way that makes it clear that you do not need to prove cheating in court of law, otherwise it becomes too costly to pursue the cheaters. The key legal question here is more or less about “burden of proof” — whether it is you who must prove that they cheated, or the accused cheaters who need to prove that they didn’t cheat. As you can guess, the latter is much better for you and much worse for cheaters. As mentioned above, one potential way to address this issue, is to have that “such decisions are at our sole discretion” clause, but be careful to check the validity of this clause depending on applicable jurisdiction.
- If possible, it is best to be able to ban without refund to create a disincentive to cheat, if not, even having a ban with refund is MUCH better than not being able to ban cheaters at all.
- DON’T worry too much about your players complaining about these kind of statements within your ToC. Most of your players won’t read the ToC anyway, and when comparing ToCs across the board, they all look pretty much the same to the general public. DON’T try to abuse this statement though — it is this which will get you into trouble really quickly.
#2. If you’re using a third-party protection engine, DO add as much protection as you can on top of it (or even better, interleaved with it)
While third-party folks-developing-protection-engines often DO know what they’re doing, in the realm of security-by-obscurity cracking these third-party protection engines becomes a target of the whole hacking community. As a result, adding something on top of them improves your position significantly.
#3. DO fight cheaters from the very beginning
As soon as you’ve got a community of cheaters who make their living from selling cheats to your game — any action of yours against them will face MUCH more resistance. In other words —
DON’T feed the cheater.
#4. DO use an authoritative server
I Really Really mean it. Moreover, DO move all the logic you can to the server. Ideally, there should be zero decisions made on the client. While this is not always 100% possible (mostly because the game is really fast-paced) DO fight vigilantly against each and every client-side decision. The more processing you have on the client, the more the attacker has to exploit. If you’re not really aggressive with pushing absolutely-everything-possible to the server side, this tends to become a Really Big Problem (I’ve seen it myself, and it has also been a subject of one of the talks at GDC 2016).
While we’re at it - DON’T hold your breath for deterministic-lockstep architectures. While deterministic-lockstep games don’t suffer too much from client-made decisions, they’re inherently vulnerable to Information Leak attacks (such as Wallhacks and Maphacks. See below on restricting your client to “need to know” information only).
#5. DO encrypt your traffic
Encrypting your traffic will help fend off quite a few attacks, including proxy-based attacks (which are next to impossible to deal with otherwise), and certain classes of bots.
Some notes in this regard:
- No, using UDP is not a valid excuse to not encrypt your game traffic. DTLS is your friend in this regard and there are some other protocols as well.
- DO link your OpenSSL library statically. OpenSSL.DLL (or whatever other DLL implementing encryption) is a surefire way to give your attacker an easy path around your encryption.
- DON’T use ADH (Anonymous Diffie-Hellman). Yes, it is written in each and every book out there, but there are still games that are trying to use it (at least one game has even mentioned ADH in their About box!).
- DO check the certificate on your client side against pre-stored root (yes, there are people out there who are not doing this).
- DON’T use root certificate from your player’s machine — run your own CA instead and embed root certificate into your client (otherwise MITM-against-himself attacks become much more viable).
- Obfuscate your root certificate within your client executable, otherwise it is another invitation for MITM-against-himself class of attacks.
#6. DO monitor publicly and commercially available cheats
Find them, get them to your lab, analyze them, and release fixes for them ASAP. For this, you will likely need a separate team as soon as you’re successful enough.
A few things to keep in mind when analyzing cheats:
Obviously, popular cheat engines should be at the very top of your list.
- Even more obviously, DON’T run cheats within your own network (this would make every-cheater’s dream come true). Create a heavily firewalled “sandbox” instead (ideally – in a different physical location with no VPNs between your internal network and “sandbox”)
#7. DON’T hire known cheaters
Well, actually there are some exceptions to this rule. First of all, it doesn’t apply to white-hat hackers. Second, it MIGHT be ok to hire known cheater(s), provided that ALL of the following stands:
- They do NOT have any special access to the internals of your program (in particular, source code should be absolutely off-limits)
- They perform ONLY a “black-box hacking” (which they can do anyway). The only difference from them doing it anyway is that they will report results of their cheating to you instead of abusing them.
- All the communications with hired cheaters MUST be via a separate e-mail account (completely isolated from the rest of your system).
- All the information from this separate email account MUST be accessible only from within that “sandbox” you’ve set up to analyze cheating programs
#8. DO think about your banning policies
Are you going to ban cheaters forever, or just for a few days? What about repeat offenders (keeping in mind that on GDC2016 it was reported that 72% of cheaters re-offend)? What kind of protection do you have against a cheater simply reopening an account (hint: PC-based FtPs have pretty much zero such protection)?
#9. As a rule of thumb, DON’T rely on your players to report cheating
Also DON’T encourage your players to report cheaters. They will report what they perceive as cheating anyway, but encouraging your players to report can easily get quite a few of them into paranoid mode. In addition, while vote-kicking MIGHT be necessary and useful in certain cases, allowing players to vote-kick an opponent is rarely a good idea.
On the other hand, DO take cheating reports seriously and review available information manually for these reports. For this, you DO need a separate team as soon as you’re successful enough. And you DO need to collect enough information (statistical and any other) to perform such an analysis. DO store such information in your DB, and DO add reports as soon as they’re requested by an anti-cheating team.
#10. DO restrict your client to only “need to know” information
In other words, DO implement so-called “interest management”. Whatever is on the client can be hacked so passing sensitive information that shouldn’t be made public is a Really Bad Idea™. Not implementing interest management is an almost guaranteed way to expose your game to See-Through-Walls cheats (a.k.a. Wallhacks), to Lifted-Fog-of-War cheats (a.k.a. Maphacks) and to so-called ESP cheats (being able to see health etc. of the opposition).
#11. DO use C++ for the client
While we’re at it, let’s note that emscripten seems to be about as good as-C++ for this purpose. At the moment, I don’t have hard evidence to support this claim, but from what I know, it certainly looks so.
#12. NEVER EVER release anything with debug information in it
Do I really need to explain it?
#13. DO go through your code and filter out all the error messages
That is, you should get rid of all the error messages that are not meaningful to anybody except for yourself (if you want to keep them, you can always replace them with their respective hashes).
I remember a case of an algorithm in use being deduced from an innocent message from a third-party library, which in turn facilitated an almost-hack. Once again, on the client we’re in the realm of obfuscation, so any information you give away can be used against you (yes, it does sound exactly as Miranda warning).
#14. DO guard your source code with your life
Your source code leaked to hackers will make 99% of your obfuscation pointless.
In particular, if you’re a large company, separate your source code into pieces, with each piece accessible only to a relevant team. Among other things, it will help to contain damage from those next-to-impossible-to-protect-from spearphishing attacks on your team.
#15. DO check the integrity of your client (including assets and/or configuration)
This should be done at least before starting your client (and ideally – while it is running too, but this is a different story).
Also make sure to report to the server if client integrity is compromised — while this is not necessarily a cheat, it is clearly a “red flag”.
#16. DO consider setting “traps” for cheaters within the client
Some typical traps include:
- The same value stored twice within the client’s memory and both stores checked occasionally to make sure they’re equal. For such a trap to be efficient, you’ll need to obfuscate the memory in at least one of these locations (for example, by XOR-ing it with some constant, though you may be MUCH more sophisticated than that). If the value happens to be not the same it is a Really Big Red Flag, though I am against using a single occurrence of such a mismatch as conclusive proof — when you have a million players, Really Strange Things do happen with some of your players from time to time.
- Checking the integrity of certain critical parts of your client code and checking for suspicious injected DLLs.
- Measuring execution times of your code and trying to find unusually long delays to realize that the client is being debugged. In this regard, using RDTSC x86/x64 instruction is an interesting option because it often allows you to measure time even when a kernel debugger is present.
- One specific type of trap is the “honey pot”. Technically, “honey pot” differs from a usual trap in that you create an artificial target for the cheater.
#17. DON’T tell the cheater exactly when you’ve got him
Instead of banning the cheater right away, it is almost universally better to delay your ban. Personally I’m not a big fan of “ban waves”, preferring random per-player delays instead, but even ban waves are MUCH better than immediate bans.
#18. Keep in mind that pretty much any Lag Compensation opens you up to time-based cheats
It is generally impossible to distinguish between genuine delay and hacked-client-induced delay a.k.a. Artificial Lag. OTOH, if you’re careful enough, you can limit the effect of such cheats on your gameplay.
#19. DON’T rely on any kind of PC identification.
While you MAY still want to implement it, but keep in mind that PC identification is trivial to get around (and make sure that your marketing/monetization/BA folks are aware of it too, otherwise they will try to rely on it to prevent promotion abuses).
#20. DO gather statistics that are potentially related to cheating
For example, if a player hits within 5% of the center of mass of his opponent 95% of the time, well, you can guess that something fishy is going on there.
As a rule of thumb, statistics SHOULD NOT be used as conclusive proof, but rather as a red flag; how to find out what to do with this red flag is a separate story.
Last but certainly not least, DON’T rely solely on statistics gathered on your client. In other words – get as much statistics on the server-side as you can.
#21. DO consider “captcha” to identify “grinding bots”
One type of protection which was seen to work reasonably well against unattended bots (usually ”grinding” bots) is “captcha”. While admittedly annoying, I’ve seen the need for captcha communicated to the community so they’ve accepted it to be a “necessary evil”, as long as it only happens once in a long while for legitimate players.
Note that for it to work, it MUST NOT be a “captcha-for-all”, instead, it MUST be activated only when one of a red flags is raised.
#22. DO be prepared to scan your player’s computers for known cheating software
This is a rather controversial issue, but much more likely than not you will need to implement it. For most of the games out there, scanning player’s computers is a MUCH lesser evil than allowing cheaters to go rampant (ruining the fun for your players and as a result – killing your game).
Overall, such scans is a very complicated matter (which is BTW pretty close to writing your own antivirus), so I will give only a few hints here:
- DO put a right-to-do-such-scans section into your ToC.
- DON’T get back any information except that absolutely necessary for cheat detection
- If you’re losing this scanning battle to cheaters, DO consider using device-driver-based stuff for this purpose (hint: there are commercial products available out there)
- While you’re at it, DO detect running-under-virtual-machine too (they’re often used to hide the software, and usually DO qualify as a red flag)
- DO consider having TWO levels of detection. The first level might be a simplistic detection of the more obvious and most popular cheats, and should result in just a friendly message “Hey, you’re using something that is not allowed by ToC. Please don’t do it again, OR ELSE”. The second level would be a full-scale detection with a ban as a result. The idea here is to stop the potential cheater before it is too late and without losing the player if possible. I’ve seen such thing working in a pretty large MOG pretty well (YMMV, batteries not included)
A Parting Shot
Of course, the list above is non-exhaustive. If your game is successful, you’re bound to find out quite a few things on this road yourself (some of them the hard way ). However, there is one all-important thing that helps us a LOT in this regard:
“You don’t have to run faster than the bear to get away. You just have to run faster than the guy next to you.”
— Jim Butcher
Applying this to our case, we can say that:
“You don’t have to be 100% cheat-proof to save your game from cheaters. You just have to do better than the guy next to you. “
— ‘No Bugs’ Hare
The economy of cheats — especially of those commercially available ones — dictates that if there are two targets, one being very juicy but very well protected, and another being moderately juicy but poorly protected, commercial cheaters are clearly going for the latter. After all, it is nothing personal, just business.
This article is a brief summary on the topic of cheating based on materials from an upcoming book “Development and Deployment of Multiplayer Online Games”. BTW, Vol. 1 of this book is currently on Kickstarter (what a coincidence ;-)), so you have a chance to get it at Kickstarter-discounted prices (KICKSTARTER SPECIAL PRICE: €15~=$17 for e-book and €25~=$28 for paperback).