What makes a good CTF challenge?

A Capture The Flag (CTF) challenge is a specific task or problem designed to test participants’ skills in various areas of cyber-security. You can see it as a compute science puzzle, problem or enigma.

No, it’s not about 0-days

While many assume that solving CTF challenges requires uncovering novel vulnerabilities and exploits, in truth, challenges only very (very) rarely demand the implementation of a 0-day exploit (actually, I’ve only seen that in Real World CTF). At most, typically for challenges under the “pwn” category, participants apply known exploits to the context of the challenge.

So, what are the ingredients for a CTF challenge participants will enjoy? I’ve been playing quite lots of CTFs, and I’m going to explain IMHO what makes a good challenge. It actually burns down to mostly 2 ingredients:

1. Implementing and deploying a concept
2. A logical solution with no guesswork

We must learn to implement and deploy a concept

There is a quite big difference between knowing a given concept (eg. sending a BLE advertising packet) and being able to implement it and run it successfully.

• The first barrier is the implementation. To implement a concept correctly, you need an in-depth understanding of it (eg. exactly which bytes to set or not).
• The second barrier pertains to deploying a piece of code into production. This often entails steps such as debugging, handling unforeseen circumstances (eg. other Bluetooth packets disturbing your device), optimizing performance etc.

A good CTF challenge makes you [learn to] implement and deploy a given concept. The process is fun both because you learn + there’s the thrill to get it to work.

Counter-example: a CTF challenge which only consists in finding the right tool that will do all the work for you and solve the challenge is no fun. You don’t learn anything (apart using a search engine…).

The solution to a challenge lies in logic, not wild guesses

The key to solving a CTF challenge lies in logic and problem-solving. While some initial exploration may involve a degree of educated guesswork, it should quickly lead to uncovering sufficient clues to proceed in the right direction. Each subsequent step of the challenge should present a clear path forward, with the difficulty lying in executing the solution rather than determining it. As participants progress, each subsequent step should similarly unveil a straightforward course of action.

When is it guessing? When is it logic?

Suppose the challenge consists in finding a password to log in as admin on a given website.

• If the password is hidden in comments of the login HTML page, this isn’t considered guesswork, as it’s reasonable to examine the HTML page to comprehend the login process.
• If the password is qwerty, the string is nowhere, but the challenge title is “RockYou”, we can consider this is a strong hint to the password list Rockyou.txt and that you need to bruteforce the password with it, or use a very common password. This is not guess work either.
• If this is a website to enter a presumed vulnerability database and the password is log4j, this is guessing because there are too many vulnerability names to try to log in. To fix this, the challenge should include a hint to direct participants towards log4j (eg. CVE-2021–44228, or at least November 2021, or a picture explaining the vulnerability etc).

Toppings for CTF challenge recipes

Other key ingredients for crafting a compelling CTF challenge are:

• A scenario that mirrors real-life situations
• Intriguing devices or techniques for participants to utilize
• An engaging narrative
• Thorough testing of your challenge!!!

Aren’t OSINT challenges just pure guessing?

The difficulty in OSINT challenge consists in finding the desired information in the middle of lots (too much) of information. I enjoy OSINT challenges when they involve tools to speed up the quest, or when there’s a path to finding the information like:

1. Oh, I found the Facebook account of the mentioned person
2. The Facebook account mentions an email hacker777@hackanonym.org and that email is found on 4 underground forums.
3. In one of those forums, a user named “Dark Chocol8” sells a ransomware against 1,000 USD on his bitcoin account. Another person answers and said he purchased and provides a link to the transaction.
4. I follow the transaction and find the flag.

In this example, there is a clear path to find information at each step.

NB. If you’re designing an OSINT challenge, make sure the information you expect is very precise (eg. define the best way you can the format of the expected flag).

— the Crypto Girl