CTFs: A Rosetta Stone in a Video Game

CTF challenges for me are, in equal parts, a learning exercise and a video game. I wouldn't call myself very good at them, but doing them has made me better at other things. The list ranges from basic shell competencies, to rabbit-holing me into Web scraping, to finally getting some exposure to Python.

For the unfamiliar, CTF stands for "capture the flag" and is essentially hacking as a game. There's a variety of purposely insecure educational platforms on the Internet that you can poke around at for fun.

For a good introductory example, you can access the challenges on Over the Wire, on a toaster, probably. Outside of the series focused on Web applications, the challenges are primarily SSH-based. And while they're largely written for Bash and CLI tooling -- at least at the early levels (that I've cleared...) -- the environment hosting them includes a variety of tools that you can use, including Python.

(This means you, too, Windows friends -- if you're on anything 10 or up, you've had ssh built into your terminal for years)

Not every level can be cleared with it -- there are limits on network access, so you can't install packages, and the standard library isn't sufficient for levels that (for instance) are about tools like ssh or tmux. (Even for network calls themselves, the http module actually recommends using the third-party requests library.) But there's a fair amount of ground you can retread with it. (And you could also use exec in many cases -- to call things like grep, for instance -- but that would defeat the point of this.)

Webapp-based challenges give you even more freedom than that -- whether they're on the Internet or on a Docker container, they're an HTTP request away, in the language (or CLI tooling) of your choice. OtW has a series called Natas that takes you through the basics of Web security vulnerabilities. You can access it through a browser and inspect it with the built-in dev tools -- in many cases requiring little (or even nothing) more than the console and some JavaScript.

But you don't have to do that in the UI. HTML, CSS, JavaScript, and JSON -- as handled by a browser -- are ultimately just text, sent over an HTTP request. If you can store that text, you then have a multitude of other choices on how to study and manipulate it. For instance, you can get even more minimal than the browser console, by making fetch requests in a backend JavaScript environment like Deno and processing the raw HTML responses. (Or feeding them into a DOM polyfill.)

And you can do this with a lot of other challenges like them -- which I'll get into, but I say now mostly because Over the Wire itself has a no spoilers policy and I'm trying to be intentional about speaking in general terms.

One way to do it in JavaScript looks like this:

const baseURL = 'https://www.example.com'
// or http
// remember, the point of these platforms
//  is that they're purposely insecure
// you also might be targeting something local

const endpoint = new URL(baseURL)

// a URL object is given properties (or not),
//  which you can then manipulate, based on
//  the string you feed into it (its `href`)
endpoint.username = 'someuser'
endpoint.password = 'placeholder'

// in this case, that string is also its `origin`
// within it is a `host` (www.example.com) and a
//  `protocol` (https)
// there's also `hash`, `query`, and various others
// as you change it, `href` will update dynamically,
// so you can either read its value directly, or
//  use `URL.toString()`

// it will read like this:
// `{protocol}://{username}:{password}@{host}`

// in a browser, the above is all unnecessary
// it would store your authenticated session data,
//  and then send it along with outbound requests

// the browser also offers `location`, which is
//  itself a URL object


// `fetch` will take a bare URL, or a URL object
const html = await fetch(endpoint)
  .then(r => r.text())
// you could also `exec(`curl ${endpoint}`)
//  here too -- but since you have to import it
//  from 'node:child_process', it's not actually
//  any shorter in JavaScript
// whereas `fetch` doesn't need one, and also
//  works in browsers
// but again, a bunch of languages have it -- or at
//  least something like it -- if you want to just
//  get the raw HTML, and not worry about how your
//  stack handles HTTP requests
// NOTE: this gets only the response body
// there are various other properties you might want

But you might also try grabbing the data with that same URL scheme using curl or Invoke-WebRequest and handling it directly within the shell of your choosing. Maybe it's another shell entirely -- like nushell, where various forms of data processing are built in. Maybe you try dumping lines of text into a SQL database (many of which can take CSV) and running queries against them. But once the raw text is on your machine, you can ultimately slice it up with any tool you feel like. And once you have one route to your flag, you can then use that as a guidepost for constructing others -- sort of like how a Metroidvania involves opening up new routes as you go, in order to enable backtracking through previous sections of the map. Or how in a fighting game learning one character might make it easier to understand similar ones in an overall archetype, opposing ends of a matchup, etc.

You can also extract data over ssh, but I'm not sure this specific platform's policy on that kind of thing for any sort of bulk file transfer. I keep mentioning this because while I say "purposely insecure," they do that on the honor system, and operate on the presumption that you, the user, won't be a Rick about it. Besides...

You don't even have to use hosted services. Some, like OWASP Juice Shop, are also open source -- so not only can you just download them yourself, but you can inspect the code directly for potential flaws, and run them locally. In fact, OWASP has a whole list of these, in a variety of different stacks.

These also don't necessarily frown upon writeups in the same way.

For some fun with encoding, CyberChef is both a versatile tool and an accompanying puzzle collection -- If you pop open the console, you'll also find a set of challenges designed so that you can solve them by using it. Some of them, like how to work with a base64 string, are also covered in the content above -- and to that same point, some languages (and the toolkits around them) will offer tools for handling base64, hexadecimal, and others out of the box.

There are also sites like HackMyVM, VulNyx, or Vulnhub, which are all... more or less what that first one sounds like. That is, they offer virtual machines that are built for you to break into. But those, again, limit you to the installed environments, unless you want to do something potentially unsafe like give the machine network access. Or, I guess, happen to have a spare machine around that you can run with no wireless netwrking or internal storage -- and feel like prepping this, plus a boot drive loaded with the OS and tools of your choice in addition to your VMs. But I'm trying to describe options with low barrier to entry here.

This is also why I'm not really talking about events. I mean, first of all, multiplayer is a different conversation I have even less experience in. But beyond that, there's more room to poke at something from a variety of different angles if it's running (or available for you to spin up locally) all the time.

These are free services, and many of them are open source software... But you can also think of them as a sort of reference material, or even test cases. You can stand many of them up yourself, and break them for fun in as many different ways as you want -- with "rules" in many cases being limited to basic concern for the safety of your host device. And if you're thorough about it, you can slowly build up a knowledge base of writeups -- not just for cybersecurity awareness, but as an interactive guide you can pick apart with every new tool you get your hands on.

(I hear Markdown and Git are great for this...)