<style> /* h2 { border-bottom: 1px solid #ddd; } h3 { border-bottom: 1px solid #eee; }*/ section { margin-bottom: 2rem; padding: 0em 1em; padding-bottom: 1em; border-radius: 4px; background-color: #f7f7f7; border: 1px solid #ccc; } center { text-align: center; } summary { font-weight: bolder; } summary:hover { text-decoration: underline; } .alert details { background-color: transparent !important; } code { background-color: rgb(244,244,245) !important; color: #ff0000; } .alert-warning { background-color: #fcf8e3 !important; color: #8a6d3b !important; } .alert-success { background-color: #dff0d8 !important; color: #3c763d !important; } </style> **[&laquo; Back to the main CSCI1660 website](https://cs.brown.edu/courses/csci1660/)** # Handin Project Setup Guide This guide is designed to help you get started with setting up the container environment you'll use for the Handin project. The primary component of the project is cs666's course infrastructure, i.e. a *very* insecure filesystem that you will attack in order to give you practice discovering some real vulnerabilities and proposing fixes. In order to let you conduct attacks in a safe, repeatable way, we provide this filesystem to you as a container image--you can run this container on your computer, which will give you access (as a student, not a TA) to cs666's course infrastructure. This guide will provide instructions on how to set up the Handin container environment and how to work with it for the project. **Some background**: This is the first year we are using containers for this course, after we learned that the method we've used in the past wasn't viable this year. While we'd always planned to make this change, we had to accelerate the process significantly. We've tested as best we can, but we appreciate your feedback and your patience as we work to find the best workflow for everyone! ### Before you start: Docker setup checkin > **New to docker?** Before you can start download the Flag container environment, you'll need Docker installed on your system. If you do not have Docker installed from the last project, you can do so using [this guide](https://hackmd.io/@cs1660/container-setup). Before continuing, now is a good time to check in and make sure your container setup is performing optimally. If your container ran slowly or had issues during Project 1, take a look at the following notes and recommendations: :::warning <details markdown="1"><summary markdown="span"><b>Updates if you had slow filesystem performance on MacOS</b></summary> If you experienced exceptionally slow filesystem performance on MacOS, see [here](https://hackmd.io/@cs1660/container-setup#Slow-filesystem-performance-on-Macs) for instructions on how to update your Docker settings for a fix. If you still encounter poor filesystem performance, or you have issues after the fix, please let us know! </details> ::: :::warning <details markdown="1"><summary markdown="span"><b>Windows users: Important instructions for making Docker more usable</b></summary> It seems that recent versions of Docker and WSL have issues with memory usage. See [these instructions](https://hackmd.io/@cs1660/container-setup#Windows-specific-issues) for a full writeup of the problem and a list of workarounds that we think will help. If you encountered issues with Docker during Project 1, we strongly recommend that you do the following steps from this guide now: 1. Update Docker to the latest version (if you installed it from a previous class) 2. [Set a memory limit on WSL](https://hackmd.io/@cs1660/container-setup#Workaround-1-Limit-WSL%E2%80%99s-memory-usage) (and thus your docker containers) The instructions also list some workarounds you can use at times to help reduce Docker/WSL's memory usage when they start to get problematic--skim these over and then keep them in mind in case you run into issues! </details> ::: ## Handin container setup <section> ### Before you start: cleaning up previous containers If you're finished with Project 2, you will likely no longer need to use its containers/images. To save disk space, you should run `./run-container clean-image` for all of the Flag containers/images that you used in Project 2, which you can do as follows: 1. **To clean up the FLAG portal:** Run `./run-container clean-image` wherever you cloned the Flag stencil repo 2. **To clean up the simple webserver** (if you used it) Run `./run-container clean-image` wherever you cloned the simple webserver repo 3. **To clean up Bob** (if you used Bob) Run `./run-container clean-image` in the `bob-router` directory of your stencil repo This should reduce the disk space Docker uses on your host machine, making room for the Handin container. Now that our environment is set up, we can download the Handin container and start working with the project! </section> ### Our container infrastructure We have one new container for this project: the **Handin container**. This container gives you access to the Handin filesystem as a student user (part of your goals for this project will be to find a way to run code with elevated privileges, which you shouldn't be able to do normally 😉). ### Reminder: super important container terminology As with Project 2, it is probable that you will make changes to the Handin filesystem in ways that will break it. This is fine--it's how you'll figure out what parts of the system are vulnerable! So, we need an easy way to make changes to the Handin filesystem and then revert it to a clean state, which is exactly the kind of workflow that containers were designed to do. Here is a reminder of some important terminology we'll use as we work with the Handin container: - When we download or set up or a container, we create a **container image** (often just called an "image"): This is a self-contained, read-only collection of files and settings that tell Docker how to run the software--think of it like a "template" for how the software runs - To actually run the code inside the image, docker creates a **container instance** (often just called a "container"), which is a specific *instance* of a container image. Unlike images, container instances are read-write. <u>Here's the crucial bit: as the code inside the container runs, only the instance changes--the original image is unaffected!</u> After testing, Docker makes it easy to discard the container instance and start over from the image, which is what we'll use in the project! ### Recommended filesystem layout (and cloning the stencil repo) You can find a link to clone the assignment's stencil code using **[this Github classroom link](https://classroom.github.com/a/HQT_Ob0G)**. To make development easier, we recommend cloning your stencil code for this project so you can access files in it from your development container. To do this, we recommend cloning the stencil in your `DEV-ENVIRONMENT/home` directory, where `DEV-ENVIRONMENT` is the name of the folder you used when you cloned the development container. Here's an example of what your filesystem might look like (based on what you might have from Projects 1 and 2): ``` - ... |--DEV-ENVIRONMENT | |--docker/ | |--home/ | | |--p01-cryptography-<your GitHub username>/ | | |--p02-flag-<your GitHub username>/ | | |--pr03-handin-<your GitHub username>/ # <----------- Clone your stencil here! | |--cs1660-run-docker | |-- ... ... ``` ### Downloading the Handin container image In the handin stencil repository, we've provided a script called `run-container`, which automates downloading the handin container image. This is probably the only container script you will need to interact with for this project. :::info **Note**: this container runs separately from all the previous containers you have used in Projects 1 and 2, though you'll interact with it in a similar way (ie, by running a script like `run-container` on your host system). ::: To download the image, do the following: 1. If you have not done so already, clone the stencil repository for the project (link in the handout) 2. Open a new terminal on your **host machine** (or, on Windows, a **WSL terminal**). 3. `cd` to the Handin project stencil directory and run the following: ```shell # Download the container image $ ./run-container setup ``` :::warning **Windows users**: If you get errors running this script the first time, you may need to run `dos2unix` on the script. See [here](https://hackmd.io/@cs1660/container-setup#Windows-line-endings-%E2%80%9Cunexpected-end-of-file%E2%80%9D) for more info. ::: This will download the latest version of the container image from our repository, which may take a few minutes. If we push updates to the image while the project is out, you can run this command again to pull the latest version (we'll post an announcement on Ed if this happens). ### Running the container Now that the image is set up, we can launch an instance of the container, which will give you a shell that you'll use to interact with the Handin filesystem. To do this: 1. In a terminal on your host machine (or a WSL terminal on Windows), run the command `./run-container` This should produce output that looks like this: ```shell $ ./run-container # Start an instance of the Handin filesystem container! Found network net-cs1660, skipping creation *************************************************************************************** Starting container: handin-container cs1660-handin-rel-00c9a9f-2023-03-16T09:42:37-04:00 This container's internal IP: 172.20.0.2 This container's internal hostname: handin-container *************************************************************************************** alice@c808c3104e5a:~$ ``` Running `./run-container` with no arguments starts up an instance of the Handin container. Once you enter the container, your username should be `alice`, and you should be redirected to your home directory. If you receive an error when launching the container, or the container otherwise quits and returns you to your terminal, please post on Ed and include any log output so we can help you debug. **Stopping/Restarting the Handin container** (don't do this now, but take note for later): You can exit out of the Handin container by running `exit` in the container's shell. If you lose this terminal for some reason, you can also stop it by running `./run-container stop` from another terminal. Once stopped, you can restart the container again by running `./run-container`. By default, any changes made to the Handin filesystem are preserved--for instructions on how to reset the container to clean state, see [here](#Resetting-the-container). ### Sharing/Editing files in the Handin container Similar to your development container, your Handin stencil container a directory called `home` that is mapped to `/home/alice` inside the container. If you want, you can use this to edit files from your host machine (or the development container). The `home` directory in the stencil also contains the code stencil for cs666's Ivy, which is the "assignment" you will submit to the cs666 autograder. For more details, see [this section](#The-cs666-Ivy-stencil). ## Working with the Handin filesystem Once you have a shell on the Handin filesystem, you can begin to poke around and start attacking cs666's course infrastructure! This part of the guide will demonstrate the most important mechanics you should know as you work on discovering vulnerabilities. ### What you know Like the FLAG portal, cs666's course infrastructure is part of a fictional grading system at Blue University (it is organised in a very similar way that the CS filesystem is organised). You've managed to get into the Blue University filesystem using the login credentials for one student -- Alice: ``` Username: alice Password: iamalice ``` :::info **Note for later**: Similar to Flag, you *also* happen to know the credentials for one other user (`bob`), which you may find useful for testing certain exploits that involve multiple users. For more info, see [here](#Logging-in-as-Bob-another-user-for-testing-exploits). ::: #### Source code All of cs666's course infrastructure lives inside the container at `/course/cs666`. You won't be able to view all of the files inside the container (since they won't be viewable to unprivileged users). However, all of cs666's code is open source and viewable here: https://github.com/brown-csci1660/handin-source You are encouraged to look at the source code to understand what the infrastructure is doing so you can figure out how to find and report on vulnerabilities. (Another example of how open systems can ultimately lead to better security :smile:) #### Your goals Given this login information, the source code, and the Handin container, your job is to explore cs666's course infrastructure and see what vulnerabilities you can find. For more information on the types of vulnerabilities we expect you to find, and specific requirements for the project and grading, please see the handout. :::danger **Remember**: for this project, your goal is to find vulnerabilities in the course infrastructure/filesytem, so **any attacks on the container infrastructure itself are out of scope and will receive no credit.** <details markdown="1"><summary markdown="span"><b><u>What does this mean?</u></b></summary> The point of this project is to help you lean about operating system vulnerabilities, meaning we want you to behave as if you are a remote attacker who can only access Blue University's filesystem. Since the container runs on your own computer, you can *easily* use `docker` commands or other methods to "break in" and get a root shell, view files, etc.--these are outside the capabilities of the attacker we want to guard against (called the "threat model") and thus are NOT considered vulnerabilities. Remember that you are being graded based on the OS vulnerabilities you find (and your explanations of them)--attacks out of this scope will not receive credit, and are not in your best interest for learning about OS vulnerabilities. </details> ::: ## CS666's Ivy (<i>not</i> CS1660's Project 3!) The handout describes an example assignment for cs666 called "Ivy", which is very similar to the Ivy project we completed for project 1. For **our** project, we'll attack cs666's autograder infrastructure to find vulnerabilities. :::info **Note:** **You should not actually implement Ivy (again)!** Your goal in this project is to exploit the vulnerabilities in cs666's course infrastructure to get yourself a good grade (or bring other students' grades down...) *without* actually implementing the assignment. ::: You know (from the handout) that Blue University decided to get students to submit assignments via a handin script, one very similar to the system sometimes used in CS courses at Brown (e.g. cs1650). For cs666, the handin script is named `cs666_handin`, which lives inside the cs666 course directory. When you run `cs666_handin`, it invokes a `setgid` binary that does the following: - Saves all files in the current working directory in a `.tar` archive in the course’s handin directory--this creates a submission - Runs the cs666's autograder, which extracts the student's code from the submission and grades it. To do this, the autograding script combines the student submission with the support code, runs the compiler, and executes the resulting binary. For more info on what the student's submission should look like, see the CS666 Ivy handout, located in the appendix of our own handout - After running the tests, the autograde script adds the grades to a course-wide database file For more details on how to run the autograder and what the output should look like, see the next section. ### How to run the CS666 Ivy autograder To hand in cs666's Ivy, you should run the `cs666_handin` script in a directory containing a file named `KEY` and the `sol.go` file we give you in the `/home/alice/ivy-stencil` directory. Again, you **should not** actually write a solution for Ivy in `sol.go` (but you *might* want to put something else there...). The autograder grades two things: - `ivy`: This is the code part: the autograder will take in your `.go` files, combine them with the support code, run the compiler, and ultimately run your code and check the output against an expected value. - `KEY`: The autograder also expects you to turn in a plain text file called `KEY`, which should contain the client's leaked wifi key. On submission, the autograder will check what's in your `KEY` file against the solution `KEY` for the user handing in (e.g. Alice) and assign a grade based on the comparison. :::info **Hint:** There are definitely some exploits you can craft to get yourself a full score on Ivy, by finding vulnerabilities with the autograder, your `KEY` file, and the solution `KEY` file... ::: So, handing in your "implementation" of cs666's Ivy (presumably, with some form of exploit called `exploit.txt` embedded) would look something like: ```shell alice@c808c3104e5a:~/ivy-stencil$ pwd /home/alice/ivy-stencil/ alice@c808c3104e5a:~/ivy-stencil$ ls <------- making sure sol.go and KEY exist exploit.txt go.mod KEY main.go Makefile sol.go alice@c808c3104e5a:~/ivy-stencil$ cs666_handin ivy ./ ./go.mod ./sol.go ./Makefile ./exploit.txt ./KEY ./main.go successfully handed in to /course/cs666/handin/ivy/alice/ff6f9982ba43c289a6a85701e2bc17bf.tar invoking autograder... Removing stencil file main.go Executing: modifydb --command grade -p "ivy:ivy" --points "0" --comment "10 tries/0 successes" -s "alice" --files main.go sol.go Executing: modifydb --command grade -p "ivy:KEY" --points "0" --comment "files do not match" -s "alice" --files main.go sol.go alice@c808c3104e5a:~/ivy-stencil$ ``` ## Resetting the container As you attack Blue University's filesystem, you will inevitably make changes to the filesystems's state in ways that will break it--this is okay, and encouraged! During this process, you'll find it useful to reset your instance of the filesystem to its default configuration to test out attacks once you've figured out a procedure, and to conduct different attacks that may conflict with each other. To reset the Handin container: 1. **Stop your Handin container** by running `exit` in the terminal. Alternatively, you can run `./run-container stop` from another terminal. 2. **Run `./run-container --clean`**. This will delete your container instance and start a new container from a fresh copy of the image. 3. You're in the filesystem as Alice again! :::warning **Super important note**: Note that any files or directories you add/delete/modify in the home directory (i.e. `/home/alice/`) **will not** be reset to their original state. This includes your Ivy stencil (`/home/alice/ivy-stencil`) or any configuration files you may modify like `/home/alice/.bashrc`. This is because `/home/alice` in the container is shared with your handin stencil repo, so the files here don't actually live inside the container! If you want to reset these files, you'll need to do it manually--since they're part of your handin repo, we recommend periodically committing your changes to git so that you can roll them back if necessary. If you want to download a fresh copy of any files, see [this repo](https://github.com/brown-csci1660/handin-stencil). ::: ## Hints and useful resources This section describes some useful tools and hints you may find useful for certain exploits, including: - The [**OS lecture demos**](https://github.com/brown-csci1660/os-demos) contain the demo code and example attacks used in lecture--look here for some example shell scripts with annotations - [**`whoami`/`cs666_whoami`**](#whoami-and-cs666_whoami): How to view your real/effective UID/GID, which is useful proving you can execute code with elevated privileges - [**Logging in as bob: another user for testing**](#Logging-in-as-Bob-another-user-for-testing-exploits): Use this if you want to test exploits involving multiple users - **[Note on running programs from a setuid program](#Running-programs-from-a-setuid-program)**: Read this if you're trying to arbitrary code execution! - [**Go modules example**](#Importing-go-modules-on-your-system): How to build a go program with local modules - [**General Linux resources**](#Useful-shell-commands): See this section for some general resources on Linux, as well shell commands and man pages you may find especially helpful ### `whoami` and `cs666_whoami` The [`whoami`](https://man7.org/linux/man-pages/man1/whoami.1.html) Linux command allows a user to see the currently logged in user. If you run this command in the Handin container, you should see an output similar to this: ```shell alice@c808c3104e5a:~$ whoami alice alice@c808c3104e5a:~$ ``` There’s a binary in your container called `cs666_whoami` which is essentially a more powerful version of the normal `whoami` command. It prints the `uid`, `euid`, `gid`, and `egid` of the process that it runs as (and thus, by default, that its parent process runs as). If you run `cs666_whoami` in your Handin container, you should see an output similar to this: ```shell alice@c808c3104e5a:~$ cs666_whoami uid: 1000 euid: 1000 gid: 1000 egid: 1000 alice@c808c3104e5a:~$ ``` :::info **Note:** Like the `cs666_handin` binary, while the `cs666_whoami` script resides in the `/course/cs666/bin` directory, you can simply run `cs666_whoami` anywhere in the container filesystem; you do not need to use the full path to the binary to use it. ::: This is a useful way to test your exploits as it allows you to prove that you can run a command with gain elevated privileges on Blue University's filesystem. For example, if you wanted to prove that you are able to run arbitrary code as the TA group, you could run the `cs666_whoami` command wherever you've found a place to inject arbitrary code. When running your injected code, if the `cs666_whoami` command prints out a `gid`/`egid` of the TA group, then congrats! You've found a way to run arbitrary code as the TA group. :::success **How to demonstrate proof:** When we grade your demo videos, using the `cs666_whoami` command to show you are executing/accessing something as the TA group would be considered as enough proof that your exploit works. When running an exploit, note that your terminal output doesn't need to be pretty--so long as we can see (from your video/or screenshots in your writeup) that the output of `cs666_whoami` is *in there somewhere* and that it's running with elevated privileges, this is enough for credit. No need to waste time pretty-printing the output! ::: ### Logging in as Bob: another user for testing exploits To test exploits, you may want to perform actions as another user--to do this, you can log in as user `bob`, who is also a student in CS666. Bob's credentials are as follows: ``` Username: bob Password: iambob ``` You can log in as Bob from any terminal in your handin container using the `su` command, like this: 1. Open a terminal in the handin container (ie, by running `./run-container` from your handin repo). By default, this should log you in as `alice`. 2. Run the command `su - bob`, which asks to switch to Bob's user 3. Enter Bob's password, as above You should now have a shell as Bob! As Bob, you can submit assignments and poke around the filesystem just like Alice. :::success **Tip**: Note that you can be logged in as both `alice` and `bob` at the same time! To do this, just start another terminal in your handin container by running `./run-container` from your Handin repo. ::: ### Running programs from a setuid program As we'll see in this assignment, `setuid` and `setgid` programs are tricky to write securely and should be treated with extreme care. This is especially true when the program is a shell script, since there are many ways to attack the program, as you'll find in this project... :wink: Because `setuid`/`setgid` shell scripts are so dangerous, shells like `/bin/bash` by default have special protections that detect if they're being run with elevated privileges and resets them back to their default values. To show what we mean, let's say you managed to get a setuid program to run a shell script you wrote, like this: ```go // **** some_setuid_program.go **** out, err = exec.Command("attack_script.sh").CombinedOutput() if err != nil { panic(err) } fmt.Printf("%s\n", out) ``` ```shell ###### attack_script.sh ###### #!/bin/bash cs666_whoami # <--- Won't run with elevated privileges! ``` This won't work because `/bin/bash` notices if it was launched with elevated privileges (ie, if `uid != euid` or `gid != egid`) and resets the process's EUID/GID back to the user's "real" UID/GID. This is a good security feature, but, for trying to construct an attack, this is annoying! However, bash has a special execution mode we can use to preserve this behavior (called [privileged mode](https://unix.stackexchange.com/a/116798), more info [here](https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html)) if we choose to use it. To turn on privileged mode, we need to run bash with `-p` in the interpreter line, like this: ```shell ###### attack_script.sh ###### #!/bin/bash -p # <--- Turn on privileged mode cs666_whoami # <--- Runs with elevated privileges ``` :::info **Note:** Notice that the autograder's main shell script [`autograder.sh`](https://github.com/brown-csci1660/handin-source/blob/main/autograde/autograde.sh#L1) turns on privileged mode because it relies on this behavior. This is probably a bad idea... :wink: ::: ### Importing go modules on your system :::info **Note**: The info here is pretty generic--it's up to you to figure out how to use it for this class! ::: Modern versions of Go (>1.13) use [*modules*](https://go.dev/blog/using-go-modules), which is a way to import and manage versions of library code in a project. In most Go programs, modules are downloaded from external repositories (eg. `go get github.com/some/module`), are checked for integrity, compiled locally, and then made available so you can import them into your code. **If you want to write your own go module**, take a look at [**this example**](https://github.com/brown-csci1660/go-modules-example), which configures a go program ("`main`") to load an external library ("`examplelib`") that lives on your local system. See the example's [README](https://github.com/brown-csci1660/go-modules-example#go-modules-example) for instructions on how to do this in your own project. As an alternative, you could also publish your own library on Github, but we don't recommend this since it adds more steps to the process. > You may have read about the environment variable `GOPATH` (usually located at `/home/user/go`, which has similar goals. While `GOPATH` still exists, its use is deprecated in modern versions of Go (like the one used in our container)--creating libraries directly in `GOPATH` in this project may have unpredictable results and isn't likely to be worth your time. ### Useful shell commands :::info **Note**: More coming soon! ::: #### General resources - **[Advanced Bash Scripting Guide](https://tldp.org/LDP/abs/html/)**: A comprehensive guide to just about every feature in Bash, with tons of examples and exercises! Also try [this guide](https://tldp.org/LDP/Bash-Beginners-Guide/html/) for more general principles on writing shell scripts. - **[CS0060](https://cs.brown.edu/courses/cs0060/)** was an experimental course at Brown on "practical systems skills", which last ran in 2019. This course focused on teaching skills working with shells and Linux--eg. the "missing manual" that introductory courses often don't have time to cover. **All of its lectures, notes, and recordings are freely available**: if you want some general background resources, we highly recommend checking it out! #### Useful man pages The following manual pages may be helpful general resources for learning about some system functionality. You can find each man page by running the command below, or searching for the equivalent man page online: - `man 7 credentials` - `man 7 symlink` #### Helpful commands - `id -u`: Get the effective user ID (UID) - `getent group <group>`: Get the GID for `<group>` - `stat`: Get detailed information about a given file or directory - `htop` or `top`: Live process viewer - `watch`: Execute a program repeatedly - `strace`: Trace system calls ## Final notes Congrats on reaching this point! This guide has provided an overview of the most important mechanics you need to work with the container for this project. If you have questions, please feel free to come to hours or post on Ed. Have fun, and happy hacking! ## FAQs/Common issues ### Windows line endings ("unexpected end of file") If you see errors along the lines of `bash: '\r': command not found` or `bash: /home/cs1660-user/.bash_profile: line 5: syntax error: unexpected end of file`, see [this section](https://hackmd.io/VvXjihLeQne4q84wq1SgWg?view#Windows-line-endings-%E2%80%9Cunexpected-end-of-file%E2%80%9D) for common fixes.