Handin Project Setup Guide

This guide is designed to help you get started with setting up the container environment you'll use for the Handin project.

The primary component of the project is cs666's course infrastructure, i.e. a very insecure filesystem that you will attack in order to give you practice discovering some real vulnerabilities and proposing fixes. In order to let you conduct attacks in a safe, repeatable way, we provide this filesystem to you as a container image–you can run this container on your computer, which will give you access (as a student, not a TA) to cs666's course infrastructure.

This guide will provide instructions on how to set up the Handin container environment and how to work with it for the project.

Some background: This is the first year we are using containers for this course, after we learned that the method we've used in the past wasn't viable this year. While we'd always planned to make this change, we had to accelerate the process significantly. We've tested as best we can, but we appreciate your feedback and your patience as we work to find the best workflow for everyone!

Before you start: Docker setup checkin

New to docker? Before you can start download the Flag container environment, you'll need Docker installed on your system. If you do not have Docker installed from the last project, you can do so using this guide.

Before continuing, now is a good time to check in and make sure your container setup is performing optimally. If your container ran slowly or had issues during Project 1, take a look at the following notes and recommendations:

Updates if you had slow filesystem performance on MacOS

If you experienced exceptionally slow filesystem performance on MacOS, see here for instructions on how to update your Docker settings for a fix.

If you still encounter poor filesystem performance, or you have issues after the fix, please let us know!

Windows users: Important instructions for making Docker more usable

It seems that recent versions of Docker and WSL have issues with memory usage.

See these instructions for a full writeup of the problem and a list of workarounds that we think will help. If you encountered issues with Docker during Project 1, we strongly recommend that you do the following steps from this guide now:

Update Docker to the latest version (if you installed it from a previous class)
Set a memory limit on WSL (and thus your docker containers)

The instructions also list some workarounds you can use at times to help reduce Docker/WSL's memory usage when they start to get problematic–skim these over and then keep them in mind in case you run into issues!

Handin container setup

Before you start: cleaning up previous containers

If you're finished with Project 2, you will likely no longer need to use its containers/images. To save disk space, you should run ./run-container clean-image for all of the Flag containers/images that you used in Project 2, which you can do as follows:

To clean up the FLAG portal:
Run ./run-container clean-image wherever you cloned the Flag stencil repo
To clean up the simple webserver (if you used it)
Run ./run-container clean-image wherever you cloned the simple webserver repo
To clean up Bob (if you used Bob)
Run ./run-container clean-image in the bob-router directory of your stencil repo

This should reduce the disk space Docker uses on your host machine, making room for the Handin container.

Now that our environment is set up, we can download the Handin container and start working with the project!

Our container infrastructure

We have one new container for this project: the Handin container. This container gives you access to the Handin filesystem as a student user (part of your goals for this project will be to find a way to run code with elevated privileges, which you shouldn't be able to do normally 😉).

Reminder: super important container terminology

As with Project 2, it is probable that you will make changes to the Handin filesystem in ways that will break it. This is fine–it's how you'll figure out what parts of the system are vulnerable!

So, we need an easy way to make changes to the Handin filesystem and then revert it to a clean state, which is exactly the kind of workflow that containers were designed to do.

Here is a reminder of some important terminology we'll use as we work with the Handin container:

When we download or set up or a container, we create a container image (often just called an "image"): This is a self-contained, read-only collection of files and settings that tell Docker how to run the software–think of it like a "template" for how the software runs
To actually run the code inside the image, docker creates a container instance (often just called a "container"), which is a specific instance of a container image. Unlike images, container instances are read-write. Here's the crucial bit: as the code inside the container runs, only the instance changes–the original image is unaffected!

After testing, Docker makes it easy to discard the container instance and start over from the image, which is what we'll use in the project!

Recommended filesystem layout (and cloning the stencil repo)

You can find a link to clone the assignment's stencil code using this Github classroom link.

To make development easier, we recommend cloning your stencil code for this project so you can access files in it from your development container. To do this, we recommend cloning the stencil in your DEV-ENVIRONMENT/home directory, where DEV-ENVIRONMENT is the name of the folder you used when you cloned the development container.

Here's an example of what your filesystem might look like (based on what you might have from Projects 1 and 2):

 - ...
 |--DEV-ENVIRONMENT
 | |--docker/ 
 | |--home/
 | | |--p01-cryptography-<your GitHub username>/
 | | |--p02-flag-<your GitHub username>/
 | | |--pr03-handin-<your GitHub username>/ # <----------- Clone your stencil here!
 | |--cs1660-run-docker
 | |-- ...
 ...

Downloading the Handin container image

In the handin stencil repository, we've provided a script called run-container, which automates downloading the handin container image. This is probably the only container script you will need to interact with for this project.

Note: this container runs separately from all the previous containers you have used in Projects 1 and 2, though you'll interact with it in a similar way (ie, by running a script like run-container on your host system).

To download the image, do the following:

If you have not done so already, clone the stencil repository for the project (link in the handout)
Open a new terminal on your host machine (or, on Windows, a WSL terminal).
cd to the Handin project stencil directory and run the following:

# Download the container image
$ ./run-container setup

Windows users: If you get errors running this script the first time, you may need to run dos2unix on the script. See here for more info.

This will download the latest version of the container image from our repository, which may take a few minutes. If we push updates to the image while the project is out, you can run this command again to pull the latest version (we'll post an announcement on Ed if this happens).

Running the container

Now that the image is set up, we can launch an instance of the container, which will give you a shell that you'll use to interact with the Handin filesystem. To do this:

In a terminal on your host machine (or a WSL terminal on Windows), run the command ./run-container

This should produce output that looks like this:

$ ./run-container   # Start an instance of the Handin filesystem container!
Found network net-cs1660, skipping creation
***************************************************************************************
Starting container:  handin-container cs1660-handin-rel-00c9a9f-2023-03-16T09:42:37-04:00
This container's internal IP:  172.20.0.2
This container's internal hostname:  handin-container
***************************************************************************************
alice@c808c3104e5a:~$

Running ./run-container with no arguments starts up an instance of the Handin container. Once you enter the container, your username should be alice, and you should be redirected to your home directory.

If you receive an error when launching the container, or the container otherwise quits and returns you to your terminal, please post on Ed and include any log output so we can help you debug.

Stopping/Restarting the Handin container (don't do this now, but take note for later): You can exit out of the Handin container by running exit in the container's shell. If you lose this terminal for some reason, you can also stop it by running ./run-container stop from another terminal. Once stopped, you can restart the container again by running ./run-container. By default, any changes made to the Handin filesystem are preserved–for instructions on how to reset the container to clean state, see here.

Sharing/Editing files in the Handin container

Similar to your development container, your Handin stencil container a directory called home that is mapped to /home/alice inside the container. If you want, you can use this to edit files from your host machine (or the development container).

The home directory in the stencil also contains the code stencil for cs666's Ivy, which is the "assignment" you will submit to the cs666 autograder. For more details, see this section.

Working with the Handin filesystem

Once you have a shell on the Handin filesystem, you can begin to poke around and start attacking cs666's course infrastructure! This part of the guide will demonstrate the most important mechanics you should know as you work on discovering vulnerabilities.

What you know

Like the FLAG portal, cs666's course infrastructure is part of a fictional grading system at Blue University (it is organised in a very similar way that the CS filesystem is organised). You've managed to get into the Blue University filesystem using the login credentials for one student – Alice:

Username:  alice
Password:  iamalice

Note for later: Similar to Flag, you also happen to know the credentials for one other user (bob), which you may find useful for testing certain exploits that involve multiple users. For more info, see here.

Source code

All of cs666's course infrastructure lives inside the container at /course/cs666. You won't be able to view all of the files inside the container (since they won't be viewable to unprivileged users). However, all of cs666's code is open source and viewable here:
https://github.com/brown-csci1660/handin-source

You are encouraged to look at the source code to understand what the infrastructure is doing so you can figure out how to find and report on vulnerabilities. (Another example of how open systems can ultimately lead to better security )

Your goals

Given this login information, the source code, and the Handin container, your job is to explore cs666's course infrastructure and see what vulnerabilities you can find. For more information on the types of vulnerabilities we expect you to find, and specific requirements for the project and grading, please see the handout.

Remember: for this project, your goal is to find vulnerabilities in the course infrastructure/filesytem, so any attacks on the container infrastructure itself are out of scope and will receive no credit.

What does this mean?

The point of this project is to help you lean about operating system vulnerabilities, meaning we want you to behave as if you are a remote attacker who can only access Blue University's filesystem. Since the container runs on your own computer, you can easily use docker commands or other methods to "break in" and get a root shell, view files, etc.–these are outside the capabilities of the attacker we want to guard against (called the "threat model") and thus are NOT considered vulnerabilities.

Remember that you are being graded based on the OS vulnerabilities you find (and your explanations of them)–attacks out of this scope will not receive credit, and are not in your best interest for learning about OS vulnerabilities.

CS666's Ivy (not CS1660's Project 3!)

The handout describes an example assignment for cs666 called "Ivy", which is very similar to the Ivy project we completed for project 1. For our project, we'll attack cs666's autograder infrastructure to find vulnerabilities.

Note: You should not actually implement Ivy (again)! Your goal in this project is to exploit the vulnerabilities in cs666's course infrastructure to get yourself a good grade (or bring other students' grades down…) without actually implementing the assignment.

You know (from the handout) that Blue University decided to get students to submit assignments via a handin script, one very similar to the system sometimes used in CS courses at Brown (e.g. cs1650).

For cs666, the handin script is named cs666_handin, which lives inside the cs666 course directory. When you run cs666_handin, it invokes a setgid binary that does the following:

Saves all files in the current working directory in a .tar archive in the course’s handin directory–this creates a submission
Runs the cs666's autograder, which extracts the student's code from the submission and grades it. To do this, the autograding script combines the student submission with the support code, runs the compiler, and executes the resulting binary. For more info on what the student's submission should look like, see the CS666 Ivy handout, located in the appendix of our own handout
After running the tests, the autograde script adds the grades to a course-wide database file

For more details on how to run the autograder and what the output should look like, see the next section.

How to run the CS666 Ivy autograder

To hand in cs666's Ivy, you should run the cs666_handin script in a directory containing a file named KEY and the sol.go file we give you in the /home/alice/ivy-stencil directory. Again, you should not actually write a solution for Ivy in sol.go (but you might want to put something else there…). The autograder grades two things:

ivy: This is the code part: the autograder will take in your .go files, combine them with the support code, run the compiler, and ultimately run your code and check the output against an expected value.
KEY: The autograder also expects you to turn in a plain text file called KEY, which should contain the client's leaked wifi key. On submission, the autograder will check what's in your KEY file against the solution KEY for the user handing in (e.g. Alice) and assign a grade based on the comparison.

Hint: There are definitely some exploits you can craft to get yourself a full score on Ivy, by finding vulnerabilities with the autograder, your KEY file, and the solution KEY file…

So, handing in your "implementation" of cs666's Ivy (presumably, with some form of exploit called exploit.txt embedded) would look something like:

alice@c808c3104e5a:~/ivy-stencil$ pwd
/home/alice/ivy-stencil/
alice@c808c3104e5a:~/ivy-stencil$ ls      <------- making sure sol.go and KEY exist
exploit.txt  go.mod  KEY  main.go  Makefile  sol.go
alice@c808c3104e5a:~/ivy-stencil$ cs666_handin ivy
./
./go.mod
./sol.go
./Makefile
./exploit.txt
./KEY
./main.go
successfully handed in to /course/cs666/handin/ivy/alice/ff6f9982ba43c289a6a85701e2bc17bf.tar
invoking autograder...
Removing stencil file main.go
Executing:
modifydb --command grade -p "ivy:ivy" --points "0"  --comment "10 tries/0 successes" -s "alice" --files main.go sol.go
Executing:
modifydb --command grade -p "ivy:KEY" --points "0"  --comment "files do not match" -s "alice" --files main.go sol.go
alice@c808c3104e5a:~/ivy-stencil$

Resetting the container

As you attack Blue University's filesystem, you will inevitably make changes to the filesystems's state in ways that will break it–this is okay, and encouraged!

During this process, you'll find it useful to reset your instance of the filesystem to its default configuration to test out attacks once you've figured out a procedure, and to conduct different attacks that may conflict with each other.

To reset the Handin container:

Stop your Handin container by running exit in the terminal. Alternatively, you can run ./run-container stop from another terminal.
Run ./run-container --clean. This will delete your container instance and start a new container from a fresh copy of the image.
You're in the filesystem as Alice again!

Super important note: Note that any files or directories you add/delete/modify in the home directory (i.e. /home/alice/) will not be reset to their original state. This includes your Ivy stencil (/home/alice/ivy-stencil) or any configuration files you may modify like /home/alice/.bashrc. This is because /home/alice in the container is shared with your handin stencil repo, so the files here don't actually live inside the container!

If you want to reset these files, you'll need to do it manually–since they're part of your handin repo, we recommend periodically committing your changes to git so that you can roll them back if necessary.

If you want to download a fresh copy of any files, see this repo.

Hints and useful resources

This section describes some useful tools and hints you may find useful for certain exploits, including:

The OS lecture demos contain the demo code and example attacks used in lecture–look here for some example shell scripts with annotations
whoami/cs666_whoami: How to view your real/effective UID/GID, which is useful proving you can execute code with elevated privileges
Logging in as bob: another user for testing: Use this if you want to test exploits involving multiple users
Note on running programs from a setuid program: Read this if you're trying to arbitrary code execution!
Go modules example: How to build a go program with local modules
General Linux resources: See this section for some general resources on Linux, as well shell commands and man pages you may find especially helpful

`whoami` and `cs666_whoami`

The whoami Linux command allows a user to see the currently logged in user. If you run this command in the Handin container, you should see an output similar to this:

alice@c808c3104e5a:~$ whoami
alice
alice@c808c3104e5a:~$

There’s a binary in your container called cs666_whoami which is essentially a more powerful version of the normal whoami command. It prints the uid, euid, gid, and egid of the process that it runs as (and thus, by default, that its parent process runs as).

If you run cs666_whoami in your Handin container, you should see an output similar to this:

alice@c808c3104e5a:~$ cs666_whoami
uid:  1000
euid: 1000
gid:  1000
egid: 1000
alice@c808c3104e5a:~$

Note: Like the cs666_handin binary, while the cs666_whoami script resides in the /course/cs666/bin directory, you can simply run cs666_whoami anywhere in the container filesystem; you do not need to use the full path to the binary to use it.

This is a useful way to test your exploits as it allows you to prove that you can run a command with gain elevated privileges on Blue University's filesystem.

For example, if you wanted to prove that you are able to run arbitrary code as the TA group, you could run the cs666_whoami command wherever you've found a place to inject arbitrary code. When running your injected code, if the cs666_whoami command prints out a gid/egid of the TA group, then congrats! You've found a way to run arbitrary code as the TA group.

How to demonstrate proof: When we grade your demo videos, using the cs666_whoami command to show you are executing/accessing something as the TA group would be considered as enough proof that your exploit works.

When running an exploit, note that your terminal output doesn't need to be pretty–so long as we can see (from your video/or screenshots in your writeup) that the output of cs666_whoami is in there somewhere and that it's running with elevated privileges, this is enough for credit. No need to waste time pretty-printing the output!

Logging in as Bob: another user for testing exploits

To test exploits, you may want to perform actions as another user–to do this, you can log in as user bob, who is also a student in CS666. Bob's credentials are as follows:

Username:   bob
Password:  iambob

You can log in as Bob from any terminal in your handin container using the su command, like this:

Open a terminal in the handin container (ie, by running ./run-container from your handin repo). By default, this should log you in as alice.
Run the command su - bob, which asks to switch to Bob's user
Enter Bob's password, as above

You should now have a shell as Bob! As Bob, you can submit assignments and poke around the filesystem just like Alice.

Tip: Note that you can be logged in as both alice and bob at the same time! To do this, just start another terminal in your handin container by running ./run-container from your Handin repo.

Running programs from a setuid program

As we'll see in this assignment, setuid and setgid programs are tricky to write securely and should be treated with extreme care. This is especially true when the program is a shell script, since there are many ways to attack the program, as you'll find in this project…

Because setuid/setgid shell scripts are so dangerous, shells like /bin/bash by default have special protections that detect if they're being run with elevated privileges and resets them back to their default values. To show what we mean, let's say you managed to get a setuid program to run a shell script you wrote, like this:

// **** some_setuid_program.go ****
out, err = exec.Command("attack_script.sh").CombinedOutput()
if err != nil {
    panic(err)
}
fmt.Printf("%s\n", out)

###### attack_script.sh ######
#!/bin/bash

cs666_whoami  # <--- Won't run with elevated privileges!

This won't work because /bin/bash notices if it was launched with elevated privileges (ie, if uid != euid or gid != egid) and resets the process's EUID/GID back to the user's "real" UID/GID.

This is a good security feature, but, for trying to construct an attack, this is annoying! However, bash has a special execution mode we can use to preserve this behavior (called privileged mode, more info here) if we choose to use it. To turn on privileged mode, we need to run bash with -p in the interpreter line, like this:

###### attack_script.sh ######
#!/bin/bash -p  # <--- Turn on privileged mode

cs666_whoami    # <--- Runs with elevated privileges

Note: Notice that the autograder's main shell script autograder.sh turns on privileged mode because it relies on this behavior. This is probably a bad idea…

Importing go modules on your system

Note: The info here is pretty generic–it's up to you to figure out how to use it for this class!

Modern versions of Go (>1.13) use modules, which is a way to import and manage versions of library code in a project. In most Go programs, modules are downloaded from external repositories (eg. go get github.com/some/module), are checked for integrity, compiled locally, and then made available so you can import them into your code.

If you want to write your own go module, take a look at this example, which configures a go program ("main") to load an external library ("examplelib") that lives on your local system. See the example's README for instructions on how to do this in your own project. As an alternative, you could also publish your own library on Github, but we don't recommend this since it adds more steps to the process.

You may have read about the environment variable GOPATH (usually located at /home/user/go, which has similar goals. While GOPATH still exists, its use is deprecated in modern versions of Go (like the one used in our container)–creating libraries directly in GOPATH in this project may have unpredictable results and isn't likely to be worth your time.

Useful shell commands

Note: More coming soon!

General resources

Advanced Bash Scripting Guide: A comprehensive guide to just about every feature in Bash, with tons of examples and exercises! Also try this guide for more general principles on writing shell scripts.
CS0060 was an experimental course at Brown on "practical systems skills", which last ran in 2019. This course focused on teaching skills working with shells and Linux–eg. the "missing manual" that introductory courses often don't have time to cover. All of its lectures, notes, and recordings are freely available: if you want some general background resources, we highly recommend checking it out!

Useful man pages

The following manual pages may be helpful general resources for learning about some system functionality. You can find each man page by running the command below, or searching for the equivalent man page online:

man 7 credentials
man 7 symlink

Helpful commands

id -u: Get the effective user ID (UID)
getent group <group>: Get the GID for <group>
stat: Get detailed information about a given file or directory
htop or top: Live process viewer
watch: Execute a program repeatedly
strace: Trace system calls

Final notes

Congrats on reaching this point! This guide has provided an overview of the most important mechanics you need to work with the container for this project. If you have questions, please feel free to come to hours or post on Ed. Have fun, and happy hacking!

FAQs/Common issues

Windows line endings ("unexpected end of file")

If you see errors along the lines of bash: '\r': command not found or bash: /home/cs1660-user/.bash_profile: line 5: syntax error: unexpected end of file, see this section for common fixes.