# Homework 3: Scripting and Working Remotely ## Scripting ### Problem 1: File Handling Roy's downloads folder is in need of a reorganization as family photos, videos, documents, etc. have accumulated over the years. He needs your help to organize his files! 1. ***cs6-work:*** Roy first wants to get a good idea of what kinds of files are in his folder. Write a script that prints out all of the file extensions present in alphabetical order and their corresponding counts. Below is an example of output: ``` csv 2 docx 5 md 10 png 24 tiff 1 ``` #### Specs - your script must run using the following command: ``` ./count_extensions.sh <path to folder to count extensions of> ``` - you **must use associative arrays** in your solution - your script cannot edit the input folder or its contents in any way - you may assume that the input folder has no subfolders - the output should be in alphabetical order and be similar to the example provided above - you may assume that all of the files have an extension - test your script on an **empty directory**; nothing should happen (and your script should not error or crash) - hint: you may find [parameter expansion](https://wiki.bash-hackers.org/syntax/pe#substring_removal) helpful for parsing file extensions Save your script in `problem1/count_extensions.sh`. 2. ***cs6-work:*** Now Roy wants to actually organize his mess! Help him write a script that can organize the contents of the folder for him by sorting files by file extension. #### Specs - your script must run using the following command: ``` ./organize.sh <path to folder to organize> <path to folder to save organized files into> ``` - your script must **copy** files from the input folder to the output folder; the files in the input should not be changed, deleted, or renamed! - you may assume that the input folder no subfolders - you must create the output folder if it does not already exist - the output folder must be sorted in the following manner: - there will be only four subfolders (and no files) in this folder: `images`, `videos`, `documents`, `other` - `images` contains all of the files with the extensions `.png`, `.jpeg`, `.jpg`, `.gif` - `videos` contains all of the files with the extensions `.mp4`, `.mkv`, `.m4v`, `.mov` - `documents` contains all of the files with the extensions `.txt`, `.docx`, `.md`, `.csv` - `other` contains all files that do not belong to the other three folders (files with no extensions or with extensions that are not mentioned before such as `.tiff` belong here) - a subfolder should be empty if and only if there were no corresponding files that belong in it - you may assume that there is at least one file for each of the specified extensions for `images`, `videos`, and `documents` Save your script in `problem1/organize.sh`. ### Problem 2: Functions Create a text file called `problem2.txt` in a new folder called `problem2` in your CS6 homework environment (under the `hw3` folder). You will write your functions here. No shebang line is needed for this problem. Open the text file in vim and define the following functions in the file. To test your functions, run `source problem2.txt` in the same directory as `problem2.txt` and call your functions directly on the command line! In the examples, the `$` represents the prompt. #### ***cs6-work:*** (1) `roll` Just like rolling a die: generate a number (integer) between two given integers inclusively. If the first argument is larger than second, raise an error appropriately (print to `stderr` and return proper "exit code"). You may assume that the two arguments are integers of reasonable magnitude and that the larger integer is positive. The output numbers should be approximately evenly distributed within the user-given range. Example: ``` $ roll 1 6 3 $ roll 1 6 6 $ roll 3 2 error: first argument larger than second ``` Useful Tips: - You may find `$RANDOM` to be useful - `exit` will be helpful for returning the correct exit code #### ***cs6-work:*** (2) `add` Sum all arguments that are passed in. You may assume that all arguments are integers (do not worry about overflow or underflow). Don't forget to `echo` your output! Example: ``` $ add 1 2 3 4 5 15 ``` Useful Tips: - You can use special parameters like `$@` or `$*` to access all arguments (see `man` page of `bash` for more [details](https://tiswww.case.edu/php/chet/bash/bashref.html#Special-Parameters)) #### ***cs6-work:*** (3) `roll-add` It's time to use one of your previously defined functions! Pick a random integer between the first two arguments (inclusively), and do that the number of times specified by the third argument (which must be an integer). Output the sum of the randomly generated numbers. This should essentially be like `roll` but with an third parameter specifying the 'number of rolls.' The output numbers of the script must be reasonable (not constant given the same arguments). This script should have the same kind of error handling as `roll`, Example: ``` $ roll-add 1 6 4 12 $ roll-add 1 6 4 9 ``` Useful Tip: - You may find **arrays** and special parameters handy here! ## Working Remotely ### Problem 3 Please fill out the following table and specify for the following URIs the scheme, hostname, path and port: | URI | scheme | hostname | path | port | |-----|--------|----------|------|------| |hdfs://nn1:8020/user/tux/contetns.xml | | | | | |http://ec2-3-92-42-224.compute-1.amazonaws.com:8080/?query=login | | | | | | s3://tuplex/job100/part_0.csv | | | | 80 | | ssh://tux@gitice.com:2222/igloo/penguin.git | | | | | ### Problem 4 ***Gradescope:*** What do the following commands do? Answer in no more than a sentence or two. Feel free to either read through the man pages or try them out on your own computer. ``` 1. scp your_username@remotehost.edu:foobar.txt /some/local/directory 2. scp foobar.txt your_username@remotehost.edu:/some/remote/directory 3. scp your_username@rh1.edu:/some/remote/directory/foobar.txt your_username@rh2.edu:/some/remote/directory/ 4. scp -r your_username@remotehost.edu:some/directory /some/local/directory ``` ### Problem 5 After having worked as a user for most of the time, now it's time to finally become an admin! For this, we would like you to spin up your first, own server! * Make an account on github.com (if you don't already have one). * If you haven't yet, register for the student pack on https://education.github.com/pack. * Make an account on https://www.digitalocean.com/ by clicking on the GitHub button under "Deploy in seconds." * Authorize the GitHub connection (you may need to put in your GitHub credentials here). * Input your billing method. GitHub provides $50 of credit, so this project won't cost you anything. Digital Ocean needs a payment method in case you try to abuse their site. DO NOT USE PAYPAL UNLESS YOU WANT TO PAY 5 DOLLARS. If you do not have a card that you can use, shoot samuel_oliphant@brown.edu an email and he'll help you out. * Input information for your first project. Make the name "cs6" and put Class project / Educational purposes under "for." * Press start * Mouse over New Droplet and click Create Droplet * Ensure that the distribution is Ubuntu and the plan is "standard." * Scroll the pricing left until you find the $5/month plan and click it. * Make sure one of the NYC datacenter regions is selected. * Click "SSH key" under authentication. * Click "new SSH key" and follow the instructions to generate and input a new key pair for your computer. There is a tuturial in the modal. * When generating the key, DO NOT put in a passphrase. Leave it blank. Similarly, it's not necessary to input a file location, the default is fine. Just make a note of where it's saved. * You may make the hostname whatever you want. * Click Create Droplet. * Hooray! Your droplet is now live. Wait a few minutes for it to go online, then you can try to SSH in with the following: ```bash ssh -i <private key path> root@<droplet IP address> ``` * Optionally, you may add an alias to your .bashrc to make SSHing into your droplet a shorter command. * Within a business day or so, you should receive an email welcoming you to the GitHub student pack. Click on the link that reads https://education.github.com/pack/offers. * Scroll to the bottom and get the promo code for Digital Ocean. * Paste the promo code into https://cloud.digitalocean.com/account/billing and apply it. It should give you $50 of credit, or enough to run your droplet for almost a year! Yay! You're all set up. Now you have access to a server. Please leave your droplet running until grades for this homework are released. ***Gradescope:*** Please put the IP address of your droplet in your handin under problem 5. You have now a server where you have unlimited power, you're root! In general, it's a bad idea to work as root. It's very easy to accidentally break the computer. For the following steps, read up on `useradd` and `groupadd` using the man pages. 1. Create a user `tux` * Select `/bin/bash` as the shell using `-s` * Don't forget the password you use. 1. Create a group `penguins` 1. Add `tux` to `penguins` 1. Create another user `sealion` * Select `/bin/bash` as the shell using `-s` * Don't forget the password you use. 1. Switch user to being sealion using `su sealion` * To switch back to being root, just type `exit` 1. In sealion's home directory (`/home/sealion`), make a .txt file with a secret message (your favorite quote will do nicely). 1. Encrypt the .txt file using openssl AES-128-cbc with the password `tux`. 1. Make sure the permissions work out so tux can't read the file, but can `ls` sealion's home directory. `su tux` and make sure that `tux` can list the file using `ls`, but can't read its contents. 1. To give the course staff access to your droplet to grade, please add our public RSA keys to your droplet. They can be found [here](https://piazza.com/class/jzdb5qijlc1594?cid=72). Add `tux_rsa.pub` and `staff_rsa.pub` to `/home/tux/.ssh/authorized_keys`. Similarly, add `sealion_rsa.pub` and `staff_rsa.pub` to `/home/sealion/.ssh/authorized_keys`. ### Problem 6 ***Gradescope:*** Which of the following practices are a good idea for maintaining a secure connection over ssh? Justify each yes/no answer. 1. Once you've generated an ssh key, e-mail it to yourself so that you can access it on all of your machines. 2. When setting up an ssh server, use the default port for incoming connections. 3. Password-based login. 4. Key-based login.