Lab 5: Java IO

--- tags: labs, spr22 --- # Lab 5: Java IO  ## Learning Objectives In this lab, you will: * learn to read from and write to files, buffers, and streams in Java * learn while loops (if you haven't already seen them) * get some practice with try/catch and exceptions **Where does this fit into the course?**: Basic input/output skills are essential in programming. Later assignments will have you work more with input/output to help you practice beyond this lab! ## Setup Before beginning this assignment, make your personal repository for this lab using the following GitHub Classroom link: https://classroom.github.com/a/YqqxTFkq You will be referring to this source code throughout the lab. Open your repository in IntelliJ to get started! ## Intro Alice just won a brand new iPod shuffle and wants to show off her cool and different taste in music at school. When she starts downloading songs to it, however, she's met with her worst nightmare: an exception that stops her efforts! Hoping to download the newest and coolest pop anthems before class, Alice puts the problem solving skills she's acquired from CS200 to use: it's time to dive into the Java IO Library! ## The Java IO Library The Java Application Program Interface (API) is the set of interfaces of the Java libraries. In this lab, you will learn about the `java.io` package, a standard Java library. IO stands for input/output, and this library handles input and output for Java programs. (This lab is self-contained, but for reference, here is the [`java.io` package summary](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/package-summary.html).) **Side Note:** *If you look at the documentation, you will notice that there are **lots** and **lots** of classes in this library. Don't spend too much time trying to understand everything in this package in detail right now; learning to read library documentation takes practice. But do come back to this later, because this particular skill (which requires an investment up front) may save you more time than any other in your future computer science endeavors. You will also need the `java.io` library in your assignments in CSCI0200.* In today's lab, you will learn about program arguments and how to use `java.io` library to read from and write to files and their cousins (buffers and streams). You will be interacting with the following seven `java.io` classes: `FileReader`,`FileWriter`, `BufferedReader`, `BufferedWriter`, `InputStreamReader`, `OutputStreamReader`, and `File`. Of primary interest to us will be the methods for working with these classes. ## Calling Programs with Arguments Much like how functions and methods take arguments as input, programs can also take arguments as input when you run them. Programs accept inputs through the `main` method's argument, `String[] args`, which declares that your program takes as input an array of `String`s called `args`. It is convention that any input to the program is passed in the form of this `String` array. Before this lab, the programs you have written in this class haven't taken in any arguments, so the `args` array has always been empty (and we have always ignored it). The `args` array refers to any strings that might be handed to a program before it is run. What sort of input might we want to give to a program when running it? We might provide the name of a data file, a number saying how many list elements to expect, or a word like "array" or "link" indicating which list data structure to use for a particular run of a program. While you can also provide such inputs in constructors and the like, sometimes you need to let users who don't have access to the code (as you do) give arguments. That's what we're enabling here. Here is an example of a class, `PrintArgs`, with only a single method `main`, that simply prints its program arguments. ```java= public class PrintArgs { public static void main(String[] args) { for (String arg: args) { System.out.println(arg); } } } ``` [comment]: <> (lila added this in, someone proofread please) Our first goal is to understand how you would run this program with arguments. ### The Command Line When you use the Run button in IntelliJ, Java checks your program for syntax errors, converts it to a form that it can run, then runs the converted version. The act of converting the code is called *compiling*. It is possible to compile your code directly, rather than through IntelliJ. We do this using two programs on our computer: `javac` (error checker and compiler) and `java` (runs the compiled program). You can access and use both of these commands through a *terminal window*. Running a program in terminal is referred to as "running from the command line". In the following example, we demonstrate compiling `PrintArgs` from the command-line and running it with the arguments `hi`, `there`, and `everybody`. When running, Java reads the arguments in as `String`'s, even though they we don't input them with quotations. With the input `hi there everybody`, the values of the the 0th, 1st, and 2nd strings in the `args` array are set to the strings `"hi"`, `"there"`, and `"everybody"`, respectively. Hence, when we run the program, we would see these words are printed out one after another as seen below. ``` hi there everybody ```  ### Providing Command-Line arguments in IntelliJ You can supply arguments directly through Intellij when running a program. To run your program with arguments in IntelliJ, go to `Run` $\rightarrow$ `Edit Configurations`, or follow the image below. ![](https://i.imgur.com/FzOdhdo.png) On the left hand menu, make sure to select the program you want to run. Then, type the names of your arguments in the `Program Arguments` box (separated by spaces), click `Apply`, and then run your program. Note: ignore the `java 11 SDK` in the image. You should still be using Java 17. ![](https://i.imgur.com/U9zC4sB.png) ::: spoiler If you're not able to find edit configurations or add program arguments when you look at `Edit Configurations`, click here! When you open`Edit Configurations` and you might see something like this: ![](https://i.imgur.com/nTccNOb.png) To solve this, you can run your program by clicking the green arrow next to a `main` method to force IntelliJ to create a configuration for you. For example, if you run the main method in the `PrintArgs` class, when you open `Edit Configurations` again you should see something that looks like this: Note: ignore the `java 13 SDK` in the image. You should still be using Java 17. ![](https://i.imgur.com/rCjAMIb.png) You can then add your program arguments in the box that says `Program arguments`. ::: \ **Task**: Open the `PrintArgs` class in the provided lab files and try running it in Intellij with different arguments. :::warning Unable to run the PrintArgs class? Try the following: 1. Create an out folder (on the same level as the sol and src foldl) 2. Copy the full path to that folder 3. Paste the full path into the compiler output field in File > Project structure 4. File > invalidate caches & restart (last step is important) ::: ## Readers, Writers, and Buffers Now, let's work towards some specific tasks. We're going to have you work on two tasks: the first will copy the lines of one file over to a new file, while the second will keep reading in words from a user in terminal and write them all out to a file. To do these, you will need to know: - how to read a line from a file - how to keep reading lines until the file ends - how to read input from a user through the terminmal - how to write to files - how to handle exceptions that might arise in the process We've put some details in spoilers to keep the handout manageable, while letting you look up what you need. Here also are links to the Java documentation that you might need: [FileReader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/FileReader.html), [FileWriter](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/FileWriter.html), [BufferedReader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/BufferedReader.html), and [BufferedWriter](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/BufferedWriter.html). ### Handling IO Exceptions A lot of things can go wrong when working with IO. Methods can easily encounter unexpected input or problems. It is your responsibility as a programmer to handle these exceptions gracefully, and protect the user from unexpected behavior and program crashes. In Java, exceptions are handled with a `try`/`catch` block. The `try` block executes first and contains any code that may potentially throw an exception. If an exception is thrown in the `try` block **and** that type of exception is caught in the `catch` block, the `catch` block will execute. One exception you may come across when working with IO is the `IOException` which is thrown from many methods that handle input and output. In order to use readers and writers while checking for this exception, you might do something like this: ```java= try { BufferedReader reader = new BufferedReader(new FileReader("/path/file")); String line = reader.readLine(); System.out.println(line); reader.close(); } catch (IOException e) { System.out.println("Encountered an error: " + e.getMessage()); } ``` This code catches the `IOException` and handles it, in this case by informing the user that the program encountered an error. Note that if in the `try` block, an exception *other* than an `IOException`, such as an `IllegalArgumentException`, had instead been thrown, the `catch` block would *not* have caught that exception. You can take a look at other possible IO exceptions in the [`java.io` package summary](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/package-summary.html). ### Concatenate/Copy  Concatenating the contents of multiple files into a new file is a useful programming tool (called `cat` in Linux/MacOS) . Here, we'll build a piece of that program, which copies the lines of one file into a new file. You'll be working in the `sol/Cat.java` file for this problem. **Reading/writing from/to a file** consists of a few basic steps: * Construct a `FileReader`/`FileWriter` object with a filename (including its path) as a String argument. A `FileReader` will read from this file and a `FileWriter` will write to this file. * Construct a `BufferedReader`/`BufferedWriter` from this `FileReader`/`FileWriter`. * Read from/write to the file using the `BufferedReader` / `BufferedWriter`. * Close the reader/writer (this tells Java you are done working with the file). Here's an example of opening a file for reading: ```java FileReader fReader = new FileReader("/Users/lilazimbalist/Desktop/cs0200/lab05-java-io-lzimbalist/songs/FireworkParody.txt"); ``` **Note:** Ensure that you use the full path of the file as the filename. *(If you want to use a so-called "relative" path that assumes Java is working in a specific directory, you have to make sure that directory is set up in the Run > Edit Configurations in IntelliJ.)* ::: spoiler **Don't know the full filepath to a certain file?** No worries, IntelliJ can help us out with that! Follow the steps below to copy the full filepath to any file. 1. Right click on the file that you want the path to in IntelliJ 2. Choose "Copy path...", you should see this ![](https://i.imgur.com/gy7aIGz.png) 3. Click on "Absolute Path" to copy it That's it! Not so bad after all :) ::: <br> `BufferedReader`s manipulate data in chunks for performance reasons. We create these from `FileReaders`: ```java BufferedReader bReader = new BufferedReader(fReader); ``` :::spoiler **More details on buffers (for those who are interested)** When you read and write data, you are reading from and writing to computer memory, calling the underlying OS directly. This is computationally expensive. **Buffering** speeds up the transfer of data by limiting the number of calls to these expensive computations. A buffer acts as a temporary data store so that data can be read or written in batches from the buffer rather than the underlying data stream. Essentially, rather than reading or writing every single piece of data from the data stream to another part of memory individually, we can store larger chunks of it in a buffer, and perform calls to the OS in bulk. A buffered input stream reads input into an internal array, refilling the buffer when it is empty. A buffered output stream similarly writes data to a buffer, then moves the data to its desired location in one go: this process is called *flushing*. For example, if you are writing to a file, the writer will not write each individual line to the file. Instead, it will wait until many lines are waiting to be written and, at that point, write all of those lines to the file at once. ::: #### Reading a file line-by-line Once you have a `BufferedReader`, you read a file line-by-line as follows: ```java= // read the first line String line = bReader.readLine(); // as long as the line isn't empty while (line != null) { // do whatever you want with the line (here, we're printing it) System.out.println(line); // get the next line from the file line = bReader.readLine(); } // close the file when we're done using it (so other programs could use it) bReader.close(); ``` :::spoiler **What is `while` and how does it work?** So far in class we have worked with `for` loops, which are used for repeating a computation a pre-determined number of times (say once per element in a list). Sometimes, we need to repeat a computation but we don't know how many times we need to repeat it upfront. We can, however, recognize when we are done repeating the computation. `while` loops are for the latter situation. The general pattern is: ```java= while (<not yet finished with repeats>) { // do the computation // advance to the next item } // after done with repetitions ``` Compare this to the sample code: we have a computation to repeat as long as there are more lines in the file. We don't know how long the file is up front, so we can't use a `for` loop. Instead, we keep reading lines (getting the next data) and printing them (the computation to repeat) as long as we haven't run out of lines (which is signaled by the line contents being `null`.) As a general rule, use `for` loops when you know up front how many repetitions are needed and `while` when you do not. ::: \ **Task:** In the `Cat` class in the stencil, fill in the `cat` method so that it will read in and print out (as in the above example) the contents of the given `BufferedReader`. **Task:** Edit the `main` method in `Cat` to create both a `FileReader` and its corresponding `BufferedReader`. Pass the `BufferedReader` to the `cat` method. Use `null` as the second argument to `cat` (for now). **Task:** Make sure you have this much working by running your program (either at the terminal or in the IntelliJ configuration) with one of the files in the `songs` directory of the stencil as input (use some string like "dummy" for the second input for now). :::spoiler Having trouble finding files or getting `FileNotFoundException`? If you're having trouble finding the files (i.e. getting a `FileNotFoundException`), try using the **absolute path**. This is a path that starts with `/` (MacOS/Linux) or `C:\` (or some other letter, Windows). Take a look at the "Don't know the full filepath to a certain file?" dropdown above to easily get the path! ::: ### Writing to File Now that you can read in the contents of a file, let's write those contents out to another file. Similar to `FileReader` and `BufferedReader`, we use analogous classes `FileWriter` and `BufferedWriter`. ```java= FileWriter fWriter = new FileWriter("/course/cs0200/lab05-java-io/file.txt") BufferedWriter bWriter = new BufferedWriter(fWriter); ``` The `write` method of a `BufferedWriter` writes to the file: ```java= bWriter.write("I am writing to a file!"); // you can write many things here // when you are done writing, include the following two lines to clean up bWriter.flush(); bWriter.close(); ``` **Task:** Modify your `cat` method to write each read-in line to the `BufferedOutput` file, rather than to the screen (as `System.out.println` does). **Task:** Modify the `main` method in `Cat` to create a `FileWriter` and `BufferedWriter` from the second argument, passing those to the `cat` method (instead of the `null` that we used earlier). **Task:** Try the program from the command line (or Run Configuration), this time providing the name of the file to write to as well as the file to read from (be sure to use a new filename, or you'll overwrite the contents of the existing filename). This means replacing the "dummy" from the reading part with an actual filename. Look at the content of the output file to make sure you're seeing the output that you expect. (Within a terminal, you can use the commands `less` or `more` (on MacOS/Linux) or `type` (on Windows) to do this.) *(Note: if the lines in your output file have all run together, rather than each input line starting on a new output line, insert a call to `BufferedWriter`'s `newLine()` method after each `write` command or add a `\n` to the end of the String you're writing.)* **Task:** We have provided you a test class to test your `cat` method; be sure to use it to ensure that your method works! **Fill in the paths with files that you test with.** We also provided sample text files of some throwback party anthems for you to test with (in the `songs` folder). :::success **Checkpoint**! Call over a TA to check your work. ::: ### Handling Exceptions Various things can go wrong when reading files, such as trying to read from files that don't exist (maybe due to a type in the filename) or trying to continue reading after the file has run out of lines. Let's practice `try/catch` blocks by checking for "file not found" errors within our `cat` program. If you give the `FileReader` constructor a path/filename that don't exist, Java will throw a `FileNotFoundException`. **Task:** Try running your `Cat/main` program with an invalid filename in the first argument. Observe the exception. Let's use a `try/catch` block to provide a more polite error message to a user. Specifically: **Task:** In `Cat/main`, put the call to the `FileReader` constructor in a `try` block. The corresponding `catch` should check whether a `FileNotFoundException` was thrown. If so, `main` should print out a message that says ``` "I'm sorry, but I can't find file <the input filename>" ``` Look back at the lecture notes on exceptions to see how to do this.  ###   ### Fun With Words Now, let's go beyond working with file and switch to working with user input: we want to let users type in a sequence of words in the terminal or console, and we'll write out the entire sequence to a file. For this problem, we'll work in the `Doc` class in the stencil. In class, we have used `System.out` to print to the console or terminal window. Java has an analogous `System.in` for reading in user input. We can use `System.in` (rather than a `FileReader`) to create a `BufferedReader`, then proceed as we did with the `cat` program. Here's how to set up a `BufferedReader` from user input: ```java BufferedReader reader = new BufferedReader(new InputStreamReader(System.in)); ``` **Task:** Write a method, ```toDocument``` inside of the `Doc` class, that repeatedly prompts the user for a word from the console, and writes those words to a file, with a space separating each word, until `null`/`EOF` is reached (at which point `toDocument` should just close the file. The code for this will look similar to what you did in the `Cat` class, so this is really just giving you a chance to really understand that code and apply the pattern again to a new problem. **Task:** Fill in the `main` method in the `Doc` class, which takes in an argument with the path of the file to write to. `main` should create an appropriate writer (which writes to the file that was specified) and pass that to `toDocument`. **Task:** Try out your program. When you are ready to stop giving new words, you have to communicate this to Java: - if you are sending input in a terminal command line, you do this by typing `Ctrl-D` (Java converts `Ctrl-D` to `null`, which is then interpreted as "end of file" (or `EOF`). - if you are sending input within the IntelliJ console, you need a different approach because IntelliJ uses`Ctrl-D` as a debugging signal. On MacOS use `Cmd-D` as a substitute, on Windows use `Ctrl-Z-Enter`. #### A Bit More on IO Exceptions Whenever we open files while programming, we have to make sure to close them. If we don't, and another program tries to work with the same program, conflicts can arise that corrupt file contents. What happens, however, if we get an exception that halts our program before we get to close the files? Because this is a real issue in practice, Java provides an alternative form of `try` that closes open files should an exception be thrown. If you want to understand more, expand the following spoiler. #### try-with-resources Java's special type of `try` statement is called [`try`-with-resources](https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html). From the documentation, "a resource is an object that must be closed after the program is finished with it" (such as a `BufferedReader`). The `try`-with-resources statement ensures that any resource given to it will be closed at the end of the statement. So we can sleep soundly knowing that our reader will be closed when we are finished. The following code demonstrates how to use `try`-with-resources: ```java= try (BufferedReader reader = new BufferedReader(new FileReader("/path/file"))) { String line = reader.readLine(); System.out.println(line); } catch (IOException e) { System.out.println("Encountered an error: " + e.getMessage()); } ``` By supplying a `BufferedReader` as an argument to `try`, Java will automatically close it at the end of the statement. Furthermore, multiple resources can be supplied to a `try` statement, and all of them will be closed! In order to supply two resources to a `try`-with-resources, just separate the declarations with semicolons. #### :::success **Checkpoint**! Call over a TA to check your work. ::: ## More Practice with While Loops (Optional/for anyone who needs/wants it) While loops are common for tasks that involve user input or an unpredictable number of iterations. If you are new to `while` loops and want more practice, try writing the following programs for practice (you can create a new `WhilePractice.java` class for this): **Task:** Write a program that prints the elements in the prefix of an array for which all numbers are larger than 0. We've gotten you started with the following code: ```java= public void posPrefix() { int[] theArr = {5, 6, 2, -1, 4}; int i = 0; while (theArr[i] > 0) { // finish the code here } } ``` **Task:** A theater uses an array of booleans to track which seats have been reserved for a performance. If `seatsArray[i]` is true, then the seat is reserved, otherwise it is free. Write a method `numFree` that takes an array index and returns the number of consecutive free seats starting from the given index (don't worry about free seats before the index). ## Just For Fun: CSV Parsing (Optional) If you finish early, or later want to know how to apply what we've learned to read CSV files, look at the following optional materials. :::spoiler CSV Parsing ### CSV Files CSV stands for comma-separated value. A CSV file contains rows of data, each row containing several datapoints separated by commas. You can think of a CSV as a spreadsheet: each datapoint in a row is the value in one cell of that row. We've provided the CIT elevator logs in the lab05 repo. The swipe data you've been given has three columns. The first contains the name of a person who swiped into the upper floors of the CIT on Sunday. The second contains the time they swiped into the elevator to go up, and the third contains the time they swiped into the elevator to go down. These times are stored in a standard CS format called Unix time, meaning they are stored in the number of seconds they occurred after midnight on January 1, 1970. We want you to look for everyone in the upper floors of the CIT, on February 16, 2020 at 2:38 PM, or in Unix time, at 1581881945, i.e. everyone who entered before and exited after this timestamp. The file is clearly too long to look through yourself. You know how to read through a file using a BufferedReader, so finding this datapoint should be manageable, but it might take a while to write. Luckily for you, CSV parsing is an incredibly common task, so there are packages that provide CSV parsers for you. You can just use one of these to quickly parse the dataset! ### Setting up the Apache CSVParser To run this project, you'll need to import the `org.apache.commons:commons-csv:1.0` from maven into your IntelliJ project. To do this, go to `File` $\rightarrow$ `Project Structure`, select `Modules` on the left sidebar, select the `Dependencies` tab, then click `+`. From there select `Library` and then you may need to select `New Library`, then `From Maven`. Then go ahead and enter `org.apache.commons:commons-csv:1.0` into the search bar and select the option from the dropdown (you may need to click search and then click the dropdown button). Select all four boxes for Transitive Dependencies, Javadocs, Sources and Annotations. After that you can just click through to install! ### Using the Apache CSVParser In order to parse our CSVs, we'll be working with the Apache CSVParser. Using a new package can be overwhelming at first, so we'll walk you through the process of using it. First, you'll need to import the classes we'll be using. **Task:** Create a class called `SuspectFinder`. At the top, import `org.apache.commons.csv.CSVFormat`, `org.apache.commons.csv.CSVParser`, and `org.apache.commons.csv.CSVRecord`. Create a method called `findSuspects` that takes in a file name and returns a `List<String>` containing the names of anyone in the CIT at the time of the crime. First, we'll have to tell the parser what specific format our CSV file is in, since there are multiple different valid formats. We do this through the use of a `CSVFormat` object. **Task:** Create a variable `format` of type `CSVFormat` in `findSuspects`. We want to use the standard CSV format, which is called `RFC4180`. This format is built in as a static field of `CSVFormat`: `CSVFormat.RFC4180`. However, this isn't enough for our format. The first row of the CSV you're using contains the names of each column in the dataset. This is called a `header`, and need to be accounted for in our format. Luckily, this is easy in Apache as well. All we'll have to do is set `format` to `CSVFormat.RFC4180.withHeader()`. Now, we'll need to set up our `CSVParser`. The constructor of `CSVParser` takes 2 arguments: a `FileReader` of the file you want to parse, and the `CSVFormat` object you just created. **Task:** Create a variable `parser` of type `CSVParser` in your method. Now, the `CSVParser` can parse the file into a series of `CSVRecord`s, one for each row, which in turn each act like a list of the `String`s from their row. You can find the documentation for `CSVParser` [here](https://commons.apache.org/proper/commons-csv/apidocs/org/apache/commons/csv/CSVParser.html) and the documentation for `CSVRecord` [here](https://commons.apache.org/proper/commons-csv/apidocs/org/apache/commons/csv/CSVRecord.html). **Task:** Finish the method, returning the names of everyone who was in the CIT at the specified time. ::: ______________________________ *Please let us know if you find any mistakes, inconsistencies, or confusing language in this or any other CSCI0200 document by filling out the [anonymous feedback form](https://docs.google.com/forms/d/e/1FAIpQLSdFzM6mpDD_1tj-vS0SMYAohPAtBZ-oZZH0-TbMKv-_Bw5HeA/viewform?usp=sf_link).*