# Lab 3: Tables

It's time to do our first multi-step data analysis! The best froggos are those who give out the best candy. To help your frog buddies choose the best candies to hand out, you've been given access to the results of a survey.

[FiveThirtyEight](https://en.wikipedia.org/wiki/FiveThirtyEight) conducted a survey in which tens of thousands of people were asked to choose between two candies; from the survey's responses, they computed the winning percentage of each candy. Then, they compiled a data-set with `Number` attributes like the winning percentage, relative price, and sugar percentage, and also `Boolean` attributes such as chocolate, fruity, caramel, and hard. This data has been published in a FiveThirtyEight article called [“The Ultimate Halloween Candy Power Ranking,"](https://fivethirtyeight.com/features/the-ultimate-halloween-candy-power-ranking/) which is definitely worth a read after the lab.

In this lab, you’ll be looking at the **relationships between these columns**.

## Setup

First, include this code at the top of your program:

provide *
provide-types *

include tables
include shared-gdrive("dcic-2021", "1wyQZj_L0qqV9Ekgr9au6RX2iqt2Ga8Ep")
include gdrive-sheets

# spreadsheet id from Google Sheets
ssid = "1XzeWZToT-lqPFVpp-RZLimdoGdGaTcVQZ_8VbDbAkOc"
data-sheet = load-spreadsheet(ssid)
candy-data = 
  load-table: name, chocolate, fruity, caramel, nutty, nougat, crisped-rice, 
    hard, bar, pluribus, sugar-percent, price-percent, win-percent
    source: data-sheet.sheet-by-name("candy-data", true)
  end

This code loads a Pyret table from a Google Sheet. Press the "Run" button in the top right corner to process the code in the definitions window. Then type `candy-data` in the interactions window to see the data as a table.

You’ll want to refer to the [CS111 Pyret Tables Documentation](https://hackmd.io/@cs111/table) for this lab (*not* the built-in Pyret documentation).

You may also want to refer to [our guide to making a plan](https://hackmd.io/@cs111/planning).

Suggestion: Use a comment-line as shown below to separate the code for each part. This can help you navigate your code file.

#-----------------------------------

## Part 1: Basic Table Filtering

Let's use the powers of filtering to learn more from our candy data. Recall the `filter-with` table operation from lecture: it takes an lambda expression that goes from a `Row` to `Boolean` as an input, and uses that expression to determine which rows to put in the output table.

**Task 1:** As a warmup to writing our lambda expression, let's write a computation on just one row. Use the code `candy-row = candy-data.row-n(0)` to get the first row of `candy-data`. Write an expression that computes whether the `sugar-percent` of `candy-row` is greater than 0.75. Verify for yourself that the value that this expression evaluates to is reasonable based on the value of the `sugar-percent` cell of `candy-row` by examining `candy-row` in the interaction window.

**Task 2:** We want to know which candies have the most sugar. Write an expression in the definitions window that produces a table containing only the candies where `sugar-percent` is greater than 0.75. The lambda expression in your `filter-with` will be your expression from Task 1, rewritten to use `r` instead of `candy-row` (this is because our lambda expression is just a function on every row `r` in our table, so we are going from the specific computation on a specific row in Task 1 to a computation on any input row `r`). [Here](https://https://hackmd.io/WulrblAaTqCwpZP2C2HzSQ) is a resource on lambda expressions you can use!

**Task 3:** Now that we know how to satisfy our sweet tooth, write an expression in the definitions window that produces a table containing only the candies where `price-percent` is greater than 0.90.

**Task 4:** How many of the candies have chocolate? Write an expression in the definitions window that outputs this number in the interactions window. You can get the length of a table by writing `<table>.length()` where `<table>` is the name of a table (or an expression that evaluates to a table).

## Part 2: Combining multiple table operations

Now that you've practiced basic table operations, let's practice computations that involve multiple table operations (or multiple uses of the same table operations). This is also a good opportunity to practice planning programs.

### Practice with Planning

The author of the article linked above was interested in combinations of ingredients. Let's say we want to figure out what proportion (percentage) of the candies with chocolate also have caramel. Before we try to *write* the code, let's step back and *plan* what we need to do.

We've seen planning (and to-do lists) in lecture. These next tasks are practice with that.

**Task 5:** Discuss with your partner what computations you will need for this problem: will you need to filter tables? Build columns? Compute lengths of tables? Create a plan on paper, based on our [How to Create a Plan](/t-b82a5vRM6JmXyDPWV7aQ) document.

---
### *CHECKPOINT:* Call over a TA once you reach this point.
___

**Task 6:** Using your plan as a guide, write an expression (not a function) to compute the proportion of chocolate candies that also have caramel.

**Task 7:** Now check for the proportion of chocolate candies that are nutty. Feel free to copy and adapt code that you've already written. How much did you have to change?

**Task 8:** Expecting that we'll want to compare several proportions of chocolate candies, let's write a function that generalizes the expressions from the previous two tasks.

This function will take in a `String` representing the name of the ingredient being paired with chocolate and a `Table` that is only candies that have the chocolate attribute. This function returns a number (which is the proportion of candies that have the given ingredient out of the candies that have chocolate).

Write a function `choc-candy-with-attr` that takes a table of candies and a string naming an ingredient. It returns the proportion of chocolate candies that also have the given ingredient. Figure out how to reuse your work on the previous two problems to help with this one! The function declaration should look like this:

`choc-candy-with-attr(table :: Table, attr :: String) -> Number:`

Experiment with your function in the interactions window (to make sure you know how to use this function).
___

### *CHECKPOINT:* Call over a TA once you reach this point.
___

## Part 3: Adding columns for analysis

For our next analysis, let's figure out the highest winning percentage of a candy that is both fruity and hard, but not a pluribus (pluribus means that there are multiple candies in a packet, e.g. Skittles, M&M’s, and Smarties)

**Task 9:** Develop a plan for this program on paper. Consider the various table operations we've seen, including `filter-with`, `length`, `build-column`, `order-by`, and `row-n`. *Hint: if you need a piece of information that isn't already in the table, considering computing a new column.*

**Task 10:** Extend the candy table with a `Boolean` column that indicates whether a candy is both fruity and hard, but not a pluribus.

**Task 11:** Finish executing your plan, ending with an expression that produces the highest winning percentage candy that is fruity, hard, and not a pluribus.

**Task 12:** Write another expression to compute the average winning percentage among these fruity-hard-single candies (use the `mean` function in our tables documentation).

___

### *CHECKPOINT:* Call over a TA once you reach this point.
___

## Part 4: Scatterplots and pie charts

**Task 13:** Recall from lecture that table columns can contain *categorical*, *numerical*, or *descriptive* data. With your partner, discuss which columns in `candy-data` are categorical and which are numerical.

Now, take a look at the [Plots and Charts section](https://hackmd.io/@cs111/table#Plots-and-Charts) of our Tables documentation. Answer the following questions *(Hint: think about how many column inputs each plotting function takes in, and why. Also think about whether these column(s) must be numerical and/or categorical.)*

- What is the difference between a freq-bar-chart and a bar chart?
- What is the difference between a freq-bar-chart and a histogram?
- What is the difference between a pie chart and a bar chart?
- What is the difference between a bar chart and a scatter plot?

**Task 14:** What’s the relationship between sugar and winning percentage? Do these two attributes seem correlated? One way to gain intuition on this is to create a scatterplot that puts one attribute on each axis. Look at the scatterplot documentation and try to figure out how to use it to generate a scatterplot of sugar versus winning percentage (it does not matter which variable goes on each axis). Write an expression in the definitions window that creates the scatterplot. In a comment, summarize the relationship in a sentence or two.

**Task 15:** Now, go back to the proportions of chocolate questions from Part 1. Using the `pie-chart` function in the tables documentation, generate a chart that shows the proportions of various other ingredients in candies with chocolate.

___

## Part 5: Misleading Data Visualizations

**Task 16:** In this lab we have created scatterplots and pie charts that visually represent the data attributes we have been working with. Now, we want to think about how the data could be misconstrued.

Below are examples of visual representations of data from [the Economist](https://www.economist.com/), which publishes about 40 charts weekly on their news platforms. [This article](https://medium.economist.com/mistakes-weve-drawn-a-few-8cdd8a42d368) from a visual data journalist at the Economist reveals that, at times, they get their visual representations of data incorrect. With your lab partner(s), consider how some charts could be misleading and how this could be avoided. Come up with 1-2 reasons explaining why the original chart (labeled “original”) was misleading, and what revisions were done to create a better chart (labeled “better”).

***Please write down your answers and share them with the TA.***

### 1.Truncating the Scale:

### 2.Cherry-picking Scales:

### 3.Choosing the visualization method:

## Part 6: Comparing Attributes

***This is a discussion and planning question -- you are NOT being asked to write code to do this analysis. In a comment, write a few sentences to summarize your ideas.***

Imagine that we want to understand the relative frequencies of the attributes in the column names. Specifically, we want to be able to choose two attributes (column names) and find out which is more frequent within the entire table.

**Task 17:** Discuss with your partner how you might go about testing this function. Write a comment describing your testing plan (what tables would you make, what rows would you need, etc).

### *CHECKPOINT:* Call over a TA once you reach this point.
___

## Part 7: Exploration

**Task 18:** Now think about questions you have about this data, or come up with interesting questions to ask. Investigate (by writing and running expressions), and write a couple sentences in a comment about what you found.

## Takeaways
This lab has mostly been about getting you comfortable working with tabular data, practicing some common operators on tables, and planning functions. It also gets you thinking about our course's focus on data: what patterns of manipulating data do we often use in computations? How does the organization of our data impact our ability to answer these questions?

Here, we see that filtering, ordering, and summarizing data are some of the key operations. So far we’ve only looked at these operations on tabular data, but these same building blocks will arise many times through this course. When you have a computation to perform around data, you should start by thinking through what combinations of filtering, sorting and summarizing will help you compute your answer.

------

> Brown University CSCI 0111 (Spring 2024)
> Feedback form: tell us about your lab experience today [here](https://forms.gle/WPXM7ja6KwHdK8by6)!