# Lab 3: Tables

It's time to do our first multi-step data analysis! Everyone knows bees love honey, but did you know that they also love sweets? To help your bee buddies choose the best candies to pick, you've been given access to the results of a survey.

[FiveThirtyEight](https://en.wikipedia.org/wiki/FiveThirtyEight) conducted a survey in which tens of thousands of people were asked to choose between two candies; from the survey's responses, they computed the winning percentage of each candy. Then, they compiled a data-set with `Number` attributes like the winning percentage, relative price, and sugar percentage, and also `Boolean` attributes such as chocolate, fruity, caramel, and hard. This data has been published in a FiveThirtyEight article called [“The Ultimate Halloween Candy Power Ranking,"](https://fivethirtyeight.com/features/the-ultimate-halloween-candy-power-ranking/) which is definitely worth a read after the lab.

In this lab, you’ll be looking at the **relationships between these columns**.

## Setup

First, include this code at the top of your program:

provide *
provide-types *

include tables
include shared-gdrive("dcic-2021", "1wyQZj_L0qqV9Ekgr9au6RX2iqt2Ga8Ep")
include gdrive-sheets

# spreadsheet id from Google Sheets
ssid = "1XzeWZToT-lqPFVpp-RZLimdoGdGaTcVQZ_8VbDbAkOc"
data-sheet = load-spreadsheet(ssid)
candy-data = 
  load-table: name, chocolate, fruity, caramel, nutty, nougat, crisped-rice, 
    hard, bar, pluribus, sugar-percent, price-percent, win-percent
    source: data-sheet.sheet-by-name("candy-data", true)
  end

This code loads a Pyret table from a Google Sheet created by the course staff, so there's no need to create your own data. Please copy the code above and press the "Run" button in the top right corner to process the code in the definitions window. Then type `candy-data` in the interactions window to see the data as a table.

You’ll want to refer to the [CS111 Pyret Tables Documentation](https://hackmd.io/@cs111/table) for this lab (*not* the built-in Pyret documentation).

You may also want to refer to [our guide to making a plan](https://hackmd.io/@cs111/planning).

Suggestion: Use a comment-line as shown **bee**-low to separate the code for each part. This can help you navigate your code file.

#-----------------------------------

## Part 1: Basic Table Filtering

<figure style="text-align: center;">
<img src="https://graphicsfactory.com/clip-art/image_files/webp/1/1751431-bob-the-bee-eating-candy.webp" alt="weird bee with lollipop" width="80%">
<figcaption><em>Image hosted by Graphics Factory.com</em></figcaption>
</figure>

Let's use the powers of filtering to learn more from our candy data. Recall the `filter-with` table operation from lecture: it takes an lambda expression that goes from a `Row` to `Boolean` as an input, and uses that expression to determine which rows to put in the output table.

**Task 1:** As a warmup to writing our lambda expression, let's write a computation on just one row. Use the code `candy-row = candy-data.row-n(0)` to get the first row of `candy-data`. Write an expression that computes whether the `sugar-percent` of `candy-row` is greater than 0.75. Verify for yourself that the value that this expression evaluates to is reasonable based on the value of the `sugar-percent` cell of `candy-row` by examining `candy-row` in the interaction window.

**Task 2:** We want to know which candies have the most sugar. Write an expression in the definitions window that produces a table containing only the candies where `sugar-percent` is greater than 0.75. The lambda expression in your `filter-with` will be your expression from Task 1, rewritten to use `r` instead of `candy-row` (this is because our lambda expression is just a function on every row `r` in our table, so we are going from the specific computation on a specific row in Task 1 to a computation on any input row `r`). [Here](https://hackmd.io/WulrblAaTqCwpZP2C2HzSQ) is a resource on lambda expressions you can use!

**Task 3:** Now that we know how to satisfy our sweet tooth, write an expression in the definitions window that produces a table containing only the candies where `price-percent` is greater than 0.90.

**Task 4:** How many of the candies have chocolate? Write an expression in the definitions window that outputs this number in the interactions window. You can get the length of a table by writing `<table>.length()` where `<table>` is the name of a table (or an expression that evaluates to a table).

## Part 2: Combining multiple table operations

Now that you've practiced basic table operations, let's practice computations that involve multiple table operations (or multiple uses of the same table operations). This is also a good opportunity to practice planning programs.

### Practice with Planning

The author of the article linked above was interested in combinations of ingredients. Let's say we want to figure out what proportion (percentage) of the candies with chocolate also have caramel. Before we try to *write* the code, let's step back and *plan* what we need to do.

We've seen planning (and to-do lists) in lecture. These next tasks are practice with that.

**Task 5:** Discuss with your partner what computations you will need for this problem: will you need to filter tables? Build columns? Compute lengths of tables? Create a plan on paper, based on our [How to Create a Plan](/t-b82a5vRM6JmXyDPWV7aQ) document.

---
### *CHECKPOINT:* Call over a TA once you reach this point.
___

**Task 6:** Using your plan as a guide, write an expression (not a function) to compute the proportion of chocolate candies that also have caramel.

**Task 7:** Now check for the proportion of chocolate candies that are nutty. Feel free to copy and adapt code that you've already written. How much did you have to change?

**Task 8:** Expecting that we'll want to compare several proportions of chocolate candies, let's write a function that generalizes the expressions from the previous two tasks to find the proportion of chocolate candies that have a specific attribute.

Write a function `choc-candy-with-attr` that takes a string naming an ingredient. It returns the proportion of chocolate candies that also have the given ingredient. Figure out how to reuse your work on the previous two problems to help with this one! The function declaration should look like this:

`choc-candy-with-attr(attr :: String) -> Number:`

Experiment with your function in the interactions window (to make sure you know how to use this function).
___

### *CHECKPOINT:* Call over a TA once you reach this point.
___

## Part 3: Adding columns for analysis

For our next analysis, let's figure out the highest winning percentage of a candy that is both fruity and hard, but not a pluribus (pluribus means that there are multiple candies in a packet, e.g. Skittles, M&M’s, and Smarties)

**Task 9:** Develop a plan for this program on paper. Consider the various table operations we've seen, including `filter-with`, `length`, `build-column`, `order-by`, and `row-n`. *Hint: if you need a piece of information that isn't already in the table, considering computing a new column.*

**Task 10:** Extend the candy table with a `Boolean` column that indicates whether a candy is both fruity and hard, but not a pluribus.

**Task 11:** Finish executing your plan, ending with an expression that produces the highest winning percentage candy that is fruity, hard, and not a pluribus.

**Task 12:** Write another expression to compute the average winning percentage among these fruity-hard-single candies (use the `mean` function in our tables documentation).

___

### *CHECKPOINT:* Call over a TA once you reach this point.
___

## Part 4: Scatterplots and pie charts

**Task 13:** Recall from lecture that table columns can contain *categorical*, *numerical*, or *descriptive* data. With your partner, discuss which columns in `candy-data` are categorical and which are numerical.

Now, take a look at the [Plots and Charts section](https://hackmd.io/@cs111/table#Plots-and-Charts) of our Tables documentation. Answer the following questions *(Hint: think about how many column inputs each plotting function takes in, and why. Also think about whether these column(s) must be numerical and/or categorical.)*

- What is the difference between a freq-bar-chart and a bar chart?
- What is the difference between a freq-bar-chart and a histogram?
- What is the difference between a pie chart and a bar chart?
- What is the difference between a bar chart and a scatter plot?

**Task 14:** What’s the relationship between sugar and winning percentage? Do these two attributes seem correlated? One way to gain intuition on this is to create a scatterplot that puts one attribute on each axis. Look at the scatterplot documentation and try to figure out how to use it to generate a scatterplot of sugar versus winning percentage (it does not matter which variable goes on each axis). Write an expression in the definitions window that creates the scatterplot. In a comment, summarize the relationship in a sentence or two.

**Task 15:** Now, go back to the proportions of chocolate questions from Part 1. Using the `pie-chart` function in the tables documentation, generate a chart that shows the proportions of various other ingredients in candies with chocolate.

___
### *CHECKPOINT:* Call over a TA once you reach this point.
___

## Part 5: Misleading Data Visualizations

**Task 16:** In this lab we have created scatterplots and pie charts that visually represent the data attributes we have been working with. Now, we want to think about how the data could be misconstrued.

**Bee**-low are examples of visual representations of data from [the Economist](https://www.economist.com/), which publishes about 40 charts weekly on their news platforms. [This article](https://medium.economist.com/mistakes-weve-drawn-a-few-8cdd8a42d368) from a visual data journalist at the Economist reveals that, at times, they get their visual representations of data incorrect. With your lab partner(s), consider how some charts could be misleading and how this could be avoided. Come up with 1-2 reasons explaining why the original chart (labeled “original”) was misleading, and what revisions were done to create a better chart (labeled “better”).

***Please write down your answers and share them with the TA.***

### 1.Truncating the Scale:

### 2.Cherry-picking Scales:

### 3.Choosing the visualization method:

## Part 6: Comparing Attributes

***This is a discussion and planning question -- you are NOT being asked to write code to do this analysis. In a comment, write a few sentences to summarize your ideas.***

Imagine that we want to understand the relative frequencies of the attributes in the column names. Specifically, we want to be able to choose two attributes (column names) and find out which is more frequent within the entire table.

**Task 17:** Discuss with your partner how you might go about testing this function. Write a comment describing your testing plan (what tables would you make, what rows would you need, etc).

___
### *CHECKPOINT:* Call over a TA once you reach this point.
___

## Takeaways
This lab has mostly been about getting you comfortable working with tabular data, practicing some common operators on tables, and planning functions. It also gets you thinking about our course's focus on data: what patterns of manipulating data do we often use in computations? How does the organization of our data impact our ability to answer these questions?

Here, we see that filtering, ordering, and summarizing data are some of the key operations. So far we’ve only looked at these operations on tabular data, but these same building blocks will arise many times through this course. When you have a computation to perform around data, you should start by thinking through what combinations of filtering, sorting and summarizing will help you compute your answer.

------

> Brown University CSCI 0111 (Fall 2024)

<iframe src="https://forms.gle/52Fi9HdMFRgW1vD67" width="640" height="372" frameborder="0" marginheight="0" marginwidth="0">Loading…</iframe>