Studio 7: Confidence Intervals and Hypothesis Testing

--- title: "Studio 7: Confidence Intervals and Hypothesis Testing" layout: post label: studio geometry: margin=2cm tags: studio --- # CS 100: Studio 7 ### Confidence Intervals and Hypothesis Testing: Two Sides of the Same Coin ##### October 26, 2022 ### Introduction The topic of this week’s studio is approval ratings. In the first part, you will investigate Presidential approval ratings, and in the second part, those of Congress. These ratings summarize polls conducted among U.S. adults. Your job will be to investigate the statistical significance of the polls’ results, using confidence intervals and hypothesis testing---two sides of the same coin. Upon completion of all tasks, a TA will give you credit for today's studio. If you do not manage to complete all the assigned work during the studio period, do not worry. You can continue to work on this assignment until Sunday, October 30 at 7 PM. Come by TA hours any time before then to show us your completed work and get credit for today's studio. ### Objectives By the end of this studio you will be able to: * Compute a confidence interval * Conduct a hypothesis test ### Data: Presidential Approval Ratings To get a sense of how U.S. presidents are perceived while in office, job approval polls are often conducted on a sample of U.S. adults. This week's studio is based on the results of one such poll. The numbers are taken from a 2019 Gallup presidential job approval center, located [here](https://news.gallup.com/interactives/185273/presidential-job-approval-center.aspx). ### Part 1: Computing Confidence Intervals On August 14, 2019, Gallup conducted a job approval [poll](http://www.gallup.com/poll/201617/gallup-daily-trump-job-approval.aspx) of President Trump. This poll found that among 1500 U.S. adults, 41% approved of President Trump’s job performance, and 54% disapproved. To quantify how closely these poll results can be expected to mirror the opinions of the population as a whole, Gallup also reported the poll’s margin of error (MoE) of 3 points, which, roughly speaking, translates into confidence intervals of size [38%, 44%] (approval) and [51%, 57%] (disapproval). Polling agencies don't generally report their confidence levels (\\(\alpha\\)), only their margins of error, but with a little trial and error, you can recover a confidence level from a MoE. In this exercise, you will determine the confidence level that Gallup used to calculate the MoE of Trump's ratings. If we assume that "approve" and "disapprove" are the only available answers to the poll (we apologize to those who have no opinion), each participant’s response can be viewed as a Bernoulli trial. By the central limit theorem, the estimates 0.41 (41% approval rating) and 0.54 (54% disapproval rating) are normally distributed, with mean \\(p\\) and standard deviation equal to the standard error, namely \\(\sqrt{\frac{p(1-p)}{n}}\\), where \\(p\\) is the proportion of successes (approvals in one case, and disapprovals in the other). The 95% confidence interval is then computed as \\(p \pm 1.96 \sqrt{\frac{p (1-p)}{n}}\\). As we do not know \\(p\\), we estimate these standard errors using \\(\hat{p}\\) instead. 0. Practice: Write a couple of lines of R code to determine the 95% confidence interval for a sample proportion. Do this by first creating variables `phat` and `n`, and assigning those variables values (e.g., 0.41 and 1500). Then input the relevant formulas, using these variables instead of hard-coded values. (You will generalize 1.96 from a hard-coded constant to a variable later in this problem.) 1. Next, we would like you to use the code you just wrote to compute a 95% confidence interval both for Trump’s approval rating and his disapproval rating. In other words we want to run your code twice, assuming two different sample proportions. Rather than repeat code, embed your code in a function that takes as input a sample proportion and a sample size, and returns the 95% confidence interval as a vector. (As above, you may hard code the value 1.96, for now.) *Hint:* The last line of your function should be something like `c(lower, upper)`, where `lower` and `upper` are variables you define in your function denoting the lower and upper bounds of the confidence interval the function computes. *Another Hint:* As 1500 is the sample size of all polls in this question, feel free to assign `n` the default value of 1500 in the function header, like this: `function(phat, n = 1500)`. Use your function to report a confidence interval that estimates the proportion of the population that approves of Trump, and a confidence interval that estimates the proportion of the population that disapproves. 2. Generalize your function to take as input a third parameter, namely a confidence level \\(\alpha\\), and calculate the confidence interval at that level. *Hint:* Use the `qnorm` function, which takes as input a percentile and returns the corresponding quantile of the standard normal distribution. For example, `qnorm(0.025)` returns `[1] -1.959964`, while `qnorm(0.5)` returns `[1] 0`. What confidence level corresponds to a MOE of 3 points for this poll? To find out, call your function with a few values of \\(\alpha\\), like \\(0.05\\), \\(0.01\\), etc. ### Part 2: Simulating Confidence Intervals The goal of this next exercise is to ensure that you understand the meaning of a confidence interval. Recall that the goal of inferential statistics is to estimate a population parameter from a sample. This population parameter is not (in frequentist statistics, anyway) assumed to be a random variable. On the contrary, the randomness derives from the sampling process. As a result, a confidence interval denotes a region in which the true parameter would be expected to lie, say 95 out of 100 times (if \\(\alpha = 0.05\\)), if an experiment (e.g., a poll) were conducted multiple times. The `rbinom` function in R generates a vector of random variables according to the binomial distribution. The inputs to this function are a vector length (the number of random variables to sample), a number of trials (1, if the binomial is actually a Bernoulli random variable), and a success probability. Try out the following examples to get a sense of how `rbinom` works. ~~~{r} rbinom(n = 1, size = 1, prob = 0.5) rbinom(n = 10, size = 1, prob = 0.5) rbinom(n = 1, size = 10, p = 0.5) ~~~ In this exercise, you are going to imagine that Gallup repeated its poll multiple, say 500, times. To simulate these 500 repeated polls of 1500 people, you should call the `rbinom` function with `n = 500` and `size = 1500`. To use the approval (resp. disapproval) rating, set `prob = 0.41` (resp. `prob = 0.54`). 1. Call the `rbinom` function with the aforementioned settings (using just one or the other of the approval or disapproval ratings), and store the results in a new vector, say `counts`. 1. Calculate the sample means and sample standard errors corresponding to all `n` simulations. Store the results in two new vectors, say `p_hats` and `std_errs`. 1. Pick a confidence level, say \\(\alpha = 0.05\\), and use the `qnorm` function as above to find the quantile corresponding to your choice of \\(\alpha\\). Use that quantile and vector arithmetic to the compute upper and lower bounds of the \\(1 - \alpha\\) confidence intervals. Store these results in two more vectors, say `lower` and `upper`. 1. Use vector logic to determine when the true probability (0.41 or 0.54) falls outside one of these confidence intervals. *Hint*: if \\(p\\) denotes the true probability, `p < lower` is a logical vector containing TRUE whenever \\(p\\) is less than a value in the vector `lower`, and likewise for `p > upper`. The probability \\(p\\) is then outside the confidence interval if ever it is either below or above one of the bounds: `p < lower | p > upper`. 1. What percentage of times does \\(p\\) fall outside the confidence interval? As you increase the number of simulations, from 500 to say 5000, this percentage should approach \\(1 - \alpha\\). (If you included the solution to this problem inside an R chunk, you can easily rerun your code multiple times to confirm this conjecture.) Hopefully, this exercise demonstrated to you the meaning of a confidence interval. If you chose \\(\alpha = 0.05\\), then the confidence interval should have included the true population parameter \\(p\\) in roughly 95% of your simulations. Likewise, based on the Gallup poll results, the range [0.38, 0.44] would be expected to include the proportion of the population that approved of Trump's activities roughly 99% of the time. ### End of Studio When you are done please call over a TA to review your work, and check you off for this studio. If you do not finish within the two hour studio period, remember to come to TA office hours to get checked off.