---
title: "Where the f is F? | Methods 1 F22, CogSci @AU"
author: "Julie, Kristiane, Gloria, Samuel, Nina aka the winning studygroup"
date: "21/11/2022"
output: pdf_document
---

# Introduction - GLORIA
This experiment was inspired by previous research done by Treisman and Gelade (1980), who investigated a feature-integration theory of attention. According to Treisman and Gelade (1980), “The integration theory suggests that attention must be directed serially to each stimulus in a display whenever conjunctions of more than one separable feature are needed to characterize or distinguish the possible objects presented.” Based on those assumptions, we conducted a study to investigate whether the similarity of visual features of letters influence the search time for finding the target letter.
If feature similarity affects visual search, then the search time for finding the target letter should differ between groups presented with letters of different levels of feature similarity.

knitr::opts_chunk$set(echo = FALSE, include = TRUE, message = FALSE, warning = FALSE)
knitr::opts_knit$set(root.dir = 'logfiles')
setwd(".")

#Loading packages
pacman::p_load(tidyverse, pastecs, stringr, dplyr, reshape2, conflicted, WRS2, ggplot2, corrplot, readr, car, ggpubr)

# Methods
## Participants - SAM
26 participants were randomly recruited for the study (M Age = 23.8 range = 20-55, SD = 6.5; Gender = 16 females, 10 males; Handedness = 18 right handed, 7 left handed, 1 ambidextrous; Visual Impairment = 10 with, 16 without).

| | Female | Male | Total |
|:------|:-------|:-----|:-----|
| Mean of Age | 24.3 | 23.1 | 23.8 |
| Handedness left| 6 | 1 | 7 |
| Handedness right| 10 | 8 | 18 |
| Handedness ambidextrous| 0 | 1 | 1 |
| Visual Impairment | 4 | 6 | 10 |
| No Visual Impairment | 12 | 4 | 16 |
| Total | 16 | 10 | 26 |

## Materials/Stimuli - SAM
The experiment was a letter searching task. Stimuli was presented to participants using custom code in PsychoPy 2022.2.4 using Python 3.11.0 running on five different devices (Windows PC (HP Model LAPTOP), MacBook Pro (13-inch, M1, 2020), MacBook Pro, (14-inch, M1, 2021), Windows PC (Asus ZenBook), Lenovo IdeaPad L340-15IRH).

Stimuli were created using Adobe Photoshop 2023.

## Procedure - SAM
Participants were presented with five displays wherein there was a distribution of three different letters - two of the letters repeated 15 times each, while one letter only appeared once. The participants were asked to find the letter that appears only once. The experiment had three conditions - one control condition and two test conditions. The repeating letters and their placement were the same in all three. The dissimilar letter placement was also the same for all three conditions. The conditions were as follows:

- **Control**: the dissimilar letter is uppercase and has little visually similar features to the surrounding, repeating letters (see figure 1)

- **Test_lower**: the dissimilar letter is lowercase, but otherwise the same letter as the one in the control condition (i.e. if the dissimilar letter in the control condition is “B”, then in this condition it is “b”) (see figure 2)

- **Test_upper**: the dissimilar letter is uppercase and has very visually similar features to the surrounding, repeating letters (see figure 3)

The participants viewed the stimuli on a computer. They were tasked with pressing the spacebar as soon as they had found the dissimilar letter. In total we had six stimuli for each condition, one being a trial stimulus. (see figure 4). We measured their search time - how long it took for the participants to find the unique letter.

# checks whether randomization has already been done: if not, run anonymization.
if (is_empty(list.files(path =  getwd(), pattern = "*logfile_snew*", full.names = T))){
  
  
  files <- list.files(path =  getwd(), pattern = "*logfile_*", full.names = T) 
  
  
  # you only need to run the below code once, when initially loading data.
  # this code is to remind you that whenever you collect data, you should make sure that:
  # subject IDs are completely random (not ordered by the date on which they were collected)
  # timestamps on the files indicating when they were collected should (generally) be removed.
  # this is to ensure complete anonymization of data (important when you run "real" studies)
  
  data_out <- list() 
  
  # determines number of files. 
  num_files <- length(files) 
  
  # randomly shuffles integers in range 1:num_files.
  rand_ids <- sample(seq(1,num_files,1)) 
  
  # initializes file counter. 
  cnt_f <- 0 
  
  # loops through files. 
  for (f in files){
    
    # updates counter. 
    cnt_f <- cnt_f + 1
    
    # reads current CSV file. 
    data_out[[f]] <- read_csv(file = f, col_names = TRUE)
    
    # generates new random ID ("sXX", where XX is number from rand_ids)
    data_out[[f]]$ID <- paste(c("snew", rand_ids[cnt_f]), collapse = "")
    
    # removes column with timestamp (if there is one).
    data_out[[f]] <- subset(data_out[[f]], select = -c(Timestamp))
    
    # defines new output file name. 
    out_name <- paste(c(getwd(), "/logfile_", unique(data_out[[f]]$ID[1]), ".csv"), collapse = "")
    
    # writes this CSV file. 
    write_csv(data_out[[f]], out_name, na = "NA")
    
    # now delete original file. 
    file.remove(f)
    
  }
  
}

# get new file names.
files <- list.files(path =  getwd(), pattern = "*logfile_snew*", full.names = T) 

# read all the files into a tibble (a fancy df) by applying read_csv() to every filename in files vector 
# and binding them together automagically
data <- map_dfr(files, read_csv, col_types = cols(Group=col_factor()))

#Converting column classes
data <- data %>%
  mutate(
    Age = as.integer(Age),
    Condition = as.factor(Condition),
    ID = as.factor(ID),
    Gender = as.factor(Gender),
    Handedness = as.factor(Handedness),
    Visual_impairment = as.factor(Visual_impairment),
  ) %>%
  rename(Index = ...1) %>% #rename Index
  rename(Feature_similarity = Condition) 

# RUN ONLY ONCE
#Renaming conditions to upper_test and lower_test
data$Feature_similarity <- gsub(pattern = "test", replacement = "upper_test", as.factor(data$Feature_similarity))
data$Feature_similarity <- gsub(pattern = "lower", replacement = "lower_test", as.factor(data$Feature_similarity))

#Previewing data
head(data)

## Analysis

Before analysing our data further, we loaded our logfiles, anonymized them, and merged them into one data set. We also prepared our data by fixing the data types and doing some renamings.

### ANOVA-Test - JULIE

In order to compare the three groups against each other, we performed an Analysis of Variance test (ANOVA), which tests whether the means are significantly different. The null hypothesis is that the mean values from all three groups are the same. The alternative hypothesis is that at least one of the groups is different. To run an ANOVA, we used the aov() function.

#Running ANOVA and storing the output
t_anova <- aov(Search_time ~ Feature_similarity, data = data)

The small p-value indicates that there is a difference between conditions.

This output still does not show where this difference lies, nor the size and direction of it. To try and answer this, we then ran the best fitting model as a linear regression to provide numeric explanation of the differences between conditions.

# running linear regression on the model
model <- lm(Search_time ~ Feature_similarity, data = data)

#### Numerical Explanation
Intercept is the base level of the predictor and in this case it is the control group. Estimate shows that the mean search time for the control group is 2.46 seconds.

The other estimates show how the lowercase and uppercase test groups compare to the control group:
The lowercase group's mean search time is 0.83 seconds faster than the control group's, which is a significant decrease $(p < .05)$.
The uppercase group's mean search time is 0.63 seconds slower than the control group's, which is a significant increase $(p < .05)$.

#### Visual Explanation - GLORIA

To support our judgement of the results of ANOVA visually, we draw boxplots using ggplot (from the Tidyverse package) with the mean as a valued point. Additionally we draw mean plots using ggline().

#Box plot
ggplot(data, aes(x=reorder(Feature_similarity, Search_time, na.rm = TRUE), y = Search_time, colour = Feature_similarity)) +
  geom_boxplot(width = 0.5) +
  stat_summary(fun = mean, geom = "point", shape = 23, colour = "Black")+
  
  #Aesthetics
  ggtitle("Search time by Feature Similarity Group")+
  labs(x = "Feature Similarity", y = "Search Time in seconds")

# Mean plots
# Plot Search_time by group
# Add error bars: mean_se
# (other values include: mean_sd, mean_ci, median_iqr, ....)

ggline(data, x = "Feature_similarity", y = "Search_time", 
       add = c("mean_se"), 
       order = c("lower_test", "control", "upper_test"),
       ylab = "Search time in seconds", xlab = "Feature similarity", 
       title = "Search time by Feature Similarity Group")
      


ggline(data, x = "Feature_similarity", y = "Search_time", 
       add = c("mean_se", "jitter"), 
       order = c("lower_test", "control", "upper_test"),
       ylab = "Search time in seconds", xlab = "Feature similarity", 
       title = "Search time by Feature Similarity Group")

### Assumptions Checking - KRISTIANE
We can only use these results for further analysis if the data is not violating any assumptions for linear regression:

#### 1. Outcome variable must be continuous (at least at the interval level)
We have three conditions that we are comparing, so this assumption is met.

#### 2. No multicollinearity (no linear relationship between 2 or more predictors)
Our model only has one predictor, so this assumption is met.
\newpage

#### 3. Linearity of residuals (linear relationship between predicted values & residuals)

plot(model, 1)

We used the function plot(model, 1) from base package to check for this assumption. From the plot we see that the assumptions are more or less met, there is no vertical pattern. But we see that there are some influential cases.
\newpage

#### 4. Normality

# This can be checked by plotting the normality of residuals in a QQ-plot
plot(model, 2)

We used the function plot(model, 2) from base package to check for this assumption. We can see that the residuals stray from the line quite a bit towards the beginning and the end. This is an indication that our normality assumption may be violated.
Because of this, we can assume that our data is non-parametric, and we are therefore going to run the non-parametric Kruskall-Wallis H-test.
\newpage

#### 5. Homoschedasticity

plot(model, 3)
bartlett.test(Search_time ~ Feature_similarity, data=data)

We used the function plot(model, 3) from base package to check for this assumption. In order to have equal variance, the residuals should in this plot be equally spread out for each group. We can see that the residuals are more spread out for the higher fitted values, which is an indication that our equal variances assumption may be violated. Though, there are more data points for the control group and the Upper_test group, which might be an explanation for this.
When running Bartlett's test to check it numerically, we see that the $p > .5$ - Non-significant result, which means the assumption is not violated.
\newpage

#### 6. No influential cases (outliers)

plot(model, 4)

We used the function plot(model, 4) from base package to check for this assumption. The outliers shown in a Cook's distance measure are the ones with a number labeled to them. So, here we have three outliers. We are trying to remove them by removing points that are three deviations away from the standard deviation.

#Removing the outliers that are 3 deviations away from the standard deviation
data_out <- data%>%
  dplyr::filter(Search_time > mean(Search_time)-3*sd(Search_time) & Search_time < mean(Search_time)+3*sd(Search_time))

#Creating a new model based on the data without outliers
model_NOout <- lm(Search_time ~ Feature_similarity, data = data_out)

#Checking the cook's distance measure plot again
plot(model_NOout, 4)

The Cook's distance measure now created three new influential cases, but we will not remove these, as we don't have that much data, and these outliers are not very significant (looking at the distance scale on the y-axis.

#### 7. Independence – the observations in each group need to be independent of each other.
Since we used a randomized between-subject design (participants were randomly assigned to the three conditions and only participated in one condition each), this assumption is met.

### Kruskal-Wallis H-test
The Kruskal-Wallis test has the following assumptions which is all met in our experiment:
1. Ordinal or continuous dependent variable
2. 2 or more categorical groups in the independent variable
3. Independence of observations = between-subject experimental design

kruskal.test(Search_time ~ Feature_similarity, data = data)

The p-value $(p <.05)$ indicates that there is a significant difference.
Because of this we are going to do a non-parametric post-hoc test to check which group means significantly differ from the others.

### Post-hoc Pairwise Comparisons
In order to narrow down exactly where the difference between the means lies, we compare the three groups in pairs with 'stricter' versions of t-tests, that is, post-hoc pairwise comparisons. We are using the non-parametric version, which is the Pairwise Wilcox Rank Sum Tests with corrections for multiple testing.

Interpretation is similar to t-tests. If there is no difference between the means, the p-values will be large, if difference between the means is present, the p-values will be small.

First, we must control the family-wise error rate with the Bonferroni adjustment by adjusting the alpha-level: .05 by three tests.

#Adjusting alpha-level
adj_alpha <- round((0.05/3), 3)
print(c("Adjusted alpha = ", adj_alpha))

# Results - JULIE and KRISTIANE

summary(model_NOout)

#Pairwise t-testing
pairwise.wilcox.test(data_out$Search_time, data_out$Feature_similarity, p.adjust.method = "bonferroni", paired = FALSE)

We found, that the relationship between search time and feature similarity can be described with a linear regression model including the effect of feature similarity on visual search time as shown below. This model accounts for 11 % of the total variance in the data (Adjusted $R^2 = .11$).

**Model_NOout: lm(Search_time ~ Feature_similarity)**

There was a significant effect of feature similarity on visual search time, $F(2, 124) = 8.71, p < .05$. The non-parametric post-hoc Bonferroni-corrected t-tests revealed that the visual search time of the upper-case test group (test_upper) was significantly slower than for the lower-case test group (test_lower) $(p < .01)$. The visual search time for the control group (control) compared to the two test groups was not found to be significantly different after adjusting the alpha-level as mentioned $(p < .001)$.

After removing some influential cases, we are not violating any assumptions other than normality, which we are accounting for by running the Kruskal-Wallis and post-hoc non-parametric tests.

# Discussion - NINA

Our investigation suggests that there are significant differences in visual search time when presented with sufficiently different stimuli. The group which was asked to search for a lower-case letter (test_lower) is faster than the group presented with upper case letters that share features with other letters shown on the screen (test_upper)

This does not really replicate the findings of Treisman and Gelade (1980), but it seems to support our hypothesis, that the lower-case condition (test_lower) leads to significantly faster search times than the one with feature similarity (test_upper). Although, the relationship between search time and feature similarity as described in our model (model_NOout), is limited by the fact, that our model only accounts for a small amount of the total variance in the data. The high level of unexplained variance could be an indicator, that our experiment design lacks at least one (unidentified) variable measure. This makes a point for further research.

We found no significant difference between the control group (control) and the upper-case test group (test_upper). One reason for this could be the choice of letter combinations we used for these two groups. Based on Gibson’s (1969) proposal for the features underlying the recognition of letters, we checked the similarity of the specific features and found two letter combinations to be problematic for our experiment design. These are “COGS” and “VXNJ” and can be found in Figure 4 in the Appendix. As seen from the table there is not one unique feature, that the first letter – the searched letter – only has in common with the letter shown in the test_upper group in these two combinations. The condition is however fulfilled by all other letter combinations.
The consequence of these higher similarities between letters in “COGS” and “VXNJ” is a higher difficulty for the control group. This could explain the insignificant difference in the overall search time of the control group and the upper_test group.

# Conclusion - SAM

To conclude, while our experiment design was slightly flawed, we could still reject our null hypothesis; we found a significant difference between the groups, which means that the visual similarities in letters had a great influence on search time.
\newpage

# Appendix
### Stimuli Example Images - GLORIA




\newpage
### Letter Recognition Features Table - NINA