---
title: DS21S04
tags: class
slideOptions:
theme: white
transition: slide
---

## <span class="censor">Data Studies 2021 // S04. Data cleaning</span>

<!--image for class-->
<img src="https://gitlab.com/xpablov/data-studies/-/raw/master/DS19/S02/record.png" width=50%>

Pablo Velasco // Information Studies // [pablov.me](https://pablov.me)

---

## Plan for the day:
* Data taxonomies
* data is, data can be, data should be: standarisation, uniqueness, outliers
* Place activity
* *Cleaning* data
* bifocalism, data feminism, and how to research cleaning practices
* Uddannelse activity

---

# DATA TAXONOMIES

----

* **Data *can* be:**
<span class="refs">Kitchin, R. (2014). The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences</span>
* Quantitative
* Nominal: categories
* Ordinal: ranks, scales
* Qualitative
* Structure
* **Structured**: consistent format
* Semi-structured: irregular format
* Unstructured
* Source
* Captured: observation, measurement
* Exhaust: produced by a machine
* Derived: from additional processing
* Type
* **Indexical**: includes identifiers
* Attribute: attributes of the identifiers
* **Metadata**: data about data

----

* **Data *is*:**
<span class="refs">Floridi, L. (2010). Information: A Very Short Introduction. OUP Oxford.
</span>
* Taxonomical: can be ordered or related
* Typological: primary, derived, metadata

* **Data *is*:**
<span class="refs">Rosenberg, D. (2013). Data before the fact.</span>
* **Abstract**: an abstraction of "reality"
* <span class="censor">"bracketing unnecessary information from diverse components within a system"</span><br><span class="refs">Cook, W. R. (2009). On understanding data abstraction, revisited</span>
* Discrete: it consists of finite elements
* Aggregative: can be combined and accumulated
* **Meaningful**: conveys some meaning

----

### "well-formed" data

* **Data *should* be FAIR:**
<span class="refs">Wilkinson et al (2016). The FAIR Guiding Principles for scientific data management and stewardship.</span>
* Findability
* Accessibility
* Interoperability
* Reusability
* **Data *should* be Smart:**
<span class="refs">Schöch, C. (2013, November 22). Big? Smart? Clean? Messy? Data in the Humanities.</span>
* Structured (or semi-structured)
* Enriched
* Small

<span class="pinky">"The chance of mistake is lowered and the chance of success is heightened. The possibility of its continued circulation is better and better for data use." Anne M.</span>

----

<img src="https://i.imgur.com/iUupQlk.jpg" width="25%">

<img src="https://i.imgur.com/TuMZ7go.png" width="65%">

<small>Examples of *wild* data</small>

----

* *Dataism*
<span class="refs">Dijck, J. van. (2014). Datafication, dataism and dataveillance: Big Data between scientific paradigm and ideology.</span>
* ideology of neutrality
<span class="censor">"Dataism thrives on the assumption that gathering data happens outside any **preset framework** (...) and data analysis happens without a preset **purpose**"</span>

<span class="pinky">Pristine data is often seen as something truthful and valid which make people forget that there lies several invisible processes conducted by other human beings behind this creation/presentation of data." Rune B H</span>

----

<img src="https://gitlab.com/xpablov/data-studies/-/raw/master/DS19/S03/PlaceFinal.png" alt="drawing" width="60%">

<small>(**Click the image for a bigger version. You can also find the gif [here](https://gitlab.com/xpablov/data-studies/-/raw/master/DS19/S03/place.gif)**)</small>

<!--place:
* 2017
* 72 hours
* 1 million squares
* 1+ million users
* 5-20 min wait
-->

----

## Short activity:
Discuss with your *naboer*

1. Identify “things” that you recognise in Place.
2. What kind of things constitute the "preset framework" for place to convey meaning? Is this framework technical, cultural, social?
3. What do you think is the purpose of Place? How does this purpose influences the final product?

---

# *CLEANING* DATA

----

* <span class="censor">"Process of detecting, diagnosing, and editing faulty data"</span>
<span class="refs">Van den Broecket al. (2005). Data Cleaning: Detecting, Diagnosing, and Editing Data</span>

<small>Most common metaphor for "data scientist" in mediacloud. Image from Catherine D'Ignazio. Data from www.mediacloud.org. (*Data feminism* ch.5) </small>

* 80% of data analysis spent cleaning and preparing data
<span class="refs">Dasu T, Johnson T (2003). Exploratory Data Mining and Data Cleaning.</span>

----

### Data, capta, sublata

<span class="pinky">"It is very much akin to cleaning silverware; one needs to make the surface presentable without damaging the delicate nature of such an object." Mikael SA</span>

* Data (*given*) != capta (*taken*)
* Not passive acceptance, but active construction
<span class="refs">Drucker, J. (2011). Humanities Approaches to Graphical Display</span>
* Knowledge as produced, more than discovered
<span class="refs">Gitelman, L. (2013). Raw Data Is an Oxymoro</span>

<span class="censor">"If we say it’s just given, that worries me a bit because it means you just seize and receive it. But in fact you actually elicit it, and eliciting is what I think this term *sublata* captures".</span>
<span class="refs">Latour, B. (2017). Gaia or knowledge without spheres </span>

----

- <span class="censor">Aesthetic practices</span>:

- data practices that elicit, extract, and select, rather than merely produce data.

<img src="https://i.imgur.com/yqobOJO.png" width="55%">

- Standarisation with consequences:
* "do not simply make data transparent and open but enable comparison of the relative performance of populations as objects of knowledge and governing."

<!-- foucault-->

<span class="refs">Ratner, H., & Ruppert, E. (2019). Producing and projecting data: Aesthetic practices of government data portals.</span>

----

<img src="https://i.imgur.com/WL7iP9K.png" width="55%">

<small>https://pudding.cool/2018/08/pockets/</small>

----

* <span class="censor">Data Feminism</span>:
<span class="refs">D’Ignazio, C., & Klein, L. F. (2020). Data Feminism</span>
* "a way of thinking about data, both their uses and their limits, that is informed by direct experience, by a commitment to action, and by intersectional feminist thought."
* "how race, class, sexuality, ability, age, religion, geography, and more are factors that together influence each person’s experience and opportunities in the world. Finally, *data feminism is about power - about who has it an who doesn't*"

* "openness is not singular in its effects but has different consequences for the populations that are enacted"
<span class="refs">Ratner, H., & Ruppert, E. (2019). Producing and projecting data: Aesthetic practices of government data portals.</span>

----


<small>Facebook gender fields circa 2014. Photo by Lauren F. Klein (Data Feminism ch.4)</small>

<span class="refs">Bivens, R. (2017). The gender binary will not be deprogrammed: Ten years of coding gender on Facebook.</span>

----

* <span class="censor">Bifocalism of knowledge</span>:
* sites of production != sites of projections
<!--marx -> construction of algorithms-->

<img src="https://i.imgur.com/3vRjeJO.png" width="65%">

<span class="refs">Latour, B. (2017). Gaia or knowledge without spheres </span>
<!-- taken from Peter Sloterdijk -->

----

### How to observe cleaning practices?

* "Showing all the **variations in processing** and making public all the ways processors “work differently” would, it was feared, under-mine the impression of **standardized** internal processing."
<span class="refs">Plantin, J.-C. (2019). Data Cleaners for Pristine Datasets: Visibility and Invisibility of Data Processors in Social Science.</span>
* observe the **role of automation** (question tools and techniques)
* notice **instances of bifocal practices** (e.g. "forced blindness")

<img src="https://i.imgur.com/SoaHwcJ.png" width="70%">

----

### How to observe the complexity of existing datasets?

<img src="https://i.imgur.com/5z8tiuc.png" width="150%">
<span class="refs">Poirier, L. (2021). Reading datasets: Strategies for interpreting the politics of data signification.</span>

----

<img src="https://i.imgur.com/8mNhfHj.png" width="60%">

<small>https://uddannelsesstatistik.dk/pages/gymnasialeudd.aspx</small>

"This is a source of data friction, as the schools cannot simply change their reporting of numbers without it having consequences for their own planning purposes (...) **Rather than changing the 'school reality', the aesthetic practice of data cleaning resolves this friction so that data can be projected in the forms required by the warehouse**"
<span class="refs">Ratner, H., & Ruppert, E. (2019). Producing and projecting data: Aesthetic practices of government data portals.</span>

----

### Short activity

Go to https://uddannelsesstatistik.dk/Pages/Reports/1693.aspx and look for information on your own gymnasium

1. Can you find any information on the collection or processing of data?
2. Is there any information on the "quality indicators"?
3. Where does the "socioøkonomiske referencer" metric come from?
4. Do you think is possible to objectively compare all gymnasiums in Denmark?

<!--SOCIOECONOMIC REFERENCE (socioøkonomiske referencer)

The socio-economic reference can be somewhat simplistically understood as "The expected grade when the students' background has been taken into account". It is a statistical expression of how students at national level, with the same background conditions as the students of the educational institution in question, have passed the tests.

‘Socioeconomic’ refers to the students' social and economic background, while ‘reference’ states that the number can be used as a basis of comparison for the grades actually obtained by the educational institution. By comparing the socio-economic reference with the actual grades of the educational institution, one can thus get a picture of whether the educational institution's students have passed the tests better, worse or at the level of students at national level with the same background. This is called the place of education
‘Lifting capacity’.

The calculation of socio-economic references has been made for graduates from one year (for example 2019) and
graduates from 3 years (eg 2017, 2018 and 2019). The calculation for the 3-year period has the smallest safety interval, as 3 graduate cohorts are included in the calculation.

The calculation of the students' socio-economic background includes the following factors at the individual level, over which the educational institution has no direct influence:
- 9th grade grade point average (bound test subjects only)
- Gender
- Age at the end of the education
- Origin
- Parents' highest completed education
- Labor market status of parents
- Parents' average gross income
- Family status (living abroad or living with a couple / single)
- Access road (what type of school does the student come from?)
- 10th grade (Has the student gone to 10th grade?)

The grade data is based on reports to STIL, while all information regarding the student's background conditions is taken from registers at Statistics Denmark.

Source (in danish) [here](https://uddannelsesstatistik.dk/Documents/Gymnasiale%20uddannelser/Datadokumentation/DVH_Gymnasiale%20uddannelser_Studenternes%20karakterer%20samt%20socio%C3%B8konomiske%20reference.pdf).

-->

<style>
.reveal{
font-family:mono;
font-size: 25px;
}
.reveal .censor{
background:black;
color:white;
}
.reveal .censorw{
background:white;
color:black;
}
.reveal .pinky{
color:#e5157d;
font-style:italic;
font-size: .8em;
}
.reveal section img {
border: none;
box-shadow: none;
}
.reveal section left{
width:50%;
}
.reveal .refs {
color: grey;
font-size: small;
text-align: left;
}
</style>