---
title: "Programming<br>Data Cleansing"
author: "Keith VanderLinden<br>Calvin University"
output:
  xaringan::moon_reader:
    nature:
      titleSlideClass: ["left", "top"]
      highlightLines: true
      ratio: "16:9"      
---

```{r child = "../../resources/scripts/slide-setup.Rmd"}
```

```{r packages, include=FALSE}
knitr::opts_chunk$set(comment = "")
options(dplyr.print_max = 10)
xaringanExtra::use_freezeframe(overlay = TRUE)
```

```{r message = FALSE, warning = FALSE}
library(tidyverse)
```

# Data Cleansing

As mentioned earlier, we distinguish between:

1. Data *Cleansing*
2. Data *Tidying*
3. Data *Wrangling*

This unit focuses on *cleansing* data, the process of transforming &ldquo;raw&rdquo; data values into forms appropriate for computation. Computation requires that data:

- be represented using an appropriate data type, either numerical and categorical.
- are consistent with respect to in-range and out-of-range values.

???
- Type examples: integers/doubles/strings; dates; factors
- Consistency examples: 
  - Date strings, e.g., "Feb 7, 1961" vs "02-07-61"
  - Category names, e.g., "Joe Biden vs Joseph R. Biden"


---
# Converting Between Types

.pull-left[
*Explicit* Type Conversion
```{r}
as.character(list(1, 2.0, "three", TRUE))
```
```{r}
as.logical(list(0, 1, -2, 3.3, "FALSE"))
```
```{r}
as.integer(list(1, 1.2, -2.3, "one", FALSE))
```
```{r}
parse_number("$1,000")
```
]
.pull-right[
*Implicit* Type Conversion
```{r}
c(1, "one", 1.0)
```
```{r}
c(1, 1.2)
```
```{r}
c(TRUE, "True")
```
]

???
Be clear on the conversion order.

Compare & contrast `as.numeric()` vs. `parse_number()`.
  - https://mdsr-book.github.io/mdsr2e/ch-dataII.html#from-strings-to-numbers

---
# Handling Special Values

.pull-left[
Doubles have three special values that represent known but undefined values.
```{r}
c(-1, 0, 1) / 0
```

`NA` represents missing values.
```{r}
l <- c(1, 2, 3, NA)

mean(l)
```
```{r}
mean(l, na.rm = TRUE)
```
]
.pull-right[
`NA` is a logical value and is treated as &ldquo;unknown&rdquo; using three-valued logic.
```{r}
typeof(NA)
```
```{r}
TRUE | NA
```
```{r}
FALSE | NA
```
```{r}
TRUE & NA
```
]

???

- `NA`: Not available
- `NaN`: Not a number
- `Inf`: Positive infinity
- `-Inf`: Negative infinity
