---
title: "Untitled"
output: html_document
---

Notes:
- Cover all the key functions required for the lab but, perhaps, simplify and/or remove some interesting exercises.
- DSBox uses the "SelectorGadget", which I'd rather not use. Use "inspect element" instead to find the appropriate HTML structural element to pull.
- Cf. https://rstudio-education.github.io/datascience-box/course-materials/slides/u2-d19-top-250-imdb/u2-d19-top-250-imdb.html


```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

```{r}
library(robotstxt)
paths_allowed("http://www.imdb.com")
paths_allowed("http://www.facebook.com")
```

```{r}
library(tidyverse)
library(rvest)
page <- read_html("https://www.imdb.com/chart/top/")
page
```

```{r}
titles <- page %>%
  html_nodes(".titleColumn a") %>%
  html_text()

titles[1:10]
length(titles)
```

```{r}
years <- page %>%
  html_nodes(".secondaryInfo") %>%
  html_text() %>%
  str_remove("\\(") %>%
  str_remove("\\)") %>%
  as.numeric()
years[1:10]
length(years)
```

```{r}
ratings <- page %>%
  html_nodes("strong") %>%
  html_text() %>%
  as.numeric()
ratings[1:10]
length(ratings)
```

```{r}
imdb_top_250 <- tibble(
  title = titles, 
  year = years, 
  rating = ratings
  )
imdb_top_250 <- imdb_top_250 %>%
  mutate(rank = 1:nrow(imdb_top_250)) %>%
  relocate(rank)  
imdb_top_250
```
The number of movies per year.

```{r}
imdb_top_250 %>% 
  count(year, sort = TRUE)
```

```{r}
imdb_top_250 %>% 
  filter(year == 2021) %>%
  print(n = 8)
```

The average rating of movies per year

```{r}
imdb_top_250 %>% 
  group_by(year) %>%
  summarise(avg_score = mean(rating)) %>%
  ggplot(aes(y = avg_score, x = year)) +
  geom_point() +
  geom_smooth(method = "lm", se = FALSE) +
  labs(x = "Year", y = "Average score")
```
