---
format:
    revealjs:
        theme: simple
        chalkboard: true
        auto-stretch: false
---

## Training Data Demos

Keith VanderLinden<br>
Calvin University

## Labeling in SLO

- SLO labeling
- Label Studio

::: {.notes}
- Run the SLO demo in a local VSCode connected to the GitHub running codespace. (I did this by opening the codespace in GitHub and choosing something like "open in local VSCode".)
    - Show the `data/coding/` datasets.
    - Explain how we just used spreadsheets, with the hyperlink to the tweet in context. The context ended up really mattering for SLO, as, indeed, it might for many applications.
    - Show my (local) kappa-computation spreadsheet.
        - Explain the neutral/non-neutral issue we had.
        - Note the low kappa score in this sheet. We ended up publishing at around 0.67.
    - Talk through the SLO auto-labeling (see `src/autocoding_preprocessor.py`).
        - Show the three rule patterns: `PTN_for`, `PTN_against`, `PTN_neutral`.
    - We worked hard to balance (rare) positive examples with (common) negative examples. Mostly, we used all the positives and sampled the negative tweets for the training set.
        - This might be classified as *probabilistic quota sampling*.
- Run the Label Studio (LS) demo locally.
    - LS has far more capability that we'll need, but it's to see the potential.
        - See the available labeling templates.
    - For the LS `TagIfAI` project, show the:
        - data, "completed" or not.
        - the labels from the two users/labelers.
        - the labeling interface.
        - the labeling configuration.
            - Show the "code" version of the configuration, which we'll copy into our repo.
            - Show the sampling setting.
        - the export.
    - This demo is pretty much the lab. So you can follow it closely. See the previous videos for the Kappa computation.

References

- <https://labelstud.io/>
:::

## Snapshot

*See the live demo.*

![](images/label-studio-demo.png)

::: {.notes}
Give snapshot of final Label Studio database dashboard.
:::

