---
format:
    revealjs:
        theme: simple
        chalkboard: true
        auto-stretch: false
---

## Model Deployment Demos

Keith VanderLinden<br>
Calvin University

::: {.notes}
- We'll follow C. Huyen's easy/play deployment (p. 192).
- Just don't forget the importance of production-strength
    scaling, monitoring, and updating.
- Cf. DMLS Figure 7-8.
:::

## Our Model Deployment Pipeline

- Model engineering pipeline
- REST API, running:
    - Locally
    - Globally

::: {.notes}
1. Demo the model engineering pipeline.
    - Walk through the basic file structure in `example/.`
    - `makefile` implements the model engineering pipeline. Notes:
        - It has extensive notes on building/deploying the service.
        - `$(DATASET)`:
            - We skip the data engineering pipeline by hard-coding `data/dataset.csv`.
            - Note how I use makefile variables.
        - `$(MLFLOW_URI)`: This passes the MLflow URI to
            the training script and REST API.
        - `$(MODELS_DIR)`: This auto-builds the models directory.
        - It auto-runs the `source .env` command, if it exists
            (locally: yes; on CodeSpaces: no).
        - Only builds the random model.
        - Includes a separate targets for MLFlow and REST.
    - `src/train_model.py` trains the model. Note that it:
        - Just loads from the CSV.
        - Builds a random model.
        - Skips the pickling; it only MLFlow-dumps the model.
    - Demo:
        - bash 1: `make mlflow`
        - bash 2: `make clean` and `make`.
        - MLFlow: show the new model/performance; set `prod`.
2. Explain the REST API code.
    - Walk through `src/api.py`. Note that it:
        - Documents the use (only) of the API.
        - `lifespan()`
            - Runs asynchronously, before the server starts.
            - Declares the variables globally.
            - Loads the production (`prod`) model from MLFlow.
            - Loads feature engineering artifacts.
            - Skips cleanup when the server stops.
        - Implements two endpoints:
            - `/` for GETing the server status.
            - `/predict` for running the model.
                - Here, we use Python type hints, which are required for handling POST data in FastAPI.
                - It uses the same (shared) text cleaning utility as the model training script (already run for `dataset.csv`). We need to preprocess texts the same way in training and inference, so we factor this out into `utilities.py`.
                - We choose POST because both DMLS and MwML use it.
                    It's not an obvious choice because predict is arguably idempotent.
3. Demo the REST API locally:
    - bash 2: `make rest`
    - browser:
        - `/` works fine.
        - `/predict` give a "method not allowed" error;  browsers only do GETs, not POSTs.
            - Show where the "internal server" messages get printed.
            - REST services tend not to give full error messages to clients.
    - bash 3:
        - Explain the cURL command in detail.
        - Demo the documented curl command - it returns the wrong answer in the right way.
        - Manufacture an error to demo live updates and error output.
    - Show how to stop the service
4. Demo the REST API in CodeSpaces
    - Use the full `data562-tagifai` implementation (with the SGD model).
    - bash 1: `make mlflow`
    - bash 2: `make rest`
        - Show where to find the REST API URI in the CodeSpaces VSCode "Ports".
        - Set the REST visibility to `public`.
        - Run commands locally. Note that we get the right answer from the SGD model.
    - Note that our fallback for the system to use DVC-stored, Pickle files.
:::

## Snapshot

*See the live demo.*

![](images/rest-demo.png)

::: {.notes}
Give snapshot of the REST API (browser + cURL)
:::
