Model Deployment Demos

Keith VanderLinden
Calvin University

• We’ll follow C. Huyen’s easy/play deployment (p. 192).
• Just don’t forget the importance of production-strength scaling, monitoring, and updating.
• Cf. DMLS Figure 7-8.

Our Model Deployment Pipeline

• Model engineering pipeline
• REST API, running:
  • Locally
  • Globally

1. Demo the model engineering pipeline.
   • Walk through the basic file structure in example/.
   • makefile implements the model engineering pipeline. Notes:
     • It has extensive notes on building/deploying the service.
     • $(DATASET):
       • We skip the data engineering pipeline by hard-coding data/dataset.csv.
       • Note how I use makefile variables.
     • $(MLFLOW_URI): This passes the MLflow URI to the training script and REST API.
     • $(MODELS_DIR): This auto-builds the models directory.
     • It auto-runs the source .env command, if it exists (locally: yes; on CodeSpaces: no).
     • Only builds the random model.
     • Includes a separate targets for MLFlow and REST.
   • src/train_model.py trains the model. Note that it:
     • Just loads from the CSV.
     • Builds a random model.
     • Skips the pickling; it only MLFlow-dumps the model.
   • Demo:
     • bash 1: make mlflow
     • bash 2: make clean and make.
     • MLFlow: show the new model/performance; set prod.
2. Explain the REST API code.
   • Walk through src/api.py. Note that it:
     • Documents the use (only) of the API.
     • lifespan()
       • Runs asynchronously, before the server starts.
       • Declares the variables globally.
       • Loads the production (prod) model from MLFlow.
       • Loads feature engineering artifacts.
       • Skips cleanup when the server stops.
     • Implements two endpoints:
       • / for GETing the server status.
       • /predict for running the model.
         • Here, we use Python type hints, which are required for handling POST data in FastAPI.
         • It uses the same (shared) text cleaning utility as the model training script (already run for dataset.csv). We need to preprocess texts the same way in training and inference, so we factor this out into utilities.py.
         • We choose POST because both DMLS and MwML use it. It’s not an obvious choice because predict is arguably idempotent.
3. Demo the REST API locally:
   • bash 2: make rest
   • browser:
     • / works fine.
     • /predict give a “method not allowed” error; browsers only do GETs, not POSTs.
       • Show where the “internal server” messages get printed.
       • REST services tend not to give full error messages to clients.
   • bash 3:
     • Explain the cURL command in detail.
     • Demo the documented curl command - it returns the wrong answer in the right way.
     • Manufacture an error to demo live updates and error output.
   • Show how to stop the service
4. Demo the REST API in CodeSpaces
   • Use the full data562-tagifai implementation (with the SGD model).
   • bash 1: make mlflow
   • bash 2: make rest
     • Show where to find the REST API URI in the CodeSpaces VSCode “Ports”.
     • Set the REST visibility to public.
     • Run commands locally. Note that we get the right answer from the SGD model.
   • Note that our fallback for the system to use DVC-stored, Pickle files.

Snapshot

See the live demo.

Give snapshot of the REST API (browser + cURL)