Engineering Process

Keith VanderLinden
Calvin University

Systems Engineering

Systems engineering conceives of, designs, builds, and deploys the entire ML/Data system to satisfy the specified business requirements.

The result must be reliable, scalable, maintainable, and adaptable.

cf. DMLS Figure 1-1

Cf. the definition of ML in the earlier slide deck (i.e., learning patterns from data for prediction). This definition includes building all the elements of that diagram.

See DMLS, p. 29-31.

Software vs ML/Data Engineering

Software Engineering

• Data/code are separate.
• Only code is versioned.
• Code-bases are small.
• Code can be unit-tested.
• Code updates are infrequent.

ML/Data Engineering

• Data/code are coupled.
• Data/models are versioned.
• Datasets can be huge.
• ML is hard to test.
• ML system updates are frequent.

E.g., Netflix re-deploys thousands of times per day.

Systems Engineering Process

Software Engineering

1. Analysis
2. Design
3. Implementation
4. Testing
5. Deployment &
   Maintenance

ML/Data Engineering

1. Project Scoping
2. Data & Model Engineering

    

    

3. System Deployment
4. System Monitoring
5. Business Analysis

The differences on the first page manifest themselves in these different views of the engineering process.

This is an iterative process. Cf. agile software development.

MAY WANT TO COVER AGILE DEVELOPMENT, INCLUDING KANBAN BOARDS NOW AS A PREPARATION FOR GITHUB PROJECTS NEXT WEEK, WHICH WOULD OFF-LOAD SOME OF THAT HEAVY GITHUB MATERIAL HERE. PERHAPS THE IDEA WOULD BE TO COVER MANAGEMENT CONCEPTS HERE (I.E., AGILE METHODS, TIME-BOXED ITERATIONS, KANBAN BOARDS, SOURCE CONTROL, PRIMARILY FOCUSING ON HOW THEY’RE UNIQUE FOR DS), SAVING THE NUTS AND BOLTS OF GITHUB PROJECTS, REPOS, BRANCHES, GIT-FLOW FOR NEXT WEEK.

This course has two-week, time-boxed iterations.

References: - DMLS, p. 33-34 - CS 262, traditional vs agile models, https://github.com/kvlinden-courses/cs262/blob/master/materials/02management/class.md

ML Product Design

Compare and contrast G. Mohandas’s: “project design” (what & why); “systems design” (how)

They’ll fill out some of the fields in a DOCX version of this chart for the lab.

• Background - ??
• Value Proposition - cf. DMLS p. 26-28.
• Objectives - ??
• Solution - cf. DMLS p. 3 (when to use ML); DMLS Figure 1-3.
• Feasibility - ??
• Data - ??
• Metrics - cf. DMLS p. 40
• Evaluation - ??
• Modeling - cf. DMLS “framing”, p. 35-40
• Experimentation - ??
• Feedback - ??
• Project - ??

See: https://madewithml.com/courses/mlops/design/