18-461/18-661: Intro to ML for Engineers


Prof. Carlee Joe-Wong cjoewong@andrew.cmu.edu
Prof. Gauri Joshi gaurij@andrew.cmu.edu


Baris Askin (Pitt)baskin@andrew.cmu.edu
Baran Atalar (Pitt)batalar@andrew.cmu.edu
Jean de Dieu Nyandwi (RW)jeandedi@andrew.cmu.edu
Tianshu Huang (Pitt)tianshu2@andrew.cmu.edu
Harry Jiang (Pitt)hhj@andrew.cmu.edu
Divyansh Jhunjhuwala (Pitt)djhunjhu@andrew.cmu.edu
April Yang (SV)yutongya@andrew.cmu.edu
Hanqing Yang (Pitt)hanqing3@andrew.cmu.edu
Kevin Sebineza (RW)ksebinez@andrew.cmu.edu
Muhammad Ustaomeroglu (Pitt)mustaome@andrew.cmu.edu
Carmel Prosper Sagbo(RW)csagbo@andrew.cmu.edu


PittsburghMon. and Wed.12:00 PM - 1:50 PM ETBH A36
Silicon ValleyMon. and Wed.9:00 AM - 10:50 AM PTB23 118
RwandaMon. and Wed.7:00 PM - 8:50 PM CATCMR F205
(6:00 PM - 7:50 PM CAT after March 10)


PittsburghFri.11:00 AM - 12:20 PM ETBH A36
Fri.3:30 PM - 4:50 PM ETWEH 4623
Silicon ValleyFri.12:30 PM - 1:50 PM PTB23 227
RwandaFri.5:00PM - 7:20PM CATCMR F205

Office Hours

Prof. Carlee Joe-WongMon1:50 pm - 3:00 pmET
Prof. Gauri JoshiThur4:00 pm - 5:00 pmET
Baran Atalar (Pitt)Mon3:00 pm - 4:00 pmET
Tianshu Huang (Pitt)Tue3:00 pm - 4:00 pmET
Divyansh Jhunjhunwala (Pitt)Thur2:00 pm - 3:00 pmET
Harry Jiang (Pitt)Thur5:00 pm - 6:00 pmET
Muhammed Ustaomeroglu (Pitt)Fri12:00 pm - 1:00 pmET
Baris Askin (Pitt)Fri2:15 pm - 3:15 pmET
Hanqing Yang (Pitt)Wed2:00 pm - 3:00 pmET
Sun10:30 am - 12:00 amET
April Yang (SV)Wed2:00 pm - 3:00 pmPT
Carmel Prosper Sagbo(RW)Thur12:00 pm - 1:00 pmCAT
Jean de Dieu Nyandwi (RW)Tue5:00 pm - 6:00 pmCAT
Kevin Sebineza (RW)Mon3:00 pm - 4:00 pmCAT
Fri2:00 pm - 3:00 pmCAT

See Piazza for the Office Hours Zoom links and locations.

Course Overview

This course provides an introduction to machine learning with a special focus on engineering applications. The course starts with a mathematical background required for machine learning and covers approaches for supervised learning (linear models, kernel methods, decision trees, neural networks) and unsupervised learning (clustering, dimensionality reduction), as well as theoretical foundations of machine learning (learning theory, optimization). Evaluation will consist of mathematical problem sets and programming projects targeting real-world engineering applications.


This course is intended for graduate students and qualified undergraduate students with a strong mathematical and programming background. Undergraduate level training or coursework in algorithms, linear algebra, calculus, probability, and statistics is suggested. A background in programming will also be necessary for the problem sets; students are expected to be familiar with python or learn it during the course. At CMU, this course is most similar to MLD's 10-601 or 10-701, though this course is meant specifically for students in engineering.


There will be no required textbooks, though we suggest the following to help you to study (all available online): We will provide suggested readings from these books in the schedule below.


We will use Piazza for class discussions. Please go to the course Piazza site to join the course forum (note: you must use a cmu.edu email account to join the forum). We strongly encourage students to post on this forum rather than emailing the course staff directly (this will be more efficient for both students and staff). Students should use Piazza to:

The course Academic Integrity Policy must be followed on the message boards at all times. Do not post or request homework solutions! Also, please be polite.

Grading Policy

Grades will be based on the following components:

Gradescope: We will use Gradescope to collect PDF submissions of each problem set. Upon uploading your PDF, Gradescope will ask you to identify which page(s) contains your solution for each problem - this is a great way to double check that you haven't left anything out. The course staff will manually grade your submission, and you'll receive feedback explaining your final marks.

Regrade Requests: If you believe an error was made during grading, you'll be able to submit a regrade request on Gradescope. For each homework, regrade requests will be open for only 1 week after the grades have been published. This is to encourage you to check the feedback you've received early!

Academic Integrity Policy

Group studying and collaborating on problem sets are encouraged, as working together is a great way to understand new material. Students are free to discuss the homework problems with anyone under the following conditions: Students are encouraged to read CMU's Academic Integrity Policy.

Using LaTeX

Students are strongly encouraged to use LaTeX for problem sets. LaTeX makes it simple to typeset mathematical equations and is extremely useful for graduate students to know. Most of the academic papers you read were written with LaTeX, and probably most of the textbooks too. Here is an excellent LaTeX tutorial and here are instructions for installing LaTeX on your machine.


This course is based in part on material developed by Fei Sha, Ameet Talwalkar, Matt Gormley, and Emily Fox. We also thank Anit Sahu and Joao Saude for their help with course development.

Schedule (Subject to Change)

1/17 Introduction [Slides] [Annotated] KM, Ch. 1
1/19 Recitation [Annotated Slides] [Python Slides] [Python Notebook] HW 1 Release
1/22 MLE/MAP, Linear Algebra Review [Slides] [Annotated] TM, Estimating Probabilities
KM, Ch. 2 (for a refresh in probability)
Math4ML (review/refresher)
Vectors, Matrices, and Least Squares
Matrix Cookbook
1/24 Linear Regression, Part I [Slides] [Annotated] KM, Ch. 7.1-7.3
Deep Learning Book, Ch. 5*
1/26 Recitation [Slides] [Annotated]
1/29 Linear Regression, Part II [Slides] [Annotated] KM, Ch. 7.4-7.6
Intro to regression
HW 2 Release
1/31 Overfitting, Bias/variance Trade-off [Slides] Deep Learning, Ch. 5.2-5.4
KM, Ch. 6.4
HW 1 due
2/2 Recitation
2/5 Naive Bayes [Slides] [Annotated] CIML, Ch. 9
KM, Ch. 3.5
2/7 Logistic Regression [Slides] [Annotated] KM, Ch. 8.1-8.4, 8.6
Discriminative vs. Generative
2/9 Recitation
2/12 SVM, Part 1 [Slides] [Annotated] ESL, Ch. 12
KM Ch. 14.5
HW 2 due
HW 3 Release
2/14 SVM, Part 2 [Slides] [Annotated] Idiot's Guide to SVM
Duality Supplement
2/16 Recitation
2/19 Nearest Neighbors [Slides] [Annotated] CIML, Ch. 3.1-3.2
2/21 Decision Trees [Slides] [Annotated] CIML, Ch. 1.3
KM, Ch. 16.2
ESL, Ch. 9.2
2/23 Recitation HW 3 due
2/26 Boosting, random forests [Slides] [Annotated] ESL, Ch. 10.1, 10.4-10.6
2/28 Midterm Exam (In-class)
3/1 Recitation
3/4 Spring Break
3/6 Spring Break
3/8 Spring Break
3/11 Graphical Models I [Slides] [Annotated] HW 4 Release
3/13 Graphical Models II [Slides] [Annotated]
3/15 Recitation
3/18 Neural Networks, Part I [Slides] [Annotated] Learning Deep Architectures for AI
3/20 Neural Networks, Part II [Slides] [Annotated] Neural Networks and Deep Learning, Ch.3
Regularization for Deep Learning
3/22 Recitation HW 5 Release
3/25 PyTorch [Slides] [Demo] HW 4 due
3/27 Neural Networks, Part III [Slides] [Annotated] Neural Networks and Deep Learning, Ch.3
Regularization for Deep Learning
3/29 Recitation
4/1 Distributed Learning [Slides] [Annotated]
4/3 Clustering, Part I [Slides] [Annotated] CIML, Ch. 15.1
HW 6 Release
4/5 Recitation [Slides]
4/8 Clustering, Part II [Slides] [Annotated] ESL, Ch. 14.3.1-14.3.9
HW 5 due
4/10 Dimensionality Reduction [Slides] [Annotated] PCA
Independent Component Analysis
4/12 Carnival
4/15 Online Learning (Bandits) [Slides] [Annotated]
4/17 Guest Lecture HW 6 due
HW 7 Release
4/19 Recitation
4/22 Reinforcement Learning, Part I [Slides] [Annotated] RL Book
4/24 Reinforcement Learning, Part II [Slides] [Annotated]
4/26 Recitation
4/30 Final Exam Week
5/1 HW 7 due