CS 4301: Numerical Methods for Machine Learning and Data Science

Fall 2020

Course Info

An introduction to computational methods in linear algebra and numerical optimization methods with the aim of preparing students for higher level electives in data science, artificial intelligence, and machine learning. Learning outcomes: Where: Online
When: MW, 10:00am-11:15am

Instructor: Nicholas Ruozzi
Office Hours: M 11:30-12:30, W 12:30pm-1:30pm, and by appointment online.

TA: Shahab Shams
Office Hours: R 10:30am - 12:30pm

Grading: problem sets (70%), midterm (15%), final (15%)

Required Prerequisites: CS3345, Data Structures and Algorithms
Recommended Prerequisites: MATH 2413 (Differential Calculus) or MATH 2417 (Calculus I) and MATH 2418 (Linear Algebra). Comfort with programming, basic probability, and algorithms is also assumed.

Schedule & Lecture Slides

Week Dates Topic Readings
1 Aug. 17 & 19 Introduction
Gradients and Multivariable Calculus
2 Aug. 24 & 26 Convex Sets and Convex Functions
Gradient Descent
Boyd 2.1-2.3
Boyd 9.1-9.3 (can skip example subsections)
3 Aug. 31 & Sept. 2 Subgradient Methods
Convex Optimization
Boyd 9.1-9.3
Boyd 4.1-4.3
4 Sept. 9 Projected Gradient
5 Sept. 14 & 16 Duality and Lagrange Multipliers
Boyd 5
6 Sept. 21 & 23 Constraint Qualification and KKT
Boyd 5
7 Sept. 28 & 30 Second Order Methods
Boyd 3.3, 9.5, 10.1-10.2
8 Oct. 5 & 7 ML Applications
9 Oct. 12 & 14 Linear Algebra Review
Positive Semidefinite Matrices
Boyd A.5
10 Oct. 19 & 21 Eigenvectors, Eigenvalues, and Semidefinite Programming. Boyd A.5

Problem Sets

All problem sets will be available on eLearning and are to be turned in there. See the homework guidelines below for homework policies.

Textbooks & References

There is no required textbook, but the following books may serve as useful references for different parts of the course.

Exams

All exams will be take home, open book, open notes.
Midterm: 10/14, Take Home
Final: TBD, during exam period

Homework Guidelines*

I expect you to try solving each problem set on your own. However, if you get stuck on a problem, I encourage you to collaborate with other students in the class, subject to the following rules:
  1. You may discuss a problem with any student in this class, and work together on solving it. This can involve brainstorming and verbally discussing the problem, going together through possible solutions, but should not involve one student telling another a complete solution.
  2. Once you solve the homework, you must write up your solutions on your own, without looking at other people's write-ups or giving your write-up to others.
  3. In your solution for each problem, you must write down the names of any person with whom you discussed it. This will not affect your grade.
  4. Do not consult solution manuals or other people's solutions from similar courses - ask the course staff, we are here to help!
Late homework will NOT be accepted except in extreme circumstances or those permitted by university policy (e.g., a religious holiday). All such exceptions MUST be cleared in advance of the due date.

UT Dallas Course Policies and Procedures

For a complete list of UTD policies and procedures, see here.


*adpated from
David Sontag