CS 4375: Introduction to Machine Learning

Fall 2023

Introduction to machine learning theory and practice. Learning outcomes:

• Ability to understand and apply basic learning algorithms: decision trees, nearest neighbor, SVMs, neural networks, etc.
• Ability to evaluate the performance of learning algorithms on real data.
• Ability to explain the trade-offs, both practical and theoretical, of different learning strategies for classification and regression.

Where: GR 2.530
When: MW, 11:30am-12:45am

Grading: problem sets (50%), midterm (20%), final (30%)

Prerequisites: CS3345 and CS3341 are required. Some familiarity with basic probability, algorithms, (differential) multivariable calculus, and linear algebra is also assumed.

Instructor: Nicholas Ruozzi
Office Hours: M 1:00pm-2:00pm, W 10:30am-11:30am, and by appointment in ECSS 3.409

TA: Vasundhara Komaragiri
Office Hours: T 2pm-4pm and R 2pm-3pm in ECSS 2.104

Where: ECSW 3.250
When: R, 5:30pm-6:30pm and 7pm-8pm
Instructor: James (Jim) Amato

Week	Dates	Topic	Readings
1	Aug. 21 & 23	Regression & Gradient Descent Perceptron	Bishop, Ch. 1 Sections 1 & 2 of Andrew Ng's Lecture Notes
2	Aug. 28 & Aug. 30	Support Vector Machines Duality & Kernel Methods	Lecture Notes by Andrew Ng Bishop, Ch. 7 for a different perspective Barber, Ch. 17.5 Boyd, Ch. 5
3	Sept. 6	More Duality and Kernel Methods	Boyd, Ch. 5
4	Sept. 11 & 13	Support Vector Machines with Slack Decision Trees	Mitchell, Ch. 3 Bishop, Ch. 7.1, SVMs & SVMs with slack Bishop, Ch. 14.4 Online Probability Notes
5	Sept. 18 & 20	More Decision Trees + k Neareset Neighbor Learning Theory & PAC Bounds	Ng, PAC Learning Notes
6	Sept. 25 & 27	VC Dimension Bias/Variance Trade-off & Bagging	Bishop, Ch. 14
7	Oct. 2 & Oct. 4	Boosting	Hastie et al., Ch. 8.7 & Ch. 15 Short Intro. to AdaBoost
8	Oct. 9 & 12	Clustering Midterm (in class Oct. 12)	Hastie et al., Ch. 14.3.6, 14.3.8, 14.3.9, 14.3.12 Bishop, Ch. 9.1
9	Oct. 16 & 18	PCA
10	Oct. 23 & 25	More PCA Bayesian Methods	Bishop, 1.5, 4.2-4.3.4 Bishop, 2-2.3.4 Bishop, 8.4.1, 9.2-9.4
11	Oct. 30 & Nov. 1	Naive Bayes Logistic Regression	Chapter 11
12	Nov. 6 & Nov. 8	Gaussian Mixture Models Neural Networks	Nielsen Ch. 1-3 Bishop 8.1, 5.1-5.3 ConvNetJS Demos
13	Nov. 13 & 15	More Neural Networks
14	Nov. 27 & 30	Reinforcement Learning More Reinforcement Learning	RL Book
15	Dec. 4 & Dec. 6	Evaluation Metrics Collaborative Filtering + Recap

All problem sets will be available on eLearning and are to be turned in there. See the homework guidelines below for homework policies.

There is no required textbook, but the following books may serve as useful references for different parts of the course.

• Introduction to Machine Learning by Ethem Alpaydin.
• Recognition and Machine Learning by Christopher M. Bishop
• Bayesian Reasoning and Machine Learning by David Barber (free online)
• Machine Learning by Tom Mitchell
• Machine Learning: a Probabilistic Perspective by Kevin Murphy

All exams will be closed book and closed notes.
Midterm: Early October, in class
Final: 12/11 at 11:00am (see Coursebook for details)

I expect you to try solving each problem set on your own. However, if you get stuck on a problem, I encourage you to collaborate with other students in the class, subject to the following rules:

1. You may discuss a problem with any student in this class, and work together on solving it. This can involve brainstorming and verbally discussing the problem, going together through possible solutions, but should not involve one student telling another a complete solution.
2. Once you solve the homework, you must write up your solutions on your own, without looking at other people's write-ups or giving your write-up to others.
3. In your solution for each problem, you must write down the names of any person with whom you discussed it. This will not affect your grade.
4. Do not consult solution manuals or other people's solutions from similar courses - ask the course staff, we are here to help!

Late homework will NOT be accepted except in extreme circumstances or those permitted by university policy (e.g., a religious holiday). All such exceptions MUST be cleared in advance of the due date.

UT Dallas Course Policies and Procedures

For a complete list of UTD policies and procedures, see here.