## CS 6375: Introduction to Machine Learning

Spring 2020### Course Info

Introduction to machine learning theory and practice. Learning outcomes:

- Ability to understand and apply basic learning algorithms: decision trees, nearest neighbor, SVMs, neural networks, etc.
- Ability to evaluate the performance of learning algorithms on real data.
- Ability to explain the trade-offs, both practical and theoretical, of different learning strategies for classification and regression.

**Where:**ECSN 2.126**When:**TR, 2:30pm-3:45pm**Instructor:**Nicholas Ruozzi**Office Hours:**T 1:30pm-2:30pm, W 11:00am-12:00pm, and by appointment in ECSS 3.409**TA:**Hailiang Dong**Office Hours:**M 6:00pm-7:00pm, R 11:00am-12pm in ECSS 2.104A1**Grading:**problem sets (50%), midterm (20%), final (30%)**Prerequisites:**CS5343 Algorithm Analysis and Data Structures. Familiarity with programming, basic probability, algorithms, multivariable calculus, and linear algebra is also assumed.### Schedule & Lecture Slides

Week | Dates | Topic | Readings |

1 | Jan. 14 & 16 | Introduction & Regression Perceptron | Bishop, Ch. 1 Sections 1 & 2 of Andrew Ng's Lecture Notes |

2 | Jan. 21 & Jan. 23 | Support Vector Machines Duality & Kernel Methods | Lecture Notes by Andrew Ng Bishop, Ch. 7 for a different perspective Barber, Ch. 17.5 Boyd, Ch. 5 |

3 | Jan. 28 & Jan. 30 | More Duality and Kernel Methods Support Vector Machines with Slack | Bishop, Ch. 7.1, SVMs & SVMs with slack |

4 | Feb. 4 & Feb. 6 | Decision Trees | Mitchell, Ch. 3 Bishop, Ch. 14.4 Online Probability Notes |

5 | Feb. 11 & 13 | k Neareset Neighbor Learning Theory & PAC Bounds | Ng, PAC Learning Notes |

6 | Feb. 18 & 20 | VC Dimension & Bias/Variance Trade-off |
Bishop, Ch. 14 |

7 | Feb. 25 & Feb. 27 | Bias/Variance Trade-off & Bagging Boosting |
Hastie et al., Ch. 8.7 & Ch. 15 Short Intro. to AdaBoost |

8 | March 3 & 5 |
Midterm More Boosting | |

9 | March 10 & 12 |
Clustering PCA |
Hastie et al., Ch. 14.3.6, 14.3.8, 14.3.9, 14.3.12 Bishop, Ch. 9.1 PCA Notes |

10 | March 31 & April 2 | More PCA Bayesian Methods | Bishop, 1.5, 4.2-4.3.4 Bishop, 2-2.3.4 Bishop, 8.4.1, 9.2-9.4 |

11 | April 7 & 9 | Naive Bayes | Bishop, 8.4.1, 9.2, 9.3, 9.4 |

12 | April 14 & 16 |
Logistic Regression Gaussian Mixture Models | Bishop 8.1, 5.1-5.3 Other Mixture Model Notes (1) (2) |

13 | April 21 & 23 | Neural Networks |
Nielsen Ch. 1-3 ConvNetJS Demos |

14 | April 28 & 30 | Reinforcement Learning |
Nielsen Ch. 3 Reinforcement Learning Notes |

### Problem Sets

All problem sets will be available on eLearning and are to be turned in there. See the homework guidelines below for homework policies.

### Textbooks & References

There is no required textbook, but the following books may serve as useful references for different parts of the course.

*Introduction to Machine Learning*by Ethem Alpaydin.*Recognition and Machine Learning*by Christopher M. Bishop*Bayesian Reasoning and Machine Learning*by David Barber (free online)*Machine Learning*by Tom Mitchell*Machine Learning: a Probabilistic Perspective*by Kevin Murphy

### Exams

All exams will be closed book and closed notes.

**Midterm:**TBD, in class**Final:**TBD, during exam period### Homework Guidelines*

I expect you to try solving each problem set on your own. However, if you get stuck on a problem, I encourage you to collaborate with other students in the class, subject to the following rules:

- You may discuss a problem with any student in this class, and work together on solving it. This can involve brainstorming and verbally discussing the problem, going together through possible solutions, but should not involve one student telling another a complete solution.
- Once you solve the homework, you must write up your solutions on your own, without looking at other people's write-ups or giving your write-up to others.
- In your solution for each problem, you must write down the names of any person with whom you discussed it. This will not affect your grade.
- Do not consult solution manuals or other people's solutions from similar courses - ask the course staff, we are here to help!

**Late homework**will**NOT**be accepted except in extreme circumstances or those permitted by university policy (e.g., a religious holiday). All such exceptions**MUST**be cleared in advance of the due date.### UT Dallas Course Policies and Procedures

For a complete list of UTD policies and procedures, see here.

**adpated from*

*David Sontag*