CS 6347: Statistical Methods in AI and ML

Spring 2015

Course Info

Where: ECSN Building 2.110
When: MW, 11:30am-12:45pm

Instructor: Nicholas Ruozzi
Office Hours: Tuesday 11am-12pm and by appointment in ECSS 3.409

TA: Travis Goodwin
Office Hours: Thursday 2pm-3pm and by appointment in Open Lab 2.103B1

Grading: problem sets (70%), final project (25%), class participation & extra credit (5%)
Attendance is MANDATORY. The instructor reserves the right to lower final grades as a result of poor attendance.

Prerequisites: some familiarity with basic probability, linear algebra, and introductory machine learning (helpful, but not required).

Schedule & Lecture Slides

Week Dates Topic Readings
1 Jan. 12 & 14 Introduction & Motivating Example
Probability Review
K&F: Ch. 1 & 2
2 Jan. 21 Bayesian Networks
K&F: Ch. 3
3 Jan. 26 & 28 Undirected Graphical Models
Variable Elimination & Belief Propagation
K&F: Ch. 4 & Ch. 9
(skip the starred sections)
4 Feb. 2 & 4 Approximate MAP Inference
MAP LP & Lagrange Multipliers
K&F: 13.1-13.5, A.5.3
Boyd: Ch. 5.1-5.5
5 Feb. 9 & 11 Variational Inference
Naive Mean Field
K&F 11.1-11.2, 11.5
Sections 1-3 of this paper
MATLAB code for mean field
6 Feb. 16 & 18 Basics of Sampling
MCMC Sampling
K&F 12.1-12.3
7 Feb. 23 & 25 Classes canceled due to weather
8 March 2 & 4 Introduction to Machine Learning
Maximum Likelihood Estimation
K&F: 17.1-17.4
9 March 9 & 11 More Maximum Likelihood Estimation
Approximate Maximum Entropy &
Conditional Gradients
K&F: 20.1-20.5
10 March 16 & 18 Spring Break
11 March 23 & 25 Expectation Maximization
LDA and Variational EM
K&F: 19.1-19.2
Box 17.E
12 March 30 & April 1 Alternative Learning Strategies
Bayesian Network
Structure Learning
K&F: 20.6
13 April 6 & April 8 Exponential Familes &
Expectation Propagation
14 April 15 Neural Networks Nielsen: Ch. 1
15 April 20 & 22 Backpropagation and RBMs
Graph Cuts
Nielsen: Ch. 2
16 April 27 & 29 Advanced Topics in MAP Inference:
Lifts of Graphs and the MAP LP

Problem Sets

All problem sets will be available on the eLearning site and are to be turned in there. See the homework guidelines below for the homework policies.

Textbooks & References

The following textbook is required: Other references that may be helpful:

Homework Guidelines*

We expect you to try solving each problem set on your own. However, when being stuck on a problem, I encourage you to collaborate with other students in the class, subject to the following rules:
  1. You may discuss a problem with any student in this class, and work together on solving it. This can involve brainstorming and verbally discussing the problem, going together through possible solutions, but should not involve one student telling another a complete solution.
  2. Once you solve the homework, you must write up your solutions on your own, without looking at other people's write-ups or giving your write-up to others.
  3. In your solution for each problem, you must write down the names of any person with whom you discussed it. This will not affect your grade.
  4. Do not consult solution manuals or other people's solutions from similar courses - ask the course staff, we are here to help!
Late homeworks will NOT be accepted except in extreme circumstances or those permitted by university policy (e.g., a religious holiday). All such extensions MUST be cleared in advance of the due date.

*adpated from David Sontag