Introduction to Statistical Machine Learning 2017

ANU COMP4670/COMP8600

Course Lecturers and Coordinator

Cheng Soon Ong (course convener) and Christian Walder (second examiner)
(email: firstname.lastname@anu.edu.au)

Tutors

Adele Jackson, Matthew Alger, Sultan Majeed

Time and Place

See the schedule below

News

See the news forum on Wattle.

Enrollment

Prerequisites: some background in elementary statistics and probabilities, numerical algorithms, and programming experience. Please have a look at the first tutorial sheet as an indication of the kind of mathematics and statistics that the course builds upon.

If you fulfil official requirements but cannot automatically enroll, please send an email to Cheng Soon Ong (firstname.lastname@anu.edu.au) that he can support your enrolment.

Assessment

The assessment consists of written assignments and a written exam:


Late assignments will incur a penalty of 20% (of the maximum grade for that assigment) per day or part thereof.

Update 27 April: The assigment 2 due date has been extended to 22/5 at 23:59 (see below). However, the maximum lateness is limited: submissions will not be accepted after 24/5 at 16:00, since the solutions will be discussed during that week's lab sessions.

Important Dates

See the schedule below for assignment due dates and the examination time.

Contact hours for students

After the lectures and per email.

Textbook

Required: Christopher M. Bishop: Pattern Recognition and Machine, Springer (selected parts)

We also recommend (ordered by priority):

Overview

Statistical Machine Learning plays a key role in science and technology. Some examples of applications using Statistical Machine Learning techniques are e.g.

Some of the basic questions raised are

This course provides a broad but thorough introduction to the methods and practice of statistical machine learning. It will focus on the mathematical and statistical foundations of machine learning. Students will learn how to implement efficient machine learning algorithms on a computer based on their mathematical formulation, and to understand machine learning from first principles. Topics covered will include Bayesian inference and maximum likelihood modeling; regression, classification, density estimation, clustering, principal and independent component analysis; parametric, semi-parametric, and non-parametric models; basis functions, neural networks, kernel methods, and graphical models; deterministic and stochastic optimisation; overfitting, regularisation, and validation.

The course will use Python3 for all examples, tutorials and assignments. In particular, we will use the jupyter notebook which combines code, text, mathematics, plots and rich media into a single document. It will be provided on all computers used in the tutorials and labs.

Schedule

(to be adapted and refined throughout the course)

Schedule for COMP4670/COMP8600 in semester 1, 2017
week Lecture Lecture Tutorial ⁄ Lab
Tue 4:00 - 5:30 pm @ PHYS T Wed 4:00 - 5:30 pm @ RS CHEM T Wed 5:30 - 7:30 pm @ CSIT N113
Thu 11 - 1 pm @ CSIT N111
Thu 6 - 8 pm @ CSIT N112
20/2 Assignment 0 available (download)
20/2 - 24/2 Overview (Slides) Introduction (Slides) none
27/2 - 3/3 Linear Algebra (Slides) Probability (Slides) Introduction to Python
(download)
solutions (download)

Linear Algebra and Optimisation
(download)
solutions (download)
6/3 - 10/3 Linear Regression 1 (Slides) Linear Regression 2 (Slides) Linear Regression
(download)

solutions (download)
10/3 at 23:59 Assignment 0 due
13/3 Assignment 1 available (download)
13/3 - 17/3 Linear Classification 1 (Slides) Linear Classification 2 (Slides) Classification
(download)


solutions (download)
20/3 - 24/3 Neural Networks 1 (Slides) Neural Networks 2 (Slides) Neural Network
(download)

solutions (download)
27/3 - 31/3 Kernel Methods (Slides) Sparse Kernel Machines (Slides) Kernel Regression
(download)

solutions (download)
3/4 - 7/4 none none none
10/4 - 14/4 none none none
17/4 - 21/4 Principal Component Analysis (Slides) Neural Networks 3 (Slides) PCA
(download)

solutions (download)
18/4 at 23:59 Assignment 1 due
24/4 Assignment 2 available (download)
24/4 - 28/4 ANZAC Day Graphical Models 1 (Slides) Discussion of Assignment 1
1/5 - 5/5 Graphical Models 2 (Slides) Graphical Models 3 (Slides) Conditional Independence
(download)

solutions (download)
8/5 - 12/5 Mixture Models and EM 1 (Slides) Mixture Models and EM 2 (Slides) Mixture Models
(download)

solutions (download)
15/5 - 19/5 Sequential Data 1 (Slides) Sequential Data 2 (Slides) Sum Product
(download)

solutions (download)
22/5 at 23:59 Assignment 2 due
22/5 - 26/5 Clustering (guest lecture) Discussion/Summary (Slides) Discussion of Assignment 2
Monday, 5 June, 9 am, Sports Hall, building 19 Written Examination (All Slides)