Instructor: Niao He

Nearly every problem in machine learning and high-dimensional statistics can be formulated in terms of optimization of some function, possibly under some constraints. In the wake of Big Data era, there is an increasingly strong need to design efficient and scalable algorithms to address optimization problems in large scale - including problems exhibiting inherently high dimensions and problems involving truly massive data sets. The key aim of this course is to expose students to a wide range of modern algorithmic developments in optimization (mainly in convex optimization) and bring them near the frontier of research in large-scale optimization and machine learning.

Courtesy warning: The course will be theory-oriented and emphasize deep understanding of structures of optimization problems and computation complexity of numerical algorithms. The course will not cover any software or tools used for big data analytics and is not an application course.

Students are expected to have strong knowledge of linear algebra, real analysis, and probability theory. Some prior exposure to optimization at a graduate level is preferred.

No book is required, but you are highly recommended to read the following:

1. Ben-Tal & Nemirovski. Lectures on Modern Convex Optimization, SIAM, 2011.
2. Nesterov. Introductory Lectures on Convex Optimization: A Basic Course, Kluwer-Academic, 2003.
3. Sra, Nowozin, Wright (eds). Optimization for Machine Learning. MIT Press, 2011.
4. Bubeck. Convex Optimization: Algorithms and Complexity. In Foundations and Trends in Machine Learning, 2015.