AMS 553.761: Nonlinear Optimization I

Nonlinear Optimization I 553.761
TTh 4:30pm - 5:45pm, BLOOMBERG HALL 274

GENERAL INFORMATION

Instructor : Amitabh Basu
Office Hours : Wednesday 5:00--7:00pm, or email for appointment. Office Hours will be in my office : Whitehead 202A.
Email : basu [dot] amitabh [at] jhu [dot] edu

Teaching Assistant : Tian Yang, Noah Wichrowski and Elizabeth Reiland will be the TAs for our class.

Tian's email is tyan4 [at] jhu [dot] edu.
Noah's email is nwichro1 [at] jhu [dot] edu.
Elizabeth's email is ereiland [at] jhu [dot] edu.

Tian's office hours will be on Fridays from 9--10am.
Noah's office hours will be on Tuesdays from 3--4pm.

Notes/Texts : I will use lecture slides prepared by Daniel P. Robinson for the course. They will be periodically posted here.

Introduction. Slides without "Notes"
Background and basics. Slides without "Notes". MATLAB demo file
Convexity. Slides without "Notes"
Newton's method. Slides without "Notes"
Optimality conditions. Slides without "Notes"
Line Search Methods. Slides without "Notes". Conjugate Gradient Notes
Trust Region Methods. Slides without "Notes".
Least Squares. Slides without "Notes".
Nonlinear Equations. Slides without "Notes".
Coordinate Minimization. Slides without "Notes".
Second Order Methods. Slides without "Notes".

Other useful textbooks and resources

Optimization:

J. Nocedal and S. Wright, Numerical Optimization, Second Edition, Springer, (2006)
A. R. Conn, N.I.M Gould, and Ph. L. Toint, Trust Region Methods, SIAM, Philadelphia, PA, (2000)
R. Fletcher, Practical Methods of Optimization, 2nd Edition, Wiley, Chichester and New York, (1987)
D. P. Bertsekas, Nonlinear Programming, Second Edition, Athena Scientific, Belmont, MA, (1999)
A. Ruszcynski, Nonlinear Optimization, Princeton University Press, Princeton, NJ (2006)
Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Kluwer Academic Publishers, Norwell, MA, (2004)

Basic Numerical Analysis:

S. D. Conte and C. de Boor, Elementary Numerical Analysis: an algorithmic approach, International Series in Pure and Applied Mathematics, McGraw-Hill Book Company, (1972).

Syllabus : The syllabus with list of topics to be covered is available HERE.
This course considers algorithms for solving various important nonlinear optimization problems and, in parallel, develops the supporting theory. Primary focus will be on unconstrained and bound-constrained optimization. Topics will include: necessary and sufficient optimality conditions; gradient, Newton, and quasi-Newton based line-search, trust-region, and cubic regularization methods; linear and nonlinear least squares problems; and linear and nonlinear conjugate gradient methods. Special attention will be paid to the large-scale case and will include topics such as limited-memory quasi-Newton methods, projected gradient methods, and subspace accelerated two-phase methods for bound-constrained optimization.

EXAM AND GRADING INFORMATION

There will one in-class Midterm and one in-class Final exam. In addition, I will regularly (approx. every two week) post homework assigments here. You will be asked to hand in some of the HW problems which will be graded (approximately every two weeks). Seriously attempting ALL the homework problems is imperative for your success in the class, and they will give an indication of the kind of problems on the tests.

Homeworks

HW I.
Please hand in your solutions in class on Tuesday, September 25, 2018.
HW II.
There was a typo in Exercise 2.3. The parameter \lambda_min should enter with an exponent of 2 in parts (i) and (ii). Thanks to Junyuan Li for catching this!
Please hand in your solutions in class on Thursday, October 11, 2018
HW III.
Please hand in your solutions in class on Thursday, November 1, 2018.
HW IV.
Please hand in your solutions in class on Thursday, November 15, 2018.
HW V.
There was an issue with original statement of Exercise 5.2. The matrix H needs to be positive definite to have a bounded minimum value. This has now been enforced. Thanks to Ronak Mehta for catching this!
Please hand in your solutions in class on Tuesday, December 4, 2018.

Midterms and Final

The Midterm will be held on Thursday, October 18, 2018 in class. The syllabus for the midterm will be everything that was covered from the beginning of the semester to the lecture on Thursday, October 11, 2018. This is everything up to the first part of the "Trust Region" slides (up to "Global convergence of a complete trust-region algorithm").
The Final Exam will be held from 9am -- 12 noon on Thursday, December 20, 2018.
The syllabus for the final is all the material covered during the course of the semester.

Grades

Your final grade will be based on the following weightage :
Homework - 30%
Midterm - 35%
Final - 35%
Blackboard will be used to post the HW and exam grades. You should be able to access your grades there.