ENGG 5501: Foundations of Optimization

2020-21 First Term

Announcements

NEW: Here is the final solution.
The take-home final examination will be released at 12pm (noon) on December 21 (Monday) and due at 11:59pm on December 22 (Tuesday). It will cover Handout 5, Handout 6 (Sections 2 and 4), and Handout 7. Although the focus of the final will be on material after the midterm, you are expected to be familiar with the material before the midterm. As usual, you should scan/take a picture of/typeset your answer script and submit a single file in pdf/doc/docx format via Blackboard. Please make sure you upload all the pages of your answer script. No submission after the deadline will be entertained.

The rules are similar to those for the midterm. You can use any reference material, provided that you cite the results you used in your answer. However, you must work on the exam on your own. In particular, no communication pertaining to the exam, whether you are the initiator or receiver, of any kind is allowed. If you have questions about the rules of the exam, please clarify with the teaching staff as soon as possible.

Here are the practice final and its solution.
Here is the solution to Homework 5.
Here is the midterm solution.
Handout 7 is posted.
Welcome to ENGG 5501! The ZOOM link has already been emailed to those who have registered in the course.
Students who are interested in taking the course but have not yet registered (or are not able to register) should contact the course instructor.
The lecture videos can be accessed from Blackboard or via the link provided on Piazza.
To better facilitate discussions and Q&As, we have set up a forum on Piazza. Please follow this link to sign up.

General Information

Instructor: Anthony Man-Cho So (manchoso at se.cuhk.edu.hk)
Office Hours: By appointment and online until further notice
Lecture Time/Location:
- Mondays 4:30pm - 6:15pm, onine via ZOOM
- Wednesdays 3:30pm - 5:15pm, onine via ZOOM

Teaching Assistants:
- Jiajin Li (jjli at se.cuhk.edu.hk)
  - Office Hours: By appointment and online until further notice
- Yuen Man Pun (ympun at se.cuhk.edu.hk)
  - Office Hours: By appointment and online until further notice
- Jiaojiao Zhang (jjzhang at se.cuhk.edu.hk)
  - Office Hours: By appointment and online until further notice

Online Q&A Forum: Follow this link.

Course Description

In this course we will develop the basic machinery for formulating and analyzing various optimization problems. Topics include convex analysis, linear and conic linear programming, nonlinear programming, optimality conditions, Lagrangian duality theory, and basics of optimization algorithms. Applications from different fields, such as combinatorial optimization, communications, computational economics and finance, machine learning, and signal and image processing, will be used to complement the theoretical developments. No prior optimization background is required for this class. However, students should have workable knowledge in multivariable calculus, real analysis, linear algebra and matrix theory.

Course Requirements

Homework sets (35%), midterm examination (30%), and final examination (35%).

General References

Ben-Tal, Nemirovski: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications, SIAM, 2001.
Boyd, Vandenberghe: Convex Optimization, Cambridge University Press, 2004.
Güler: Foundations of Optimization, Springer, 2010.
Luenberger, Ye: Linear and Nonlinear Programming (4th Edition), Springer, 2016.
Nesterov: Introductory Lectures on Convex Optimization: A Basic Course, Kluwer Academic Publishers, 2004.

Handouts

Handout A: Information Sheet
Handout B: Linear Algebra Cheat Sheet
Handout C: Real Analysis Cheat Sheet

The mathematical prerequisites for this course are summarized in Handouts B and C. Students are strongly advised to go through them carefully.
Handout 1: Introduction
Handout 2: Elements of Convex Analysis
Handout 3: Elements of Linear Programming
Handout 4: Some Applications of Linear Programming
Handout 5: Elements of Conic Linear Programming
Handout 6: Some Applications of Conic Linear Programming
Handout 7: Optimality Conditions and Lagrangian Duality

Lecture Notes

Homework Sets

Homework 1 (due September 30, 2020) Solution
Homework 2 (due October 15, 2020) Solution
Homework 3 (due November 4, 2020) Solution
Homework 4 (due December 2, 2020) Solution
Homework 5 (due December 14, 2020) Solution

Last Updated: December 24, 2020