Recommended book for Optimisation?

[Sherry Darlin]

I'm looking to do a course on Optimisation, however there was no prescribed textbook and I'm a bit wary of doing a course without a textbook to reference. There was a generalised list given, of like 10 textbooks, but this is a bit too much, especially with 3 other subjects to do!

Here is the general outline, perhaps someone can recommend 1 - 2 books?

Overview: Optimization is the study of problems in which we wish to optimize (either
maximize or minimize) a function (usually of several variables) often subject to a
collection of restrictions on these variables.

The restrictions are known as constraints
and the function to be optimized is the objective function. Optimization problems are
widespread in the modelling of real world systems, and cover a very broad range of
applications.

Problems of engineering design (such as the design of electronic circuits
subject to a tolerancing and tuning provision), information technology (such as the
extraction of meaningful information from large databases and the classication of
data), nancial decision making and investment planning (such as the selection of
optimal investment portfolios), and transportation management and so on arise in
the form of a multi-variable optimization problem or an optimal control problem.

Introduction: What is an optimization problem? Areas of applications of optimization.
Modelling of real life optimization problems.

Multi-variable optimization. Formulation of multi-variable optimization problems; Struc-
ture of optimization problems: objective functions and constraints. Mathematical
background: multi-variable calculus and linear algebra; (strict) local and (strict)
global minimizers and maximizers; convex sets, convex and concave functions; global
extrema and uniqueness of solutions.

Optimality conditions: First and second order conditions for unconstrained prob-
lems; Lagrange multiplier conditions for equality constrained problems; Kuhn-Tucker
conditions for inequality constrained problems.

Numerical Methods for Unconstrained Problems: Steepest descent method,

Newton's method, Conjugate gradient methods.

Numerical Methods for Constrained Problems: Penalty Methods.

Optimal Control: What is an optimal control problem? Areas of applications of optimal
control. Mathematical background: ordinary differential equations and systems of
linear differential equations.

The Pontryagin maximum principle: Autonomous control problems; unbounded
controls

 

[Nick Alger]

Numerical Optimization by Nocedal and Wright, and Convex Optimization by Boyd are both truly excellent.

 

[Kreizhn]

I definitely agree with Nick Alger with regards to Nocedal and Wright. It is a fantastic book for doing numerical compution: very clear, well motivated and explained, and very comprehensive.

On the other hand, neither of the above suggested books covers optimal control or Pontryagin's principle. Perhaps the most obvious reference here is Pontryagin's The Mathematical Theory of Optimal Processes. It is not exactly user friendly though, but is great if you need a reference for some examples or the actual proof of the PMP. Enid Pinch has a book Optimal Control and the Calculus of Variations which isn't too bad. You can find free course notes online (see attached). Finally, while unlikely, there is (in my opinion) the ultimate control theory book of all time, Jurdjevic's book Geometric Control Theory, though this might be too advanced.