Model-Based Derivative-Free Optimization: Algorithms and Approximation Theory (Part 1)

Derivative-free optimization (DFO) refers to nonlinear optimization algorithms that do not rely on the availability of gradient or Hessian information. It is primarily designed for settings when functions are black-box, expensive to evaluate and/or noisy, relevant to applications including climate science and quantum computing. A widely used and studied class of DFO methods for local optimization is model-based DFO (MBDFO), where the general principles from nonlinear optimization algorithms are followed, but with local approximations to the objective constructed by polynomial interpolation (rather than, e.g. Taylor series). In these lectures, we will cover the foundational approximation theory and algorithms behind MBDFO, and look at recent extensions to constrained and noisy problems.

This is part 1 of a 3-part lecture series.