Derivative-Free Optimisation Methods for Nonlinear Least-Squares Problems

Derivative-free optimisation (DFO) algorithms are a category of optimisation methods for situations when one is unable to compute or estimate derivatives of the objective. The need for DFO arises in applications where techniques such as finite differencing or algorithmic differentiation are inaccurate or impractical, such as when the objective has noise (e.g. Monte Carlo simulations in finance) or is very expensive (e.g. climate simulations). \n