Randomised Subspace Methods for Scalable Derivative-Free Optimisation [slides available]

Abstract

Most algorithms for optimising nonlinear functions rely on access to (possibly stochastic) derivative information. However, for problems including adversarial example generation for neural networks and fine-tuning large language models, good derivative information can be difficult to obtain and “derivative-free” optimisation (DFO) algorithms are beneficial. Although there are many approaches for DFO, they generally struggle to solve large-scale problems such as those arising in machine learning. In this talk, I will introduce new scalable DFO algorithms based on random subspaces and develop a novel average-case analysis of such algorithms.

Lindon Roberts

Lecturer

My research is in numerical analysis and data science, particularly nonconvex and derivative-free optimization.