Hosted by Mark Peletier

Abstract

In the last ten years, 2nd order ODEs from classical mechanics has been fruitfully applied to study existing accelerated optimization methods and to develop new ones, and in this framework the objective function plays the role of the potential energy. Typically, continuous-time models for accelerated methods are mechanical systems with damping, which is in charge of dissipating the energy of the system and yields convergence. In this talk, we present an optimization method based on a conservative mechanical system (i.e., no damping). Due to the absence of damping, the convergence of this method completely relies on the following adaptive restart strategy:

1) The initial velocity is set equal to 0;

2) By the conservation of the mechanical energy, part of the initial potential energy is transformed into kinetic energy;

3) When a proper restart condition is met, the velocity is reset to zero and the kinetic energy at the restart time is instantly dissipated.

We prove the convergence result both for the continuous-time method and for the discrete-time version.

Finally, we discuss some extensions to the nonsmooth case, with particular focus on l1 regularization.