Date/Time
Date(s) - 22/11/2024
11:00 am - 12:00 pm

Location: Zoom

Speaker: Maksim Velikanov

Title: A spectral perspective on Gradient Descent dynamics for quadratic problems

Abstract: Classical optimization research often emphasizes algorithmic development without explicitly considering problem structure. However, many modern optimization problems exhibit specific structures that can be linked to quadratic problems with power-law spectral distributions. In this talk, I will focus on such problems, primarily discussing the non-stochastic case while briefly addressing the effects of mini-batch sampling noise. The goal is to illustrate how incorporating problem-specific structure give more fine-grained insights into optimization dynamics, eventually leading to accelerated algorithms. We will see how the interaction between optimization algorithms and learning on different spectral scales gives a useful and constructive  perspective on the problem.