Guest Speaker: Lorenzo Rosasco
All dates for this event occur in the past.
Lorenzo Rosasco
DL 480
2015 Neil Avenue
Columbus, Ohio 43210
(Un)conventional regularization for efficient large scale machine learning
Regularization is classically designed by penalizing or imposing explicit constraints to an empirical objective function. This approach can be derived from different perspectives and has optimal statistical guarantees. However, it postpones computational considerations to a separate analysis. In large scale scenarios, considering independently statistical and numerical aspects often leads to prohibitive and unrealistic computational requirements. It is then natural to ask whether different regularization principles exist or can be derived to encompass both aspects at once.
In this talk, we will present several ideas in this direction, showing how procedures typically developed to perform efficient computations can often be seen as a form of implicit regularization. We will discuss how iterative optimization of an empirical objective leads to regularization, and analyze the effect of acceleration and stochastic approximations. We will further discuss the regularization effect of sketching/subsampling methods by drawing a connection to classical regularization projection methods common in inverse problems.
We will show how these forms of implicit regularization obtain optimal statistical guarantees, with dramatically reduced computational properties.
Joint work will Alessandro Rudi, Silvia Villa, Junhong Lin, Luigi Carratino.
Bio: Lorenzo Rosasco is associate professor at University of Genova and Visiting professor at the Massachusetts Institute of Technology (MIT). He is also a researcher at the Italian Technological Institute (IIT), where he coordinates the joint IIT-MIT Laboratory for Computational and Statistical Learning. He received his PhD in 2006 from the University of Genova, after being a visiting student at the Center for Biological and Computational Learning at MIT, the Toyota Technological Institute at Chicago (TTI-Chicago) and the Johann Radon Institute for Computational and Applied Mathematics. Between 2006 and 2009 he has been a postdoctoral fellow at the Center for Biological and Computational Learning at MIT. His research focuses on theory and algorithms for machine learning.
Host: Misha Belkin