Professor Mikhail Belkin published in PNAS

Posted: 

The paper titled Reconciling modern machine-learning practice and the classical bias–variance trade-off authored by Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Manda was published in August 2019 in PNAS.

While breakthroughs in machine learning and artificial intelligence are changing society, our fundamental understanding has lagged behind. It is traditionally believed that fitting models to the training data exactly is to be avoided as it leads to poor performance on unseen data. 

However, powerful modern classifiers frequently have near-perfect fit in training, a disconnect that spurred recent intensive research and controversy on whether theory provides practical insights. In this work, we show how classical theory and modern practice can be reconciled within a single unified performance curve and propose a mechanism underlying its emergence. We believe this previously unknown pattern connecting the structure and performance of learning architectures will help shape design and understanding of learning algorithms.

PNAS is one of the world’s most-cited and comprehensive multidisciplinary scientific journals, publishing more than 3,200 research papers annually. The Proceedings of the National Academy of Sciences (PNAS), the official journal of the National Academy of Sciences (NAS), is an authoritative source of high-impact, original research that broadly spans the biological, physical, and social sciences. The journal is global in scope and submission is open to all researchers worldwide.

Click here for more information.