Jones Seminar: How Do Neural Networks Learn Features from Data?

Optional ZOOM LINK
Meeting ID: 963 3025 4065
Passcode: 956285

The ability of neural networks to learn patterns from data, or features, has been central to their success. In this talk, I will present a unifying mechanism that characterizes feature learning across neural network architectures. Namely, features learned by neural networks are captured by a statistical operator known as the average gradient outer product (AGOP). More generally, the AGOP enables feature learning in machine learning models that have no built-in feature learning mechanism (e.g., kernel methods). I will present two applications of this line of work. First, I will show how AGOP can be used to steer LLMs and vision-language models, guiding them towards specified concepts and shedding light on vulnerabilities in these models. I will then discuss how AGOP connects feature learning with independence testing and how we used AGOP to develop a scalable, nonlinear measure of dependence known as the InterDependence Score (IDS). I will conclude with an application of IDS to million-scale text and genomics datasets, where we use it to identify subpopulations of interest.  

Hosted by Professor Bijan Mazaheri.

About the Speaker(s)

Adit Radhakrishnan
Assistant Professor of Applied Mathematics, MIT

Adit Radhakrishnan is an assistant professor of applied mathematics at MIT and an associate member of the Broad Institute of MIT and Harvard. He was previously an Eric and Wendy Schmidt Center Postdoctoral Fellow at the Broad Institute and a George F. Carrier Postdoctoral Fellow in Harvard's School of Engineering and Applied Sciences. Radhakrishnan completed his PhD in electrical engineering and computer science at MIT. 

His theoretical research focuses on understanding how AI systems learn features from data and how we can design novel, computationally-efficient feature learning algorithms. On the applied side, he works on developing algorithms to discover and characterize structure from heterogeneous biomedical data. Most recently, he has been working on methods to discover cellular programs from million-scale single-cell RNA sequencing data.  

Contact

For more information, contact Amos Johnson at amos.l.johnson@dartmouth.edu.