“Accuracy of High-Dimensional Deep Learning Networks”

Speaker: Andrew Barron
Professor of Statistics and Data Science at Yale University

Monday, September 17, 4:15-5:30pm

Location: Yale Institute for Network Science, 17 Hillhouse Ave, Room 328

Talk summary: It has been experimentally observed in recent years that multi-layer artificial neural networks have a surprising ability to generalize, even when trained with far more parameters than observations. Is there a theoretical basis for this? The best available bounds on their metric entropy and associated complexity measures are essentially linear in the number of parameters, which is inadequate to explain this phenomenon. Here we examine the statistical risk (mean squared predictive error) of multi-layer networks with L1 controls on their parameters and with ramp activation functions (also called lower-rectified linear units). In this setting, the risk is shown to be upper-bounded by [(L^3 log d)/n]^{1/2}, where d is the input dimension to each layer, L is the number of layers, and n is the sample size. In this way, the input dimension can be much larger than the sample size and the estimator can still be accurate, provided the target function has such L1 controls and that the sample size is at least moderately large compared to L^3 log d. The heart of the analysis is the development of a sampling strategy that demonstrates the accuracy of a sparse covering of deep ramp networks. Lower bounds show that the identified risk is minimax optimal, this being so already in the subclass of functions with L = 2. This is joint work with Jason Klusowski.

“Improving Generative Adversarial Networks using Game Theory and Statistics”

Speaker: Constantinos Daskalakis
Professor of Computer Science and Electrical Engineering at MIT

Wednesday, September 12, 12:00-1:00pm

Location: Yale Institute for Network Science, 17 Hillhouse Ave, Room 328

Talk summary: Generative Adversarial Networks (aka GANs) are a recently proposed approach for learning samplers of high-dimensional distributions with intricate structure, such as distributions over natural images, given samples from these distributions. They are obtained by setting up a two-player zero-sum game between two neural networks, which learn statistics of a target distribution by adapting their strategies in the game using gradient descent. Despite their intriguing performance in practice, GANs pose great challenges to both optimization and statistics. Their training suffers from oscillations, and they are difficult to scale to high-dimensional settings. We study how game-theoretic and statistical techniques can be brought to bear on these important challenges. We use Game Theory towards improving GAN training, and Statistics towards scaling up the dimensionality of the generated distributions.

The GSP Analysis Centers co-organized the 2018 Harvard Program in Quantitative Genomics Conference on Biobanks: Study Design and Data Analysis. See https://www.hsph.harvard.edu/2018-pqg-conference/. Below is the conference announcement. The GSP investigators are welcome to attend it if interested.

Biobanks: Study Design and Data Analysis

November 1-2, 2018
Harvard Medical School Conference Center | Boston, MA
#HarvardPQG18

The impetus for the 2018 Conference of the Program in Quantitative Genomics of Harvard T.H. Chan School of Public Health comes from the proliferation of large scale biobanks worldwide. Biobanks are composed of massive genetic and genomic data, epidemiological data, Electronic Medical Records, wearable devices and imaging data. Examples of large biobanks include UK Biobank, China Kadoorie Biobank, eMERGE, Finland Biobank, Million Veteran Program, and MyCode Project of the Geisinger Health System, among others. The use of biobanks is becoming an essential and potentially revolutionary approach to biomedical research, with the capacity to improve the prevention, diagnosis, and treatment of a wide range of illnesses, and to advance personalized health. To take full advantage of the enormous opportunities presented in biobanks, there is an urgent need for discussing important quantitative issues, leveraging interdisciplinary expertise, and designing studies with increased scale and power. This conference aims at discussing several key quantitative challenges in biobank studies, including designing biobanks to meet a wide array of needs, developing strategies for improving phenotyping accuracy, harmonizing data across biobanks, and developing analytic methods for biobank data.

The conference will be centered on the following three topics:

SESSION 1: Types of Biobanks
SESSION 2: Biobank Data Analysis
SESSION 3: Phenotyping and Harmonization Across Biobanks

Keynote Speakers:
Tianxi Cai Harvard T.H. Chan School of Public Health
Gil McVean Oxford University
Catherine Sudlow UK Biobank, University of Edinburgh

Distinguished Speakers:
Zhengming Chen China Kadoorie Biobank, Oxford University
Kelly Cho Brigham and Women’s Hospital, Harvard Medical School Joshua Denny Vanderbilt University
David Ledbettter Geisinger Health System
Seunggeun Shawn Lee University of Michigan
Shawn Murphy Massachusetts General Hospital, Harvard Medical School Benjamin Neale Massachusetts General Hospital, Harvard Medical School Samuli Ripatti University of Helsinki, Institute for Molecular Medicine Finland (FIMM)
Manuel Rivas Stanford University

The conference program includes time for scientific presentations and a poster session and reception for submitted abstracts. Please visit the conference website for more details.

Registration + travel awards will be provided to support junior researchers who submit abstracts.
*Three abstracts will be selected to be presented as 10-minute platform talks

REGISTER

Submit Abstract

Copyright © 2018 Department of Biostatistics, Harvard T.H. Chan School of Public Health, All rights reserved.
Our weekly newsletter for past and present members of the department, and for those who have opted in to receive our mailings.

Our mailing address is:

Department of Biostatistics, Harvard T.H. Chan School of Public Health

655 Huntington Ave

Building 2, 4th Floor

Boston, Ma 02115