Can Deep Learning Succeed at Small Scales?

Dr. Rebekka Burkholz
CISPA Helmholtz Center for Information Security,
Saarbrücken, Germany

Deep learning continues to achieve impressive breakthroughs across disciplines but relies on increasingly large neural network models that are trained on massive data sets. Their development inflicts costs that are only affordable by a few labs and prevent global participation in the creation of related technologies. In this talk, we will ask the question if it really has to be like this and discuss some of the major challenges that limit the success of deep learning on smaller scales. We will give three examples of complimentary approaches that could help us address the underlying issues: (i) early neural network sparsification, (ii) the integration of useful inductive bias in the design of problem specific neural network architectures, and (iii) the improvement of training from scratch in the context of graph neural networks.

CRC 1461: Neurotronics
Colloquium: 25-April-2024_29, Thursday, 3:30 pm to 4:00 pm (CET)
The colloquium will start at 4:00 pm

Link to the zoom meeting

Invited by Sonja Reich, referral by Thomas Strunskus
Kiel University, Faculty of Engineering

Download the announcement here.