RoMaDS - Events

Click here for a list of our past events and here for a presentation of the Rome Centre on Mathematics for Modelling and Data ScienceS.

 

Upcoming events

08.02.2023 - Seminar: Francesco Tudisco (Gran Sasso Science Institute), Efficient training of low-rank neural networks
Aula Dal Passo, 14h00. Link to streaming.
Abstract

Neural networks have achieved tremendous success in a variety of applications. However, their memory footprint and computational demand can render them impractical in application settings with limited hardware or energy resources. At the same time, overparametrization seems to be necessary in order to overcome the highly nonconvex nature of the optimization problem. An optimal trade-off is then to be found in order to reduce networks' dimensions while maintaining high performance. Popular approaches in the literature are based on pruning techniques that look for "winning tickets", smaller subnetworks achieving approximately the initial performance. However, these techniques are not able to reduce the memory footprint of the training phase and can be unstable with respect to the input weights. In this talk, we will present a training algorithm that looks for "low-rank lottery tickets" by interpreting the training phase as a continuous ODE and by integrating it within the manifold of low-rank matrices. The low-rank subnetworks and their ranks are determined and adapted during the training phase, allowing the overall time and memory resources required by both training and inference phases to be reduced significantly. We will illustrate the efficiency of this approach on a variety of fully connected and convolutional networks.
The talk is based on:
S Schotthöfer, E Zangrando, J Kusch, G Ceruti, F Tudisco
Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations
NeurIPS 2022
https://arxiv.org/pdf/2205.13571.pdf


15.02.2023 - Seminar: Francesco Vaccarino (Politecnico di Torino), Title TBA
Aula Dal Passo, 14h00.
Abstract

TBA


22.03.2023 - Seminar: Federico Camia (NYU Abu Dhabi), Title TBA
Aula Dal Passo, 14h00.
Abstract

TBA