» I-X Seminar Series: Understanding message passing: limitations of the paradigm and new possibilities with Francesco Di Giovanni

Speaker

Francesco Di Giovanni

I am a Senior Research Associate in the Department of Computer Science at the University of Oxford, working under the supervision of Michael Bronstein. Specialising in Geometric Deep Learning (GDL), my research has primarily revolved around Graph Neural Networks. I have made contributions in identifying limitations of the current message-passing paradigm and have developed extensions to enhance their efficacy, particularly for tasks involving long-range interactions. My current research also spans generative AI and a theoretical reassessment of conventional GDL methodologies. Before joining Oxford, I served as a Research Associate at the University of Cambridge and as a Machine Learning Researcher at Twitter. I earned my PhD in Mathematics from UCL in 2021, under the guidance of Jason Lotay, with a thesis on Ricci flow. My general goal is understanding how symmetries and geometry of the data affect the power and optimization of deep learning models, with the ambition of working towards the next generation of AI for non-Euclidean domains, which are common in Life Sciences.

Talk Title

Understanding message passing: limitations of the paradigm and new possibilities

Talk Summary

Message passing (MP) stands as a cornerstone in Geometric Deep Learning, driving the success of Graph Neural Networks (GNNs) in analysing both graphs and point clouds, and leading to empirical achievements in many scientific domains. Despite its widespread application, the theoretical underpinnings of MP’s successes and limitations remain underexplored. This talk delves into a key systematic limitation known as over-squashing, which significantly impacts problems involving long-range interactions. I will discuss how over-squashing emerges from a disconnect between data topology and the intended task, highlighting its role in revealing shortcomings of existing approaches to study the expressive power of GNN models. A major focus will be on graph rewiring, a novel approach in MP that treats topology as a critical design element, offering a solution to mitigate over-squashing and expand the capabilities of MP. The talk will conclude by exploring exciting future directions, including applications in generative AI and the sparsification of large Transformer models, promising new horizons for MP’s evolution in Geometric Deep Learning.