Colloquium

Louis ten Bosch

AI, Automatic Speech Recognition and phonetics

Over the last couple of years, we witness the enormous impact of AI-inspired end-to-end approaches for Automatic Speech Recognition. In this talk I will discuss the progress in the field, by sketching a number of experiments that aim to find phonetic relations in the latent representations of Wav2vec2.0. Also the use of end-to-end audio decoding will be discussed for the early detection of neurodegenerative diseases (Alzheimer, Parkinson etc.) from the speech signal. Finally, we will address the topic of interpretability of a foundation model, by discussing various techniques that can be used to ‚crack‘ the black box. A nice discussion point is insofar/whether the old ‚classical‘ knowledge about speech is represented in some way in the parameters of a trained end-to-end model, and how.
 

Tiago Pimentel

Promises and Limitations of Causality for Machine Learning Interpretability

How can we move from observing what a model does to understanding why it does it? In this talk, I argue that causality is necessary but not sufficient to uncover the mechanisms underlying model predictions. First, I examine a “macro” view of model analysis, showing how econometric tools—such as regression discontinuity or difference-in-differences—can isolate the causal impact of specific design choices, like tokenisation and training-data selection, on a model’s outputs. Second, I turn to a “micro” view of mechanistic interpretability, focusing on causal abstraction as a method to establish whether a model implements a high-level algorithm. I demonstrate that this approach faces a critical limitation: without strict assumptions about how models encode information, the framework becomes vacuous, implying that any model implements any algorithm. This reveals that the ability to make counterfactual predictions about a model is not, on its own, sufficient to guarantee that we understand it. I conclude with a short discussion of how causality can be used to develop more principled interpretability methods, and with an open question about what interpretability is currently missing.

Xing Han Lu

From a dozen clicks to a thousand steps: The evolution of digital agents and the future of work

A year ago, mentioning the word “agent” in a conversation would’ve yielded a confused stare. So how come everyone is now talking about agents? In this talk, we take a look at earlier developments of agents designed for navigating websites, which can be controlled through a conversational UI. To understand how to train those models, we examine gym environments designed for web agents, with a focus on LLM judges, which are the keys for unlocking environment-based training (such as Reinforcement Learning). We look at how those environments and judges can be combined to design a simple but effectively pipeline to distill frontier models into smaller models, and discuss how to extend them to long-horizon computer-use tasks. Finally, we conclude this talk with a discussion about the pace of improvements of agents capable of doing increasingly more complex work in browsers and on desktops, and muse about how the future of knowledge work might look like.