Mathematisches Kolloquium: Capabilities and Limits of Large Language Models

Am 23. Oktober um 14 Uhr spricht Prof. Dr. Michael Hahn (UdS) im Rahmen des mathematischen Kolloquiums über das Thema Capabilities and Limits of Large Language Models:

AI systems based on Large Language Models (LLMs) are showing stunning progress. What are the abilities and limits of these models? I will present an emerging theory, from my group and others, on the capabilities of the underlying neural network architecture — the Transformer. First, I will study which functions this architecture is able to generalizably learn from finite data. Then, I will show bounds on the number of tokens that a Transformer needs to produce to perform specific computations. While the focus is on mathematical results, I will illustrate them with empirical data from Transformers and real-world Large Language Models.