Hi, I'm Maxime. I'm a PhD student in machine learning.
I am interested in using probabilistic machine learning for making decisions. In particular, how to effectively train, fine-tune or continually update neural networks using insights from deep learning theory and Bayesian deep learning; how to model arbitrary and complex distributions with neural density estimation; and how to reliably account for uncertainty and risk in model predictions using tools such as conformal prediction.
Some other topics and areas I work on include:
- things involving stochastic processes in ML, including Bayesian nonparametrics and diffusion processes,
- deep learning theory and understanding training dynamics and architectural choices,
- reinforcement learning and other forms of active learning which involve making sequences of decisions under uncertainty,
- using LLMs to make 'judgemental forecasts' or answer questions with uncertainty; so as to leverage large pre-trained models and unstructured data as a supplement to traditional numerical data in forecasting or decison making
Blog Articles
I write articles related to things I'm working on. Here are the latest ones:
-
Bayesian Low-Rank Adaptation for Large Language Models
An overview of some recent work, published in ICLR 2024, where we estimate the uncertainty and marginal likelihoods in LLMs using Bayesian LoRA adapters. We focus on the fine-tuning setting, and scale our method to LLMs using a Laplace approximation with low-rank K-FAC.
-
Second-Order Methods in Machine Learning
A motivation of the Hessian from an optimisation perspective (and the related Generalised Gauss-Newton / Fisher Information Matrix), an introduction to Kronecker-factored approximate curvature, and applications of the curvature in machine learning.
-
Of VJPs and JVPs
Some intuitions and visualisations of vector-Jacobian products and Jacobian-vector products, to help you avoid confusing the two again.
Papers
Some papers I have written or co-authored.
-
Bayesian Reward Models For LLM Alignment
Adam X. Yang, Maxime Robeyns, Thomas Coste, Jun Wang, Haitham Bou-Ammar, Laurence Aitchison
2024 – Pre-print
https://arxiv.org/pdf/2402.13210.pdf
-
Bayesian Low-rank Adaptation for Large Language Models
Adam X. Yang, Maxime Robeyns, Xi Wang, Laurence Aitchison
2024 – ICLR
https://arxiv.org/abs/2308.13111
-
Taylor TD-learning
Michele Garibbo, Maxime Robeyns, Laurence Aitchison
2023 – NeurIPS
https://arxiv.org/abs/2302.14182
-
A Theory of Representation Learning in Deep Neural Networks Gives a Deep Generalisation of Kernel Methods
Adam X. Yang, Maxime Robeyns, Edward Milsom, Nandi Schoots, Laurence Aitchison
2023 – ICML
https://proceedings.mlr.press/v202/yang23k/yang23k.pdf
-
Fast Estimation of Physical Galaxy Properties using Simulation-Based Inference
Maxime Robeyns, Sotiria Fotopolou, Mike Walmsley, Laurence Aitchison
2022 – ICML 2022 Workshop on Machine Learning for Astrophysics
https://ml4astro.github.io/icml2022/assets/12.pdf
Fragments
Any other drivel and shower thoughts end up in my "fragments". Here are the last ones: