Transformer models (e.g. LLMs) have revolutionized many aspects of life. However, progress in tabular data has been slow. In this talk I introduce Tabular Foundation Models, a class of transformer models designed and trained to excel in sparse tabular data. I introduce the models and benchmark their performance in classification and regression against established models. Further, I provide real-world evidence for when they are a substantial improvement and give advice on how to implement them.
