Tabular data is prevalent across many scientific domains, particularly in medicine and healthcare, where datasets are often heterogeneous, small but high-dimensional, and noisy. Despite recent advances in tabular foundation models, learning representations that capture and preserve underlying data structures remains a significant challenge.

In this talk, I will present some of our recent work on tabular foundation models, focusing on the role of structural inductive biases in data generation, evaluation, domain adaptation, and inference. I will introduce novel approaches for synthetic data generation and augmentation, from leveraging pre-trained in-context models as energy-based models to extending their capabilities towards transferable representations for generative modelling. Complementing these approaches, we introduce a framework for evaluating the structural fidelity of tabular data, assessing the quality of the structural relationships beyond conventional evaluation metrics. Finally, I will discuss how structural relationships can also be explicitly grounded in domain knowledge, such as the one represented in knowledge graphs, providing additional information that may be difficult to infer from data alone, particularly in data-scarce scenarios. Taken together, these works highlight the importance of learning and leveraging structural information to address some of the fundamental challenges in developing general-purpose tabular foundation models.

Zoom link: Click me.