Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets
Zenodo (CERN European Organization for Nuclear Research)
Abstract
Recently, pre-trained foundation models have shown significant advancements in multiple fields. However, the lack of datasets with labeled features and codebases has hindered the development of a supervised foundation model for molecular tasks. Here, we have carefully curated seven datasets specifically tailored for node- and graph-level prediction tasks to facilitate supervised learning on molecules. Moreover, to support the development of multi-task learning on our proposed datasets, we created the Graphium graph machine learning library. Our dataset collection encompasses two distinct categories. Firstly, the TOYMIX category modifies three small existing datasets with additional data for multi-task learning. Secondly, the LARGEMIX category includes four large-scale datasets with 344M graph-level data points and 409M node-level data points from ∼5M unique molecules. Finally, the ultra-large dataset contains 2,210M graph-level data points and 2,031M node-level data points coming from 86M molecules. Hence our datasets represent an order of magnitude increase in data volume compared to other 2D-GNN datasets. In addition, recognizing that molecule-related tasks often span multiple levels, we have designed our library to explicitly support multi-tasking, offering a diverse range of multi-level representations, i.e., representations at the graph, node, edge, and node-pair level. We equipped the library with an extensive collection of models and features to cover different levels of molecule analysis. By combining our curated datasets with this versatile library, we aim to accelerate the development of molecule foundation models. Datasets and code are available at https://github.com/datamol-io/graphium.
Authors 26
-
Mila - Quebec Artificial Intelligence Institute · Université de Montréal
Affiliation as printed
Mila, Valence, University of Montreal
-
Affiliation as printed
Mila, Valence, McGill University
-
Affiliation as printed
Graphcore
-
Affiliation as printed
Graphcore
-
Affiliation as printed
Mila, Valence, McGill University
-
Mila - Quebec Artificial Intelligence Institute · Université de Montréal
Affiliation as printed
Mila, Valence, University of Montreal
-
Affiliation as printed
Graphcore
-
Affiliation as printed
Graphcore
-
New Jersey Institute of Technology
Affiliation as printed
Mila, Valence, New Jersey Institute of Technology
-
Mila - Quebec Artificial Intelligence Institute
Affiliation as printed
Mila
-
Mila - Quebec Artificial Intelligence Institute · Université de Montréal
Affiliation as printed
Mila, University of Montreal
-
Affiliation as printed
Valence
-
Mila - Quebec Artificial Intelligence Institute · Université de Montréal
Affiliation as printed
Mila, University of Montreal
-
Mila - Quebec Artificial Intelligence Institute · Université de Montréal
Affiliation as printed
Mila, University of Montreal
-
Mila - Quebec Artificial Intelligence Institute · Université de Montréal
Affiliation as printed
Mila, University of Montreal
-
Affiliation as printed
Valence
-
Mila - Quebec Artificial Intelligence Institute · Université de Montréal
Affiliation as printed
Mila, University of Montreal, CIFAR AI Chair
-
Mila - Quebec Artificial Intelligence Institute · McGill University
Affiliation as printed
Mila, McGill University, CIFAR AI Chair
-
HEC Montréal · Mila - Quebec Artificial Intelligence Institute
Affiliation as printed
Mila, HEC Montreal, CIFAR AI Chair
-
Mila - Quebec Artificial Intelligence Institute · Université de Montréal
Affiliation as printed
Mila, University of Montreal
-
Mila - Quebec Artificial Intelligence Institute · Université de Montréal
Affiliation as printed
Mila, University of Montreal, CIFAR AI Chair
-
Affiliation as printed
Valence
-
Affiliation as printed
Valence
-
Affiliation as printed
Graphcore
-
Affiliation as printed
Graphcore
-
Affiliation as printed
Graphcore
Cited by 0 stored of 0
No patents citing this paper on Lens.org (checked 2026-10-06).