Machine Learning for Atomistic Simulation

This document aims to serve as a concise overview of literature that discusses the application of machine learning to the problem of atomistic simulation. It is not a comprehensive index but rather aims to provide a sense of the development of the field and highlight key papers.

Please feel free to open an issue or pull requests.

Table of Contents

• Embeddings and Representations
  • Atomic Cluster Expansion (ACE)
  • Smooth Overlap of Atomic Position (SOAP)
  • Gaussian Approximation Potentials (GAP)
  • Moment Tensor Potentials (MTP)
  • Atom-Centered Symmetry Functions (ACSF)
  • FCHL
  • Wavelet Scattering
  • Miscellaneous
  • Meta-literature
• Tools and Architectures
  • e3nn Precursor
  • e3nn
  • Equivariant Graph Neural Networks (EGNN)
  • SchNet
  • Preferred Networks
  • Symmetric Gradient Domain Machine Learning (sGDML)
  • DeepPot
  • DimeNet
  • Miscellaneous
  • Meta-literature

Embeddings and Representations

Below are various techniques for representing atomic systems and materials in a manner that is intended for usage with machine learning systems.

Atomic Cluster Expansion (ACE)

Title & Link	Author(s)	Pub. Date
Atomic cluster expansion for accurate and transferable interatomic potentials	Drautz	Jan 2019
Atomic cluster expansion of scalar, vectorial and tensorial properties and including magnetism and charge transfer	Drautz	Feb 2020
Performant implementation of the atomic cluster expansion: Application to copper and silicon	Lysogorskiy, van der Oord, Bochkarev, Menon, Rinaldi, Hammerschmidt, Mrovec, Thompson, Csányi, Ortner, Drautz	Mar 2021
Efficient parametrization of the atomic cluster expansion	Bochkarev, Lysogorskiy, Menon, Qamar, Mrovec, Drautz	Jan 2022
Atomic cluster expansion: Completeness, efficiency and stability	Dusson, Bachmayr, Csanyi, Drautz, Etter, van der Oord, Ortner	Apr 2022
Multilayer atomic cluster expansion for semi-local interactions	Bochkarev, Lysogorskiy, Ortner, Csanyi, Drautz	May 2022
Atomic cluster expansion and wave function representations	Drautz, Ortner	Jun 2022
Permutation-adapted complete and independent basis for atomic cluster expansion descriptors	Goff, Sievers, Wood, Thompson	Aug 2022

Smooth Overlap of Atomic Position (SOAP)

Title & Link	Author(s)	Pub. Date
On representing chemical environments	Bartok, Kondor, Csanyi	Mar 2013
Comparing molecules and solids across structural and alchemical space	De, Bartok, Csanyi, Ceriotti	Apr 2016
Automatic Selection of Atomic Fingerprints and Reference Configurations for Machine-Learning Potentials	Imbalzano, Anelli, Giofré, Klees, Behler, Ceriotti	Apr 2018
Machine-learning of atomic-scale properties based on physical principles	Ceriotti, Willatt, Csányi	Jan 2019
Atom-density representations for machine learning	Willatt, Musil, Ceriotti	Apr 2019
Atomic-scale representation and statistical learning of tensorial properties	Grisafi, Wilkins, Willatt, Ceriotti	Apr 2019
Optimizing many-body atomic descriptors for enhanced computational performance of machine learning based interatomic potentials	Caro	May 2019
Efficient implementation of atom-density representations	Musil, Veit, Goscinski, Fraux, Willatt, Stricker, Junge, Ceriotti	Jan 2021

Gaussian Approximation Potentials (GAP)

(Many applications paper omitted; please see here for a more comprehensive list)

Title & Link	Author(s)	Pub. Date
Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons	Bartok, Payne, Kondor, Csanyi	Oct 2009
Gaussian Approximation Potential: an interatomic potential derived from first principles Quantum Mechanics	Bartok	Mar 2010
Accuracy and transferability of GAP models for tungsten	Szlachta, Bartok, Csanyi	May 2014
Localized Coulomb Descriptors for the Gaussian Approximation Potential	Barker, Bulin, Hamaekers, Mathias	Nov 2016
Many-Body Coarse-Grained Interactions using Gaussian Approximation Potentials	John	Nov 2016
Machine-learned Interatomic Potentials for Alloys and Alloy Phase Diagrams	Rosenbrock, Gubaev, Shapeev, Pártay, Bernstein, Csányi, Hart	Jun 2019
Combining phonon accuracy with high transferability in Gaussian approximation potential models	George, Hautier, Bartok, Csanyi, Deringer	May 2020
Gaussian Moments as Physically Inspired Molecular Descriptors for Accurate and Scalable Machine Learning Potentials	Zaverkin, Kastner	July 2020
Massively Parallel Fitting of Gaussian Approximation Potentials	Klawohn, Kermode, Bartók	Jul 2022
Atomistic structure search using local surrogate mode	Rønne, Christiansen, Slavensky, Tang, Brix, Pedersen, Bisbo, Hammer	Aug 2022

Moment Tensor Potentials (MTP)

Title & Link	Author(s)	Pub. Date
Moment Tensor Potentials: a class of systematically improvable interatomic potentials	Shapeev	Dec 2015
Moment tensor Potentials as a Promising Tool to Study Diffusion Processes	Novoselov, Yanilkin, Shapeev, Podryabinkin	Dec 2018
Machine-learning potentials enable predictive and tractable high-throughput screening of random alloys	Hodapp, Shapeev	Jul 2021

Atom-Centered Symmetry Functions (ACSF)

Title & Link	Author(s)	Pub. Date
Generalized Neural-Network Representation of High-Dimensional Potential-Energy Surfaces	Behler, Parrinello	Apr 2007
Atom-centered symmetry functions for constructing high-dimensional neural network potentials	Behler	Feb 2011
An implementation of artificial neural-network potentials for atomistic materials simulations: Performance for TiO2 (aenet)	Artrith, Urban	Mar 2016
Amp: A modular approach to machine learning in atomistic simulations	Khorshidi, Peterson	Oct 2016
Efficient and accurate machine-learning interpolation of atomic energies in compositions with many species	Artrith, Urban, Ceder	Jul 2017
wACSF—Weighted atom-centered symmetry functions as descriptors in machine learning potentials	Gastegger, Schwiedrzik, Bittermann, Berzsenyi, Marquetand	Mar 2018
Optimized symmetry functions for machine-learning interatomic potentials of multicomponent systems	Rostami, Amsler, Ghasemi	Sep 2018
Augmenting machine learning of energy landscapes with local structural information	Honrao, Xie, Hennig	Aug 2020
A fourth-generation high-dimensional neural network potential with accurate electrostatics including non-local charge transfer	Wai Ko, Finkler, Goedecker, Behler	Jan 2021
Unified theory of atom-centered representations and message-passing machine-learning schemes	Nigam, Pozdnyakov, Fraux	May 2022

FCHL

(Acronym based off of initial authors' last names)

Title & Link	Author(s)	Pub. Date
Alchemical and structural distribution based representation for universal quantum machine learning	Faber, Christensen, Huang, von Lilienfeld	Mar 2018
FCHL revisited: Faster and more accurate quantum machine learning	Christensen, Bratholm, Faber, von Lilienfeld	Jan 2020

Wavelet Scattering

Title & Link	Author(s)	Pub. Date
Solid harmonic wavelet scattering	Eickenberg, Exarchakis, Hirn, Mallat	Dec 2017
Steerable Wavelet Scattering for 3D Atomic Systems with Application to Li-Si Energy Prediction	Brumwell, Sinz, Jin Kim, Qi, Hirn	Nov 2018
Wavelet Scattering Networks for Atomistic Systems with Extrapolation of Material Properties	Sinz, Swift, Brumwell, Liu, Jin Kim, Qi, Hirn	Jun 2020

Miscellaneous

Title & Link	Author(s)	Pub. Date
Simultaneous fitting of a potential-energy surface and its corresponding force fields using feedforward neural networks	Pukrittayakamee, Malshe, Hagan, Raff, Narulkar, Bukkapatnum, Komanduri	Apr 2009
Fourier series of atomic radial distribution functions: A molecular fingerprint for machine learning models of quantum chemical properties	von Lilienfeld, Ramakrishnan, Rupp, Knoll	Jul 2013
How to represent crystal structures for machine learning: Towards fast prediction of electronic properties	Schutt, Glawe, Brockherde, Sanna, Muller, Gross	May 2014
A fingerprint based metric for measuring similarities of crystalline structures	Zhu, Amsler, Fuhrer, Schaefer, Faraji, Rostami, Ghasemi, Sadeghi, Grauzinyte, Wolverton, Goedecker	Jan 2016
Unified Representation of Molecules and Crystals for Machine Learning (MBTR)	Huo, Rupp	Apr 2017
An Atomistic Fingerprint Algorithm for Learning Ab Initio Molecular Force Fields	Tang, Zhang, Karniadakis	Dec 2017
A novel approach to describe chemical environments in high-dimensional neural network potentials	Kocer, Mason, Erturk	Mar 2019
Embedded Atom Neural Network Potentials: Efficient and Accurate Machine Learning with a Physically Inspired Representation (EAM)	Zhang, Hu, Jiang	Aug 2019
Continuous and optimally complete description of chemical environments using Spherical Bessel descriptors (SB)	Kocer, Mason, Erturk	Jan 2020
TUCAN: A molecular identifier and descriptor applicable to the whole periodic table from hydrogen to oganesson	Brammer, Blanke, Kellner, Hoffmann, Herres-Pawlis, Schatzschneider	Mar 2022

Meta-literature

Title & Link	Author(s)	Pub. Date
SISSO: A compressed-sensing method for identifying the best low-dimensional descriptor in an immensity of offered candidates	Ouyang, Curtarolo, Ahmetcik, Scheffler, Ghiringhelli	Aug 2018
A Performance and Cost Assessment of Machine Learning Interatomic Potentials	Zuo, Chen, Li, Deng, Chen, Behler, Csányi, Shapeev, Thompson, Wood, Ong	Jul 2019
DScribe: Library of descriptors for machine learning in materials science	Himanen, Jäger, Morooka, Canova, Ranawat, Gao, Rinke, Foster	Nov 2019
Descriptors representing two- and three-body atomic distributions and their effects on the accuracy of machine-learned interatomic potentials	Jinnouchi, Karsai, Verdi, Asahi, Kresse	Apr 2020
Sensitivity and Dimensionality of Atomic Environment Representations used for Machine Learning Interatomic Potentials	Onat, Ortner, Kermode	Jun 2020
An assessment of the structural resolution of various fingerprints commonly used in machine learning	Karamad, Magar, Shi, Siahrostami, Gates, Farimani	Aug 2020
Incompleteness of Atomic Structure Representations	Pozdnyakov, Willatt, Bartok, Ortner, Csanyi, Ceriotti	Oct 2020
Manifolds of quasi-constant SOAP and ACSF fingerprints and the resulting failure to machine learn four-body interactions	Parsaeifard, Goedecker	Feb 2021
The role of feature space in atomistic learning	Goscinski, Fraux, Imbalzano, Ceriotti	Apr 2021
Strategies for the Construction of Machine-Learning Potentials for Accurate and Efficient Atomic-Scale Simulations	Miksch, Morawietz, Kästner, Urban, Artrith	May 2021
Compressing local atomic neighbourhood descriptors	Darby, Kermode, Csányi	Dec 2021

Tools and Architectures

These are methods and networks that are intended to process materials or molecular data for the purposes of atomistic simulation (occasionally, property prediction)

e3nn Precursor

These works were developed independently of each other but propose very similar ideas which were mostly consolidated into the e3nn framework.

Title & Link	Author(s)	Pub. Date
Group Equivariant Convolutional Networks	Cohen, Welling	Feb 2016
Spherical CNNs	Cohen, Geiger, Koehler, Welling	Jan 2018
Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds	Smidt, Thomas, Kearnes, Yang, Li, Kohlhoff, Riley	Feb 2018
N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials	Kondor	Mar 2018
Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network	Kondor, Lin, Trivedi	Jun 2018
3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data	Weiler, Geiger, Welling, Boomsma, Cohen	Jul 2018
SE(3)-Equivariant prediction of molecular wavefunctions and electronic densities	Unke, Bogojeski, Gastegger, Geiger, Smidt, Müller	Jun 2021

e3nn

Title & Link	Author(s)	Pub. Date
Finding symmetry breaking order parameters with Euclidean neural networks	Smidt, Geiger, Miller	Jul 2020
Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties	Miller, Geiger, Smidt, Noe	Aug 2020
E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials (NequIP)	Batzner, Musaelian, Sun, Geiger, Mailoa, Kornbluth, Molinari, Smidt, Kozinsky	Jan 2021]
Direct Prediction of Phonon Density of States With Euclidean Neural Networks	Chen, Andrejevic, Smidt, Z. Ding, Q. Xu, Y. Chi, Q. Nguyen, Alatas, Kong, M. Li	Mar 2021
Cracking the Quantum Scaling Limit with Machine Learned Electron Densities	Rackers, Tecot, Geiger, Smidt	Jan 2022
Learning Local Equivariant Representations for Large-Scale Atomistic Dynamics (Allegro)	Musaelian, Batzner, Johansson, Sun, Owen, Kornbluth, Kozinsky	Apr 2022
The Design Space of E(3)-Equivariant Atom-Centered Interatomic Potentials (BOTNet)	Batatia, Batzner, Kovács, Musaelian, Simm, Drautz, Ortner, Kozinsky, Csányi	May 2022
MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields	Batatia, Kovács, Simm, Ortner, Csányi	Jun 2022
Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs	Liao, Smidt	Jun 2022
e3nn: Euclidean Neural Networks	Geiger, Smidt	Jul 2022
Machine Learning Magnetism Classifiers from Atomic Coordinates	Merker, Heiberger, Q. Nguyen, Liu, Chen, Andrejevic, Drucker, Okabe, S.E. Kim, Wang, Smidt, M. Li	Sep 2022
High Accuracy Uncertainty-Aware Interatomic Force Modeling with Equivariant Bayesian Neural Networks	Rensmeyer, Craig, Kramer, Niggemann	Apr 2023

Equivariant Graph Neural Networks (EGNN)

Title & Link	Author(s)	Pub. Date
E(n)-Equivariant Graph Neural Networks	Satorras, Hoogeboom, Welling	Feb 2021
E(n)-Equivariant Normalizing Flows	Satorras, Hoogeboom, Fuchs, Posner, Welling	May 2021
Geometric and Physical Quantities Improve E(3)-Equivariant Message Passing (SEGNN)	Brandstetter, Hesselink, Pol, Bekkers, Welling	Oct 2021

SchNet

Title & Link	Author(s)	Pub. Date
SchNet: A continuous-filter convolutional neural network for modeling quantum interactions	Schutt, Kindermans, Sauceda, Chmiela, Tkatchenko, Muller	Jun 2017
SchNet – a deep learning architecture for molecules and materials	Schutt, Sauceda, Kindermans, Tkatchenko, Muller	Mar 2018
Analysis of Atomistic Representations Using Weighted Skip-Connections	Nicoli, Kessel, Gastegger, Schutt	Oct 2018
SchNetPack: A Deep Learning Toolbox For Atomistic Systems	Schutt, Kessel, Gastegger, Nicoli, Tkatchenko, Muller	Nov 2018
Learning representations of molecules and materials with atomistic neural networks	Schutt, Tkatchenko, Muller	Dec 2018
Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions (SchNOrb)	Schutt, Gastegger, Tkatchenko, Muller, Maurer	Nov 2019
Combining SchNet and SHARC: The SchNarc Machine Learning Approach for Excited-State Dynamics	Westermayr, Gastegger, Marquetand	Apr 2020
A deep neural network for molecular wave functions in quasi-atomic minimal basis representation	Gastegger, McSloy, Luya, Schutt, Maurer	May 2020

Preferred Networks

Title & Link	Author(s)	Pub. Date
TeaNet: universal neural network interatomic potential inspired by iterative electronic relaxations	Takamoto, Izumi, Li	Dec 2019
Towards Universal Neural Network Potential for Material Discovery Applicable to Arbitrary Combination of 45 Elements	Takamoto, Shinagawa, Motoki, Nakago, Li, Kurata, Watanabe, Yayama, Iriguchi, Asano, Onodera, Ishii, Kudo, Ono, Sawada, Ishitani, Ong, Yamaguchi, Kataoka, Hayashi, Charoenphakdee, Ibuka	Jun 2021

Symmetric Gradient Domain Machine Learning (sGDML)

Title & Link	Author(s)	Pub. Date
Machine learning of accurate energy-conserving molecular force fields	Chmiela, Tkatchenko, Sau	May 2017
Towards exact molecular dynamics simulations with machine-learned force fields	Chmiela, Sauceda, Müller, Tkatchenko	Sep 2018
Molecular force fields with gradient-domain machine learning: Construction and application to dynamics of small molecules with coupled cluster forces	Sauceda, Chmiela, Poltavsky, Muller, Tkatchenko	Feb 2019
sGDML: Constructing accurate and data efficient molecular force fields using machine learning	Chmiela, Sauceda, Poltavsky, Muller, Tkatchenko	Jul 2019
Construction of Machine Learned Force Fields with Quantum Chemical Accuracy: Applications and Chemical Insights	Sauceda, Chmiela, Poltavsky, Muller, Tkatchenko	Sep 2019
Accurate Molecular Dynamics Enabled by Efficient Physically-Constrained Machine Learning Approaches	Chmiela, Sauceda, Tkatchenko, Muller	Dec 2019
Ensemble learning of coarse-grained molecular dynamics force fields with a kernel approach	J. Wang, Chmiela, Muller, Noe, Clementi	May 2020
Molecular force fields with gradient-domain machine learning (GDML): Comparison and synergies with classical force fields	Sauceda, Gastegger, Chmiela, Muller, Tkatchenko	Sep 2020
Towards Linearly Scaling and Chemically Accurate Global Machine Learning Force Fields for Large Molecules	Kabylda, Vassilev-Galindo, Chmiela, Poltavsky, Tkatchenko	Sep 2022

DeepPot

Title & Link	Author(s)	Pub. Date
Deep Potential Molecular Dynamics: A Scalable Model with the Accuracy of Quantum Mechanics	Zhang, Han, Wang, Car, Weinan	Apr 2018
End-to-end Symmetry Preserving Inter-atomic Potential Energy Model for Finite and Extended Systems (DeepPot-SE)	Zhang, Han, Wang, Saidi, Car, Weinan	May 2018
DP-GEN: A concurrent learning platform for the generation of reliable deep learning based potential energy models	Zhang, Wang, Chen, Zeng, Zhang, Wang, Weinan	Oct 2019

DimeNet

Title & Link	Author(s)	Pub. Date
Directional Message Passing for Molecular Graphs (DimeNet)	Gasteiger, Groß, Günnemann	Mar 2020
Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules (DimeNet++)	Gasteiger, Giri, Margraf, Günnemann	Nov 2020

Miscellaneous

Title & Link	Author(s)	Pub. Date
ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost	Smith, Isayev, Roitberg	Feb 2017
Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals	Chen, Ye, Zuo, Zheng, Ping Ong	Dec 2018
Cormorant: Covariant Molecular Neural Networks	Anderson, Hy, Kondor	Jun 2019
Graph dynamical networks for unsupervised learning of atomic scale dynamics in materials (GDyNets)	Xie, France-Lanord, Wang, Shao-Horn, Grossman	Jun 2019
Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks (FermiNet)	Pfau, Spencer, de G. Matthews, Foulkes	Sep 2019
A fast neural network approach for direct covariant forces prediction in complex multi-element extended systems	Mailoa, Kornbluth, Batzner, Samsonidze, Lam, Vandermause, Ablitt, Molinari, Kozinsky	Sep 2019
SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks	Fuchs, Worrall, Fischer, Welling	Jun 2020
Equivariant message passing for the prediction of tensorial properties and molecular spectra (PaiNN)	Schutt, Unke, Gastegger	Feb 2021
ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations	Hu, Shuaibi, Das, Goyal, Sriram, Leskovec, Parikh, Zitnick	Mar 2021
Accurate and scalable graph neural network force field and molecular dynamics with direct force architecture (GNNFF)	Park, Kornbluth, Vandermause, Wolverton, Kozinsky, Mailoa	May 2021
Informing Geometric Deep Learning with Electronic Interactions to Accelerate Quantum Chemistry (OrbNet-Equi)	Qiao, Christensen, Welborn, Manby, Anandkumar, Miller III	May 2021
GemNet: Universal Directional Graph Neural Networks for Molecules	Gasteiger, Becker, Günnemann	Jun 2021
Rotation Invariant Graph Neural Networks using Spin Convolutions (SpinConv)	Shuaibi, Kolluru, Das, Grover, Sriram, Ulissi, Zitnick	Jun 2021
SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects	Unke, Chmiela, Gastegger, Schütt, Sauceda, Müller	Dec 2021
Convergence acceleration in machine learning potentials for atomistic simulations	Bayerl, Andolina, Dwaraknath, Saidi	Jan 2022
NNP/MM: Fast molecular dynamics simulations with machine learning potentials and molecular mechanics	Galvelis, Varela-Rial, Doerr, Fino, Eastman, Markland, Chodera, Fabritiis	Jan 2022
TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentials	Tholke, Fabritiis	Feb 2022
A Universal Graph Deep Learning Interatomic Potential for the Periodic Table (M3GNet)	Chen, Ping Ong	Feb 2022
Equivariant Graph Attention Networks for Molecular Property Prediction	Le, Noe, Clevert	Feb 2022
Learning Atomic Multipoles: Prediction of the Electrostatic Potential with Equivariant Graph Neural Networks	Thurlemann, Boselt, Riniker	Feb 2022
NewtonNet: a Newtonian message passing network for deep learning of interatomic potentials and forces	Haghighatlari, Li, Guan, Zhang, Das, Stein, Zadeh, Liu, M. Head-Gordon, Bertels, Hao, Leven, T. Head-Gordon	Feb 2022
MolNet: A Chemically Intuitive Graph Neural Network for Prediction of Molecular Properties	Y. Kim, Jeong, J. Kim, E.K. Lee, W.J. Kim, I. Choi	Feb 2022
Equivariant graph neural networks for fast electron density estimation of molecules, liquids, and solids	Jørgensen, Bhowmik	Aug 2022

Meta-literature

Title & Link	Author(s)	Pub. Date
Challenges for machine learning force fields in reproducing potential energy surfaces of flexible molecules	Vassilev-Galindo, Fonseca, Poltavsky, Tkatchenko	Mar 2021
Symmetry Group Equivariant Architectures for Physics	Bogatskiy, Ganguly, Kipf, Kondor, Miller, Murnane, Offermann, Pettee, Shanahan, Shimmin, Thais	Mar 2022
How Robust are Modern Graph Neural Network Potentials in Long and Hot Molecular Dynamics Simulations?	Stocker, Gasteiger, Becker, Gunnemann, Margraf	Apr 2022
Forces Are Not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations	X. Fu, Z. Wu, W. Wang, T. Xie, Keten, Gomez-Bombarelli, Jaakkola	Sep 2022
Neural Scaling of Deep Chemical Models	Frey, Soklaski, Alexrod, Samsi, Gomez-Bombarelli, Coley, Gadepally	May 2022
Incompleteness of graph neural networks for points clouds in three dimensions	Pozdnyakov, Ceriotti	Nov 2022