You should be able to implement, on your own, automatic differentiation, your own neural network library, build your own linear algebra backend for CPU and GPU and should be comfortable implementing your own MLPs, ConvNets, RNN, and Transformer architectures. You should understand hardware acceleration.
- (2022/03) Carnegie Mellon - Deep Learning Systems
- (2022/03) Andrey Karpathy - Neural Networks: Zero to Hero
- You should have an intuition for softmax
- Linear Algebra
- Machine Learning
- Differential Calculus
- Youtube - 3Blue1Brown - Essence of Calculus
- KhanAcademy - College Calculus AB 4%
- Coursera - DeepLearningAI - Calculus for Machine Learning and Data Science
- https://www.coursera.org/specializations/precalculus-data-modelling
- https://www.coursera.org/specializations/differential-calculus-data-modeling
- https://www.edx.org/xseries/mitx-18.01x-single-variable-calculus