Hi there,
If you are seeing this, this means either I have recommended you this list myself, or you are a web scraper bot from the famous search engine.
The list below contains some resources, including papers, websites, articles, blog posts, YouTube videos, or any other kind of stuffs, on machine learning and neural networks, where everyone can read and understand from zero to hero. This list should evolve over time, as long as I don't become lazy at some point and forgot to update this. Basically I can throw in anything that I think is interesting in the field of ML and NN.
I also intend to have this list to be appropriate to everyone, from the beginners of ML and NN, to a person like me who wants to read on new techniques of machine learning. I am not the god of ML and NN, so I do not know everything, and this list is not going to be the exhaustive list of everything. This is going to be just another list from me.
Thanks!
- Python 3 tutorial is a free tutorial on Python 3 provided by Python Software Foundation. Nowadays Python is a very popular programming language of choice due to its ease of use. There are also a myriad of libraries that can be used with Python, like NumPy, SciPy, Matplotlib, etc., that can help you do anything from software development to data analysis and machine learning, and many of those packages are free! Free! As in free beer! You don't need stupid proprietary software just to perform matrix multiplication or solve equations with an archaic and utterly terrible editor. *Ahem* MATLAB.
- Download Python 3 here The reason I stressed Python 3 is that you might encounter some outdated Python tutorials that used Python 2. When Python 3 was released to the world in 2006, it had a lot of changes that can break existing code in Python 2, so the developers had a grace period to transition their libraries to Python 3. That grace period ended on 1 January 2020, and that means Python 2 is no longer supported. All Python packages you can download these days are made for Python 3 anyway, so you should be using that as well.
- NumPy quick start NumPy is a Python library that lets you handle calculations in the form of arrays and matrices, pretty much like MATLAB. Unlike MATLAB, however, you can use NumPy with many other packages in Python. If you used MATLAB before and wanted to try something better, here is NumPy guide for MATLAB users.
- Create your first neural network with TensorFlow! This tutorial uses TensorFlow, one of the most popular machine learning packages out there, with the goal of classifying types of clothing from Fashion MNIST dataset. You are not going to create a very complex neural network there, just a simple network with some hidden layers and nothing fancy. There are also more tutorials like this at TensorFlow tutorials page
- ML Zero to Hero series part 1, part 2, part 3, part 4 This series of YouTube videos is a great introduction to machine learning using TensorFlow, starting from the most basic concepts of machine learning, all the way to convolutional neural networks. It is however a roller coaster and skips many fundamental ideas of neural networks, so you may want some more materials to reinforce your understanding.
- M. A. Nielsen, "Neural Networks and Deep Learning", Determination Press, 2015 is the book I first read about neural networks. This book teaches you the mathematics behind neural networks, with an example built with NumPy. When we work with neural networks, however, we do not reinvent the wheel and rewrite the network with numpy. Instead we use TensorFlow or Keras to design the network.
- A. Zhang et al. "Dive into Deep Learning", arXiv:2106.11342 is a very extensive book about deep learning presenting the basics of neural networks starting from linear models, all the way to several advanced concepts in machine learning.
- Scikit-learn Scikit-learn is a machine learning library on Python, containing a vast set of machine learning methods ranging from linear regression to PCA and t-SNE. However, if you want to design your own neural network with some more flexibility, Keras or PyTorch would be better off. This library also gives you some tools that is essential to neural network training as well as other useful variable transformation methods.
- TensorFlow TensorFlow is a Python package that revolves around calculating tensors, which is essential to neural networks in general. It also contains some stuffs like dataset loader and whole Keras API that helps you create your neural network much easier.
- Keras Keras is a Python package that helps you build neural networks very easily. It can use either TensorFlow or Theano as a backend for tensor calculations. You will also notice the name "keras" in TensorFlow documentation because TensorFlow has the whole Keras API within it as a helper API.
- PyTorch PyTorch is a neural network framework that gives you flexibility in neural network design, letting you control almost every aspect of neural network forwarding and backpropagating. With this blessing, however, it also gives you the curse of having to define everything from feedforwarding steps to loss calculation and backpropagating steps. (However, PyTorch still provides a lot of tools so that you don't have to reinvent the wheel.) There is also PyTorch Lightning that promises you to help training your network faster, but Keras still does it better when it comes to ease of use.
- This X does not exist Generative adversarial networks (GANs) are a class of neural networks that generates something to mimic the training data. After the training, however, we can use this type of network to generate that something based on a random noise. This web page is a collection of stuffs that are generated using GANs, and they do not exist.
- The Unreasonable Effectiveness of Recurrent Neural Networks Recurrent neural networks are a class of neural networks that can generate results in series. One important feature of this class of networks is the output of the network can be fed as an input in the next time step. This blog post shows you how effective this type of networks can generate any type of text (depending on the training dataset of course). The effectiveness is unreasonable, however, because this work does not touch any complex ideas of Natural Language Processing (NLP), and still it works! It's a fun read, you should try it.
- Distill Many concepts in neural networks can be hard and complex to think, and reading papers on new neural network techniques can lead you to nowhere. This website contains several beautiful interactive illustrations that helps explaining some interesting concepts, such as Feature Visualization of a neural network that is trained to understand networks.
Any paper that does not have any explanation from me means I have not read it yet.
- ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network arXiv:2002.10200
- Efficient, reliable and fast high-level triggering using a bonsai boosted decision tree arXiv:1210.6861
- Wasserstein GAN arXiv:1701.07875
- Auto-Encoding Variational Bayes arXiv:1312.6114
- Graph neural networks: A review of methods and applications arXiv:1812.08434
- LHCb Topological Trigger Reoptimization arXiv:1510.00572
- A guide to convolution arithmetic for deep learning arXiv:1603.07285 Convolutional networks are a class of neural networks that uses a trainable kernel scanning through the data. It is usually applied in 2D data, especially images, and determining the size of output from each convolutional layer can be hard. This guide has several formulas to calculate the output size of each convolutional layer in many different use cases, so you do not have to guess the shape of the output every time.
- beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework https://openreview.net/forum?id=Sy2fzU9gl
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling arXiv:1412.3555
- TensorFlow from the ground up (ML Tech Talks) This talk explains what TensorFlow is doing when we train neural networks. Basically when we train neural networks, we are playing around with tensors, and this talk explains how neural networks based on TensorFlow are trained in a technical level.