8S50x_L13.ipynb

{"cells": [{"cell_type": "markdown", "id": "1a7c0d85", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<hr style=\"height: 1px;\">\n", "<i>This notebook was authored by the 8.S50x Course Team, Copyright 2022 MIT All Rights Reserved.</i>\n", "<hr style=\"height: 1px;\">\n", "<br>\n", "\n", "<h1>Lesson 13: Deep Learning</h1>\n"]}, {"cell_type": "markdown", "id": "6a6864bf", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='section_13_0'></a>\n", "<hr style=\"height: 1px;\">\n", "\n", "\n", "## <h2 style=\"border:1px; border-style:solid; padding: 0.25em; color: #FFFFFF; background-color: #90409C\">L13.0 Overview</h2>\n"]}, {"cell_type": "markdown", "id": "291c0c79", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<h3>Navigation</h3>\n", "\n", "<table style=\"width:100%\">\n", "    <tr>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#section_13_1\">L13.1 Machine Learning</a></td>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#exercises_13_1\">L13.1 Exercises</a></td>\n", "    </tr>\n", "    <tr>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#section_13_2\">L13.2 Procedure and Example Algorithms</a></td>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#exercises_13_2\">L13.2 Exercises</a></td>\n", "    </tr>\n", "    <tr>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#section_13_3\">L13.3 Logistic Regression Algorithm</a></td>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#exercises_13_3\">L13.3 Exercises</a></td>\n", "    </tr>\n", "    <tr>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#section_13_4\">L13.4 Introduction to Neural Networks</a></td>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#exercises_13_4\">L13.4 Exercises</a></td>\n", "    </tr>\n", "    <tr>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#section_13_5\">L13.5 What Do Neural Networks Do?</a></td>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#exercises_13_5\">L13.5 Exercises</a></td>\n", "    </tr>\n", "    <tr>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#section_13_6\">L13.6 Training a Neural Network</a></td>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#exercises_13_6\">L13.6 Exercises</a></td>\n", "    </tr>\n", "    <tr>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#section_13_7\">L13.7 Training a Neural Network with PyTorch</a></td>\n", "        <td style=\"text-align: left; vertical-align: top; font-size: 10pt;\"><a href=\"#exercises_13_7\">L13.7 Exercises</a></td>\n", "    </tr>\n", "</table>"]}, {"cell_type": "markdown", "id": "d8712c6e", "metadata": {"tags": ["learner", "catsoop_00", "md"]}, "source": ["<h3>Learning Objectives</h3>\n", "\n", "Now, we will introduce important core concepts in deep learning!\n", "\n", "This lesson covers several topics related to machine learning, including the logistic regression algorithm (which is a popular algorithm used for classification tasks), introduction to neural networks, what neural networks do, and training a neural network using Pytorch, which is a popular machine learning library in Python.\n"]}, {"cell_type": "markdown", "id": "76f61f6c", "metadata": {"tags": ["learner", "md"]}, "source": ["<h3>Data</h3>\n", "\n", "Download the directory where we will save data."]}, {"cell_type": "code", "execution_count": null, "id": "0b796a1c", "metadata": {"tags": ["learner", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.0-runcell00\n", "\n", "!git init\n", "!git remote add -f origin https://github.com/mitx-8s50/nb_LEARNER/\n", "!git config core.sparseCheckout true\n", "!echo 'L13' >> .git/info/sparse-checkout\n", "!git pull origin main"]}, {"cell_type": "markdown", "id": "62d337d6", "metadata": {"tags": ["learner", "md"]}, "source": ["<h3>Importing Libraries</h3>\n", "\n", "Before beginning, run the cells below to import the relevant libraries for this notebook. "]}, {"cell_type": "code", "execution_count": null, "id": "dbb547d9", "metadata": {"tags": ["learner", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.0-runcell01\n", "\n", "#do the following if running this notebook locally (if needed)\n", "#within your conda environment, run the following\n", "#conda install pytorch"]}, {"cell_type": "code", "execution_count": null, "id": "c517e659", "metadata": {"tags": ["learner", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.0-runcell02\n", "\n", "import torch                      #https://pytorch.org/\n", "import numpy as np                #https://numpy.org/doc/stable/ \n", "import matplotlib.pyplot as plt   #https://matplotlib.org/3.5.3/api/_as_gen/matplotlib.pyplot.html\n", "from scipy import stats           #https://docs.scipy.org/doc/scipy/reference/stats.html\n", "import scipy.optimize as opt      #https://docs.scipy.org/doc/scipy/reference/optimize.html"]}, {"cell_type": "markdown", "id": "7dda174b", "metadata": {"tags": ["learner", "md"]}, "source": ["<h3>Setting Default Figure Parameters</h3>\n", "\n", "The following code cell sets default values for figure parameters."]}, {"cell_type": "code", "execution_count": null, "id": "a3785d45", "metadata": {"tags": ["learner", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.0-runcell03\n", "\n", "#set plot resolution\n", "%config InlineBackend.figure_format = 'retina'\n", "\n", "#set default figure parameters\n", "plt.rcParams['figure.figsize'] = (9,6)\n", "\n", "medium_size = 12\n", "large_size = 15\n", "\n", "plt.rc('font', size=medium_size)          # default text sizes\n", "plt.rc('xtick', labelsize=medium_size)    # xtick labels\n", "plt.rc('ytick', labelsize=medium_size)    # ytick labels\n", "plt.rc('legend', fontsize=medium_size)    # legend\n", "plt.rc('axes', titlesize=large_size)      # axes title\n", "plt.rc('axes', labelsize=large_size)      # x and y labels\n", "plt.rc('figure', titlesize=large_size)    # figure title\n"]}, {"cell_type": "markdown", "id": "da500ade", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='section_13_1'></a>\n", "<hr style=\"height: 1px;\">\n", "\n", "## <h2 style=\"border:1px; border-style:solid; padding: 0.25em; color: #FFFFFF; background-color: #90409C\">L13.1 Machine Learning</h2>  \n", "\n", "| [Top](#section_13_0) | [Previous Section](#section_13_0) | [Exercises](#exercises_13_1) | [Next Section](#section_13_2) |\n"]}, {"cell_type": "markdown", "id": "68e041c6", "metadata": {"tags": ["learner", "md"]}, "source": ["*The material in this section is discussed in the video **<a href=\"https://courses.mitxonline.mit.edu/learn/course/course-v1:MITxT+8.S50.2x+2T2023/block-v1:MITxT+8.S50.2x+2T2023+type@sequential+block@seq_LS13/block-v1:MITxT+8.S50.2x+2T2023+type@vertical+block@vert_LS13_vid1\" target=\"_blank\">HERE</a>.** You are encouraged to watch that video and use this notebook concurrently.*"]}, {"cell_type": "markdown", "id": "f304d347", "metadata": {"tags": ["learner", "md"]}, "source": ["<h3>Slides</h3>\n", "\n", "Run the code below to view the slides for this section, which are discussed in the related video. You can also open the slides in a separate window <a href=\"https://mitx-8s50.github.io/slides/L13/slides_L13_01.html\" target=\"_blank\">HERE</a>."]}, {"cell_type": "code", "execution_count": null, "id": "45f162e7", "metadata": {"tags": ["learner", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.1-slides\n", "\n", "from IPython.display import IFrame\n", "IFrame(src='https://mitx-8s50.github.io/slides/L13/slides_L13_01.html', width=970, height=550)"]}, {"cell_type": "markdown", "id": "133524d2", "metadata": {"tags": ["learner", "md", "lect_01"]}, "source": ["<h3>Machine Learning</h3>\n", "\n", "Machine learning (ML) broadly describes the methods by which computers are able to learn mathematical models to describe data. If this sounds a little familiar, don't be surprised: ML at its core is fitting.\n", "\n", "ML involves three main components:\n", "\n", "- Model\n", "    - chosen mathematical model (depends on the task / available data)\n", "- Learning\n", "    - estimate statistical model from data \n", "- Prediction/Inference\n", "    - use statistical model to make predictions on new data points and infer properties"]}, {"cell_type": "markdown", "id": "fc617052", "metadata": {"tags": ["learner", "md", "lect_01"]}, "source": ["<h3>Supervised Learning</h3>\n", "\n", "Supervised learning is a type of machine learning that involves learning from labeled data. In supervised learning, the algorithm is trained on a dataset where the inputs are labeled with their corresponding outputs. The algorithm uses this labeled data to learn the relationship between the inputs and outputs, and then uses this knowledge to predict the output for new, unseen inputs.\n", "\n", "Given N examples with features ${x_i\\in X}$ and targets ${y_i\\in Y}$, learn function mapping $h(x)=y$."]}, {"cell_type": "markdown", "id": "99fbf75e", "metadata": {"tags": ["learner", "md", "lect_01"]}, "source": ["The code cell below generates the set of blue and red points mentioned in the video. Feel free to try various values of the `noise` parameter, but be sure to go back and rerun it with `noise=1` before continuing."]}, {"cell_type": "code", "execution_count": null, "id": "35eae3b6", "metadata": {"tags": ["learner", "py", "lect_01", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.1-runcell01\n", "\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "#NOTE: this sets the random seed\n", "#the random data will look slightly different from the video\n", "np.random.seed(42)\n", "\n", "#define points\n", "x1 = np.random.normal(0, 5, 100) #mean, sigma, num_points\n", "x2 = np.random.normal(0, 5, 100)\n", "\n", "noise = 1. #sets the noise scale\n", "x1noise = np.random.normal(0, noise, 100)\n", "x2noise = np.random.normal(0, noise, 100)\n", "\n", "mask_pos = x2 > (2. - 3.*x1)\n", "mask_neg = x2 <= (2. - 3.*x1)\n", "\n", "x1 = x1 + x1noise\n", "x2 = x2 + x2noise\n", "\n", "# let's define a true boundary between the two classes\n", "# by x_2 = 2 - 3 x_1\n", "x1_pos = x1[mask_pos]\n", "x2_pos = x2[mask_pos]\n", "x1_neg = x1[mask_neg]\n", "x2_neg = x2[mask_neg]\n", "\n", "\n", "#look at things\n", "plt.plot(x1_pos, x2_pos, 'r+')\n", "plt.plot(x1_neg, x2_neg, 'b.')\n", "\n", "plt.show()"]}, {"cell_type": "markdown", "id": "dce66512", "metadata": {"tags": ["learner", "md", "lect_01"]}, "source": ["<h3>Types of Supervised Learning</h3>\n", "\n", "\n", "<h4>Classification</h4>\n", "\n", "In classification tasks, $Y$ is a finite set of labels (i.e. classes). Can be:\n", "\n", "- Binary classification ($Y={0,1}$) e.g. blue vs. red as we have here, or signal vs. background\n", "\n", "- Multi-class classification ($Y={c_1,c_2,...,c_n}$) e.g. blue vs. red vs. green, or is this particle an electron or pion or photon? \n", "    - Multi-class labels are typically represented using \"one-hot-encoding\" or \"one-hot-vectors\": $y_i=(0,0,...,1,...,0)$ where the $k$th element is 1 and all other elements are 0 for class $c_k$.\n", "        - For case of blue vs. red vs. green we could encode blue as $c_0=(1,0,0)$, red as $c_1=(0,1,0)$, green as $c_2=(0,0,1)$\n", "\n", "<h4>Regression</h4>\n", "\n", "In regression tasks, $Y$ is the real numbers\n", "    - Instead of color, a real number is assigned at each point (equivalent to a 3D scatter plot)\n", "\n"]}, {"cell_type": "markdown", "id": "a33ed572", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='exercises_13_1'></a>     \n", "\n", "| [Top](#section_13_0) | [Restart Section](#section_13_1) | [Next Section](#section_13_2) |\n"]}, {"cell_type": "markdown", "id": "0c43e5b3", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.1.1</span>\n", "\n", "Change the `noise` parameter in the code that you previously ran. What do you observe about the red and blue points as the `noise` value is *increased*? **Afterwards set the noise parameter back to its original value, `noise = 1.`, as other problems will depend on the original form.**\n", "\n", "Choose the best answer from the following:\n", "\n", "A) Increasing the noise adds vertical jitter to the points.\\\n", "B) Increasing the noise adds more points to the plot.\\\n", "C) Increasing the noise causes the points to mix across the line that defines the boundary between the two classes.\\\n", "D) Increasing the noise simply makes the random selection of point locations different from run to run.\\"]}, {"cell_type": "markdown", "id": "48384344", "metadata": {"tags": ["learner", "md"]}, "source": [">#### Follow-up 13.1.1a (ungraded)\n", ">\n", ">Try defining a new boundary that separates the classes. Feel free to play around with this! **Afterwards, set everything back to the intitial state, as other problems will depend on the original form.**\n"]}, {"cell_type": "markdown", "id": "9f489916", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.1.2</span>\n", "\n", "Which of the following statements best describes the difference between machine learning and supervised learning?\n", "\n", "A) Machine learning is a type of learning where the algorithm learns from labeled data, while supervised learning is a broader term that includes different types of learning.\\\n", "B) Supervised learning is a type of learning where the algorithm learns from labeled data, while machine learning is a broader term that includes different types of learning.\\\n", "C) Machine learning and supervised learning are interchangeable terms that describe the same thing."]}, {"cell_type": "markdown", "id": "4b52e45c", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.1.3</span>\n", "\n", "Consider a data set with 10 classes. What would be the length of the one-hot-encoding vector that describes one of the classes? Enter your answer as an integer."]}, {"cell_type": "markdown", "id": "2c44db91", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='section_13_2'></a>\n", "<hr style=\"height: 1px;\">\n", "\n", "## <h2 style=\"border:1px; border-style:solid; padding: 0.25em; color: #FFFFFF; background-color: #90409C\">L13.2 Procedure and Example Algorithms</h2>  \n", "\n", "| [Top](#section_13_0) | [Previous Section](#section_13_1) | [Exercises](#exercises_13_2) | [Next Section](#section_13_3) |\n"]}, {"cell_type": "markdown", "id": "8ac7b13a", "metadata": {"tags": ["learner", "md"]}, "source": ["*The material in this section is discussed in the video **<a href=\"https://courses.mitxonline.mit.edu/learn/course/course-v1:MITxT+8.S50.2x+2T2023/block-v1:MITxT+8.S50.2x+2T2023+type@sequential+block@seq_LS13/block-v1:MITxT+8.S50.2x+2T2023+type@vertical+block@vert_LS13_vid2\" target=\"_blank\">HERE</a>.** You are encouraged to watch that video and use this notebook concurrently.*"]}, {"cell_type": "markdown", "id": "591e55ea", "metadata": {"tags": ["learner", "md", "lect_02"]}, "source": ["<h3>Procedure for Supervised Learning</h3>\n", "\n", "- Design a function (model) with adjustable parameters\n", "- Design a loss function\n", "- Find best parameters which minimize loss\n", "    - Use a labeled *training-set* to compute loss\n", "    - Adjust parameters to reduce loss function\n", "    - Repeat until parameters stabilize (minima)\n", "- Estimate final performance on *test-set*\n"]}, {"cell_type": "markdown", "id": "6e9a796a", "metadata": {"tags": ["learner", "md", "lect_02"]}, "source": ["<h3>Example 1: Perceptron Algorithm</h3>\n", "\n", "For our first simplest discrimination, we are going to define a new function that determines whether we can discriminate something. For this algorithm, what we will do is define a label for red, which we set to 1, and a label for blue, which we set to -1. The label, we can then assign to all of our data points above. We can call this label for each i-th point $y_{i}$. We can then define a \"loss\" effectively negative log-likelihood-like thing that we would like to minimize. This loss will be low if we have good discrimination, and high if we don't. \n", "\n", "For the perceptron, we can define loss by first defining a function that is +1 if points are above a line, and -1 if points are below the line, or in other words: \n", "\n", "$$\n", "f(x|w,b) = \\textrm{sign}(w^{T}\\vec{x}+b)  = \\textrm{sign}(w_1x_1+w_2x_2+b)\n", "$$\n", "\n", "where here $x_{1}$ and $x_{2}$ are our coordinates in the space above. \n", "\n", "Finally, we can define a loss that will just be the product of the truth with the function above. If we get both right we have either $+1 \\times +1$ or $-1 \\times -1$, which is just $+1$. The sum over all of these will be largest if we have the best line. The negative sum over all of these will be smallest if we have the best line, hence we take the negative to minimize. Now for simplicity of the problem, we can take the max relative to $0$. \n", "\n", "<h4>Characteristics</h4>\n", "\n", "Basic classification: two inputs, two classes (denoted by -1 and +1)\n", "\n", "Linear model: $f(x|w,b) = \\textrm{sign}(w^{T}x+b)$ = $\\textrm{sign}(w_1x_1+w_2x_2+b)$\n", "\n", "Loss: $L_i = \\textrm{max}[0,-y_i f(x_i|w,b)]$,  $y_i\\in{-1,+1}$\n", "\n", "Issue: No way to distinguish two solutions. Is $x_2 = 2 - 3.001 x_1$ a better solution that $x_2 = 2 - 3 x_1$ in this case?"]}, {"cell_type": "code", "execution_count": null, "id": "fb775721", "metadata": {"tags": ["learner", "py", "lect_02", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.2-runcell01\n", "\n", "def perceptron_loss(x,y,w,b):\n", "    loss = 0.\n", "    for i in range(len(x[0])):\n", "        fm = np.sign(x[0][i]*w[0]+x[1][i]*w[1]+b)\n", "        loss = loss + max(0.,-1.*y[i]*fm)\n", "    return loss\n", "\n", "x_list = [np.concatenate((x1_pos,x1_neg)),np.concatenate((x2_pos,x2_neg))]\n", "y_list = [1.]*len(x1_pos) + [-1.]*len(x1_neg)\n", "\n", "\n", "#NOTE: because our random data are different from the video,\n", "#these results will look slightly different from the video\n", "print(\"Perceptron loss (w1=3, w2=1, b=-2)\")\n", "print(perceptron_loss(x_list,y_list,[3.,1.],-2.))\n", "print()\n", "print(\"Perceptron loss (w1=3, w2=1, b=-10.)\")\n", "print(perceptron_loss(x_list,y_list,[3.,1.],-10.))\n", "print()\n", "print(\"Perceptron loss (w1=3.001, w2=1, b=-2)\")\n", "print(perceptron_loss(x_list,y_list,[3.,1.],-2.1))"]}, {"cell_type": "markdown", "id": "8e2e373e", "metadata": {"tags": ["learner", "md", "lect_02"]}, "source": ["From the results shown above, you can see that the perceptron loss is a discrete function, and it classifies one side as good, and the other side as bad. Let's go ahead and plot this discrete setup. "]}, {"cell_type": "code", "execution_count": null, "id": "1a4d56a0", "metadata": {"tags": ["learner", "py", "lect_02", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.2-runcell02\n", "\n", "#create grid\n", "x1_list = np.linspace(-20., 20., 100)\n", "x2_list = np.linspace(-20., 20., 100)\n", "x1_grid, x2_grid  = np.meshgrid(x1_list, x2_list)\n", "\n", "#fill with model value\n", "h_grid = np.sign(3.*x1_grid + x2_grid - 2. )\n", "\n", "#draw 2d mesh\n", "plt.pcolormesh(x1_grid, x2_grid, h_grid, cmap = 'bwr', shading='auto')\n", "\n", "plt.plot(x1_pos, x2_pos, 'k+')\n", "plt.plot(x1_neg, x2_neg, 'k.')\n", "\n", "plt.show()"]}, {"cell_type": "markdown", "id": "37656a9e", "metadata": {"tags": ["learner", "md", "lect_02"]}, "source": ["<h3>Example 2: Support Vector Machine</h3>\n", "\n", "Now that we have done a very basic discrimination with a perceptron algorithm, we realize this is not the most useful setup since it gives us a discrete classification of what is good and what is bad. As a consequence, to really go further, we need to avoid the discretization and make a classifier that carries some notion of distance away from the line. The simplest way to do this is with something called a support vector machine. \n", "\n", "We can write this as: \n", "\n", "$$\n", "f(x|w,b) = w^{T}\\vec{x}+b\n", "$$\n", "\n", "\n", "Now, similar to what was done before, we have a function which is positive for points above the line and negative for points below the line, and we again multiply by the truth to get +1 for correctly identified points. However, now we have a loss which is larger the farther a point is away from the line. \n", "\n", "\n", "$$\n", "\\textrm{max}[0,1-y_i f(x_i|w,b)],  y_i\\in{-1,+1}\n", "$$"]}, {"cell_type": "markdown", "id": "4d942ed4", "metadata": {"tags": ["learner", "md", "lect_02"]}, "source": ["<h4>Characteristics</h4>\n", "\n", "Similar to what was done for the perceptron algorithm, we modify the loss to select maximally discriminant parameters:\n", "\n", "\n", "Linear model: $f(x|w,b) = w^{T}x+b$\n", "\n", "Loss: $L_i = \\textrm{max}[0,1-y_i f(x_i|w,b)]$,  $y_i\\in{-1,+1}$\n"]}, {"cell_type": "code", "execution_count": null, "id": "f7ae62a2", "metadata": {"tags": ["learner", "py", "lect_02", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.2-runcell03\n", "\n", "def svm_loss(x,y,w,b):\n", "    loss = 0.\n", "    for i in range(len(x[0])):\n", "        fm = x[0][i]*w[0]+x[1][i]*w[1]+b\n", "        loss = loss + max(0.,1.-1.*y[i]*fm)\n", "    return loss\n", "\n", "#NOTE: because our random data are different from the video,\n", "#these results will look slightly different from the video\n", "print(\"SVM loss (w1=3, w2=1, b=-2)\")\n", "print(svm_loss(x_list,y_list,[3.,1.],-2.))\n", "print()\n", "print(\"SVM loss (w1=3, w2=1, b=-1.9)\")\n", "print(svm_loss(x_list,y_list,[3.,1.],-1.9))\n", "print()\n", "print(\"SVM loss (w1=3, w2=1, b=-2.1)\")\n", "print(svm_loss(x_list,y_list,[3.,1.],-2.1))"]}, {"cell_type": "markdown", "id": "b951b320", "metadata": {"tags": ["learner", "md", "lect_02"]}, "source": ["Now, we see that when we are close to the line and moving around it, our values for the loss start to change, this avoids the discrete issue we had above.\n", "\n", "Let's go ahead and visualize it."]}, {"cell_type": "code", "execution_count": null, "id": "cd59d801", "metadata": {"tags": ["learner", "py", "lect_02", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.2-runcell04\n", "\n", "#create grid\n", "x1_list = np.linspace(-20., 20., 100)\n", "x2_list = np.linspace(-20., 20., 100)\n", "x1_grid, x2_grid  = np.meshgrid(x1_list, x2_list)\n", "\n", "#fill with model value\n", "h_grid = np.sign(3.*x1_grid + x2_grid - 2. )\n", "\n", "\n", "#draw 2d mesh\n", "plt.pcolormesh(x1_grid, x2_grid, h_grid, cmap = 'bwr', shading='auto')\n", "\n", "plt.plot(x1_pos, x2_pos, 'k+')\n", "plt.plot(x1_neg, x2_neg, 'k.')\n", "\n", "plt.show()"]}, {"cell_type": "markdown", "id": "9ddb42ea", "metadata": {"tags": ["learner", "md", "lect_02"]}, "source": ["Better... but what we really want is a probabilistic model. For data points we haven't seen, and more complex data, we want the algorithm to also output its confidence in the prediction. We'll look at such a model next."]}, {"cell_type": "markdown", "id": "7423d891", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='exercises_13_2'></a>     \n", "\n", "| [Top](#section_13_0) | [Restart Section](#section_13_2) | [Next Section](#section_13_3) |\n"]}, {"cell_type": "markdown", "id": "29aa3bef", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.2.1</span>\n", "\n", "A loss function that returns zero indicates that the prediction is:\n", "\n", "A) correct\\\n", "B) incorrect"]}, {"cell_type": "markdown", "id": "b6709ce3", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.2.2</span>\n", "\n", "In the example in this section, the main difference between the perceptron loss function and the support vector machine loss function is that:\n", "\n", "A) The perceptron loss function is only suitable for linearly separable data, while the support vector machine loss function can handle non-linearly separable data.\\\n", "B) The support vector machine loss function only considers the samples that are closest to a boundary, while the perceptron loss function takes into account all samples.\\\n", "C) The perceptron loss function is a step function (correct vs. incorrect), whereas the support vector machine loss function assigns a continuous value to a data point, based on how close it is to a boundary."]}, {"cell_type": "markdown", "id": "8983a5e0", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.2.3</span>\n", "\n", "Write code in the cell provided in the notebook to compute the perceptron loss for a slope of the dividing line within the range of 2 to 4 (while keeping the constant term at -2), and then vary the constant within the range of -2 to -4 (while keeping the slope at 3). Do you still see the same loss? What is the minimum?\n", "\n", "Specifically, varying which parameter will produce the GREATER variation in loss?\n", "    \n", "A) varying slope\\\n", "B) varying constant"]}, {"cell_type": "code", "execution_count": null, "id": "fcc3e872", "metadata": {"tags": ["draft", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>EXERCISE: L13.2.3\n", "# Use this cell for drafting your solution (if desired),\n", "# then enter your solution in the interactive problem online to be graded.\n", "\n"]}, {"cell_type": "markdown", "id": "a9f3dfb7", "metadata": {"tags": ["learner", "md"]}, "source": [">#### Follow-up 13.2.3a (ungraded)\n", ">\n", ">For large randomness (i.e. an increased value of the `noise`), find both the perceptron and SVM loss. At what randomness does the loss break down? **Afterwards, set everything back to the intitial state, as other problems will depend on the original form.**\n", "\n"]}, {"cell_type": "markdown", "id": "7e4b0260", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.2.4</span>\n", "\n", "Let's say we have a single point in our distribution, at `(100,100)`. How does this change the svm loss if it is correct vs. incorrect?\n", "\n", "Calculate the svm loss for both cases, using the code cell in your notebook, and report your answer as a list of numbers with precision 1e-1: `[correct, incorrect]`"]}, {"cell_type": "code", "execution_count": null, "id": "9e0c241c", "metadata": {"tags": ["draft", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>EXERCISE: L13.2.4\n", "# Use this cell for drafting your solution (if desired),\n", "# then enter your solution in the interactive problem online to be graded.\n", "\n"]}, {"cell_type": "markdown", "id": "7b4d97d8", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='section_13_3'></a>\n", "<hr style=\"height: 1px;\">\n", "\n", "## <h2 style=\"border:1px; border-style:solid; padding: 0.25em; color: #FFFFFF; background-color: #90409C\">L13.3 Logistic Regression Algorithm</h2>  \n", "\n", "| [Top](#section_13_0) | [Previous Section](#section_13_2) | [Exercises](#exercises_13_3) | [Next Section](#section_13_4) |\n"]}, {"cell_type": "markdown", "id": "115070da", "metadata": {"tags": ["learner", "md"]}, "source": ["*The material in this section is discussed in the video **<a href=\"https://courses.mitxonline.mit.edu/learn/course/course-v1:MITxT+8.S50.2x+2T2023/block-v1:MITxT+8.S50.2x+2T2023+type@sequential+block@seq_LS13/block-v1:MITxT+8.S50.2x+2T2023+type@vertical+block@vert_LS13_vid3\" target=\"_blank\">HERE</a>.** You are encouraged to watch that video and use this notebook concurrently.*"]}, {"cell_type": "markdown", "id": "a3e4924a", "metadata": {"tags": ["learner", "md", "lect_03"]}, "source": ["<h3>Overview</h3>\n", "\n", "The last exercise in the previous section identified a disadvantage to the \"distance\" used in the support vector machine example, namely that one outlier can spoil the whole optimization process. This can be avoided by using a loss function that peaks at +/- 1, even if a point is very far away. To accomplish this, let's define a new function that tries to capture discrimination in terms of probability. \n", "\n", "\n", "$$\n", "f(x|w,b)=\\frac{1}{1+e^{-(w^{T}\\vec{x}+b)}}\n", "$$\n", "\n", "The value of $f(x|w,b)=0$ when $x\\rightarrow-\\infty$, and $1$ when $x\\rightarrow\\infty$. Thus, this function is nicely bounded and gives us the opportunity to write down a loss that we can minimize. For this scenario, instead of labelling blue and red as $\\pm$1, what we will do is label red as $1$ and blue as $0$. A loss with label 1 will give us the function $f(x|w,b)$ that we will want to set close to 1. A loss with label 0, will give us the same thing if we take $1-f(x|w,b)$. \n", "\n", "Now, we can go one step further, and recall from statistical mechanics that the Gibb's entropy, which we can write with Boltzmann's constant as: \n", "\n", "\n", "$$\n", "S_{G} = k_{B} \\sum_{i} p_{i} \\log (p_{i})\n", "$$\n", "\n", "where we have given $n$ states with probability $p_{i}$ for each state, defines the $\\log$ of the number of allowed states following the entropy definition.   \n", "\n", "We can do the same thing for machine learning by considering two probabilities $p$ and $q$. We want these probabilities to match as close as possible, or in other words we want to maximize the number of states that are correct, and minimize the number of states that are incorrect. We can write this in terms of a likelihood as\n", "\n", "$$\n", "\\mathcal{L}(\\theta) = \\prod_{i} ({\\rm Estimated \\,probability\\, of\\, occurence})^{(\\rm number\\, of \\,occurence)} \\\\\n", "\\mathcal{L}(\\theta) = \\prod_{i} (q_{i})^{N p_{i}} \\\\\n", "\\log(\\mathcal{L}(\\theta)) = N \\sum_{i} p_{i} \\log(q_{i})\n", "$$\n", "\n", "\n", "and define the cross entropy noting the \"estimated\" probability is $q_{i}$ for an $i$-th instance, with the chances of this happening given by the $N$ \"empirical\" probability $N p_{i}$ where $p_{i}$ is the empirical probability. Effectively, this is playing off our best fit function $f$ against the observed data $p$ and $N$.\n", "\n", "For this scenario, our observed probability is red $1$ or blue $0$, which is just the truth label, or in this case $y_{i}=[0,1]$ for blue and red respectively. Likewise, we can make the predicted probability $q_{i}(\\theta)=f(x|w,b)$ apply for one scenario (blue). The cross entropy then can be written \n", "\n", "\n", "$$\n", "\\log(\\mathcal{L_{\\rm red}}(\\theta))  = \\sum_{i} y_{i} \\log(f(x_i|w,b)) \\\\\n", "\\log(\\mathcal{L_{\\rm blue}}(\\theta)) = \\sum_{i} (1-y_{i}) \\log(1-f(x_i|w,b)) \\\\\n", "L_{i} = -\\log(\\mathcal{L_{\\rm red}}(\\theta))  - \\log(\\mathcal{L_{\\rm blue}}(\\theta))\n", "$$\n", "\n", "In other words, we sum over both samples of red and blue and aim to maximize the probability that our system is correct. \n", "\n", "Let's go ahead and run over the regression and see how it looks. "]}, {"cell_type": "markdown", "id": "5f19950f", "metadata": {"tags": ["learner", "md", "lect_03"]}, "source": ["<h3>Example 3: Logistic regression</h3>\n", "\n", "Change from \\[-1,1\\] to \\[0,1\\] for simplicity\n", "\n", "Model: $f(x|w,b)=\\frac{1}{1+e^{-(w^{T}x+b)}}$\n", "\n", "Loss: $L_i = y_i \\log(f(x_i|w,b)) + (1-y_i)\\log(1-f(x_i|w,b))$\n", "    (known as *Binary Cross Entropy*)"]}, {"cell_type": "code", "execution_count": null, "id": "14b5b9a0", "metadata": {"tags": ["learner", "py", "lect_03", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.3-runcell01\n", "\n", "#switch to [0,1] from [-1,1]\n", "y_list_0 = [1.]*len(x1_pos) + [0.]*len(x1_neg)\n", "\n", "def lr_loss(x,y,w,b,scale=1.):\n", "    loss = 0.\n", "    for i in range(len(x[0])):\n", "        expon = -1.*scale*(x[0][i]*w[0]+x[1][i]*w[1]+b)\n", "        if (expon > 99.):\n", "            expon = 99.\n", "        fm = 1./(1.+np.exp(expon))\n", "        if (fm < 1. and fm > 0.):\n", "            loss = loss - y[i]*np.log(fm) - (1.-y[i])*np.log(1.-fm)\n", "    return loss\n", "\n", "\n", "#NOTE, the outputs in this notebook will be different from the video\n", "#because we have used a different random seed\n", "print('>>>Setting scale = 1')\n", "print(\"Logistic regression loss (w1=3, w2=1, b=-2) [scale = 1]\")\n", "print(lr_loss(x_list,y_list_0,[3.,1.],-2.))\n", "print()\n", "print(\"Logistic regression loss (w1=3, w2=1, b=-1.9) [scale = 1]\")\n", "print(lr_loss(x_list,y_list_0,[3.,1.],-1.9))\n", "print()\n", "print(\"Logistic regression loss (w1=3, w2=1, b=-2.1) [scale = 1]\")\n", "print(lr_loss(x_list,y_list_0,[3.,1.],-2.1))\n", "print()\n", "print('>>>Setting scale = 1.5')\n", "print(\"Logistic regression loss (w1=3, w2=1, b=-2) [scale = 1.5]\")\n", "print(lr_loss(x_list,y_list_0,[3.,1.],-2.,1.5))\n", "print()\n", "print('>>>Setting scale = 0.5')\n", "print(\"Logistic regression loss (w1=3, w2=1, b=-2) [scale = 0.5]\")\n", "print(lr_loss(x_list,y_list_0,[3.,1.],-2,0.5))"]}, {"cell_type": "markdown", "id": "0550f9b2", "metadata": {"tags": ["learner", "md", "lect_03"]}, "source": ["Notice that the code above includes an additional `scale` term, which changes the impact of the slope on the exponential. What this means is: \n", "\n", " * A large scale will increase confidence\n", " * A small scale will decrease the confidence of discrimination\n", " \n", "Let's visualize this below: "]}, {"cell_type": "code", "execution_count": null, "id": "ad25a791", "metadata": {"tags": ["learner", "py", "lect_03", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.3-runcell02\n", "\n", "#fill with model value - try changing the scale parameter\n", "h_grid = 1./(1.+np.exp(-1.*0.5*(x2_grid - (2. - 3.*x1_grid))))\n", "\n", "#draw 2d mesh\n", "plt.pcolormesh(x1_grid, x2_grid, h_grid, cmap = 'bwr', shading='auto')\n", "plt.colorbar()\n", "\n", "plt.plot(x1_pos, x2_pos, 'k+')\n", "plt.plot(x1_neg, x2_neg, 'k.')\n", "\n", "plt.show()"]}, {"cell_type": "markdown", "id": "be550e22", "metadata": {"tags": ["learner", "md", "lect_03"]}, "source": ["The above is a sort of neural network with an output from 0 to 1. The output function is what we call a sigmoid. The matrix multiply of the inputs is what we call the input layer. That's it! \n", "\n", "However, there's a problem. With this simple model (two feature inputs, one output with a sigmoid activation, and no hidden layers), we can never predict anything more than a line boundary. We will examine more complex architectures next!\n", "\n", "In the meantime, here is a <a href=\"https://playground.tensorflow.org/\" target=\"_blank\">fun tool</a> to build your own neural networks. Have a go!"]}, {"cell_type": "markdown", "id": "b14d0f1f", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='exercises_13_3'></a>     \n", "\n", "| [Top](#section_13_0) | [Restart Section](#section_13_3) | [Next Section](#section_13_4) |\n"]}, {"cell_type": "markdown", "id": "e591c07a", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.3.1</span>\n", "\n", "Complete the code below to plot the logistic curve in one dimension: $f(x|scale)=\\dfrac{1}{1+e^{-\\mathrm{scale}*x}}$\n", "\n", "Afterwards, try varying the parameters of the plot."]}, {"cell_type": "code", "execution_count": null, "id": "941fee53", "metadata": {"tags": ["draft", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>EXERCISE: L13.3.1\n", "# Use this cell for drafting your solution (if desired),\n", "# then enter your solution in the interactive problem online to be graded.\n", "\n", "def logistic_curve(x,scale):\n", "    return #YOUR CODE HERE\n", "\n", "# Define the x values\n", "x = np.linspace(-10, 10, 1000)\n", "\n", "# Define the y values\n", "scale = 0.5\n", "y = logistic_curve(x, scale)\n", "\n", "# Plot the function\n", "plt.plot(x, y)\n", "plt.xlabel('x')\n", "plt.ylabel('y')\n", "plt.title('Logistic Curve with Scale = {}'.format(scale))\n", "plt.grid(True)\n", "plt.show()\n"]}, {"cell_type": "markdown", "id": "0c1cf0d7", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.3.2</span>\n", "\n", "How do the features of the logistic curve change as you increase the scale parameter?\n", "\n", "A) The curve shifts to the left\\\n", "B) The curve shifts to the right\\\n", "C) The curve becomes steeper\\\n", "D) The curve becomes flatter"]}, {"cell_type": "markdown", "id": "e7f342da", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.3.3</span>\n", "\n", "Consider the plot produced by code cell `L13.3-runcell02`. That code had a fixed factor of 0.5 in the exponent, rather than an adjustable `scale` parameter. Write code in your notebook to reproduce the plot using modified values of the `scale` parameter, first to 100 larger, then to 100 times smaller. How does this impact the value at a position on the lower left corner of the plot (-20,-20)? Choose the best answer from the following:\n", "\n", "\n", "A) Modifying the scale parameter has no effect on the z-axis value at (-20,-20).\\\n", "B) Increasing the scale parameter leads to a higher value at (-20,-20).\\\n", "C) Increasing the scale parameter has no effect on the z-axis value at (-20,-20).\\\n", "D) Increasing the scale parameter leads to a lower value at (-20,-20).\\\n", "E) Decreasing the scale parameter leads to a higher value at (-20,-20).\\\n", "F) Decreasing the scale parameter has no effect on the z-axis value at (-20,-20).\\\n", "G) Decreasing the scale parameter leads to a lower value at (-20,-20).\n", "\n"]}, {"cell_type": "code", "execution_count": null, "id": "3ed27a2f", "metadata": {"tags": ["draft", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>EXERCISE: L13.3.3\n", "# Use this cell for drafting your solution (if desired),\n", "# then enter your solution in the interactive problem online to be graded.\n"]}, {"cell_type": "markdown", "id": "bddbad5a", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='section_13_4'></a>\n", "<hr style=\"height: 1px;\">\n", "\n", "## <h2 style=\"border:1px; border-style:solid; padding: 0.25em; color: #FFFFFF; background-color: #90409C\">L13.4 Introduction to Neural Networks</h2>  \n", "\n", "| [Top](#section_13_0) | [Previous Section](#section_13_3) | [Exercises](#exercises_13_4) | [Next Section](#section_13_5) |\n"]}, {"cell_type": "markdown", "id": "27f3cbde", "metadata": {"tags": ["learner", "md"]}, "source": ["*The material in this section is discussed in the video **<a href=\"https://courses.mitxonline.mit.edu/learn/course/course-v1:MITxT+8.S50.2x+2T2023/block-v1:MITxT+8.S50.2x+2T2023+type@sequential+block@seq_LS13/block-v1:MITxT+8.S50.2x+2T2023+type@vertical+block@vert_LS13_vid4\" target=\"_blank\">HERE</a>.** You are encouraged to watch that video and use this notebook concurrently.*"]}, {"cell_type": "markdown", "id": "72146725", "metadata": {"tags": ["learner", "md"]}, "source": ["<h3>Slides</h3>\n", "\n", "Run the code below to view the slides for this section, which are discussed in the related video. You can also open the slides in a separate window <a href=\"https://mitx-8s50.github.io/slides/L13/slides_L13_04.html\" target=\"_blank\">HERE</a>."]}, {"cell_type": "code", "execution_count": null, "id": "005c8934", "metadata": {"tags": ["learner", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.4-slides\n", "\n", "from IPython.display import IFrame\n", "IFrame(src='https://mitx-8s50.github.io/slides/L13/slides_L13_04.html', width=970, height=550)"]}, {"cell_type": "markdown", "id": "f4e2b57d", "metadata": {"tags": ["learner", "md", "lect_04"]}, "source": ["<h3>Overview</h3>\n", "\n", "A neural network is composed of multiple layers of interconnected processing units, or neurons, that perform a sequence of computations on the input data to produce an output. Each layer in a neural network consists of multiple neurons that process the input data and produce an output.\n", "\n", "The input layer is the first layer in the neural network, which takes the raw input data and passes it to the next layer. The output of the input layer is then fed into one or more hidden layers, which perform a sequence of transformations on the input data. Each neuron in a hidden layer takes the outputs of the previous layer, applies a linear transformation (i.e., a weighted sum of the inputs), and passes the result through a non-linear activation function.\n", "\n", "The activation function is a critical component of each neuron, as it introduces non-linearity into the network and enables the model to learn complex patterns in the data. Commonly used activation functions include the sigmoid, linear and tanh functions, as well as the ReLU (Rectified Linear Unit) and LeakyReLU functions. For multiclass cases, a Softmax function might be used.\n", "\n", "The output layer of the neural network takes the output of the final hidden layer and produces the final output of the model. The activation function used in the output layer depends on the type of problem being solved. For example, a binary classification problem might use a sigmoid function in the output layer to produce a probability score between 0 and 1, while a multi-class classification problem might use a Softmax function to produce a probability distribution over the classes."]}, {"cell_type": "markdown", "id": "0100f6fc", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='exercises_13_4'></a>     \n", "\n", "| [Top](#section_13_0) | [Restart Section](#section_13_4) | [Next Section](#section_13_5) |\n"]}, {"cell_type": "markdown", "id": "6e863fc6", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.4.1</span>\n", "\n", "Having introduced the concept of activation functions, let's try a different activation function in our output node, for the data set we have been studying. Specifically, we will replace the logistic function (i.e., sigmoid) with a `tanh` activation function.\n", "\n", "Since the `tanh` function outputs values over the range [-1,1], it is not suitable to use for calculating the Binary Cross Entropy (since BCE requires probabilities between [0,1]). Therefore, let's calculate the loss using the Support Vector Machine algorithm. Let's also convert the binary data back to the range [-1,1] (for now).\n", "\n", "**Fill in the code below to calculate the loss for the parameters given, and enter your answer as a number with precision 1e-2.**\n", "\n", "Try varying the parameters to see where the loss is minimized, as we have done previously. Does this loss function give results that are similar to the Binary Cross Entropy with sigmoid activation (i.e., what we did in `L13.3-runcell01`)?"]}, {"cell_type": "code", "execution_count": null, "id": "c13f3673", "metadata": {"tags": ["draft", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>EXERCISE: L13.4.1\n", "# Use this cell for drafting your solution (if desired),\n", "# then enter your solution in the interactive problem online to be graded.\n", "\n", "def tanh_loss(x, y, w, b, scale=1.):\n", "    loss = 0.\n", "    for i in range(len(x[0])):\n", "        fm = #YOUR CODE HERE\n", "        loss = loss + (1-1.*y[i]*fm)        \n", "    return loss\n", "\n", "#using y_list with values from [-1,1]\n", "print(\"tanh loss (w1=3, w2=1, b=-2) [scale = 1]\")\n", "print(tanh_loss(x_list,y_list,[3.,1.],-2.))"]}, {"cell_type": "markdown", "id": "2bae4aa4", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='section_13_5'></a>\n", "<hr style=\"height: 1px;\">\n", "\n", "## <h2 style=\"border:1px; border-style:solid; padding: 0.25em; color: #FFFFFF; background-color: #90409C\">L13.5 What Do Neural Networks Do?</h2>     \n", "\n", "| [Top](#section_13_0) | [Previous Section](#section_13_4) | [Exercises](#exercises_13_5) | [Next Section](#section_13_6) |\n"]}, {"cell_type": "markdown", "id": "445f7e68", "metadata": {"tags": ["learner", "md"]}, "source": ["*The material in this section is discussed in the video **<a href=\"https://courses.mitxonline.mit.edu/learn/course/course-v1:MITxT+8.S50.2x+2T2023/block-v1:MITxT+8.S50.2x+2T2023+type@sequential+block@seq_LS13/block-v1:MITxT+8.S50.2x+2T2023+type@vertical+block@vert_LS13_vid5\" target=\"_blank\">HERE</a>.** You are encouraged to watch that video and use this notebook concurrently.*"]}, {"cell_type": "markdown", "id": "b396e90f", "metadata": {"tags": ["learner", "md"]}, "source": ["<h3>Slides</h3>\n", "\n", "Run the code below to view the slides for this section, which are discussed in the related video. You can also open the slides in a separate window <a href=\"https://mitx-8s50.github.io/slides/L13/slides_L13_05.html\" target=\"_blank\">HERE</a>."]}, {"cell_type": "code", "execution_count": null, "id": "ad79573d", "metadata": {"tags": ["learner", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.5-slides\n", "\n", "from IPython.display import IFrame\n", "IFrame(src='https://mitx-8s50.github.io/slides/L13/slides_L13_05.html', width=970, height=550)"]}, {"cell_type": "markdown", "id": "82f208a1", "metadata": {"tags": ["learner", "md", "lect_05"]}, "source": ["<h3>Overview</h3>\n", "\n", "During training, the neural network adjusts the weights and biases of each neuron using a process called backpropagation. The objective of backpropagation is to minimize the difference between the predicted output of the model and the true output, as measured by a loss function.\n", "\n", "The backpropagation algorithm works by propagating the error backward through the network, starting from the output layer and moving back through each layer in reverse order. The goal is to compute the gradient of the loss function with respect to each weight and bias in the network.\n", "\n", "To calculate the gradient, the algorithm uses the chain rule of differentiation to compute the derivative of the loss function with respect to each output of each neuron in the network. This derivative is then used to compute the derivative of the loss function with respect to each weight and bias in the network.\n", "\n", "The backpropagation algorithm is usually combined with a technique called gradient descent, which updates the weights and biases of the network in the direction of the negative gradient of the loss function. This process is repeated iteratively until the loss function is minimized, or until a stopping condition is met."]}, {"cell_type": "markdown", "id": "824b6631", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='exercises_13_5'></a>   \n", "\n", "| [Top](#section_13_0) | [Restart Section](#section_13_5) | [Next Section](#section_13_6) |"]}, {"cell_type": "markdown", "id": "bd65e1ad", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.5.1</span>\n", "\n", "Which of the following statements accurately describes how backpropagation works in neural networks?\n", "\n", "A) Backpropagation calculates the forward pass of the neural network to generate predictions, and then updates the weights and biases based on the difference between the predicted and true values.\n", "\n", "B) Backpropagation computes the gradient of the loss function with respect to the weights and biases of the neural network, and uses this gradient to update the parameters of the model.\n", "\n", "C) Backpropagation calculates the error of each individual neuron in the neural network, and adjusts the weights and biases of each neuron based on its individual error.\n", "\n", "D) Backpropagation randomly samples a subset of the training data and updates the weights and biases of the neural network based on the error on this subset, repeating the process until convergence is reached."]}, {"cell_type": "markdown", "id": "05bc938a", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.5.2</span>\n", "\n", "Let's return to the alternate activation function that we previously considered: the `tanh` function. Which of the following statements accurately describes the usefulness of using the `tanh` activation function, compared to other functions, for an output node in a binary classification task? Consider what we have just learned about the training process.\n", "\n", "Select all that apply:\n", "\n", "A) The tanh activation function is a symmetric function that captures both positive and negative patterns equally well.\\\n", "B) The gradients of the tanh function are relatively steeper compared to the sigmoid function, especially in the region closer to the origin. This can lead to faster convergence during training.\\\n", "C) When the input values to the tanh function are large, the gradients become close to zero, leading to faster convergence.\\\n", "D) None of the above."]}, {"cell_type": "markdown", "id": "fbbafcac", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='section_13_6'></a>\n", "<hr style=\"height: 1px;\">\n", "\n", "## <h2 style=\"border:1px; border-style:solid; padding: 0.25em; color: #FFFFFF; background-color: #90409C\">L13.6 Training a Neural Network</h2>  \n", "\n", "\n", "| [Top](#section_13_0) | [Previous Section](#section_13_5) | [Exercises](#exercises_13_6) | [Next Section](#section_13_7) |\n"]}, {"cell_type": "markdown", "id": "590a8ce8", "metadata": {"id": "wirAdBUP4nbD", "tags": ["learner", "md", "lect_06"]}, "source": ["<h3>Overview</h3>\n", "\n", "*Note: There is no corresponding video for this section.*\n", "\n", "How do we train the network? How do we handle more complicated data sets?\n", "\n", "Before we go into real training of our network, let's just treat our neural network as if we were to run a fit. On top of what we did above, let's just run a minimizer tool and minimize our loss.\n", "\n", "What we can do is take our loss above, which we used for the logistic regression algorithm, and solve for the best fit parameters using all the minimizer tools that we have been using in class! \n", "\n", "This process is what we call **Training** a neural network. "]}, {"cell_type": "code", "execution_count": null, "id": "498121e5", "metadata": {"colab": {"base_uri": "https://localhost:8080/", "height": 564}, "id": "-oK2xgwHxVsy", "outputId": "64c83ae6-1abb-43e5-f112-3be23f91efba", "tags": ["learner", "py", "lect_06", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.6-runcell01\n", "\n", "#the elements of the array x0 are defined as the following:\n", "#x0[0] -> w[0]\n", "#x0[1] -> w[1]\n", "#x0[2] -> b\n", "#x0[3] -> scale\n", "x0 = np.array([0,0,0,1.])\n", "\n", "\n", "def fun(inputs):    \n", "  #recall the inputs of lr_loss(): lr_loss(x,y,w,b,scale=1.)\n", "  loss=lr_loss(x_list,y_list_0,[inputs[0],inputs[1]],inputs[2],inputs[3])\n", "  return loss\n", "\n", "sol=opt.minimize(fun,x0)\n", "print(sol)\n", "print()\n", "\n", "#we print the weights and b,\n", "#rescaled such that they correspond to our original definition of the boundary\n", "print(\"(w1, w2, b):\", sol.x[3]*sol.x[0], sol.x[3]*sol.x[1], sol.x[3]*sol.x[2])\n", "\n", "\n", "#Now plot it\n", "h_grid = 1./(1.+np.exp(-1.*sol.x[3]*(sol.x[0]*x1_grid+sol.x[1]*x2_grid + sol.x[2] )))\n", "\n", "plt.pcolormesh(x1_grid, x2_grid, h_grid, cmap = 'bwr', shading='auto')\n", "plt.colorbar()\n", "\n", "plt.plot(x1_pos, x2_pos, 'k+')\n", "plt.plot(x1_neg, x2_neg, 'k.')\n", "\n", "plt.show()"]}, {"cell_type": "markdown", "id": "e0338763", "metadata": {"tags": ["learner", "md", "lect_06"]}, "source": ["Notice that we also floated our scale parameter in the fit done by the code shown above. We did this because we can.\n", "\n", "More importantly, notice that the code does not print the actual parameter values found in the fit, but rather those parameters multiplied by the fitted values of the scale (parameter `x[3]`):\n", "\n", "<pre>\n", "print(\"(w1, w2, b):\", sol.x[3]*sol.x[0], sol.x[3]*sol.x[1], sol.x[3]*sol.x[2]\n", "</pre>\n", "\n", "As mentioned in the code, this is necessary to get back to the parameters of the actual line dividing the two regions.\n", "\n", "One other thing that we can do with our minimizer is to look at the uncertainties on our fitted parameters. This is actually something we traditionally don't do with neural networks, the reason being that we will quickly make the networks very complicated with a lot of parameters. This makes it hard to compute the Hessian (it takes $\\mathcal{O}(N^{2})$ derivative computations where $N$ is the number of variables). \n", "\n"]}, {"cell_type": "markdown", "id": "34c68dfd", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='exercises_13_6'></a>     \n", "\n", "| [Top](#section_13_0) | [Restart Section](#section_13_6) | [Next Section](#section_13_7) |\n"]}, {"cell_type": "markdown", "id": "ed755350", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.6.1</span>\n", "\n", "What happens to the best fit parameters of the logistic regression model when you force the scale to be 1? Try this yourself in the code cell provided!\n", "\n", "Choose the best answer from the options below:\n", "\n", "A) Rescaled parameters are exactly the same, to within many decimal places.\\\n", "B) Rescaled parameters change a little, but not significantly.\\\n", "C) The rescaled parameters all decrease.\\\n", "D) The rescaled parameters all increase."]}, {"cell_type": "code", "execution_count": null, "id": "6ea5fcc9", "metadata": {"tags": ["draft", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>EXERCISE: L13.6.1\n", "# Use this cell for drafting your solution (if desired),\n", "# then enter your solution in the interactive problem online to be graded.\n", "\n"]}, {"cell_type": "markdown", "id": "b5bef474", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='section_13_7'></a>\n", "<hr style=\"height: 1px;\">\n", "\n", "## <h2 style=\"border:1px; border-style:solid; padding: 0.25em; color: #FFFFFF; background-color: #90409C\">L13.7 Training a Neural Network with PyTorch</h2>  \n", "\n", "\n", "| [Top](#section_13_0) | [Previous Section](#section_13_6) | [Exercises](#exercises_13_7) |"]}, {"cell_type": "markdown", "id": "c1f0378c", "metadata": {"tags": ["learner", "md", "lect_07"]}, "source": ["<h3>Overview</h3>\n", "\n", "*Note: There is no corresponding video for this section.*\n", "\n", "What we did in the previous sections was to simply fit a function, but this is effectively all we are doing when training a neural network. Let's go ahead and do the **exact same thing**, but now with the neural network. There are many ways to do this, but here we will focus on the `pytorch` approach to building a deep learning algorithm. \n", "\n", "The first step is to declare the network. We will label this as an \"MLP\" or multilayer perceptron. That is the technical definition of this network. Let's create it, and just print out the parameters, which take on random initial values.\n", "\n", "**Note:** Importantly, we will fix the initial random values with the command `torch.random.manual_seed(42)`. This is only to make YOUR results comparable to OUR results. Different initial random values may produce slightly different behavior (e.g., rate of convergence, final results, etc.). However, when trained for many iterations, this should not really matter."]}, {"cell_type": "code", "execution_count": null, "id": "15c47628", "metadata": {"colab": {"base_uri": "https://localhost:8080/"}, "id": "Zc_UW4dSOCJ0", "outputId": "32dc5f8d-a657-4a09-ac77-a5662df9f002", "tags": ["learner", "py", "lect_07", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.7-runcell01\n", "\n", "#now let's do this with a neural network\n", "\n", "torch.random.manual_seed(42) # Set the random seed\n", "\n", "class simple_MLP(torch.nn.Module):\n", "    def __init__(self):\n", "        super().__init__()\n", "        self.fc1 = torch.nn.Linear(2,1)\n", "        self.output = torch.nn.Sigmoid()\n", "\n", "    def forward(self, x):\n", "        x = self.fc1(x)\n", "        x = self.output(x)\n", "        return x\n", "    \n", "\n", "simple_model = simple_MLP()\n", "print(simple_model.named_parameters('fcl'))\n", "print('----------')\n", "print(simple_model.state_dict())"]}, {"cell_type": "markdown", "id": "96bc242d", "metadata": {"tags": ["learner", "md", "lect_07"]}, "source": ["<h3>About the model</h3>\n", "\n", "This code defines a simple multilayer perceptron (MLP) neural network using the PyTorch library. The MLP consists of one input layer, one hidden layer with one neuron, and one output layer with one neuron.\n", "\n", "The `fc1` attribute (the hidden layer) is defined as a linear layer that maps 2 input features to 1 output feature, and the output attribute is defined as a sigmoid activation function.\n", "\n", "In the `forward` method of the `simple_MLP` class, the input x is passed through the `fc1` linear layer and then through the output sigmoid activation function.\n", "\n", "The `simple_model` object is then instantiated as an instance of the `simple_MLP class`.\n", "\n", "The `state_dict` method is called on the model instance to print the state dictionary of the model, which contains the parameter names and their corresponding tensor values. The state dictionary can be used to save and load the parameters of the model.\n", "\n", "This model is just the same as before, but now we have put it into pytorch, which enables us to use all sorts of bells and whistles going forward within the pytorch framework, which has a whole slew of deep learning functionalities. \n"]}, {"cell_type": "markdown", "id": "819468f3", "metadata": {"tags": ["learner", "md", "lect_07"]}, "source": ["<h3>Training the model</h3>\n", "\n", "The first thing we will do with this network is \"train it\". This just means fitting it to a particular dataset, except that now we fit with a simplified minimizer that actually doesn't compute Hessians. This makes it a lot faster. It also does less sophisticated step calculations, which also makes it more robust. \n", "\n", "The last thing is that we don't necessarily tell the minimizer when to stop, we just let it go for a while and then we take the best fitted value. I know this all seems a little weird compared to fitting, but there is nothing special to this. "]}, {"cell_type": "code", "execution_count": null, "id": "f9ad0b0e", "metadata": {"colab": {"base_uri": "https://localhost:8080/", "height": 495}, "id": "ZinJyWitO_PL", "outputId": "3547379d-2666-4005-cd36-bc6bba74ed8f", "scrolled": true, "tags": ["learner", "py", "lect_07", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.7-runcell02\n", "\n", "\n", "#------------------------\n", "#Define lists and grids (this was done previously in this notebook)\n", "#create grid\n", "x1_list = np.linspace(-20., 20., 100)\n", "x2_list = np.linspace(-20., 20., 100)\n", "x1_grid, x2_grid  = np.meshgrid(x1_list, x2_list)\n", "#------------------------\n", "\n", "\n", "#Create a loss and minimizer \n", "simple_criterion = torch.nn.BCELoss() #BCE => Binary Cross Entropy\n", "simple_optimizer = torch.optim.Adam(simple_model.parameters(), lr=0.01) \n", "simple_history = {'loss':[]}\n", "\n", "#pytorch\n", "x_list_np = np.transpose(np.vstack((x_list[0],x_list[1])))\n", "y_list_np = (np.array(y_list)+1)/2\n", "x_torch=torch.from_numpy(x_list_np).float()\n", "y_torch=torch.from_numpy(y_list_np.reshape(100,1)).float()\n", "\n", "for epoch in range(100):\n", "    simple_optimizer.zero_grad()\n", "    outputs = simple_model(x_torch)\n", "    loss = simple_criterion(outputs, y_torch)\n", "    loss.backward()\n", "    simple_optimizer.step()    \n", "    # add loss statistics\n", "    current_loss = loss.item()\n", "    print('[%d] loss: %.5f  ' % (epoch + 1,  current_loss))\n", "    simple_history['loss'].append(current_loss)\n", "            \n", "print('Finished Training')\n", "torch.save(simple_model.state_dict(), 'data/L13/simple_model.pt')\n", "print(simple_model.state_dict())"]}, {"cell_type": "code", "execution_count": null, "id": "d545c7c0", "metadata": {"colab": {"base_uri": "https://localhost:8080/", "height": 495}, "id": "ZinJyWitO_PL", "outputId": "3547379d-2666-4005-cd36-bc6bba74ed8f", "scrolled": false, "tags": ["learner", "py", "lect_07", "learner_chopped"]}, "outputs": [], "source": ["#>>>RUN: L13.7-runcell03\n", "\n", "#Now plot it\n", "x_grid_np = np.transpose(np.vstack((x1_grid.flatten(),x2_grid.flatten())))\n", "x_torch=torch.from_numpy(x_grid_np).float()\n", "h_grid = simple_model(x_torch)\n", "h_grid=h_grid.detach().numpy()\n", "h_grid=h_grid.reshape(100,100)\n", "print(h_grid.shape)\n", "plt.pcolormesh(x1_grid, x2_grid, h_grid, cmap = 'bwr', shading='auto')\n", "plt.colorbar()\n", "\n", "plt.plot(x1_pos, x2_pos, 'k+')\n", "plt.plot(x1_neg, x2_neg, 'k.')\n", "\n", "plt.show()\n", "\n", "print(simple_model.state_dict())"]}, {"cell_type": "markdown", "id": "aa905a58", "metadata": {"tags": ["learner", "md", "lect_07"]}, "source": ["Let's discuss what this code is doing.\n", "\n", "First, a loss function (`BCELoss`) and an optimizer (`Adam`) are created. The optimizer is configured to update the parameters of the model (`simple_model.parameters()`) with a learning rate of `lr=0.01`. A dictionary is also created to store the training history.\n", "\n", "The learning rate in a neural network is a hyperparameter that controls the step size of the optimization algorithm during the training process. Specifically, it determines how much to adjust the weights of the neural network with respect to the gradient of the loss function.\n", "\n", "Next, the input and target data are prepared for training. The `x_list` and `y_list` data are converted from NumPy arrays to PyTorch tensors, and the `y_list` data is transformed to be in the range `[0,1]` instead of `[-1,1]`.\n", "\n", "The training loop is then executed for 100 epochs. In each epoch, the optimizer gradients are set to zero (`simple_optimizer.zero_grad()`), and the model's predictions (`outputs`) are computed using the current input (`x_torch`). The loss is then computed between the predictions and the targets (`simple_criterion(outputs, y_torch)`), and the gradients of the loss with respect to the model parameters are computed (`loss.backward()`). Finally, the optimizer is used to update the model parameters (`simple_optimizer.step()`).\n", "\n", "During training, the current loss is printed for each epoch, and the current loss is appended to the `simple_history` dictionary. After training is complete, the state dictionary of the trained model is saved to a file (`simple_model.pt`) using the PyTorch save function. The state dictionary contains the values of the model parameters after training. The 3 parameters (2 slopes in the `weight` tensor and one intercept in the `bias` tensor) correspond to the same 3 parameters (`w1`, `w2`, and `b`) we were fitting earlier.\n", "\n", "Notice how quickly this code ran. For fun, try running the code again (it will start from the last set of parameters) or reinitialize using `L13.7-runcell01` and increasing the number of epochs, and see if the loss continues to decrease. The next few questions will focus on this type of exploration."]}, {"cell_type": "markdown", "id": "db77d12a", "metadata": {"tags": ["learner", "md", "lect_07"]}, "source": ["That's it! We have now used pytorch to classify the datapoints as red or blue. Simple datapoints with only two possible values are not a particularly interesting thing to classify. We can also classify much more complicated things and use this technique for all sorts of interesting physics problems. "]}, {"cell_type": "markdown", "id": "65162d2b", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["<a name='exercises_13_7'></a>     \n", "\n", "| [Top](#section_13_0) | [Restart Section](#section_13_7) |"]}, {"cell_type": "markdown", "id": "8552a25d", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.7.1</span>\n", "\n", "Run code cell `L13.7-runcell02` until the optimizer reaches a minimum loss value which no longer changes. There are two ways to do this. One option is to simply rerun `L13.7-runcell02` over and over, since it will start each time with the final parameters found in the previous run. Another option is to first run code cell `L13.7-runcell01` to reinitialize the parameters, and then increase the number of epochs when running `L13.7-runcell02`. See the solution for a more detailed discussion of these two options. Rerunning `L13.7-runcell02` is a fine approach.\n", "\n", "When the loss is minimized, what are the best fit slope parameters of the neural network? How do they compare to the output of our minimizer in the previous section? Print the values from `simple_model.fc1.weight`, using the code cell provided.\n", "\n", "Enter your answer as a list of two numbers with precision 1e-2: `[w1, w2]`"]}, {"cell_type": "code", "execution_count": null, "id": "62081ce6", "metadata": {"tags": ["draft", "py", "learner_chopped"]}, "outputs": [], "source": ["#>>>EXERCISE: L13.7.1\n", "# Use this cell for drafting your solution (if desired),\n", "# then enter your solution in the interactive problem online to be graded.\n", "\n", "print(\"w1:\", #YOUR CODE HERE)\n", "print(\"w2:\", #YOUR CODE HERE)"]}, {"cell_type": "markdown", "id": "828c6cb8", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.7.2</span>\n", "\n", "Vary the learning rate, which is the parameter `lr` in the function `torch.optim.Adam(simple_model.parameters(), lr=0.01)`. In order to see what happens when you don't start with the previously found best-fit (or in some cases very bad fit) values of the parameters, run code `cell L13.7-runcell01` to first reinitialize the parameters. Do this each time you change `lr` in code cell `L13.7-runcell02`. \n", "\n", "What happens when the learning rate is very large (e.g., approaching 1)?\n", "\n", "\n", "A) The network does not converge to a solution.\\\n", "B) The network converges more quickly.\\\n", "C) The learning rate does not affect the rate of convergence.\n", "\n"]}, {"cell_type": "markdown", "id": "47606638", "metadata": {"tags": ["learner", "md", "learner_chopped"]}, "source": ["### <span style=\"border:3px; border-style:solid; padding: 0.15em; border-color: #90409C; color: #90409C;\">Ex-13.7.3</span>\n", "\n", "Try changing the number of epochs and the number of times that you run the cell to see what effect this has on how quickly the network minimizes the loss. When trying different number of epochs, re-initialize the model by first running code cell `L13.7-runcell01`.\n", "\n", "What did you discover after varying the different ways that the neural network is run? Choose the best option below:\n", "\n", "A) The network always minimizes the loss on the first run.\\\n", "B) The network is not sensitive to the way in which it is run.\\\n", "C) The network will always find a minimum if the epoch is long enough.\\\n", "D) The network will always find a minimum if it is reinitialized.\\\n", "E) None of the above."]}], "metadata": {"celltoolbar": "Tags", "kernelspec": {"display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.9"}}, "nbformat": 4, "nbformat_minor": 5}