NEU 171L Stats intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [NEU 171L] Introduction statistical analysis with Python\n",
    "\n",
    "*Estimated Time: 30 minutes*\n",
    "\n",
    "### Table of Contents:\n",
    "\n",
    "\n",
    "**Part I: Some Statistics**\n",
    "1. [The Mean](#section_mean)\n",
    "2. [The Distribution](#section_distr)\n",
    "\n",
    "**Part II: Statistical Analysis**\n",
    "1. [The Bootstrap](#section_bootstrap)\n",
    "2. [Confidence Interval](#section_ci)\n",
    "3. [The P-Value](#section_pval)\n",
    "\n",
    "\n",
    "# The Jupyter  Notebook <a id='section_jupyter'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "    \n",
    "Just a few reminders about how to use Jupyter notebooks before we get into statistical analysis. It might be helpful to look back at the introductory notebook to review some basic commands.\n",
    "\n",
    "A notebook is composed of rectangular sections called `cells`. There are 2 kinds of cells: markdown and code. A `markdown cell`, such as this one, contains text. A `code cell` contains code in Python, a programming language that we will be using for the remainder of this module. You can select any cell by clicking it once. After a cell is selected, you can navigate the notebook using the up and down arrow keys.\n",
    "\n",
    "To run a code cell once it's been selected:\n",
    "\n",
    "<ul>\n",
    "\n",
    "  <li>press Shift-Enter, or</li>\n",
    "  <li>click the Run button in the toolbar at the top of the screen.</li>\n",
    "\n",
    "</ul>  \n",
    "\n",
    "If a code cell is running, you will see an asterisk (\\*) appear in the square brackets to the left of the cell. Once the cell has finished running, a number will replace the asterisk and any output from the code will appear under the cell.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Saving and Loading\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "    \n",
    "Your notebook can record all of your text and code edits, as well as any graphs you generate or calculations you make. You can save the notebook in its current state by clicking Control-S, clicking the floppy disc icon in the toolbar at the top of the page, or by going to the File menu and selecting \"Save and Checkpoint\".\n",
    "\n",
    "The next time you open the notebook, it will look the same as when you last saved it.\n",
    "\n",
    "<br><br>\n",
    "<i><b>Note:</b></i> after loading a notebook you will see all the outputs (graphs, computations, etc) from your last session, but you won't be able to use any variables you assigned or functions you defined. You can get the functions and variables back by re-running the cells where they were defined- the easiest way is to highlight the cell where you left off work, then go to the Cell menu at the top of the screen and click \"Run all above\". You can also use this menu to run all cells in the notebook by clicking \"Run all\".</br></br>\n",
    "</div>\n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "\n",
    "Before we begin, we'll need a few extra tools to conduct our analysis. Run the next cell to load some code packages that we'll use later.\n",
    "\n",
    "\n",
    "<br><br>\n",
    "<i><b>Note: </b></i>this cell MUST be run in order for most of the rest of the notebook to work.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dependencies: THIS CELL MUST BE RUN\n",
    "\n",
    "import numpy as np\n",
    "import math\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('fivethirtyeight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 1: Statistics + DataFrames <a id='section 2'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "    \n",
    "We can also apply our mathematical and statistical background along with our coding experience to find out basic descriptive stats regarding our data (i.e. mean, median, standard deviation, etc.). In this section, we will use DataFrames and functions to calculate these stats.\n",
    "    \n",
    "To start with, let's work with the data we used in the Intro notebook last time. This is data from the Allen Brain Connectivity atlas, which we will be using later in the course.    \n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell to load in the data\n",
    "primary_auditory = pd.read_csv('data/primary_auditory_area.csv')\n",
    "primary_auditory.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "Now we will make a series called <code>proj_density</code> again by selecting the <code>projection_density</code> from the <code>primary_auditory</code> DataFrame. This is exactly what we did in the Intro notebook.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Run this cell to select data from the Projection Density column\n",
    "proj_density = primary_auditory['projection_density']\n",
    "proj_density"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Part 1.1: The Mean<a id='section_mean'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's first find the **mean** projection densities at the primary auditory area. The formula for the mean is:\n",
    "\n",
    "$$M = \\frac{\\sum_{i=1}^N x_i}{N}$$\n",
    "\n",
    "\n",
    "In other words, we add up all the values in the projection density array, then divide the sum by the total number of values in the array. For example, in the context of this problem, we can assign these variables as follows:\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "* $M$ = Average projection density in the Primary Auditory Area\n",
    "* $x_i$ = Data: i.e. each projection density value in the array\n",
    "* $N$ = Number of values Total Number of projection densities in the array\n",
    "\n",
    "To do this, we can use **proj_density** which we defined earlier and use `.values` to get the list of values. Then, we can sum up the values using the `sum()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "sum_values = sum(proj_density.values)\n",
    "sum_values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " \n",
    "Next, we can use the `len()` function to find the number of values in the list of projection densities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "num_values = len(proj_density.values)\n",
    "num_values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "    \n",
    "Finally, we can divide the summed up values from `sum_values` with the number of values from `num_values` to get the mean."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "mean = sum_values / num_values\n",
    "mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "**Practice:** Now, try and find the `mean projection volumes` of the primary auditory area. First, assign `proj_volumes` to a list of projection volumes using `.values`. Then, assign `sum_volumes` to the sum of these values using the `sum()` function. Next, assign `num_values` to number of values using the `len()` function. Finally, assign mean_volume to `sum_volumes` divided by `num_volumes`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "proj_volumes = primary_auditory['volume'].values\n",
    "sum_volumes = sum(...)\n",
    "num_volumes = len(...)\n",
    "mean_volume = \n",
    "mean_volume"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2. The Distribution<a id='section_distr'></a>\n",
    "\n",
    "Lastly, we can visualize the distribution of the projection density values. A distribution shows us how the data or values are distributed by drawing a nice smooth curve. There are three types of distributions: \n",
    "* A **right skewed** distribution has the majority of values concentrated in the left with a tail off to the right. \n",
    "* A **left skewed** distribution has the majority of values concentrated in the right with a tail off to the left. \n",
    "* A **normal (symmetric)** distribution has values concentrated at the center. Also known as the \"bell curve.\"\n",
    "\n",
    "Notice in the graph below, taken from Lumen Learning that shows the how values can be distributed.\n",
    "\n",
    "<img src='./images/dist.png' width=\"700px\"/>\n",
    "\n",
    "Run the cell below to output a visual distribution of projection densities in the primary auditory area."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "plt.xlim(0, 0.21)\n",
    "sns.distplot(proj_density, hist=False);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can clearly see that this is a distribution skewed to the right because the projection density values are highly concentrated in the left and has a tail going off to the right. One example where you can see this is in U.S. Household Incomes. You can learn more about distributions [here](https://www.statisticshowto.datasciencecentral.com/probability-and-statistics/skewed-distribution/).\n",
    "\n",
    "Recall the stats we computed for the mean, median, and standard deviation:\n",
    "\n",
    "* **Mean:** 0.0055\n",
    "* **Median:** 3.6405e-05\n",
    "* **Standard Deviation:** 2.7076e-16\n",
    "\n",
    "Compare these values to the figure above and the visual distribution. Notice that the data we have contains many zeros as well as high values less than 0.025, which in turn affect our outcome when caculating the stats."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 2: Statistical Analysis "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Part 2.1: The Bootstrap<a id='section_bootstrap'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "    \n",
    "### Background"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "    \n",
    "Before we dive into what bootstrapping is, we're going to define some terms that we will be using throughout the rest of the intro lab and the lab you will perform.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "        \n",
    "- **Sample**: A sample is a set of data collected from a population by a defined procedure. In the next cells, you will be able to see a sample of a population compared with the original population that we obtain by using the `.sample` function.\n",
    "\n",
    "   - *The type of sample that data scientists use often is a simple   random sample, where each member of the sample has an equal chance of getting chosen.*\n",
    "\n",
    "\n",
    "- **Statistic**: A single measure of some attribute of a sample. In the next cells, you will be able to see how we obtain the mean of the population and the sample using the `.mean()` function.\n",
    " \n",
    "   - *An example of a statistic that is used many times is the mean.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "        \n",
    "Run the next couple of cells. In these cells, we will show you how a sample is done and the statistic that is drawn from these samples. You do not need to understand the code in this section, you are only responsible for understanding the concept and analysis behind Bootstrapping.\n",
    "\n",
    "We will be using the grade distribution for the class Math 110 in Fall 2018. That semester, 402 people received a grade, as shown in the x-axis. The grades are shown by the amount of GPA points awarded per grade (an A and A+ being a 4.0 to an F being 0.0).\n",
    "\n",
    "We will also find a statistic of this data, the mean.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "math_110 = pd.read_csv('../data/math_110_grade_distribution.csv')\n",
    "math_110.hist();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "math_110_mean = math_110.mean()[0]\n",
    "math_110_mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "       \n",
    "In the next cell, we will take a sample of the students who took the class. Our sample will be of 100 people. We will then obtain the mean and compare it with that of the population.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "sample_1 = math_110.sample(100)\n",
    "sample_1.hist();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample_1.mean()[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "        \n",
    "How do the distributions from the population and the sample differ? How does the statistic differ? \n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*If you run the two cells above again, you most likely will get a different distribution and a different statistic. This is because the statistic is only based off the sample and not the entire population.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "    \n",
    "### Bootstrapping"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "       \n",
    "A data scientist is using the data in a random sample to estimate an unknown parameter. She uses the sample to calculate the value of a statistic that she will use as her estimate.\n",
    "\n",
    "Once she has calculated the observed value of her statistic, she could just present it as her estimate and go on her merry way. But she’s a data scientist. She knows that her random sample is just one of numerous possible random samples, and thus her estimate is just one of numerous plausible estimates.\n",
    "\n",
    "By how much could those estimates vary? To answer this, it appears as though she needs to draw another sample from the population, and compute a new estimate based on the new sample. But she doesn’t have the resources to go back to the population and draw another sample.\n",
    "\n",
    "It looks as though the data scientist is stuck.\n",
    "\n",
    "Fortunately, a brilliant idea called the bootstrap can help her out. Since it is not feasible to generate new samples from the population, the bootstrap generates new random samples by a method called resampling: the new samples are drawn at random from the original sample.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/bootstrap.png\" width=\"5000px\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "       \n",
    "In our example given above, the grades for all 402 students enrolled in Math 110 in Fall 2018 is our population. Bootstrapping is used when we don't have the overall population and instead we only have the sample drawn. \n",
    "\n",
    "Pretending we don't have the population, in the next cells, we will use bootstrap on a sample of 100 students provided by us to estimate the population statistic (which we have).\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "math_110_sample = pd.read_csv('../data/math_110_sample.csv')\n",
    "math_110_sample.hist();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "sample_mean = math_110_sample.mean()[0]\n",
    "sample_mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Our overall goal is to use the resamples to calculate an interval in which the statistic (the mean) of the population is likely to be. To do this, we need to find the statistic for all the resamples, which we will do in the following cells.\n",
    "\n",
    "Before we start, there are three keys to resampling that you should know about when performing a bootstrap:\n",
    "- Draw at random from the original sample \n",
    "  - This just means that we are getting a random sample of the sample and that every individual has an equal possibility of being chosen. \n",
    "- Draw with replacement (replace = True)\n",
    "  - Pretend you are drawing a card from the deck of cards. You draw the first card. If you were to draw without replacement, then you would have 51 cards to draw from for the next turn. However, if you draw with replacement, you draw from the 52 cards, meaning that you can draw the same card that you drew the first time again.\n",
    "- Draw as many values as the original sample contained \n",
    "  - For every resample, you will draw as many cards as the first sample (in this case 100).\n",
    "\n",
    "We will apply these rules when doing bootstrapping in the cells below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In this cell, we will use something called a **for loop** to easily \n",
    "# get the resamples and find the mean of each one. We want a large number\n",
    "# of resamples, so we set it to 5000.\n",
    "means = []\n",
    "for i in np.arange(5000):\n",
    "    resampled = math_110_sample.sample(n = 100, replace = True)\n",
    "# notice that we used n = 100 because the sample had 100 students and that\n",
    "# we set replace = True\n",
    "    mean = resampled.mean()\n",
    "    means = np.append(means, mean)\n",
    "# We obtain the means of the resamples in a list, shown below\n",
    "means"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "        \n",
    "Run the cell below to make a DataFrame with these means. \n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "sample_table = pd.DataFrame(data={'Math 110 Averages': means})\n",
    "sample_table.hist();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Part 2.2: Confidence Interval<a id='section_ci'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "    \n",
    "<p>In the case of our lab, after bootstrapping is done, we want to find out the confidence interval. Bootstrapping produces an interval of estimates, to account for chance variability in the random sample. However, it happens that out of 100 resamples, around 5 of them do not include the statistic of the population. Because of this, we are only 95% confident that the bootstrap contains our population parameter.</p> \n",
    "\n",
    "<p>To account for this, we create a <code>95% Confidence Interval</code> of the bootstrap. We do this by using the <code>.quantile</code> function, which takes in the left and right parameters. These, we choose to be <i>0.025</i> and <i>0.975</i> so that we may be the medium 95% of the bootstrap values.</p>\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "left_95 = 0.025\n",
    "right_95 = 0.975\n",
    "percentile = list(sample_table.quantile([left_95, right_95])['Math 110 Averages'])\n",
    "percentile"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "       \n",
    "Run the next cell to see the 95% Confidence Interval overlayed onto the bootstrap. \n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "sample_table.hist()\n",
    "plt.hlines(y=0, xmin=percentile[0], xmax=percentile[1], linewidth=10, color = 'y');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Part 2.3: The P-value<a id='section_pval'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Background"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "\n",
    "Before diving into the p-value, we need to go over some basic necessary background, namely the<code>null hypothesis</code> and the<code>alternative hypothesis</code>.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "    \n",
    "All statistical tests attempt to choose between two views of the world. Specifically, the choice is between two views about how the data were generated. These two views are called hypotheses.\n",
    "\n",
    "<code>The null hypothesis</code>: This is a clearly defined model about chances. It says that the data were generated at random under clearly specified assumptions about the randomness. The word “null” reinforces the idea that if the data look different from what the null hypothesis predicts, the difference is due to nothing but chance.\n",
    "\n",
    "<code>The alternative hypothesis</code>: This says that some reason other than chance made the data differ from the predictions of the model in the null hypothesis.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "    \n",
    "Let's say that in Fall 2018, the head of the math department wanted to sample the scores of 100 students from one of the math classes on campus. He did this, but forgot to write down what math class it was. The mean that he obtained from these 100 students was 3.408, which we will assign to <code>math_mean</code>. Did he take the sample from Math 110, or was it from another class?\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "math_mean = 3.408"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "    \n",
    "The first step is to come up with a <code>null hypothesis</code> and a <code>alternative hypothesis</code>.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "\n",
    "<p><b>null hypothesis</b>: math_mean comes from the same sample distribution as that of Fall 2018 Math 110. Any variation is purely due to chance.</p>\n",
    "\n",
    "<p><b>alternative hypothesis</b>: math_mean does not come from the same sample distribution as that of Fall 2018 Math 110.</p> \n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "    \n",
    "We can find out what hypothesis is correct using something called a p-value. The P-value of a test is the chance, based on the model in the null hypothesis, that the test statistic will be equal to the observed value in the sample or even further in the direction that supports the alternative.\n",
    "\n",
    "If a P-value is small, that means the tail beyond the observed statistic is small and so the observed statistic is far away from what the null predicts. This implies that the data support the alternative hypothesis better than they support the null. By convention, we say that anything less than 5% is \"statistically significant\".\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "sample_table.hist(bins = 20)\n",
    "plt.axvline(math_mean, color = 'k');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "p_value = np.average(sample_table > math_mean)\n",
    "p_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "    \n",
    "Our P-value ends up being 0.0. Because out P-value is less than 0.05, we reject the <code>null hypothesis</code>. This means that math_mean does not come from the same sample distribution as that of Fall 2018 Math 110 and <code>math_mean</code> was too large to reflect chance variation alone. We, however do not accept the <code>alternative hypothesis</code> because it could be true, but we can never totally be sure. We can just show that the <code>null hypothesis</code> is highly improvable given our data, which is why we reject it. \n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "    \n",
    "After doing some organizing in her office, the head of the math department found out what course had been sampled, it was actually Math 125A! We were right, the mean of the sampled class had not come from the same distribution as that of our own, Math 110!\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"border-left: 3px solid #003262; padding: 1px; padding-left: 10px; background: #ffffff; \">\n",
    "    \n",
    "It is important that you review these concepts before your lab. Know the basics of operating python and how to utilize some of the functions like <code>.sort_values</code>, <code>len()</code>, <code>sum()</code>, <code>.values</code> and <code>[...]</code>. Don't worry too much about the code, but do know conceptually what a bootstrap does, what a 95% Confidence Interval is, and what the significance of a P-value is. \n",
    "</div>\n",
    "\n",
    "<i>Notebook Developed by: Elias Saravia & Daniel Lopez</i>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}