[YCMI_CBDS_Summer_Course]_Summative_Activity_and_Applying_Methods_Learned.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "[YCMI_CBDS Summer Course] Summative Activity and Applying Methods Learned",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o6k1iS6FNJq9",
        "colab_type": "text"
      },
      "source": [
        "For this exercise, we'll need several things from `sklearn`:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GEUNfyJoNKtq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn import decomposition\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "from sklearn.model_selection import cross_val_score\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.metrics import confusion_matrix"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t_nXgPU4rbK8",
        "colab_type": "text"
      },
      "source": [
        "We'll also need three other modules: `pandas`, `plotnine`, `sqlite3`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zwtfi4AQr4jO",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lFUYGCwrr6PA",
        "colab_type": "text"
      },
      "source": [
        "Run the following cell.\n",
        "\n",
        "It copies a database file called `friday.db` from the web into the file system hosting this notebook. This data is modified from that used in [Smith et al, 1988](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2245318/)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pw3nPPSjPbG-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!wget https://github.com/YCMI/summer-course-2020/raw/master/friday.db"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pT5-G-3MP0xg",
        "colab_type": "text"
      },
      "source": [
        "The database contains three tables, with columns as below:\n",
        "\n",
        "**demographics**: patient_id, dob, gender, children, state\n",
        "\n",
        "**patienthistory**: patient_id, diabetespedigree, diagnoses\n",
        "\n",
        "**testresults**: patient_id, diastolic, systolic, bicepskinthickness, seruminsulin, glucose, bmi\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FjsR2GJUVoPQ",
        "colab_type": "text"
      },
      "source": [
        "Connect to the `friday.db` database and look at a sample of each table to see how the data is structured."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QxMKHFLTV46N",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qVBip2kOV5Ch",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VPG9Yf2nV5JB",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "st-kyRTOctp8",
        "colab_type": "text"
      },
      "source": [
        "Make a bar chart comparing the number of male and female patients per state in our dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IL8gJlQJc_1z",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xGBU8leSV5k7",
        "colab_type": "text"
      },
      "source": [
        "We wish to test the hypotheses that for women, age, number of children, diabetespedigree (family history), blood pressure, serum insulin, glucose levels, and BMI are predictive of whether or not they have diabetes.\n",
        "\n",
        "Use SQL to get a DataFrame of just the females with information on their: dob, number of children, diabetespedigree, diagnoses, diastolic, systolic, bicepskinthickness, seruminsulin, glucose, and bmi."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iiMt499fXiFS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GMPYDdgsZIh5",
        "colab_type": "text"
      },
      "source": [
        "Let's clean up this data a bit."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XlGH3tcLXidq",
        "colab_type": "text"
      },
      "source": [
        "Create a new column `age` and estimate it from the date of birth by assuming that every year has exactly 365.25 days.\n",
        "\n",
        "Hint: You'll want to subtract dates and divide by `pd.to_timedelta('365.25 days')`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fuKR35noRQyk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NV7IbYd2Yyg8",
        "colab_type": "text"
      },
      "source": [
        "Derive a new column `has_diabetes` (with `True` or `False` values) based on the `diagnoses` column.\n",
        "\n",
        "Hint: you can do this in one line."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o4ivEraxZBbz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8nHLwKngZZFD",
        "colab_type": "text"
      },
      "source": [
        "The `bicepskinthickness` column has values with two different units -- cm and mm. Standardize this into a new column `bicepskinthickness_cm` measured in cm without units.\n",
        "\n",
        "Hints: you could do this with the `re` module, but you don't need it. You'll also want a for loop, and an if statement to build the new column one entry at a time, although you can. You can convert a string `my_numstr` to a number using e.g. `my_num = float(my_numstr)`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_tQJ3RQvaVz5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "03rM-F6ga4jG",
        "colab_type": "text"
      },
      "source": [
        "Are any of the columns missing data? Are they missing a little data or a lot of data? Should we keep the column(s) with missing data?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f0L_72ocbo9u",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Tn1r8YysGU0u",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ukbzmzhweCqp",
        "colab_type": "text"
      },
      "source": [
        "While we're at it, drop the redundant columns: `dob`, `diagnoses`, and `bicepskinthickness`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7X9_ohHPGUJG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-gAOPLdseZs1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3ADk-BZhcIPI",
        "colab_type": "text"
      },
      "source": [
        "Standardize your numeric data (should be everything except the `has_diabetes` column) to have a mean of 0 and a standard deviation of 1.\n",
        "\n",
        "Hint: You can do this with simple arithmetic operations as we did on Wednesday or use a `StandardScaler` as you learned yesterday. The arithmetic solution can be done in two lines."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AXxXgno-dY5P",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "aDBaU-E_fUZO"
      },
      "source": [
        "### PCA\n",
        "\n",
        "A good starting place when looking at a new dataset is running PCA and visualizing at a couple of the first principle components.\n",
        "\n",
        "Remember, PCA is simply a rotation of our data that allows us to visualize the axis with the greatest variation. This can elucidate some separation in our data if it exists on those axis, but it may not. \n",
        "\n",
        "**Adapt yesterday's PCA code to work with today's data and visualize the data in PCA space. Color-code by `has_diabetes`.** (For concreteness, plot all possible pairs from the first three principal components... that is, show three graphs PC1-PC2, PC1-PC3, and PC2-PC3.)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uVilLlOKfuzy",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d5y9qxagJMQ-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "lCMakrs2JM4E",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nNGlSBl8iPe0",
        "colab_type": "text"
      },
      "source": [
        "What do you make of the graphs?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ac04tEm2gC2j",
        "colab_type": "text"
      },
      "source": [
        "# Training and testing sets"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0K16fV3OfUBf",
        "colab_type": "text"
      },
      "source": [
        "Split your data into training and test sets with 20% of the data being used for the test set. To ensure we all work with the same random split, let's specify a `random_state`, namely `random_state=42`.\n",
        "\n",
        "If your predictor variables are in `X` and your outcome vector is in `y`, this can be done using:\n",
        "\n",
        "```\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "```\n",
        "\n",
        "Hint: how can you split your DataFrame into a predictor matrix `X` and outcome vector `y`?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W0TCdY1rhG51",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8-bzGUNxlZHw",
        "colab_type": "text"
      },
      "source": [
        "# Cross-validation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X6SSk_i6lbP2",
        "colab_type": "text"
      },
      "source": [
        "Using 5-fold cross-validation (`cross_val_score`) on your *training* data only, plot the relationship between the number of estimators (between 10 and 100 counting by 10s) with a max depth of 4 and the average regression score for a corresponding `RandomForest`.\n",
        "\n",
        "Hint: you did cross-validation yesterday in the disease classification exercises with logistic regression."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Yb9OntH7mX6P",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2MFjeFPZmYMB",
        "colab_type": "text"
      },
      "source": [
        "# Testing\n",
        "\n",
        "Using the number of estimators that gave the best score in the validation phase, train a `RandomForest` on your entire training set, and `.score` its performance with the test set."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kBa7Ec4vmtOk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C3kpmt6-mtZ4",
        "colab_type": "text"
      },
      "source": [
        "Print the confusion matrix. What does it mean?\n",
        "\n",
        "Hint: looking at `y_test.value_counts()` might help with the interpretation."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4InslPDzmvRN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CKtEYXjNmvqo",
        "colab_type": "text"
      },
      "source": [
        "Finally, make a horizontal bar graph showing the fitted `RandomForest`'s `feature_importances_`, sorted in order of increasing importance.\n",
        "\n",
        "Hints: you used `feature_importances_` in the Warfarin exercises. You'll need to specify a `stat` to `geom_bar` and you'll need to modify the coordinate system to make the graphs horizontal."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XQS5qd8tTC31",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H6HG-tRfnuGo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}