15_bikeshare_exercise.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise with Capital Bikeshare data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "- Capital Bikeshare dataset from Kaggle: [data](https://github.com/JamesByers/GA-SEA-DAT1/blob/master/data/bikeshare.csv), [data dictionary](https://www.kaggle.com/c/bike-sharing-demand/data)\n",
    "- Each observation represents the bikeshare rentals initiated during a given hour of a given day"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.cross_validation import cross_val_score\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.tree import DecisionTreeRegressor, export_graphviz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# read the data and set \"datetime\" as the index\n",
    "url = '../data/bikeshare.csv'\n",
    "bikes = pd.read_csv(url, index_col='datetime', parse_dates=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# the feature name \"count\"is also the name of a method, so to avoid confusion it's\n",
    "# best to rename that column\n",
    "bikes.rename(columns={'count':'total'}, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# create \"hour\" as its own feature\n",
    "bikes['hour'] = bikes.index.hour"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>season</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>weather</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>humidity</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>casual</th>\n",
       "      <th>registered</th>\n",
       "      <th>total</th>\n",
       "      <th>hour</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>datetime</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-01-01 00:00:00</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>9.84</td>\n",
       "      <td>14.395</td>\n",
       "      <td>81</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>13</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 01:00:00</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>9.02</td>\n",
       "      <td>13.635</td>\n",
       "      <td>80</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>32</td>\n",
       "      <td>40</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 02:00:00</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>9.02</td>\n",
       "      <td>13.635</td>\n",
       "      <td>80</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>27</td>\n",
       "      <td>32</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 03:00:00</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>9.84</td>\n",
       "      <td>14.395</td>\n",
       "      <td>75</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>13</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 04:00:00</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>9.84</td>\n",
       "      <td>14.395</td>\n",
       "      <td>75</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     season  holiday  workingday  weather  temp   atemp  \\\n",
       "datetime                                                                  \n",
       "2011-01-01 00:00:00       1        0           0        1  9.84  14.395   \n",
       "2011-01-01 01:00:00       1        0           0        1  9.02  13.635   \n",
       "2011-01-01 02:00:00       1        0           0        1  9.02  13.635   \n",
       "2011-01-01 03:00:00       1        0           0        1  9.84  14.395   \n",
       "2011-01-01 04:00:00       1        0           0        1  9.84  14.395   \n",
       "\n",
       "                     humidity  windspeed  casual  registered  total  hour  \n",
       "datetime                                                                   \n",
       "2011-01-01 00:00:00        81          0       3          13     16     0  \n",
       "2011-01-01 01:00:00        80          0       8          32     40     1  \n",
       "2011-01-01 02:00:00        80          0       5          27     32     2  \n",
       "2011-01-01 03:00:00        75          0       3          10     13     3  \n",
       "2011-01-01 04:00:00        75          0       0           1      1     4  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bikes.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>season</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>weather</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>humidity</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>casual</th>\n",
       "      <th>registered</th>\n",
       "      <th>total</th>\n",
       "      <th>hour</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>datetime</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2012-12-19 19:00:00</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>15.58</td>\n",
       "      <td>19.695</td>\n",
       "      <td>50</td>\n",
       "      <td>26.0027</td>\n",
       "      <td>7</td>\n",
       "      <td>329</td>\n",
       "      <td>336</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-12-19 20:00:00</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>14.76</td>\n",
       "      <td>17.425</td>\n",
       "      <td>57</td>\n",
       "      <td>15.0013</td>\n",
       "      <td>10</td>\n",
       "      <td>231</td>\n",
       "      <td>241</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-12-19 21:00:00</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>13.94</td>\n",
       "      <td>15.910</td>\n",
       "      <td>61</td>\n",
       "      <td>15.0013</td>\n",
       "      <td>4</td>\n",
       "      <td>164</td>\n",
       "      <td>168</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-12-19 22:00:00</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>13.94</td>\n",
       "      <td>17.425</td>\n",
       "      <td>61</td>\n",
       "      <td>6.0032</td>\n",
       "      <td>12</td>\n",
       "      <td>117</td>\n",
       "      <td>129</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2012-12-19 23:00:00</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>13.12</td>\n",
       "      <td>16.665</td>\n",
       "      <td>66</td>\n",
       "      <td>8.9981</td>\n",
       "      <td>4</td>\n",
       "      <td>84</td>\n",
       "      <td>88</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     season  holiday  workingday  weather   temp   atemp  \\\n",
       "datetime                                                                   \n",
       "2012-12-19 19:00:00       4        0           1        1  15.58  19.695   \n",
       "2012-12-19 20:00:00       4        0           1        1  14.76  17.425   \n",
       "2012-12-19 21:00:00       4        0           1        1  13.94  15.910   \n",
       "2012-12-19 22:00:00       4        0           1        1  13.94  17.425   \n",
       "2012-12-19 23:00:00       4        0           1        1  13.12  16.665   \n",
       "\n",
       "                     humidity  windspeed  casual  registered  total  hour  \n",
       "datetime                                                                   \n",
       "2012-12-19 19:00:00        50    26.0027       7         329    336    19  \n",
       "2012-12-19 20:00:00        57    15.0013      10         231    241    20  \n",
       "2012-12-19 21:00:00        61    15.0013       4         164    168    21  \n",
       "2012-12-19 22:00:00        61     6.0032      12         117    129    22  \n",
       "2012-12-19 23:00:00        66     8.9981       4          84     88    23  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bikes.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **hour** ranges from 0 (midnight) through 23 (11pm)\n",
    "- **workingday** is either 0 (weekend or holiday) or 1 (non-holiday weekday)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 1\n",
    "\n",
    "Run these two `groupby` statements and figure out what they tell you about the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "workingday\n",
       "0    188.506621\n",
       "1    193.011873\n",
       "Name: total, dtype: float64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# mean rentals for each value of \"workingday\"\n",
    "bikes.groupby('workingday').total.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "hour\n",
       "0      55.138462\n",
       "1      33.859031\n",
       "2      22.899554\n",
       "3      11.757506\n",
       "4       6.407240\n",
       "5      19.767699\n",
       "6      76.259341\n",
       "7     213.116484\n",
       "8     362.769231\n",
       "9     221.780220\n",
       "10    175.092308\n",
       "11    210.674725\n",
       "12    256.508772\n",
       "13    257.787281\n",
       "14    243.442982\n",
       "15    254.298246\n",
       "16    316.372807\n",
       "17    468.765351\n",
       "18    430.859649\n",
       "19    315.278509\n",
       "20    228.517544\n",
       "21    173.370614\n",
       "22    133.576754\n",
       "23     89.508772\n",
       "Name: total, dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# mean rentals for each value of \"hour\"\n",
    "bikes.groupby('hour').total.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 2\n",
    "\n",
    "Run this plotting code, and make sure you understand the output. Then, separate this plot into two separate plots conditioned on \"workingday\". (In other words, create two new plots. One plot should display the hourly trend for \"workingday=0\", and the other should display the hourly trend for \"workingday=1\".)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# mean rentals for each value of \"hour\"\n",
    "bikes.groupby('hour').total.mean().plot()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# hourly rental trend for \"workingday=0\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# hourly rental trend for \"workingday=1\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# combine the two plots onto one new plot\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 3\n",
    "\n",
    "Fit a linear regression model to the entire dataset, using \"total\" as the response and \"hour\" and \"workingday\" as the only features. Then, print the coefficients and interpret them. What are the limitations of linear regression in this instance?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# create X and y\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# fit a linear regression model and print coefficients\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 4\n",
    "\n",
    "Use 10-fold cross-validation to calculate the RMSE for the linear regression model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# save the 10 MSE scores output by cross_val_score to a new variable named scores\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# convert MSE to RMSE, and then calculate the mean of the 10 RMSE scores\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 5\n",
    "\n",
    "Use 10-fold cross-validation to evaluate a decision tree model with those same features (fit to any \"max_depth\" you choose)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# evaluate a decision tree model with \"max_depth=7\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 6\n",
    "\n",
    "Fit a decision tree model to the entire dataset using \"max_depth=3\", and create a tree diagram using Graphviz. Then, figure out what each leaf represents. What did the decision tree learn that a linear regression model could not learn?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# fit a decision tree model with \"max_depth=3\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# create a Graphviz file\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Note: Graphviz needs to be installed on your laptop \n",
    "to be available to convert .dot to .png http://www.graphviz.org/Download..php in the next step\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# At the command line, run this to convert to PNG:\n",
    "#   dot -Tpng tree_bikeshare.dot -o tree_bikeshare.png"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display the .png file in this notebook using markdown"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}