intergroup_difference.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intergroup Difference Explanations\n",
    "\n",
    "**Example.** In this example, we create a model to predict whether a criminal will recidivate within two years of release. We find that our model predicts higher recidivism rates for Black criminals than for White criminals, then use G-SHAP to identify which features are most responsible for this difference. We find that prior convictions, age, and race are the most important variables.\n",
    "\n",
    "**General formulation.** More generally, suppose we have two groups of observations, $X_0\\subseteq X$ and $X_1\\subseteq X$, and a measure $d$ of how different our model output is between groups. We can ask why our model makes different predictions for different groups. Here, groups $X_0$ and $X_1$ are White and Black criminals respectively, the model output is a classification which indicates that a criminal will recidivate within two years, and the difference measure is the relative difference in recidivism rates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-18T00:06:43.003590Z",
     "start_time": "2020-08-18T00:06:41.721709Z"
    }
   },
   "outputs": [],
   "source": [
    "import gshap\n",
    "from gshap.datasets import load_recidivism\n",
    "from gshap.intergroup import IntergroupDifference\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "import random\n",
    "import os\n",
    "\n",
    "np.random.seed(0)\n",
    "random.seed(0)\n",
    "\n",
    "# directory for saving plots\n",
    "# set to `None` to avoid saving plots\n",
    "PLOTS_DIR = 'plots'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Model.** We train Scikit-learn's support vector classifier with default settings on the Correctional Offender Management Profiling for Alternative Sanctions dataset, selecting only Black and White criminals ($n_\\text{Black}=3565,n_\\text{White}=1904$), using a 75\\%-25\\% train-test split. The purpose of this dataset is to predict whether a criminal will recidivate within two years of release based on demographic information and criminal history. Our classifier achieves 68\\% test accuracy. The predicted recidivism rate is 63\\% higher for Black criminals (65\\%) than for White criminals (40\\%)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-18T00:06:44.609620Z",
     "start_time": "2020-08-18T00:06:43.006124Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test score: 0.6780\n",
      "Predicted two-year recidivism rate for Whites:  0.40\n",
      "Predicted two-year recidivism rate for Blacks:  0.65\n",
      "Relative difference:  0.63\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAc4UlEQVR4nO3dfZxVZb338c+XCeKWID0IyeFpUBEDFdQB9OTtQ6RRdiSfAoxUrMhjlDdaRC+78eHYqexRjdsiUyL1IM7JoqK88ykPiMJMijIoifjATKIDlqJHj+D8zh97DW3GeVgzzNqbmf19v177tfd1rWut9dvzgvnNuta1rksRgZmZla4exQ7AzMyKy4nAzKzEORGYmZU4JwIzsxLnRGBmVuLeVewA2mv//feP8vLyYodhZtalVFdXb42IAc1t63KJoLy8nKqqqmKHYWbWpUh6rqVt7hoyMytxTgRmZiUu00QgabKkDZI2SprXzPbvS3o0ef1Z0t+yjMfMzN4ps3sEksqABcDJQC2wRtKyiFjf2CYi5uS1/wJwZEfOtWPHDmpra3nzzTf3MOruo3fv3gwZMoSePXsWOxQz28tlebN4ArAxIjYBSFoCTAHWt9B+OnB5R05UW1tL3759KS8vR1KHgu1OIoJt27ZRW1vLiBEjih2Ome3lsuwaGgxszivXJnXvIGk4MAK4t4XtsyRVSaqqr69/x/Y333yT/v37OwkkJNG/f39fIZlZKnvLzeJpQGVEvN3cxohYGBEVEVExYECzw2CdBJrwz8PM0soyEdQBQ/PKQ5K65kwD/j3DWMzMrAVZJoI1wEhJIyT1IvfLflnTRpIOBfYDVmUYS2Y++tGP8re/tT7Y6T3veU+z9eeffz6VlZVZhGXWpc2dO5dzzz2XuXPnFjuUkpDZzeKI2ClpNnAXUAbcFBE1kq4CqiKiMSlMA5ZEF1shJyKICJYvX17sUMy6nS1btlBX11IHgnW2TO8RRMTyiDgkIg6KiK8ndfPzkgARcUVEvOMZg0KZN28eCxYs2FW+4ooruPrqq5k0aRJHHXUUhx9+OL/61a8AePbZZxk1ahTnnnsuhx12GJs3b6a8vJytW7cC8PGPf5yjjz6aMWPGsHDhwt3OM2fOHMaMGcOkSZNo7oZ3dXU1J5xwAkcffTQf/vCHeeGFFzL81mZmf7e33CwumqlTp7J06dJd5aVLl3Leeedx55138qc//Yn77ruPSy+9lMYLlqeeeoqLLrqImpoahg8fvtuxbrrpJqqrq6mqquK6665j27ZtALz++utUVFRQU1PDCSecwJVXXrnbfjt27OALX/gClZWVVFdXc8EFF3DZZZdl/M3NzHK63KRzne3II4/kpZde4i9/+Qv19fXst99+HHDAAcyZM4cHHniAHj16UFdXx4svvgjA8OHDOeaYY5o91nXXXcedd94JwObNm3nqqafo378/PXr0YOrUqQDMmDGDM844Y7f9NmzYwLp16zj55JMBePvttxk0aFBWX9nMbDclnwgAzj77bCorK9myZQtTp07l1ltvpb6+nurqanr27El5efmuMfl9+vRp9hj3338/d999N6tWrWKfffbhxBNPbHEcf9OhnRHBmDFjWLWqS94vN7MuruS7hiDXPbRkyRIqKys5++yzeeWVVxg4cCA9e/bkvvvu47nnWpy9dZdXXnmF/fbbj3322Ycnn3yShx56aNe2hoaGXaODbrvtNo477rjd9h01ahT19fW7EsGOHTuoqanpxG9oZtYyJwJgzJgxbN++ncGDBzNo0CA++clPUlVVxeGHH87ixYs59NBD2zzG5MmT2blzJ+9///uZN2/ebt1Hffr0YfXq1Rx22GHce++9zJ8/f7d9e/XqRWVlJV/5ylcYO3Ys48aN48EHH+z072lm1hx3DSUef/zxXZ/333//Frtp1q1bt1v52Wef3fX5d7/7XbP7vPbaa83WL1q0aNfncePG8cADD6SM1sys8/iKwMysxDkRmJmVOHcNme1Fjv7y4mKHsFfou3U7ZcDzW7f7ZwJUf/vcTI/vKwIzsxLnRGBmVuKcCMzMSly3vEfQ2X2KWffPmZkVk68IurBFixYxe/bsYodhZl2cE8Fe6O23m12x08wsE04EnWD+/Pn84Ac/2FW+7LLLuPbaa9/R7v777+f444/n1FNPZdSoUVx44YU0NDQAuVXMLr30UsaOHcuqVau45ZZbmDBhAuPGjeNzn/vcruRw8803c8ghhzBhwgRWrlxZmC9oZt2aE0EnuOCCC1i8OHdfoqGhgSVLljBjxoxm265evZrrr7+e9evX8/TTT/OLX/wCyK1ZMHHiRNauXUv//v25/fbbWblyJY8++ihlZWXceuutvPDCC1x++eWsXLmSFStWsH79+oJ9RzPrvrrlzeJCKy8vp3///jzyyCO8+OKLHHnkkfTv37/ZthMmTODAAw8EYPr06axYsYKzzjqLsrIyzjzzTADuueceqqurGT9+PABvvPEGAwcO5OGHH+bEE09kwIABQG7W1D//+c8F+IZm1p05EXSSz3zmMyxatIgtW7ZwwQUXtNiu6VoEjeXevXtTVlYG5NYnOO+88/jGN76xW9tf/vKXnRy1mVk3TQTFGO55+umnM3/+fHbs2MFtt93WYrvVq1fzzDPPMHz4cG6//XZmzZr1jjaTJk1iypQpzJkzh4EDB/Lyyy+zfft2Jk6cyMUXX8y2bdvo168fd9xxB2PHjs3ya5lZCeiWiaAYevXqxUknncS+++676y/75owfP57Zs2ezceNGTjrpJE4//fR3tBk9ejRXX301p5xyCg0NDfTs2ZMFCxZwzDHHcMUVV3Dsscey7777Mm7cuCy/klnRNPTqs9u7ZSvTRCBpMnAtUAbcGBHfbKbNJ4ArgADWRsQ5WcaUlYaGBh566CHuuOOOVtv169eP3/zmN++ob7pmwdSpU3etc5xv5syZzJw5c8+CNdvLvT7ylGKHUFIyGzUkqQxYAHwEGA1MlzS6SZuRwFeBD0TEGOD/ZBVPltavX8/BBx/MpEmTGDlyZLHDMTNrlyyvCCYAGyNiE4CkJcAUIH/M42eBBRHxV4CIeCnDeDIzevRoNm3atKv8+OOP86lPfWq3Nu9+97t3jfoxM9ubtJkIJH0M+FdgeNJeQEREvzZ2HQxszivXAhObtDkkOcdKct1HV0TE75uJYRYwC2DYsGHNniwi3jEip1gOP/xwHn300aLGEBFFPb+ZdR1puoZ+AJwH9I+IfhHRN0USSOtdwEjgRGA68BNJ+zZtFBELI6IiIioax9Dn6927N9u2bfMvv0REsG3bNnr37l3sUMysC0jTNbQZWBft/y1bBwzNKw9J6vLVAg9HxA7gGUl/JpcY1rTnREOGDKG2tpb6+vp2hth99e7dmyFDhhQ7DDPrAtIkgrnAckl/BP67sTIivtfGfmuAkZJGkEsA04CmI4J+Se5K4GZJ+5PrKtpEO/Xs2ZMRI0a0dzczMyNdIvg68BrQG+iV9sARsVPSbOAucv3/N0VEjaSrgKqIWJZsO0XSeuBt4MsRsa29X8LMzDouTSL4x4g4rCMHj4jlwPImdfPzPgdwSfIyM7MiSHOzeLkkP91hZtZNpUkE/wL8XtIbkl6VtF3Sq1kHZmZmhdFm11BE9C1EIGZmVhxtXhFI+oCkPsnnGZK+J6n5p7rMzKzLSdM1dAPwX5LGApcCTwM/zzQqMzMrmDSJYGcyumcK8MOIWAC4u8jMrJtIM3x0u6SvAjOA4yX1AHpmG5aZmRVKmiuCqeSeKP50RGwhN1XEtzONyszMCibNqKEtwPfyys8Di7MMyszMCqfFRCBpRUQcJ2k7udXDdm0i3TTUZmbWBbSYCCLiuOTdN4bNzLqxNM8RXCfp2EIEY2ZmhZfmZnE18H8lPS3pO5Iqsg7KzMwKp81EEBE/i4iPAuOBDcC3JD2VeWRmZlYQaa4IGh0MHEpu7eInswnHzMwKLc09gmuSK4CrgHVARUT8c+aRmZlZQaR5svhp4NiI2Jp1MGZmVnitPUdwaEQ8SW7t4WFNZxyNiD9lHZyZmWWvtSuCS4HPAt9tZlsAH8wkIjMzK6jWHij7bPJ+UuHCMTOzQmuta+iM1naMiF90fjhmZlZorXUNNY4MGgj8E3BvUj4JeBBoMxFImgxcC5QBN0bEN5tsP5/cTKZ1SdUPI+LGtMGbmdmea61raCaApP8PjI6IF5LyIGBRWweWVAYsAE4GaoE1kpZFxPomTW+PiNkdC9/MzPZUmgfKhjYmgcSLQJo1iycAGyNiU0S8BSwht8qZmZntRdIkgnsk3SXp/KQr57fA3Sn2GwxszivXJnVNnSnpMUmVkoY2dyBJsyRVSaqqr69PcWozM0srzVxDs4EfA2OT18KI+EInnf/XQHlEHAH8AfhZCzEsjIiKiKgYMGBAJ53azMwg3ZPFjSOE2jtKqA7I/wt/CH+/Kdx43G15xRuBa9p5DjMz20MtXhFIWpG8b5f0at5ru6RXUxx7DTBS0ghJvYBpwLIm5xiUVzwNeKL9X8HMzPZEZiuURcROSbOBu8gNH70pImokXQVURcQy4IuSTgN2Ai8D53fkXGZm1nFtdg1JOgaoiYjtSbkvueGkD7e1b0QsB5Y3qZuf9/mrwFfbG7SZmXWeNKOGbgBeyyu/ntSZmVk3kCYRKCKisRARDaS8yWxmZnu/NIlgk6QvSuqZvC4GNmUdmJmZFUaaRHAhubmG6sg9FDYRmJVlUGZmVjhtdvFExEvkhn6amVk3lGbN4kMk3SNpXVI+QtLXsg/NzMwKIU3X0E/IDfHcARARj+ErBDOzbiPN6J99ImK1pPy6nRnFYwU0d+5ctmzZwgEHHMA113h2D7NSlSYRbJV0ELl1ipF0FvBC67tYV7Blyxbq6urabmhm3VqaRPB5YCFwqKQ64Bngk5lGZWZmBdNqIkhWGbsoIj4kqQ/Qo3GqCTMz6x5aTQQR8bakxsnnXi9MSGZmVkhpuoYekbQMuIPcPEPArjUKzMysi0uTCHoD24AP5tUF7V+oxszM9kJpniyeWYhAzMysONI8UGZmZt2YE4GZWYlrNRFI6iHpE4UKxszMCq/VRJAsQjO3QLGYmVkRpOkaulvSlyQNlfQPja/MIzMzs4JIM3x0avL++by6AA7s/HDMzKzQ2rwiiIgRzbxSJQFJkyVtkLRR0rxW2p0pKSRVtCd4MzPbc6kWoZd0GDCa3MNlAETE4jb2KQMWACeTW+JyjaRlEbG+Sbu+wMXAw+0L3czMOkOaFcouB65PXicB1wCnpTj2BGBjRGyKiLeAJcCUZtr9K/At4M20QZuZWedJc0VwFjAWeCQiZkp6H3BLiv0GA5vzyo0L3+8i6ShgaET8VtKXWzqQpFnALIBhw4alOHXrjv5yqxczJaPv1u2UAc9v3e6fCVD97XOLHYJZUaQZNfRGMox0p6R+wEvA0D09saQewPeAS9tqGxELI6IiIioGDBiwp6c2M7M8aa4IqiTtS27t4mrgNWBViv3q2D1hDEnqGvUFDgPuT5bBPABYJum0iKhKcXwzM+sEaSaduyj5+CNJvwf6JQvYt2UNMFLSCHIJYBpwTt5xXwH2byxLuh/4kpOAmVlhpblZLEkzJM2PiGeBv0ma0NZ+EbETmA3cBTwBLI2IGklXSUpzs9nMzAogTdfQ/wMayK1HcBWwHfgPYHxbO0bEcmB5k7r5LbQ9MUUsZmbWydIkgokRcZSkRwAi4q+SemUcl5mZFUiaUUM7kofDAkDSAHJXCGZm1g2kSQTXAXcCAyV9HVgB/FumUZmZWcGkGTV0q6RqYBIg4OMR8UTmkZmZWUG0mQgkfRf4aUQsKEA8ZmZWYGm6hp4AfiLpYUkXSnpv1kGZmVnhpJmG+saI+ABwLlAOPCbpNkknZR2cmZllL9Xi9cmooUOT11ZgLXCJpCUZxmZmZgWQ5h7B94GPAfcC/xYRq5NN35K0IcvgzMwse2keKHsM+FpEvN7MtjanmjAzs71bmnsENzcmAUkLm2x7JavAzMysMFItVZnHawp3Iw29+uz2bmalqb2J4KVMorCieH3kKcUOwcz2AqlGDTWKiMlZBWJmZsWRZj2CIZLulFQv6SVJ/yFpSCGCMzOz7KW5IrgZWAYMAv4R+HVSZ2Zm3UCaRDAgGTm0M3ktAryCvJlZN5EmEWxLlqosS14zgG1ZB2ZmZoWRJhFcAHwC2AK8AJwFzMwyKDMzK5w0w0dfjAgvNm9m1k2lSQTrJL0I/GfyWuEnis3Muo80U0wcDEwHHgdOBdZKejTNwSVNlrRB0kZJ85rZfqGkxyU9KmmFpNHt/QJmZrZnUj1HAHwA+N/AkUANcHuK/cqABcBHgNHA9GZ+0d8WEYdHxDjgGuB77QvfzMz2VJquoeeBNeSmoL6wHceeAGyMiE0AydoFU4D1jQ0i4tW89n2AaMfxzcysE6RJBEcCxwHnJN07TwF/jIiftrHfYGBzXrkWmNi0kaTPA5cAvYAPNncgSbOAWQDDhg1LEbKZmaWV5h7BWuBn5J4mvhc4AZjfWQFExIKIOAj4CvC1FtosjIiKiKgYMMDPspmZdaY0K5RVAe8GHiQ3auj4iHguxbHrgKF55SFJXUuWADekOK6ZmXWiNF1DF+UtTwmApBER8Uwb+60BRkoaQS4BTAPOaXKckRHxVFI8lVy3k5mZFVCaJ4t/1ExdZVs7RcROYDZwF/AEsDQiaiRdJanxAbXZkmqS4aiXAOeljNvMzDpJi1cEkg4FxgDvlXRG3qZ+QO80B4+I5cDyJnXz8z5f3K5ozcys07XWNTQK+BiwL/DPefXbgc9mGZSZmRVOi4kgIn4F/ErSsRGxqoAxmZlZAaUZPuokYGbWjbVrzWIzM+t+nAjMzEpca6OGLmltx4jwBHFmZt1Aa6OG+ibvo4Dx5Bawh9wIotXN7mFmZl1Oa6OGrgSQ9ABwVERsT8pXAL8tSHRmZpa5NPcI3ge8lVd+K6kzM7NuIM1cQ4uB1ZLuTMofJzcbqZmZdQNtJoKI+Lqk35FboQxgZkQ8km1YZmZWKGmHj+4DvBoR1wK1yYyiZmbWDaRZs/hycovGfDWp6gnckmVQZmZWOGmuCE4HTgNeB4iIv/D3oaVmZtbFpUkEb0VEkCwsL6lPtiGZmVkhpUkESyX9GNhX0meBu4Ebsw3LzMwKJc2ooe9IOhl4ldxTxvMj4g+ZR2ZmZgWRZvH6b0XEV4A/NFNnZmZdXJquoZObqftIZwdiZmbF0drso/8CXAQcJOmxvE19gQezDszMzAqjta6h24DfAd8A5uXVb4+IlzONyszMCqbFrqGIeCUingWuBV6OiOci4jlgp6SJaQ4uabKkDZI2SprXzPZLJK2X9JikeyQN7+gXMTOzjklzj+AG4LW88mtJXasklQELyN1PGA1MlzS6SbNHgIqIOAKoBK5JE7SZmXWeNIlAyQNlAEREA+lmLZ0AbIyITRHxFrAEmJLfICLui4j/SooPAUPShW1mZp0lTSLYJOmLknomr4uBTSn2GwxszivXJnUt+TS5exJmZlZAaRLBhcA/AXXkfplPBGZ1ZhCSZgAVwLdb2D5LUpWkqvr6+s48tZlZyUvzZPFLwLQOHLsOGJpXHpLU7UbSh4DLgBMi4r9biGEhsBCgoqIimmtjZmYd09pzBHMj4hpJ15NMOJcvIr7YxrHXACOTtQvqyCWTc5qc40jgx8DkJOGYmVmBtXZF8ETyXtWRA0fETkmzgbuAMuCmiKiRdBVQFRHLyHUFvQe4QxLA8xFxWkfOZ2ZmHdNiIoiIXyfvHV6fOCKWA8ub1M3P+/yhjh7bzMw6R2tdQ7+mmS6hRv7L3cyse2ita+g7yfsZwAH8fXnK6cCLWQZlZmaF01rX0B8BJH03IiryNv1aUofuG5iZ2d4nzXMEfSQd2FhIRgF5uUozs24izVQRc4D7JW0CBAwHPpdpVGZmVjBpHij7vaSRwKFJ1ZMtPfhlZmZdT5tdQ5L2Ab4MzI6ItcAwSR/LPDIzMyuINPcIbgbeAo5NynXA1ZlFZGZmBZUmERwUEdcAOwCSaaOVaVRmZlYwaRLBW5L+F8nDZZIOAnyPwMysm0gzauhy4PfAUEm3Ah8Azs8yKDMzK5xWE4GkHsB+5J4uPoZcl9DFEbG1ALGZmVkBtJoIIqIhmY56KfDbAsVkZmYFlOYewd2SviRpqKR/aHxlHpmZmRVEmnsEU5P3z+fVBXBgM23NzKyLSfNk8YhCBGJmZsXRZiKQ1Bu4CDiO3JXAfwI/iog3M47NzMwKIE3X0GJgO3B9Uj4H+DlwdlZBmZlZ4aRJBIdFxOi88n2S1mcVkJmZFVaaUUN/knRMY0HSRDq4oL2Zme190lwRHA08KOn5pDwM2CDpcSAi4ojMojMzs8ylSQSTO3pwSZOBa4Ey4MaI+GaT7ccDPwCOAKZFRGVHz2VmZh2TZvjocx05sKQyYAFwMlALrJG0LCLy7y88T27eoi915BxmZrbn0lwRdNQEYGNEbAKQtASYAuxKBBHxbLKtIcM4zMysFWluFnfUYGBzXrk2qTMzs71Ilomg00iaJalKUlV9fX2xwzEz61ayTAR1wNC88pCkrt0iYmFEVERExYABAzolODMzy8kyEawBRkoaIakXMA1YluH5zMysAzJLBBGxE5gN3AU8ASyNiBpJV0k6DUDSeEm15Kar+LGkmqziMTOz5mU5aoiIWA4sb1I3P+/zGnJdRmZmViRd4maxmZllx4nAzKzEORGYmZU4JwIzsxLnRGBmVuKcCMzMSpwTgZlZiXMiMDMrcU4EZmYlzonAzKzEORGYmZU4JwIzsxLnRGBmVuKcCMzMSpwTgZlZiXMiMDMrcU4EZmYlzonAzKzEORGYmZU4JwIzsxLnRGBmVuKcCMzMSlymiUDSZEkbJG2UNK+Z7e+WdHuy/WFJ5VnGY2Zm75RZIpBUBiwAPgKMBqZLGt2k2aeBv0bEwcD3gW9lFY+ZmTUvyyuCCcDGiNgUEW8BS4ApTdpMAX6WfK4EJklShjGZmVkT78rw2IOBzXnlWmBiS20iYqekV4D+wNb8RpJmAbOS4muSNmQScWnanyY/71Kl75xX7BBsd/63meikf5vDW9qQZSLoNBGxEFhY7Di6I0lVEVFR7DjMmvK/zcLJsmuoDhiaVx6S1DXbRtK7gPcC2zKMyczMmsgyEawBRkoaIakXMA1Y1qTNMqDxmucs4N6IiAxjMjOzJjLrGkr6/GcDdwFlwE0RUSPpKqAqIpYBPwV+Lmkj8DK5ZGGF5S4321v532aByH+Am5mVNj9ZbGZW4pwIzMxKnBNBiWpr+g+zYpF0k6SXJK0rdiylwomgBKWc/sOsWBYBk4sdRClxIihNaab/MCuKiHiA3ChCKxAngtLU3PQfg4sUi5kVmROBmVmJcyIoTWmm/zCzEuFEUJrSTP9hZiXCiaAERcROoHH6jyeApRFRU9yozHIk/TuwChglqVbSp4sdU3fnKSbMzEqcrwjMzEqcE4GZWYlzIjAzK3FOBGZmJc6JwMysxDkRmLVAUnlzM2BKul9SuxdVl3S+pB92TnRmnceJwMysxDkRmLXuXZJulfSEpEpJ++RvlHSDpCpJNZKuzKsfL+lBSWslrZbUt8l+p0paJWn/Qn0Rs5Zktni9WTcxCvh0RKyUdBNwUZPtl0XEy8kaD/dIOgJ4ErgdmBoRayT1A95o3EHS6cAlwEcj4q+F+RpmLXMiMGvd5ohYmXy+Bfhik+2fkDSL3P+lQeQW+gnghYhYAxARrwJIAvggUAGc0lhvVmzuGjJrXdM5WHaVJY0AvgRMiogjgN8Cvds43tNAX+CQzgzSbE84EZi1bpikY5PP5wAr8rb1A14HXpH0PnJLfwJsAAZJGg8gqa+kxqvv54AzgcWSxmQevVkKTgRmrdsAfF7SE8B+wA2NGyJiLfAIuXsCtwErk/q3gKnA9ZLWAn8g70ohIp4EPgncIemgAn0PsxZ59lEzsxLnKwIzsxLnRGBmVuKcCMzMSpwTgZlZiXMiMDMrcU4EZmYlzonAzKzE/Q8qd6Cn9hfexAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# model training and evaluation\n",
    "recid = load_recidivism()\n",
    "X, y = recid.data, recid.target\n",
    "X_train, X_test, y_train, y_test = train_test_split(recid.data, recid.target, test_size=1000)\n",
    "clf = SVC()\n",
    "clf.fit(X_train, y_train)\n",
    "print('Test score: %.4f' % clf.score(X_test, y_test))\n",
    "\n",
    "# racial difference\n",
    "df = pd.concat((X_test, y_test), axis=1)\n",
    "df['y_pred'] = clf.predict(X_test)\n",
    "p_non_black, p_black = df.groupby('black')['y_pred'].mean()\n",
    "print('Predicted two-year recidivism rate for Whites: % .2f' % p_non_black)\n",
    "print('Predicted two-year recidivism rate for Blacks: % .2f' % p_black)\n",
    "print('Relative difference: % .2f' % (p_black/p_non_black - 1))\n",
    "\n",
    "df['id'] = df.index\n",
    "df = df.melt(('id', 'black'), value_vars='y_pred')\n",
    "sns.barplot(y='value', x='black', hue='variable', data=df)\n",
    "plt.ylabel('predicted two-year recidivism')\n",
    "if PLOTS_DIR is not None:\n",
    "    plt.savefig(os.path.join(PLOTS_DIR, 'recidivism_differnce.png'))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We plot the correlations of all variables in our dataset with race (Black) and two-year recidivism. Visually, it is unclear which variables lead our model to believe that Black criminals are at higher risk for recidivism."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-18T00:06:45.191982Z",
     "start_time": "2020-08-18T00:06:44.613471Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaMAAAFgCAYAAAAW6RbuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deZhdVZ3u8e/LICggIEREpIwis4FACgQuYoQ0oiKQi4iixgBNbETR5iGtV22FoC0abzvgRKQRaGhA0KAXWhCZCVMSQggzaAYb6FZGCWCA8N4/9qp4cqiqnEoNu4b38zznqT2s4bdP4PzOWnufvWWbiIiIOq1RdwARERFJRhERUbsko4iIqF2SUURE1C7JKCIiapdkNIgccMABBvLKK69Vv2KYSTIaRB577LG6Q4iIqEWSUURE1G6tugOIwWPc1HPqDiGCudMn1R1C1CAjo4iIqF2SUURE1C7JKCIiapdkFBERtUsyioiI2iUZRURE7ZKMIiKidklGhaTRku7qZPu1ktpXo73Jkn7QN9FFRAxvSUYREVG7JKOVrSXpPEn3SrpY0msad0r6saQ5ku6WdHLD9t0k3SRpvqTbJG3QVO/9km6WtOlAHUhExFCS2wGtbFvgaNuzJJ0JfKpp/5dsPyFpTeAqSTsB9wEXAofbni3ptcDzHRUkTQROAN5n+8nmDiVNAaYAtLW19ctBtWrmBtN7Vb/tKwv6KJKIGGkyMlrZH23PKsvnAns37f+QpNuBecCOwA5UCexR27MBbP/F9kul/L7A54H3d5aISvkZttttt48aNaqPDyciYmhIMlpZ83NSVqxLegtwIrCf7Z2Ay4B1V9He74ENgG36MsiIiOEmyWhlbZL2LMtHADc27Hst8CzwtKTNgPeW7fcDm0vaDUDSBpI6pj8XA4cC50jasd+jj4gYopKMVnY/cJyke4GNgR937LA9n2p67j7gP4BZZfsLwOHAaZLmA1fSMGKyfR/wUeAiSVsN0HFERAwpuYChsL0I2K6TXeMbykzuou5sYI+mzWeVF7bnUZ1fioiITmRkFBERtUsyioiI2iUZRURE7ZKMIiKidklGERFRuySjiIioXS7tjhVyb7mIqEtGRhERUbsko4iIqF2SUURE1C7JKCIiapcLGGKFcVPPGbC+5k6fNGB9RcTgl5FRRETULskoIiJql2QUERG1SzKKiIjaJRlFRETtkowiIqJ2SUYREVG7JKOIiKjdkEhGkqZJmlB3HKtD0mhJR9QdR0TEYDbok5GkNW1/xfbvelKnP2PqodFAklFERDdku77OpdHA5cBcYFfgbmAScA9wIfB3wLeAA4BLbV8saT/g21S3MpoNHGt7maRFTXVeD/wD8BJwj+0PdxHD+sBpQDtg4GTbv5D0EeCLgIDLbH++lF9qe/2y/EHgQNuTJZ0F/KW08wbgn0q8twDbAwuBs21/p6n/KcAUgLa2tnGLFy9evTezDyyZNqa2vnsqz14a8VR3ANG3BsO96bYFjrY9S9KZwKfK9sdt7wog6YDyd13gLGA/2w9IOgc4FvhuJ3UeAd5SEtVG3fT/z8DTtseUehtLeiPwTWAc8CTwW0mH2L5kFceyObA3sB3wa+Bi4AvAibYP7KyC7RnADID29vb6vhlERNRoMEzT/dH2rLJ8LtWHOVSjnGbbAgttP1DWzwb2adjfWOdO4DxJH6MaHXVlAvDDjhXbTwK7Adfa/rPtl4DzmvrpyiW2X7Z9D7BZC+UjIoLBkYyaRwMd68+uRluNdd5PlWR2BWZL6qtRYGO86zbtW9awnGmEiIgWDYZk1CZpz7J8BHBjN2XvB0ZLeltZ/zhwXXMhSWsAW9q+Bvg8sCGwfhdtXgkc11B3Y+A24F2SNi0XQ3ykoZ//kbR96WNiC8f3DLBBC+UiIkaswZCM7geOk3QvsDHw464K2v4rcCRwkaQFwMvATzopuiZwbikzD/i+7ae6aPZrwMaS7pI0H3i37UepzvVcA8wH5tr+VSn/BeBS4Cbg0RaO705guaT5kv6xhfIRESPOYLia7lLbb68tiEGkvb3dc+bMqa3/XE0XQ0imwYeZwTAyioiIEa7WS7ttLwIGZFQk6Ujgs02bZ9k+rrPyERExcAbD74wGhO2fAT+rO46IiHilTNNFRETtRszIKFYtFwVERF0yMoqIiNolGUVERO2SjCIionZJRhERUbsko4iIqF2uposVxk09p8/bnDt9Up+3GRHDT0ZGERFRuySjiIioXZJRRETULskoIiJql2QUERG1SzKKiIjaJRlFRETtkowiIqJ2q0xGkkZLuquT7dMkTVhF3ZMkndibAOsg6aZV7P9PSRsNVDwREcPdat+BwfZX+jKQgSRJgGy/3Nl+23t1V9/2+/olsIiIEarVZLSmpJ8CewEPAwcDPwYutX2xpPcB/wo8C8wC3mr7wFJ3B0nXAm3Ad21/v7MOJK0H/Bx4E7AmcIrtCyUtAtptPyapHfi27fGSTgK2At4GbAp8y/ZPS1tTgQ8B6wAzbX9V0mjgCuBWYBzwc0nr255a6kwu/Xxa0lLb60vaHLgQeG15r461fUNTTCcAR5XDOMP2d0tfvwFubHzPbD/fyXFPAaYAtLW1rerfoV/N3GB6n7e5ZFr3beaBfhEBrZ8z2hr4oe0dgaeAQzt2SFoXOB14r+1xwKimutsB7wF2B74qae0u+jgAeMT2zrbfDlzeQlw7AfsCewJfkfRGSfuXeHcHxgLjJO3TcBw/KsfxI2BiQ1uHAxc0tX8EcIXtscDOwB2NOyWNA44E3gHsARwjaZeGvjp9zxrZnmG73Xb7qFHNb11ExMjQajJaaLvjg3guMLph33bAH2wvLOvnN9W9zPYy248BfwI266KPBcDfSfqmpHfafrqFuH5l+/nS9jVUCWj/8poH3F7i27qUX2z7FgDbfwb+IGkPSZuUcrOa2p8NHFlGYWNsP9O0f2+qkdeztpcCvwTeWfZ1955FRESDVpPRsobl5fTsXFNLdW0/AOxKlZS+JqnjnNRLDXGu21ytk3UB37A9trzeZvvfyv5nm8pfQDWddyhVUlmpPdvXA/tQTbOdJaknt6DuzXsWETGi9MWl3fcDby3nSaCa7uoxSW8EnrN9LjCdKjEBLKI6xwOvnOo6WNK6ZWQznmokcwVwlKT1S7tbSHp9F93OpDr/9RFeOUWHpDcD/1PORZ3REFOHG4BDJL2mnPOaWLZFREQP9Prbuu3nJX0KuFzSs1QJYXWMAaZLehl4ETi2bD8Z+DdJpwDXNtW5k2p6blOqCx4eAR6RtD1wc3XRHEuBj1GNTppjf1LSvcAOtm/rJKbxwFRJL5Z2JjXVv13SWUBH3TNsz2tIzBER0QI1zUytXiPVVWlLyyXTPwQetP2dXjfcfZ8nAUttf7s/+xlI7e3tnjNnTm39L5k2ZsD7zNV0sZpUdwDRt/rqDgzHSLoDuBvYkOrquoiIiJb0yUn1MgpqaSRUzu9c1cmu/Ww/3oM+T2q1bEREDG4DfoVXSThjB7rfiIgYvHKj1IiIqF1++xIr5GKCiKhLRkYREVG7JKOIiKhdklFERNQuySgiImqXZBQREbXL1XSxwrip5/Sq/tzpPbmpeUTE32RkFBERtUsyioiI2iUZRURE7ZKMIiKidklGERFRuySjiIioXZJRRETULskoIiJq1yfJSNJGkj7VF20NVZLaJX2/i32LJG060DFFRAwVfTUy2ggYMslI0pp9UaaR7Tm2j1/9qCIiRq6+uh3QqcBWku4A5gEzbf9a0kzgSdtHSToK2Mr2lySdABxV6p5h+7udNSppGvBEx35JXwf+ZPt7kqYCHwLWKf19tZS5BNgSWBf4nu0ZZftS4HRgAnAccGMn/S0CLgT+DviWpCeAk0sfvweOtL1U0m7A94D1gGXAfsA44ETbB0raBDgf2AK4GVBXb5ykKcAUgLa2tq7f4QEwc4Ppvaq/ZFrv6kdAHvI4UvXVyOgLwO9tjwWuAN5Ztm8B7FCW3wlcL2kccCTwDmAP4BhJu3TR7pnAJABJawAfBs6VtD+wNbA7MBYYJ2mfUuco2+OAduD4khigShy32t7Z9isSUYPHbe8K/A74MjChrM8BTpD0KqqE9VnbO1Mlt+eb2vgqcKPtHYGZQJdZxvYM2+2220eNGtVNWBERw1d/3Cj1BuBzknYA7gE2lrQ5sCdwPNWIaKbtZwEk/ZIqUc1rbsj2IkmPl2S1GTDP9uMlGe3fUGd9quR0PVUCmli2b1m2Pw4sB37RQvwXlr97UCXSWZIAXkU1ytkWeNT27BLjX8pxNLaxD/C/y/7LJD3ZQr8RESNWnycj2w9L2gg4gCo5vI5qOm2p7WeaPrRbcQYwGXgD1UgJqmmvb9g+vbGgpPFUI5U9bT8n6Vqq6TqAv9pe3kJ/zzb0caXtjzT1MaanBxAREd3rq2m6Z4ANGtZvAT5HlYxuAE4sfyl/D5H0GknrARMb9nVmJlVi241qCpDy9yhJ6wNI2kLS64ENqc5RPSdpO6rRzeq6Bfhfkt5W+lhP0jbA/cDm5bwRkjaQ1JzUrweOKPvfC2zcizgiIoa9PhkZlamzWZLuAn5DlVz2t/2QpMVUo6MbStnbJZ0F3Faqn2H7FVN0DW2/IOka4KmOkY3t30raHri5jLSWAh8DLgf+QdK9VEnjll4c058lTQbOl7RO2fxl2w9IOhw4TdKrqc4XTWiqfnKpdzdwE7BkdeOIiBgJZLvuGLpVLly4HTjM9oN1x9Of2tvbPWfOnNr6XzItM5BRvxavpuvxfH8MboP6DgzlIoiHgKuGeyKKiBjJBsVjx8vl11d1sms/22/th/5mAm9p2vx521d0Vj4iIvrXoEhGth+n+r3QQPU3cdWlIiJioAzqabqIiBgZBsXIKAaH3IYlIuqSkVFERNQuySgiImqXZBQREbVLMoqIiNolGUVERO1yNV2sMG7qOatdd+70SX0YSUSMNBkZRURE7ZKMIiKidklGERFRuySjiIioXZJRRETULskoIiJql2QUERG1G7HJSNLxku6VdF4X+8dLunSg44qIGIlG8o9ePwVMsP1fdQcSETHSjciRkaSfAG8FfiPpS5LOlHSbpHmSDu6k/OskXSLpTkm3SNqpbD+p1L1W0h8kHV+2T5P0uYb6X5f02YE6voiIoUa2646hFpIWAe3ACcA9ts+VtBFwG7ALsBtwou0DJZ0GPGb7ZEn7Av9qe6ykk4D9gXcDGwD3A28AtgB+aXtXSWsADwK7l8erN8cxBZgC0NbWNm7x4sX9etzdWTJtTK/q5+F8MYBUdwDRt0byNF2H/YGDJJ1Y1tcF2prK7A0cCmD7akmbSHpt2XeZ7WXAMkl/AjazvUjS45J2ATYD5nWWiEp7M4AZAO3t7SPzm0FEjHhJRtU3rENt37/SRmmzFusva1hezt/e0zOAyVQjpTN7GWNExLA2Is8ZNbkC+IwkAZTRTLMbgI+W/eOppuz+sop2ZwIHUE33XdFn0UZEDEMZGcEpwHeBO8v5nYXAgU1lTgLOlHQn8BzwiVU1avsFSdcAT9le3rchR0QMLyM2Gdke3bD6yU72XwtcW5afAA7ppMxJTetv71guiW0P4LA+CDciYljLNF0/kLQD8BBwle0H644nImKwG7Ejo/5k+x6q3zFFREQLMjKKiIjaJRlFRETtkowiIqJ2OWcUK+R2PhFRl4yMIiKidklGERFRuySjiIioXZJRRETULskoIiJql6vpYoVxU8/pcZ250yf1QyQRMdJkZBQREbVLMoqIiNolGUVERO2SjCIionZJRhERUbsko4iIqF2SUUTEICDpPyVttIoyS7vYfpakD/ZPZAMjvzOKiKiRJAGy/b66Y6lTr0dGkm7qi0B60F+7pO/3oHyt3xgkHSJph7r6j4iBIelUScc1rJ8k6cuSrpJ0u6QFkg4u+0ZLul/SOcBdwJaSFknatOy/RNJcSXdLmtLUz3fK9qskjeokjnGSriv1r5C0ef8eed/odTKyvVdfBNKD/ubYPn4g++ylQ4Ako4jh70LgQw3rHwLOBiba3hV4N/B/y0gIYGvgR7Z3tL24qa2jbI8D2oHjJW1Stq8HzLG9I3Ad8NXGSpLWBk4DPljqnwl8vc+OsB/1epquzGEeCJxo+8Cy7QfAHOC/gaNtH1a2j28s10VbPwbeBzwKfBH4FtAGfM72rxvbkPQu4HuluoF9gKVU/xh/B/wReGEV8e9W2lgPWAbsB7xY4mgHXgJOsH2NpMlAu+1Pl7qXAt+2fW2J/XvlvXgeOBjYCjgIeJekLwOH2v59U/9TgCkAbW1t3YXa72ZuML3HdZZM+1udPJwvRjLb8yS9XtIbgVHAk1Sfgd+RtA/wMrAFsFmpstj2LV00d7ykiWV5S6rE9Xhp48Ky/Vzgl031tgXeDlxZct6aVJ+lg15/X8DwO+AdktYr64cDF3RTfj3g6pL1nwG+RpVUJgLTOil/InCc7bHAO6mSwESqf5AdgElAlyM3Sa+i+of9rO2dgQmljeMA2x4DfAQ4W9K6qzjW9YBbSjvXA8fYvgn4NTDV9tjmRETVyQzb7bbbR416xYg7IoaWi4APUn3WXQh8lCoxjSufU/8DdHyWPNtZA+UL9wRgz/J5Mq+hTjM3VwfuLp83Y22Psb1/L45nwPRrMrL9EnA58AFJawHvB37VTZUXSnmABcB1tl8sy6M7KT8L+FdJxwMblf72Ac63vdz2I8DV3fS3LfCo7dkl3r+UNvam+taB7fuAxcA2qzjcF4BLy/LcLuKNiOHtQuDDVAnpImBD4E+2X5T0buDNLbSxIfCk7eckbQfs0bBvjdI2wBHAjU117wdGSdoTqmk7STuu9tEMoL5KRi81tdWYxS+gmjvdl2qu85lu2nnRdkemf5lq2gzbL9PJlKLtU4G/B14NzCr/cP2pu+NsjH05uVIxYsSxfTewAfCw7UeB84B2SQuoZmrua6GZy4G1JN0LnAo0TuU9C+wu6S6qz9SVZoxsv0CVrL4paT5wB93MDg0mffWBuRjYQdI6VIlhP/6Wsa+jOol2DN1P0fWYpK1sLwAWlHM/21FNkX1S0tnA66lOGv5HF03cD2wuaTfbsyVtQDVNdwPV8PpqSdtQnbO6H3gt8ClJa1DN/e7eQpjPUP3HGREjQJne71h+DNizi6Jvb6o3umH1vV20vX4X2yc3LN9BNUM0pPRFMrLtP0r6OdUligup5jg7di4vJ/onA5/og/4afa4MfV8G7gZ+QzVdti9wD7AEuLmbwF+QdDhwmqRXUyWiCcCPgB+XbzMvAZNtL5M0qxzfPcC9wO0txHgB8NMylfjBzs4bRUSMdPrbzNJqVK4uN7zddivzoLEK7e3tnjNnTm39L5k2ZtWFupGr6WIAadVFYihZ7XNG5fLFm4Fv9104ERExEq32NF25Um1VV5h1StKtwDpNmz9ezv/0C0kzgbc0bf687Sv6q8+IiGhNLVd82X5HDX1OXHWpiIioQ+7aHRERtctvYWKFXIAQQ924qees/hVZnZg7fdIqL5SQNBq41Pbbm7ZfS3Xrsh5dldR827GRIiOjiIioXZJRRETvrSXpPEn3SrpY0msad0r6saQ55dEPJzds303STZLmS7qt/PC+sd77Jd3c8WiJ4SzTdBERvbct1RMKZkk6E/hU0/4v2X5C0prAVZJ2oro10IXA4eUOMK+l+uE9AOWu3ScA77P95MAcRn2SjCIieu+PtmeV5XOB5meufag8LmYtYHOqpwqYphs1A5RHP+xL9Qib/Tu2D3eZpouI6L3mCydWrEt6C9XjbvazvRNwGV0/EqLD76nuablav+UcijIyihXGTT2nR+XnTp/UT5FEDDltkva0fTN/e7TDB8q+11LdbftpSZtR3QT1Wrq+UTNUN5+eCvxS0mHlbuDDWpJRRAwbrVyK3U/uB44r54vuoXpS9AcAbM+XNI/qHNEfqZ7D1t2Nmin775P0UeAiSR8Y7jdZTjKKiOgF24uoHl/TbHxDmcld1J3Nyg/PAzirvLA9j+r80rCXc0YREVG7JKOIiKhdklFERNQuySgiImqXZBQREbVLMoqIiNoN+KXdkm6yvVcftXUYMA34b9vv7qLMaDq5vftAkbQRcITtH9XRf8RIsmTamD59hETbVxbU9bulEWfAR0Z9lYiKo4FjukpEg8RGvPKmiRExDEjaSNKI/v9bUruk73exb1GrdxyvY2S0FDiQ6qFTB5ZtPwDmAP9Ndefbw8r28Y3lmtr5CrA38G+Sfg18ATiV6odm6wA/tH16C/GsCXwTOAB4Gfip7dMk7Qd8m+o9mg0ca3uZpEVUD756TFI78G3b4yWdBLQBby1/v2v7+yWmrSTdAVxpe2pT/1OAKQBtbW0tvIP9Z+YG03tUfsm0qnweyhcjWMeXzSEx8yFpTdvLe1umUXl4YI8eINiZwXbO6HfAOyStV9YPBy7orKDtaVRvwEfLB/zRwNO2dwN2A44pNyhclSnAaGBsuYnheZLWpfoF9OG2x1AlpGNbaGs74D3A7sBXJa1NlSR/b3tscyIqxzHDdrvt9lGjRrXQRUQMIiu+bEr6maSDACTNLLcGQtJRkr5elk+QdFd5fa6rRiVNa9wv6euSPluWp0qaLenOpmcjXSJpbnlm0pSG7Usl/V9J84E9u+hvkaRvSrodOEzS/uU5SrdLukjS+qXcK56/JGm8pEvL/k0k/bbEcAbQ8jTnoEpGtl8CLgc+IGkt4P3Ar1qsvj8wqYxAbgU2AbZuod4E4PTSN7afoHo2yULbD5QyZwP7tNDWZbaX2X4M+BOwWYuxR8TQtOLLJnAF8M6yfQv+dhufdwLXSxoHHAm8g+oWQMdI2qWLds8EJgFIWgP4MHCupP2pPtd2B8YC4yR1fDYdZXsc1aMnjpe0Sdm+HnCr7Z1t39jNsTxue1eqQcGXgQllfQ5wgqRXUT1/6bO2d6b67Hy+qY2vAjfa3hGYSTVL1JK67k33EisnwsbbqV8AfBp4Aphj+5kW2xTwGdtXrLSxuoChLzXG3nwb+GUNy8vJvf8iRpIbgM9J2oHqZqkbS9qcajRyPHAUMNP2swCSfkmVqOY1N2R7kaTHS7LaDJhn+/GSjPZvqLM+VXK6nioBTSzbtyzbH6f6LPpFC/FfWP7uQZVIZ5VnK70KuJnqS3pXz1/qsA/wv8v+yyS1/FDAuj4sFwM7SFoHeDWwH9Ut1wGuo/pWcAxdTNF14QrgWElX235R0jbAwy3UuxL4pKRrbL8k6XVUd+AdLeltth8CPl7iAlgEjAN+AxzaQvvPUD2XJCKGMdsPl6tnD6BKDq8DPgQstf1M04d2K84AJgNvoPpMhOpL9zeaz4eX8+sTgD1tPyfpWv72ZfmvLZ4Derahjyttf6SpjzE9PYCeqCMZ2fYfJf0cuAtYSMM3A9vLy/zjZOATPWj3DKpzP7er+lf/M3BIi/W2Ae6U9CLVBQw/kHQk1a3bOy5g+EkpfzLVRROnUD2TpFvl28wsSXcBv+nsvFFE9I0aLsVu/rJ5C/A5qie1bgJcXF5QjZzOknQq1Qf+RKovul2ZSfXTlbWpnpEE1ZfuUySdZ3uppC2AF4ENgSdLItqOV94JvCduAX7Y8WW8nMPfgu6fv9Th+hLr1yS9F9i41U4HNBmVOcwnAGz/E/BPnZWz/Wmqqbpu2R7fsPwy8MXyavQ00OVvjMq5ohPKq3H7VcAr5nNt30AnT1+0fVLT+tsblo9oLh8RQ1/zl02qhLN/+RBfTDU6uqGUvV3SWcBtpfoZ5RERXbX9gqRrgKc6Rja2fytpe+DmMtJaCnyM6lz7P0i6lypp3NKLY/qzpMnA+WX2CuDLth9QN89fKk4u9e4GbgKWtNqv7D79jVjXHUlvpBpJnGb7tAHpdIhpb2/3nDm9vkJytS2Ztnqj8FzaHTUY9j9GLRcu3A4cZvvBuuPpbwM2MrL9CKv5PHdJt1L9dqjRx223/Cko6T1UvydqtND2xM7KR0TUpVwEcSnVBQ/DPhHBELnay/Y7+qCNK6jmWyMiBoVy6uKqTnbtZ/ut/dDfTKD595efb74KuQ5DIhlFRAxHth+n+r3QQPU3aGeCkoxihZz7iYi6DKo7MERExMiUZBQREbVLMoqIiNolGUVERO2SjCIiona5mi5WGDf1nJbLzp0+qR8jiYiRJiOjiIioXZJRRETULskoIiJql2QUERG1SzKKiIjaJRlFRETtkowiIqJ2SUZ9SNJ4SZfWHUdExFCTZBQREbVLMmoiabSk+ySdJekBSedJmiBplqQHJe1eXjdLmifpJknbdtLOepLOlHRbKXdwHccTETEUyHbdMQwqkkYDDwG7AHcDs4H5wNHAQcCRwCTgOdsvSZoAHGv7UEnjgRNtHyjpX4B7bJ8raSPgNmAX28829TcFmALQ1tY2bvHixQNwlJ1bMm1Mn7WVB/VFP1PdAUTfyr3pOrfQ9gIASXcDV9m2pAXAaGBD4GxJWwMG1u6kjf2BgySdWNbXBdqAexsL2Z4BzABob2/PN4OIGJGSjDq3rGH55Yb1l6nes1OAa2xPLCOpaztpQ8Chtu/vvzAjIoaHnDNaPRsCD5flyV2UuQL4jCQBSNplAOKKiBiSkoxWz7eAb0iaR9ejy1Oopu/uLFN9pwxUcBERQ02m6ZrYXgS8vWF9chf7tmmo9uWy/1rKlJ3t54FP9mOoERHDRkZGERFRuySjiIioXZJRRETULskoIiJql2QUERG1y9V0sUJu4RMRdcnIKCIiapdkFBERtUsyioiI2iUZRURE7ZKMIiKidrmaLlYYN/WcFctzp0+qMZKIGGkyMoqIiNolGUVERO2SjCIionZJRhERUbsko4iIqF2SUURE1C7JKCIiapdkFBERtVvtZCTppr4MpIs+vtiwPFrSXf3dZ18rcR9RdxwREYPZaicj23v1ZSBd+OKqi7RGUl13mxgNJBlFRHRDtlevorQUOBA40faBZdsPgDnAfwNH2z6sbB/fWK6Ttj5ClXgEXGb785JOBaYCC4C7gS8BvwFuBPYCHgYOtv28pK2AHwKjgOeAY2zfJ+ks4K/ALsAs2yd00vf6wGlAO2DgZNu/6CymjuO2vX5Z/iBwoO3Jpa+/lHbeAPyT7Ysl3QJsD+G3BVsAAAycSURBVCwEzrb9nab+pwBTANra2sYtXrx41W9+P1kybUyv6ufhfDGAVHcA0bf665zR74B3SFqvrB8OXNBZQUlvBL4J7AuMBXaTdIjtLwDP2x5r+6Ol+NbAD23vCDwFHFq2zwA+Y3sccCLwo4Yu3gTs1VkiKv4ZeNr2GNs7AVd3FVMLx705sDdVkj61bPsCcEM5ju80V7A9w3a77fZRo0a10EVExPDTL8nI9kvA5cAHyvTY+4FfdVF8N+Ba238u9c4D9umi7ELbd5TlucDoMrLZC7hI0h3A6VRJocNFtpd3E+4EqlFVR+xP9jCmRpfYftn2PcBmLZSPiAh6f9ful1g5oa3bsHwB8GngCWCO7Wd62RfAsobl5cCrS/9P2R7bRZ1n+6DfRo3zmus27WuML9MIEREt6u3IaDGwg6R1JG0E7New7zpgV+AYupiiK24D3iVpU0lrAh8pdQFelLR2dwHY/guwUFLH+SlJ2rkHx3AlcFzHiqSNVxHT/0jaXtIawMQW2n8G2KAH8UREjDi9SUa2/Ufg58Bd5e+8hp3LgUuB95a/XTXyKNV5lWuA+cBc2x1TejOAOyWdt4pYPgocLWk+1cUOB/fgOL4GbCzprlL/3auI6QvleG4CHm2h/TuB5ZLmS/rHHsQVETFirNbVdJI2AW63/ea+D2nkam9v95w5c2rrP1fTxRCSafBhpscjo3Kl2c3At/s+nIiIGIl6fAGD7UeAbVanM0m3Aus0bf647X7/Si3pSOCzTZtn2T6us/IRETFwBvSuBLbfMZD9NfX9M+BndfUfERFdy41SIyKidnXdry0GoVyAEBF1ycgoIiJql2QUERG1SzKKiIjaJRlFRETtkowiIqJ2uZpuBBk39Zxu98+dPmmAIomIWFlGRhERUbsko4iIqF2SUURE1C7JKCIiapdkFBERtUsyioiI2iUZRURE7ZKMekjSGyVd3MM6Z0n6YH/FFBEx1CUZdUPSWs3rth+xncQSEdGHhmUykjRa0n1lRPKApPMkTZA0S9KDknYvr5slzZN0k6RtS93Jkn4t6Wrgqk7WR0u6q5RdU9J0SbMl3Snpk2W7JP1A0v2Sfge8vrY3IyJiCBjOtwN6G3AYcBQwGzgC2Bs4CPgiMAl4p+2XJE0A/gU4tNTdFdjJ9hOSJjetj27o42jgadu7SVoHmCXpt8AuwLbADsBmwD3AmZ0FKWkKMAWgra2tb468WDJtzErrMzdYVY3cDigi6jGck9FC2wsAJN0NXGXbkhYAo4ENgbMlbQ0YWLuh7pW2n+hmvcP+wE4N54M2BLYG9gHOt70ceKSMqjplewYwA6C9vd2rcZwREUPecE5GyxqWX25Yf5nquE8BrrE9sYx2rm0o/2xTW83rHQR8xvYVK22U3rd6IUdEjEzD8pxRizYEHi7Lk1ezjSuAYyWtDSBpG0nrAdcDh5dzSpsD7+5tsBERw9lITkbfAr4haR6rP0I8g+p80O3loobTS1szgQfLvnOAm3sfbkTE8CU7pykGi/b2ds+ZM6fP2mu+gGFV2r6yoM/6juhnqjuA6FsjeWQUERGDRJJRRETULskoIiJql2QUERG1SzKKiIjaDecfvY54uTouIoaKjIwiIqJ2SUYREVG7JKOIiKhdklFERNQuySgiImqXq+mGoHFTz+mXdudOz8P1IqIeGRlFRETtkowiIqJ2SUYREVG7JKOIiKhdklFERNQuySgiImqXZBQREbVLMoqIiNolGUVERO2SjHpA0iWS5kq6W9KUsu1oSQ9Iuk3STyX9oGwfJekXkmaX1/+qN/qIiMErtwPqmaNsPyHp1cBsSZcB/wzsCjwDXA3ML2W/B3zH9o2S2oArgO2bGyxJbQpAW1vbKgNYMm0MMzfoi0PpTG4HFBH1SDLqmeMlTSzLWwIfB66z/QSApIuAbcr+CcAOkjrqvlbS+raXNjZoewYwA6C9vd39HH9ExKCUZNQiSeOpEsyetp+TdC1wH52Mdoo1gD1s/3VgIoyIGLpyzqh1GwJPlkS0HbAHsB7wLkkbS1oLOLSh/G+Bz3SsSBo7oNFGRAwhSUatuxxYS9K9wKnALcDDwL8AtwGzgEXA06X88UC7pDsl3QP8w4BHHBExRGSarkW2lwHvbd4uaY7tGWVkNBO4pJR/DDh8YKOMiBiaMjLqvZMk3QHcBSykJKOIiGhdRka9ZPvEumOIiBjqMjKKiIjaJRlFRETtkowiIqJ2OWc0xLR9ZUHdIURE9LmMjCIionZJRhERUTvZuTfnYCHpz8DiGkPYFHisxv5XZbDHB4M/xsEeH7QW42O2DxiIYGJgJBnFCuVuEu11x9GVwR4fDP4YB3t8MDRijL6XabqIiKhdklFERNQuySgazag7gFUY7PHB4I9xsMcHQyPG6GM5ZxQREbXLyCgiImqXZBQREbVLMhohJB0g6X5JD0n6Qif715F0Ydl/q6TRDfv+T9l+v6T31BTfCZLuKU/OvUrSmxv2LZd0R3n9uqb4Jkv6c0Mcf9+w7xOSHiyvT/RHfC3G+J2G+B6Q9FTDvoF4D8+U9CdJd3WxX5K+X+K/U9KuDfsG5D2MGtnOa5i/gDWB3wNvBV4FzAd2aCrzKeAnZfnDwIVleYdSfh3gLaWdNWuI793Aa8rysR3xlfWlg+D9mwz8oJO6rwP+UP5uXJY3riPGpvKfAc4cqPew9LEPsCtwVxf73wf8BhCwB3DrQL6HedX7yshoZNgdeMj2H2y/AFwAHNxU5mDg7LJ8MbCfJJXtF9heZnsh8FBpb0Djs32N7efK6i3Am/o4hl7F1433AFfafsL2k8CVQH/cOaCnMX4EOL8f4uiS7euBJ7opcjBwjiu3ABtJ2pyBew+jRklGI8MWwB8b1v+rbOu0jO2XgKeBTVqsOxDxNTqa6ht0h3UlzZF0i6RD+ji2nsR3aJleuljSlj2sO1AxUqY43wJc3bC5v9/DVnR1DAP1HkaN8giJGFIkfQxoB97VsPnNth+W9FbgakkLbP9+gEP7f8D5tpdJ+iTVKHPfAY6hVR8GLra9vGHbYHgPYwTLyGhkeBjYsmH9TWVbp2UkrQVsCDzeYt2BiA9JE4AvAQfZXtax3fbD5e8fgGuBXQY6PtuPN8R0BjCu1boDFWODD9M0RTcA72ErujqGgXoPo051n7TKq/9fVCPgP1BNzXSc3N6xqcxxrHwBw8/L8o6sfAHDH+j7CxhaiW8XqhP0Wzdt3xhYpyxvCjxINyfu+zG+zRuWJwK3lOXXAQtLnBuX5dfV8W9cym0HLKL84H2g3sOGvkbT9QUM72flCxhuG8j3MK96X5mmGwFsvyTp08AVVFddnWn7bknTgDm2fw38G/Dvkh6iOsn84VL3bkk/B+4BXgKO88rTOwMV33RgfeCi6roKltg+CNgeOF3Sy1Qj/VNt31NDfMdLOojqPXqC6uo6bD8h6RRgdmlumu3uTuL3Z4xQ/bteYLvx1iv9/h4CSDofGA9sKum/gK8Ca5f4fwL8J9UVdQ8BzwFHln0D8h5GvXI7oIiIqF3OGUVERO2SjCIionZJRhERUbsko4iIqF2SUURE1C7JKKILkk6SdOIqyhwiaYeG9Wnlx7kR0QNJRjGilLtLdLm+Gg6hurM5ALa/Yvt3vWwzYsRJMoohS9KkcmPS+ZL+XdJoSVc3PPOorZQ7S9JPJN0KfKuT9a0kXS5prqQbJG3XSV/HSJpd+vqFpNdI2gs4CJhengO0VWn7g6XOfpLmSVpQnuWzTtm+SNLJkm4v+17RX8RIk2QUQ5KkHYEvA/va3hn4LHAacLbtnYDzgO83VHkTsJftEzpZnwF8xvY44ETgR510+Uvbu5W+7gWOtn0T8Gtgqu2xbrixqKR1gbOAw22Pobpdz7EN7T1me1fgx6XPiBEtySiGqn2Bi2w/BtUtY4A9gf8o+/8d2Luh/EVNtzG6yPZySesDe1HdZugO4HRg8076e3sZNS0APkp1z77ubAsstP1AWT+b6uFyHX5Z/s6lul9bxIiWe9PFSPFsF+trAE/ZHruK+mcBh9ieL2ky1T3WeqPjDt/Lyf+HERkZxZB1NXCYpE0AJL0OuIlyg1eq0csNq2rE9l+AhZIOK+1I0s6dFN0AeFTS2qXtDs+Ufc3uB0ZLeltZ/zhw3SqPKmKESjKKIcn23cDXgeskzQf+FfgMcKSkO6k+/D/bYnMfBY4u7dxN54/r/mfgVmAWcF/D9guAqeVCha0a4vsr1V2nLypTey8DP+nBIUaMKLlrd0RE1C4jo4iIqF2SUURE1C7JKCIiapdkFBERtUsyioiI2iUZRURE7ZKMIiKidv8fZYLPKp39qMcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 471x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "corr_df = recid.df.corr().sort_values('black', ascending=False).reset_index()\n",
    "corr_df = corr_df.melt('index', value_vars=('black', 'two_year_recid'))\n",
    "sns.catplot(\n",
    "    x='value',\n",
    "    y='index',\n",
    "    hue='variable',\n",
    "    data=corr_df,\n",
    "    kind='bar',\n",
    "    orient='h'\n",
    ")\n",
    "plt.xlabel('correlation')\n",
    "plt.ylabel('')\n",
    "if PLOTS_DIR is not None:\n",
    "    plt.savefig(os.path.join(PLOTS_DIR, 'recidivism_corr.png'), bbox_inches='tight')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**G-SHAP values.** We compute the G-SHAP values, where $g$ measures the difference in model outputs between groups. In this case, $d$ is the relative difference between group means. Intuitively, this measures the relative difference in predicted recidivism rates between Black and White criminals. In the binary case, relative mean difference is known as *disparate impacts* in the algorithm fairness literature.\n",
    "\n",
    "\\begin{equation}\n",
    "g(f,X,\\Omega) \n",
    "= d(E_{x\\in X_0}[f(x)],E_{x\\in X_1}[f(x)])\n",
    "= \\frac{n_\\text{Black}^{-1}\\sum_{x_i\\in X_\\text{Black}}\\mathbf{1}(f(x_i)=\\text{Recidivate})}\n",
    "{n_\\text{White}^{-1}\\sum_{x_i\\in X_\\text{White}}\\mathbf{1}(f(x_i)=\\text{Recidivate})}\n",
    "-1\n",
    "\\end{equation}\n",
    "\n",
    "Where additional arguments to $g$ indicate group membership; $\\Omega=\\{X_0,X_1\\}$.\n",
    "\n",
    "Although we express the intergroup difference measure as a function of group means, our method easily extends to intergroup difference measures more generally; for example, a difference between group medians or a Wasserstein distance between sample distributions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-18T00:07:48.673602Z",
     "start_time": "2020-08-18T00:06:45.194500Z"
    }
   },
   "outputs": [],
   "source": [
    "g = IntergroupDifference(group=X_test['black'], distance='relative_mean_distance')\n",
    "explainer = gshap.KernelExplainer(clf.predict, X_train, g)\n",
    "gshap_values = explainer.gshap_values(X_test, nsamples=32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Interpretation.** For interpretation, we compare our test data $X$ to a background dataset $Z$ (the shuffled training data) in terms of $g$. Consistent with our previous result, the relative difference in predicted recidivism rates is 63\\%. The relative difference for our shuffled background data is 0\\% (because the background data are shuffled, the model has no way to statistically discriminate between groups). Each G-SHAP value $\\phi_g^j$ is the number of percentage points of this difference (63 percentage points) explained by feature $j$.\n",
    "\n",
    "For intergroup difference explanations more broadly, our model predicts the distance between groups is $d(E_{x\\in X_0}[f(x)],E_{x\\in X_1}[f(x)])$ for our sample, and 0 for the shuffled background data. Each G-SHAP value $\\phi_g^j$ is the amount of this difference explained by feature $j$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-18T00:09:26.610894Z",
     "start_time": "2020-08-18T00:07:48.676989Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Relative mean difference for test data: 0.6339\n",
      "Relative mean difference for (shuffled) background data: 0.0007\n",
      "Sum of G-SHAP values: 0.6541\n"
     ]
    }
   ],
   "source": [
    "g_comparison, g_background = explainer.compare(X_test, bootstrap_samples=1000)\n",
    "print('Relative mean difference for test data: %.4f' % g_comparison)\n",
    "print('Relative mean difference for (shuffled) background data: %.4f' % g_background)\n",
    "print('Sum of G-SHAP values: %.4f' % gshap_values.sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Results.** This plot tells us which features explain the relative difference in predicted recidivism rates between Black and White criminals. We see that the most important features are number of prior convictions, age, and race."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-18T00:09:26.929020Z",
     "start_time": "2020-08-18T00:09:26.612909Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcYAAAEGCAYAAAD/ijWdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de5gdVZ3u8e8rOAQBASFyYDS0yE1ACNBcZLgKoyCDwAFkwFuQA14QdDz4yJGjg3gZEOc4o+AlKgYE5eYAHjwDghDEgEKH3IGAJqADPk4LgsBgIMl7/tirpbLZ3b27sy/d6ffzPPvpqlVr1fpV5en+ZVXVriXbRERERM3Luh1ARETEWJLEGBERUZHEGBERUZHEGBERUZHEGBERUbF2twOI1bfpppu6p6en22FERIwbs2fP/oPtyY22JTGuAXp6eujr6+t2GBER44akRwbblkupERERFRkxBrt//NJuhxARMWKzL3hPW/abEWNERERFEmNERERFEmNERERFEmNERERFEmNERETFGp0YJZ0r6ZBuxzEaknokndjtOCIiJpo1NjFKWsv2p23fMpI27YxphHqAJMaIiA4bl4mxjKYekHS5pPslXSPpFZIelnS+pHuB4yTNkHRsaXOwpDmSFki6WNI6pby+zRmS7pM0X9IVQ8SwvqTvlv3Nl3RMKT+hlC2UdH6l/jOV5WMlzSjLMyR9RdKdkpYMxAucB+wnaa6kf2jQ/6mS+iT19ff3r/Y5jYiImvH8Bf/tgJNtz5J0MfChUv647d0AJB1afk4CZgAH235Q0qXAB4F/adDmMeB1tpdJ2miI/j8FPGX7jaXdxpK2AM4Hdgf+CPxE0lG2rxvmWDYH9gW2B34EXAOcBZxp++8aNbA9HZgO0Nvb62H2HxERTRqXI8bit7ZnleXLqCUWgCsb1N0OWGr7wbJ+CbB/ZXu1zXzgcknvApYP0f8hwEUDK7b/COwBzLTdb3s5cHldP4O5zvZK2/cBmzVRPyIi2mQ8J8b6UdLA+rOj2Fe1zeHUEt5uwD2SWjWqrsY7qW7bssqyWtRfRESMwnhOjFMkvaksnwj8fIi6i4EeSVuX9XcDt9dXkvQy4LW2bwM+AWwIrD/IPm8GTqu03Ri4GzhA0qblQZ4TKv38XtIbSh9HN3F8TwMbNFEvIiJaaDwnxsXAaZLuBzYGvj5YRdt/Bk4Crpa0AFgJfKNB1bWAy0qdOcBXbD85yG4/B2xcHrKZBxxk+3fU7g3eBswDZtu+vtQ/C7gBuBP4XRPHNx9YIWleo4dvIiKiPcbzwzfLbb+rrqynumJ7WmX5p8Cu9Tux3VNZfoEX71UOyfYzwHsblP8A+EGD8muoPVRTXz6tbn39SixvbiaWiIhonfE8YoyIiGi5cTlitP0wsFMn+pJ0EvCRuuJZtk9rVD8iIsY32fkK3HjX29vrvr6+bocRETFuSJptu7fRtlxKjYiIqEhijIiIqEhijIiIqBiXD99Ea/3m3Dd2O4SINdKUTy/odggxChkxRkREVCQxRkREVCQxRkREVCQxRkREVCQxRkREVCQxRkREVCQxRkREVCQxdoCk6yTNlrRI0qml7GRJD0q6W9K3JF1YyidL+qGke8rnb7obfUTExJIv+HfG+2w/IWld4B5JPwY+BewGPA3cSm1iY4B/Bb5s++eSpgA3AW+o32FJsKcCTJkypQOHEBExMSQxdsYZko4uy68F3g3cbvsJAElXA9uW7YcAO0gaaPtKSeuXiZH/wvZ0YDrUZtdoc/wRERNGEmObSTqQWrJ7k+3/kjQTeIAGo8DiZcDetv/cmQgjIqIq9xjbb0PgjyUpbg/sDawHHCBpY0lrA8dU6v8EOH1gRdLUjkYbETHBJTG2343A2pLuB84DfgE8CnwBuBuYBTwMPFXqnwH0Spov6T7gAx2POCJiAsul1DazvQw4rL5cUp/t6WXEeC1wXan/B+D4zkYZEREDMmLsnnMkzQUWAkspiTEiIrorI8YusX1mt2OIiIiXyogxIiKiIiPGyCzjEREVGTFGRERUJDFGRERUJDFGRERU5B5j8Ddf7fwEHrNOn9XxPiMimpERY0REREUSY0REREUSY0REREUSY0REREUSY0REREUS42qS1CNpYYPymZJ6R7G/aZIubE10ERExUkmMERERFUmMrbG2pMsl3S/pGkmvqG6U9HVJfZIWSfpMpXwPSXdKmifpbkkb1LU7XNJdkjbt1IFEREx0SYytsR3wNdtvAP4EfKhu+9m2e4GdgQMk7Szpr4ArgY/Y3gU4BHhuoIGko4GzgLeVyYtXIenUkmz7+vv723NUERETUBJja/zW9sCrXC4D9q3b/g5J9wJzgB2BHagl09/ZvgfA9p9sLy/13wx8Ajjc9h8bdWh7uu1e272TJ09u8eFERExcSYyt4cHWJb0OOBM42PbOwI+BScPs79fABsC2rQwyIiKGl8TYGlMkvaksnwj8vLLtlcCzwFOSNgMOK+WLgc0l7QEgaQNJA++ufQQ4BrhU0o5tjz4iIv4iibE1FgOnSbof2Bj4+sAG2/OoXUJ9APg+MKuUPw8cD3xV0jzgZiojSdsPAO8Erpb0+g4dR0TEhJfZNVaT7YeB7RtsOrBSZ9ogbe8B9q4rnlE+2J5D7X5kRER0SEaMERERFUmMERERFbmUGpk0OCKiIiPGiIiIiiTGiIiIiiTGiIiIiiTGiIiIijx8E9y+/wFDbj/gZ7d3KJKIiO7LiDEiIqIiiTEiIqIiiTEiIqIiiTEiIqIiiTEiIqKibYlRUo+khQ3Kz5V0yDBtz5F0ZrtiaxdJdw6z/f9J2qhT8URExMh1/Osatj/d6T5bRZIA2V7ZaLvtfYZqb/ttbQksIiJapt2XUteS9C1JiyT9RNK6kmZIOhZA0tskPSBptqSvSLqh0nYHSTMlLZF0xmAdSFpP0o8lzZO0UNLxpfxhSZuW5V5JM8vyOZK+J+kuSQ9JOqWyr49LukfSfEmfKWU9khZLuhRYCHxK0gWVNtMkXViWnyk/N5f0M0lzS0z7NYjpY2XbQkkfrfR1f/05G+S4T5XUJ6mvv79/ZP8qERExqHYnxm2Ai2zvCDwJHDOwQdIk4JvAYbZ3BybXtd0eeCuwJ/CPkl4+SB+HAo/Z3sX2TsCNTcS1M/Bm4E3ApyVtIektJd49ganA7pL2rxzH18pxfA04urKv44Er6vZ/InCT7anALsDc6kZJuwMnAXtRm6j4FEm7VvpqeM6qbE+33Wu7d/Lk+lMXERGj1e7EuNT2QFKYDfRUtm0PLLG9tKz/oK7tj20vs/0H4D+BzQbpYwHwt5LOl7Sf7aeaiOt628+Vfd9GLRm+pXzmAPeW+LYp9R+x/QsA2/3AEkl7S9qk1Kuft+ke4CRJ5wBvtP103fZ9gWttP2v7GeDfgP3KtqHOWUREtFm7E+OyyvIKRnZPs6m2th8EdqOWID8naeAe5nJePL5J9c0arAv4J9tTy2dr298p25+tq38F8A5qo7lrba+yP9s/A/YHHgVmSHrP4If5EqtzziIiYjV18+sai4GtJPWU9eNHsxNJWwD/Zfsy4AJqSRLgYWD3slx/OfJISZPKiO9AaiO8m4D3SVq/7PevJb16kG6vBY4ETuCll1GRtCXwe9vfAr5diWnAHcBRkl4haT1ql2bvaO6IIyKinbo2GrH9nKQPATdKepZachqNNwIXSFoJvAB8sJR/BviOpM8CM+vazKd2CXVT4LO2HwMek/QG4K7aw6c8A7yL2qitPvY/Srof2MH23Q1iOhD4uKQXyn7eU9f+XkkzgIG237Y9p/KfhIiI6BLVXQXsbOfS+rafKV+DuAh4yPaX29znOcAztr/Uzn46qbe31319faNun9k1ImKikTTbdm+jbd1+880pkuYCi4ANqT2lGhER0TVdfbCjjA6bGiGW+4E/bbDpYNuPj6DPc5qtGxERE8+4eeKxJL+p3Y5jTZRLpRERL+r2pdSIiIgxJYkxIiKiIokxIiKiIokxIiKioqmHbyS9HvgP28skHUjtJdyX2n6yncFFZ1z4P//vX5Y//M9HdDGSiIjua3bE+ENghaStgenAa4Hvty2qiIiILmk2Ma60vZzaOz2/avvjwObtCysiIqI7mk2ML0g6AXgvMDCZ8GDzI0ZERIxbzSbGk6hN6vt520slvQ74XvvCioiI6I6mEqPt+4BPUJvAF9tLbZ/fzsAmEkkHSrph+JoREdFuTSVGSUcAc4Eby/pUST9qZ2ARERHd0Oyl1HOAPYEnAWzPBbZqU0zjkqQeSQ9ImiHpQUmXSzpE0ixJD0nas3zukjRH0p2Stmuwn/UkXSzp7lLvyG4cT0TERNX0wze2n6orW9nqYNYAWwP/DGxfPicC+wJnAp8EHgD2s70r8GngCw32cTZwq+09gYOoTcK8Xn0lSadK6pPU19/f35aDiYiYiJqdXWORpBOBtSRtA5wB3Nm+sMatpbYXAEhaBPzUtiUtAHqozTl5STmHpvGTvW8B3i7pzLI+CZgC3F+tZHs6te+U0tvb273ZpiMi1jDNjhhPB3YEllH7Yv9TwEfbFdQ4tqyyvLKyvpLaf0I+C9xmeyfgCGpJr56AY2xPLZ8ptu9vUC8iItpg2BGjpLWAH9s+iNplvhi9DYFHy/K0QercBJwu6fQy2tzV9pyORBcREcOPGG2vAFZK2rAD8azpvgj8k6Q5DP6fks9Su8Q6v1yO/WyngouIiObvMT4DLJB0M/DsQKHtM9oS1Thk+2Fgp8r6tEG2bVtp9r/L9pnAzLL8HPD+NoYaERFDaDYx/lv5RERErNGaSoy2L2l3IBEREWNBs/MxLqX29YJV2M6X/CMiYo3S7KXU3sryJOA44FWtDye6IZMTR0S8qNmXiD9e+Txq+1+Aw9scW0RERMc1eyl1t8rqy6iNIJsdbUZERIwbzSa3f64sLweWAu9ofTgRERHd1WxiPNn2kmpBmaw41gCff9exAJx92TVdjiQiovuafVdqo7+Y+SsaERFrnCFHjJK2p/by8A0l/ffKplfS+AXYERER49pwl1K3A/4O2IjabBADngZOaVdQERER3TJkYrR9PXC9pDfZvqtDMUVERHRNsw/fzJF0GrXLqn+5hGr7fW2JKiIiokuaffjme8B/A94K3A68htrl1AlH0hmS7pd0+SDbD5R0Q6fjioiI1mh2xLi17eMkHWn7EknfB+5oZ2Bj2IeAQ2z/R7cDiYiI1mt2xPhC+fmkpJ2ozUT/6vaENHZJ+gawFfDvks6WdLGkuyXNkXRkg/qvknSdpPmSfiFp51J+Tmk7U9ISSWeU8nMlfbTS/vOSPtKp44uIiOYT43RJGwOfAn4E3EdtNvoJxfYHgMeAg4D1gFtt71nWL5C0Xl2TzwBzbO8MfBK4tLJte2qXpvcE/lHSy4GLgfcASHoZ8PfAZY1ikXSqpD5Jff39/a06xIiICa/Z+Ri/XRZvpzZiCngL8HZJZ5b1ScCUujr7AscA2L5V0iaSXlm2/dj2MmCZpP8ENrP9sKTHJe0KbEYtqT7eqHPb04HpAL29vS+ZEiwiIkan2ZeIbwZ8AdjC9mGSdgDeZPs7bY1ubBNwjO3FqxTWzlUzllWWV/Div8W3gWnUHna6eDVjjIiIEWr2UuoM4CZgi7L+IPDRQWtPDDcBp0sSQBnl1bsDeGfZfiDwB9t/Gma/1wKHAnuUPiIiooOaTYyb2r4KWAlgezm1Uc5E9lng5cB8SYvKer1zgN0lzQfOA9473E5tPw/cBlxle6Kf44iIjmv26xrPStoEMICkvYGn2hbVGGa7p7L6/gbbZwIzy/ITwFEN6pxTt77TwHJ56GZv4LgWhBsRESPUbGL8GLWnUV8vaRYwGTi2bVFNUOXe7Q3AtbYf6nY8ERET0XCza0yx/Rvb90o6gNpLxQUstv3CUG1j5GzfR576jYjoquHuMV5XWb7S9iLbC5MUIyJiTTXcpVRVljOSWUOdfVnmnI6IGDDciNGDLEdERKyRhhsx7iLpT9RGjuuWZcq6bb9y8KYRERHjz3ATFa/VqUAiIiLGgma/4B8RETEhJDFGRERUJDFGRERUJDFGRERUJDFGRERUJDFGRERUdDwxSrqzA318srLcI2lhu/tstRL3id2OIyJioul4YrS9Twe6+eTwVZojqdkZSFqtB0hijIjosG6MGJ+RdKCkGyplF0qaJulQSVdXylep12BfJ0haIGmhpPNL2XnU3tIzV9Llpepakr4laZGkn0hat9R9vaQbJc2WdIek7Uv5DEnfkPRL4IuD9L2+pO+W/udLOmawmAaOu7J8rKQZlb6+IulOSUskDUzndR6wXzmOf2j+DEdExOoYa/cYbwH2krReWT8euKJRRUlbAOcDbwamAntIOsr2WcBztqfafmepvg1wke0dgSeBY0r5dOB027sDZwJfq3TxGmAf2x8bJNZPAU/ZfqPtnYFbB4upiePeHNgX+DtqCRHgLOCOchxfbnD8p0rqk9TX39/fRBcREdGMMZUYbS8HbgSOKJcwDweuH6T6HsBM2/2l3eXA/oPUXWp7blmeDfRIWh/YB7ha0lzgm9QS1ICrba8YItxDgIsqsf9xhDFVXWd7ZZmPcbMm6mN7uu1e272TJ09upklERDShW/fPlrNqUp5UWb4C+DDwBNBn++kW9LessrwCWLf0/6TtqYO0ebYF/VZVZyeZVLetGp+IiIiu6daI8RFgB0nrSNoIOLiy7XZgN+AUBrmMWtwNHCBpU0lrASeUtgAvSHr5UAHY/hOwVNJxAKrZZQTHcDNw2sCKpI2Hien3kt4g6WXA0U3s/2lggxHEExERLdCNxGjbvwWuAhaWn3MqG1cANwCHlZ+D7eR31O7D3QbMA2bbHrjsOh2YX3n4ZjDvBE6WNA9YBBw5guP4HLBxechmHnDQMDGdVY7nTuB3Tex/PrBC0rw8fBMR0TmyOzf/sKRNgHttb9mxTieA3t5e9/X1dTuMiIhxQ9Js272NtnVsxFie2LwL+FKn+oyIiBipjj18Y/sxYNvRtC3fJ1ynrvjdthesdmDD930S8JG64lm2T2tUPyIixrduPZU6Irb36mLf3wW+263+IyKis8bU9xgjIiK6LYkxIiKiIokxIiKiIokxIiKiIokxIiKiIokxIiKiIokxIiKiIokxIiKiIokxIiKiIokxIiKiYtwkRkl3tnBfx0m6X9JtQ9TpkbSwVX2OlKSNJH2oW/1HRExU4yYx2t6nhbs7GTjF9kEt3GerbQQkMUZEdNi4SYySnpF0oKQbKmUXSpom6VBJV1fKV6lXt59PA/sC35F0gaS1ys97JM2X9P4m41lL0pfKRMXzJZ1eyg+WNEfSAkkXS1qnlD8sadOy3CtpZlk+p9SbKWmJpDNKF+cBr5c0V9IFDfo/VVKfpL7+/v5mQo6IiCaMm8Q4jFuAvSStV9aPB65oVNH2uUAf8E7bH6c2enzK9h7AHsApkl7XRJ+nAj3AVNs7A5dLmgTMAI63/UZqs5d8sIl9bQ+8FdgT+EdJLwfOAn5te2qJs/44ptvutd07efLkJrqIiIhmrBGJ0fZy4EbgCElrA4cD1zfZ/C3AeyTNBX4JbAJs00S7Q4Bvlr6x/QSwHbDU9oOlziXA/k3s68e2l9n+A/CfwGZNxh4RES02LuZjrFjOqsl8UmX5CuDDwBNAn+2nm9yngNNt37RKodQz+jAbqsY+qW7bssryCsbfv0tExBpjvI0YHwF2kLSOpI2Agyvbbgd2A05hkMuog7gJ+GC5fImkbSuXZIdyM/D+MkJF0quAxUCPpK1LnXeXuAAeBnYvy8c0sf+ngQ2aOoKIiGiZ8ZQYbfu3wFXAwvJzTmXjCuAG4LDys1nfBu4D7i1fz/gmzY3Yvg38BpgvaR5wou0/AycBV0taAKwEvlHqfwb4V0l91EaFQ7L9ODCrPNzzkodvIiKiPWS72zEMS9ImwL22t+x2LGNRb2+v+/r6uh1GRMS4IWm27d5G28b8iFHSFsBdwJe6HUtERKz5xvxDHrYfA7YdTVtJvwTWqSt+t+0FI9jHW4Hz64qX2j56NDFFRMTYNuYT4+qwvVcL9nETtQd0IiJiAhjzl1IjIiI6KYkxIiKiIokxIiKiIolxgrvq6j27HUJExJiSxBgREVGRxBgREVGRxBgREVGRxBgREVGRxBgREVGRxNghkraQdM0I28yQdGy7YoqIiJdKYmyDgTkaq+u2H7OdJBcRMcYlMVZI6pH0QBmpPSjpckmHSJol6SFJe5bPXZLmSLpT0nal7TRJP5J0K/DTBus9Zb5HJK0l6QJJ90iaL+n9pVySLpS0WNItwKu7djIiIiaoNfol4qO0NXAc8D7gHuBEYF/g7cAngfcA+9leLukQ4AvAMaXtbsDOtp+QNK1uvafSx8nAU7b3kLQOtQmJfwLsCmwH7ABsRm0C5YsbBSnpVOBUgClTprTmyCMiIomxgaUD01JJWgT81LYlLQB6gA2BSyRtAxh4eaXtzbafGGJ9wFuAnSv3DzcEtgH2B35gewXwWBltNmR7OjAdahMVj+I4IyKigSTGl1pWWV5ZWV9J7Xx9FrjN9tFlFDizUv/Zun3Vrw8QcHqZ0urFQultows5IiJaJfcYR25D4NGyPG2U+7gJ+KCklwNI2lbSesDPgOPLPcjNgYNWN9iIiBiZJMaR+yLwT5LmMPoR97ep3T+8tzyQ882yr2uBh8q2S4G7Vj/ciIgYCdm5PTXe9fb2uq+vb1Rtr7p6T95x3N0tjigiYmyTNNt2b6NtGTFGRERUJDFGRERUJDFGRERUJDFOcLm/GBGxqiTGiIiIiiTGiIiIiiTGiIiIiiTGCW6Xa24avlJExASSxBgREVGRxBgREVGRxBgREVGRxBgREVGRxBgREVGRxBgREVHRtcQo6c4O99cr6SsjqD9D0rHtjGmY/o+StEO3+o+ImKi6lhht79Ph/vpsn9HJPlfTUUASY0REh3VzxPiMpAMl3VApu1DSNEmHSrq6Ur5KvUH2dYGkRZJukbSnpJmSlkh6e/0+JB0gaW75zJG0gWoulLRY0i3Aq4eJfw9Jd0qaJ+nuso9Jkr4raUHZ70Gl7jRJF1ba3iDpwErsny/7+YWkzSTtA7wduKDE+PoG/Z8qqU9SX39/f3MnPSIihjVW7zHeAuwlab2yfjxwxRD11wNutb0j8DTwOeBvgaOBcxvUPxM4zfZUYD/guVJ3O2qjtPcAg45oJf0VcCXwEdu7AIeUfZwG2PYbgROASyRNGuZY1wN+UfbzM+AU23cCPwI+bnuq7V/XN7I93Xav7d7JkycP00VERDRrTCZG28uBG4EjJK0NHA5cP0ST50t9gAXA7bZfKMs9DerPAv6PpDOAjUp/+wM/sL3C9mPArUP0tx3wO9v3lHj/VPaxL3BZKXsAeATYdpjDfR4YGA3PHiTeiIjokG4nxuV1MVRHV1cA7wDeDPTZfnqI/bxg22V5JbAMwPZKYO36yrbPA/4HsC4wS9L2oz6C5gx1nNXYV9Ag3oiI6JxuJ8ZHgB0krSNpI+Dgyrbbgd2AUxj6MuqISXq97QW2zwfuAbandhnzeElrSdocOGiIXSwGNpe0R9nfBmVkewfwzlK2LTCl1H0YmCrpZZJeC+zZRJhPAxuM6gAjImLUujk6se3fSroKWAgsBeZUNq4oD8tMA97b4r4/Wh6MWQksAv6d2iXNNwP3Ab8B7hoi8OclHQ98VdK61O4vHgJ8Dfi6pAXURonTbC+TNKsc333A/cC9TcR4BfCtcrn32Eb3GSMiovX04lW8DnYqbQLca3vLjne+Burt7XVfX9+o2u5yzU3MO/atLY4oImJskzTbdm+jbR2/lCppC2qjsS91uu+IiIjhdPxSannic7gnNRuS9Etgnbrid9tesNqBDd7ntcDr6oo/YTsz/EZErIHG1ROQtvfqQp9Hd7rPTspl1IiIVXX7qdSIiIgxpSsP30RrSeqn9tWX0doU+EOLwmm38RLreIkTEmu7jJdYx0uc0NpYt7Td8LVhSYyBpL7Bns4aa8ZLrOMlTkis7TJeYh0vcULnYs2l1IiIiIokxoiIiIokxgCY3u0ARmC8xDpe4oTE2i7jJdbxEid0KNbcY4yIiKjIiDEiIqIiiTEiIqIiiXENJ+lQSYsl/UrSWQ22ryPpyrL9l5J6Ktv+VylfLKmtr8gZbZySeiQ9J2lu+XyjnXE2Gev+ku6VtFzSsXXb3ivpofJp9awxrY51ReW8/qjLcX5M0n2S5kv6qaQtK9vG2jkdKtaOndMmY/2ApAUlnp9L2qGyrWO//6sTa1v+BtjOZw39AGsBvwa2Av4KmAfsUFfnQ8A3yvLfA1eW5R1K/XWovSv218BaYzDOHmDhGDunPcDOwKXUpgwbKH8VsKT83LgsbzwWYy3bnhlD5/Qg4BVl+YOVf/+xeE4bxtrJczqCWF9ZWX47cGNZ7tjvfwtibfnfgIwY12x7Ar+yvcT289TmeDyyrs6RwCVl+RrgYEkq5VfYXmZ7KfArmptgudNxdtqwsdp+2PZ8avN9Vr0VuNn2E7b/CNwMHDpGY+2kZuK8zfZ/ldVfAK8py2PxnA4Wa6c1E+ufKqvrAQNPY3by9391Y225JMY1218Dv62s/0cpa1jH9nLgKWCTJtuOhTgBXidpjqTbJe3XphhHEms72o7G6vY3SVKfpF9IOqq1oa1ipHGeTG1y8dG0XV2rEyt07pxCk7FKOk3Sr4EvAmeMpG0LrU6s0OK/AeNqdo2IBn4HTLH9uKTdgesk7Vj3v8sYnS1tPyppK+BWSQts/7qbAUl6F9ALHNDNOJoxSKxj7pzavgi4SNKJwP8G2n6fdrQGibXlfwMyYlyzPQq8trL+mlLWsI6ktYENgcebbNv1OMulnscBbM+mdp9iVPN9tjDWdrQdjdXqz/aj5ecSYCawayuDq2gqTkmHAGcDb7e9bCRtW2h1Yu3kOW061oorgIFR7Jg8rxV/ibUtfwPadTM1n+5/qF0RWELt5vnADe0d6+qcxqoPtVxVlndk1ZvvS2jfwzerE+fkgbio3bh/FHhVN89ppe4MXvrwzVJqD4lsXJbHaqwbA+uU5U2Bh6h7GKLD//67UvuDt01d+Zg7p0PE2rFzOoJYtxM5OFAAAALFSURBVKksHwH0leWO/f63INaW/w1oy0HmM3Y+wNuAB8sv6tml7Fxq/5MFmARcTe3m+t3AVpW2Z5d2i4HDxmKcwDHAImAucC9wxBg4p3tQu0fyLLXR96JK2/eVY/gVcNJYjRXYB1hQ/kAtAE7ucpy3AL8v/85zgR+N4XPaMNZOn9MmY/3Xyu/PbVSSUSd//1cn1nb8Dcgr4SIiIipyjzEiIqIiiTEiIqIiiTEiIqIiiTEiIqIiiTEiIqIiiTEiXkLSZpK+L2mJpNmS7pJ0dIN6e6s228lcSfdLOqeUT5N0YV3dmZJ6K+tTJVnSoXX1BmagWCjpakmvaNNhRjSUxBgRqygvZ78O+JntrWzvTu2lCo1ehn0JcKrtqcBOwFUj6OoE4OflZ9Vztqfa3gl4HvjASI8hYnXkXakRUe/NwPO2/zKvne1HgK82qPtqau+qxPYK4L5mOijJ9zjgb4E7JE2y/ecGVe+gNi1WRMdkxBgR9Xak9gaRZnwZWCzpWknvlzSpsu34yuSxc6m9UHvAPsBS116gPRM4vH7H5Z24h1F7S0xExyQxRsSQJF0kaZ6ke+q32T6XWsL7CXAicGNl85XlkujUcqm1r7LtBGovgqb8rF5OXbck0j7gN8B3Wnc0EcPLK+EiYhWSDgY+bfuAStmm1BLVbdRekv2Y7bfVtVsb6Ae2pvaS517bH65snwmcCcyh9n7W5cAKQNTm1tzc9tOSnrG9fvuOMGJoGTFGRL1bqU2o+8FK2SsAbJ9URoBvA5B0eLlfCLANtUT35DD7PxiYb/u1tntsbwn8EHjJU68R3ZDEGBGrcO0y0lHAAZKWSrqb2tOnn2hQ/d3U7jHOBb4HvLM8hDOUE4Br68p+yEufTo3oilxKjYiIqMiIMSIioiKJMSIioiKJMSIioiKJMSIioiKJMSIioiKJMSIioiKJMSIiouL/A7Jlnw7bABTpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df = pd.DataFrame({'Features': X.columns, 'G-SHAP': gshap_values})\n",
    "df = df.sort_values('G-SHAP', ascending=False)\n",
    "sns.barplot(y='Features', x='G-SHAP', data=df)\n",
    "if PLOTS_DIR is not None:\n",
    "    plt.savefig(os.path.join(PLOTS_DIR, 'recidivism_gshap.png'), bbox_inches='tight')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-18T00:09:26.941998Z",
     "start_time": "2020-08-18T00:09:26.931178Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Features</th>\n",
       "      <th>G-SHAP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>priors_count</td>\n",
       "      <td>0.354922</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>age</td>\n",
       "      <td>0.251491</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>black</td>\n",
       "      <td>0.034307</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>high_supervision</td>\n",
       "      <td>0.009620</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>male</td>\n",
       "      <td>0.004049</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>felony</td>\n",
       "      <td>0.002129</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>juv_other_count</td>\n",
       "      <td>0.000502</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>juv_fel_count</td>\n",
       "      <td>-0.000277</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>married</td>\n",
       "      <td>-0.001286</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>juv_misd_count</td>\n",
       "      <td>-0.001338</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           Features    G-SHAP\n",
       "5      priors_count  0.354922\n",
       "1               age  0.251491\n",
       "7             black  0.034307\n",
       "9  high_supervision  0.009620\n",
       "0              male  0.004049\n",
       "6            felony  0.002129\n",
       "4   juv_other_count  0.000502\n",
       "2     juv_fel_count -0.000277\n",
       "8           married -0.001286\n",
       "3    juv_misd_count -0.001338"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}