Lecture3_OverfittingRevisitedAndValidation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to Tackle a Machine Learning Competition\n",
    "\n",
    "\n",
    "## Lecture 3: Overfitting revisited and Validation\n",
    "\n",
    "Arianna Di Bernardo, arianna.dibernard@edu.unito.it\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_______________________________________________________________________________________________________________________________\n",
    "## Outline\n",
    "Click on these links to jump to a particular section of this notebook.\n",
    "- [1. Introduction, packages, prerequisites](#1)\n",
    "- [2. Overfitting review](#2)\n",
    "    - [2.1 Overfitting causes](#2-1)\n",
    "    - [2.2 How to combact overfitting?](#2-2)\n",
    "- [3. Validation](#3)\n",
    "    - [3.1 Train, Validation, Test sets](#3-1)\n",
    "    - [3.2 Validation strategies](#3-2)\n",
    "    - [3.3 Data Splitting strategies](#3-3)\n",
    "    - [3.4 Problems occurring during validation](#3-4)\n",
    "- [4. References and additional materials](#4)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_______________________________________________________________________________________________________________________________\n",
    "<a name='1'></a>\n",
    "# 1. Introduction, packages and prerequisites"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook we review some basic but essential concepts in machine learning: _**overfitting**_ and _**validation**_. \n",
    "\n",
    "In fact, when you design a ML algorithm, the main challenge is to avoid overfitting, a phenomenon that causes poor performance. Whit validation, one can keep track of the quality of the model, notice when it overfits and prevent the resulting bad solution.\n",
    "\n",
    "In this lesson, we'll make use of the following packages:\n",
    "- [numpy](https://docs.scipy.org/doc/numpy/), a popular library for scientific computing\n",
    "- [matplotlib](https://matplotlib.org/3.1.1/contents.html), a plotting library compatible with numpy\n",
    "- [pandas](https://pandas.pydata.org/docs/) is what we'll use to manipulate our data\n",
    "- [sklearn](https://scikit-learn.org/stable/index.html) will be used to put into practice what we have seen in the theory "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import sklearn as sk\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before you start with this notebook, make sure you are practical with basic commands in scikit learn and with the usual ML algorithms. \n",
    "\n",
    "Also we'll make use of some concepts seen in the first two notebooks of the series _How to Tackle a Machine Learning Competition:_ Lecture1: FeatureProcessing and Lecture2: EDA\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_______________________________________________________________________________________________________________________________\n",
    "<a name='2'></a>\n",
    "# 2. Overfitting review\n",
    "\n",
    "**Overfitting** means _fitting the data more than is warranted._\n",
    "\n",
    "It is the phenomenon where fitting the observed data well no longer indicates that we will get a good performance, and may actually lead to the opposite effect: in fact, many Machine Learning models have the ability to memorize the training set, which can lead to poor efficiency on unseen data.\n",
    "\n",
    "In those cases, we obtain a model that perfectly approximates your data, but it doesn't have the ability to generalize to new examples.\n",
    "\n",
    "The most important part when you design a model is recognizing overfitting and understanding the underlying causes, in order to combat it.  Remember: *Overfitting is your worst enemy!* \n",
    "\n",
    "\n",
    " **Underfitting** is the opposite problem: it occurs when we fit data not enough, leading to bad approximation in addition to poor generalization.\n",
    " \n",
    " ![overfitting_2.png](https://github.com/MLJCUnito/ProjectX2020/blob/master/HowToTackleAMLCompetition/img/Lecture3/overfitting_2.png)\n",
    " \n",
    "> So, to choose the best model, we basically want to avoid underfitting on the one side and overfitting on the other.\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='2-1'></a>\n",
    "## 2.1 Overfitting causes\n",
    "\n",
    "Let's start by identifying the causes of overfitting:\n",
    "\n",
    "- Dataset size\n",
    "\n",
    "- Stochastic noise\n",
    "\n",
    "- Deterministic noise\n",
    "\n",
    "**_Dataset size_** is the first culprit of overfitting: the fewer examples for training, the more models can fit our data. Consider an extreme example, a sample with only one training point: any model will be able to \"explain\" it!  As the training set size increases, fewer models are able to explain them, leading to decreased overfitting.\n",
    "\n",
    "**_Stochastic noise_** is due to measurement errors or random fluctuations in your data. It is stochastic in the sense that it is random: each time we generate the sample, the stochastic noise is different. We cannot directly intervene to limit this kind of noise, it doesn't depend on the model and therefore on our choices.\n",
    "\n",
    "**_Deterministic noise,_** in contrast, depends on the target function complexity and it is fixed for a given sample. It is the part of the target function which cannot be modeled: the learning algorithm shouldn't try to fit them, however it can't distinguish noise from the real signal.\n",
    "\n",
    "![overfit%20depends%20on.PNG](https://github.com/MLJCUnito/ProjectX2020/blob/master/HowToTackleAMLCompetition/img/Lecture3/overfit%20depends%20on.PNG)\n",
    "\n",
    "One can think to contrast noise (stochastic and deterministic) by using a complex model. But is that really the case? _**NO!**_\n",
    "\n",
    "If the level of noise is high and the dataset is small, using a complex model makes it worse. In fact, _a complex model makes use of its additional degrees of freedom to fit noise,_ which can result in overfitting.\n",
    "\n",
    "\n",
    "Let's understand this concept in a very simple example of a one dimensional regression problem with polynomials features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- ### Example: polynomial fitting with linear regression ###"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.pipeline import Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#generate the target function, ie the function to predict \n",
    "#to predict a function starting from data points is a ML problem, generally solved with linear regression\n",
    "\n",
    "def target(X):\n",
    "    return np.sin(2 *np.pi * X)        #deterministic noise is intrinsic in the target function complexity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "degrees = [1, 3, 50]  #degrees of the models we will use"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#generate the sample (the training set)\n",
    "\n",
    "n_samples = 30    #dataset size\n",
    "\n",
    "X = np.sort(np.random.rand(n_samples))    #generate random points\n",
    "y = target(X) + np.random.randn(n_samples) * 0.1   #generate samples and add noise (what happens without noise?)\n",
    "\n",
    "X_test = np.linspace(0, 1, 100)   #generate random points for testing our model. What will be the predictions for that points?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAykAAAE8CAYAAADE7v5rAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gUVffA8e/sbpJNhzQ6hF5Dr9KLVCmKUmxYqCoivlhfCwLqawP1Z0FFBVtAsVEEadJBegm9hV7Se7LZ3fn9MUkEA2nbssn5PA8PJLs7c3YJhzlz77lXUVUVIYQQQgghhCgtdK4OQAghhBBCCCGuJ0WKEEIIIYQQolSRIkUIIYQQQghRqkiRIoQQQgghhChVpEgRQgghhBBClCpSpAghhBBCCCFKFSlShBBCCCGEEKWKFCluRFGUaEVRMhRFSVEUJVFRlK2KokxUFMUt/x4VRZmpKMpBRVHMiqJMd3U8QpQnZTCf/KUoSoyiKMmKouxXFGWoq2MSoqwqg/kj9/2k5vxa9a/HpyqKckVRlCRFUb5SFMXLVbGWJ275w1TODVZV1R+oBfwPeA740hEnUhRF74jjXuck8Cyw3MHnEULcXFnKJ1OAKqqqBgDjge8URani4HMKUZ6VpfwB2vvxy/nV97pz9wOeB3oD4UAd4DUnxFPuSZHiplRVTVJVdQkwEhijKEozAEVRvBRFeVdRlHOKolxVFGWuoijeua9TFOVZRVEuK4pySVGUsYqiqIqi1Mt5bL6iKJ8qivKHoihpQM8iHO8ORVH2XXcnpXkx3sMCVVVXACn2+lyEEMVXRvLJAVVVzblfAh5ADds/HSFEQcpC/ijEGOBLVVUPqaqaAMwEHrLTsUUBpEhxc6qq7gAuAF1zvvUW0ABoCdQDqgGvACiK0h94GuiT81j3mxzyXuB1wB/YXMjxWgNfAROAYOAzYEnuMKiiKJ8oivKJXd+wEMJh3D2fKIqyTFGUTOBvYD2wq5gfgRCihNw9fwDfK9qU0VWKorS47vtNgf3Xfb0fqKQoSnAhxxM2kiKlbLgEBCmKogDjgKmqqsarqpoCvAGMynneCODrnLsB6dx8uPJ3VVW3qKpqBbIKOd444DNVVf9WVdWiquqCnNd0BFBV9TFVVR9zzFsWQjiI2+YTVVXvQLugGQj8mXNeIYTzuGv+uA9tKlct4C/gT0VRKuQ85gckXffc3D/7F+UDESVncHUAwi6qAfFAKOAD7NbyAwAKkDuXsyo33lk8f5NjXf+9wo5XC21od/J1r/HMOY8Qwj25dT5RVTUbWKEoyhRFUU7lTEMRQjiHW+YPVVW3XPflm4qijEEbEVoKpAIB1z2e+2eZqu5gUqS4OUVR2qElhc1ALJABNFVV9eJNnn4ZqH7d1zebr61e9+fCjnceeF1V1ddLErsQonQpY/nEANS107GEEIUoY/lDRSuCAA4BLYAfc75uAVxVVTXOTucStyDTvdyUoigBiqLcASwEvlNV9WDOkOgXwBxFUcJynldN0VamAO0f2MOKojRWFMWHnLmct1KE430BTFQUpYOi8VUUZZCiKEUaAlUUxUNRFCPaz6FBURSj4pwVPIQQ13H3fKIoSiNFUQYoiuKdk1fuB7oBG4r9YQghiqUM5I+aiqJ0VhTFM+c65BkgBMgdXfkGeFRRlCaKolQEXgLmF/XzESUnRYr7WaooSgraXYP/ArOBh697/Dm0pX23K4qSDKwBGgLkrKT1Idp8y5PAtpzXZBVwvoKOtwttHuhHQELO8x7KfWHOyhtzCzj2F2h3RkbnvJcM4IEC370Qwp7KSj5RgOnANSAGbTnikaqq7insAxBClFhZyR/+wKc5r7sI9AcG5I6UqKq6Eng7J9azOb9eLSBOYSeKqqqFP0uUSYqiNAaiAK/rlu4UQohik3wihCgpyR/iZmQkpZxRFOXOnCHNimjL+S2VhCCEKAnJJ0KIkpL8IQojRUr5MwFtOsQpwAJMcm04Qgg3JvlECFFSkj9EgWS6lxBCCCGEEKJUkZEUIYQQQgghRKkiRYoQQgghhBCiVHHIZo4hISFqeHi4Iw4thCih3bt3x6qqGurqOIpL8okQpY/kEzgXn05GtoWGlYq0NViJmcxWjl1NoXpFbyr6eDr0XMK+0k0WTsWkEh7si7/ROfunX0jIIDXLTKPKt/65jLqYRIi/F5UDjDd9/ODFJCr5GwkL8HJUmHkKyiUO+cTCw8PZtWuXIw4thCghRVHOujqGkpB8IkTpI/kEJny7i+jYdP6c2s0ux7uVq8mZdHhjLTPvjODeDjUdei5hX3vPJXDnJ1uZ+3A7ejYMc8o5n128n00nYtn2Qu9bPqfBf1fwaNfaPNe/Ub7HrFaVOi/+wdQ+DZjSp74jQwUKziUy3UsIIYQQophMZiueBsdfRnnlnCPLbHH4uYR9ydJUtpEiRQghhBCimEwWZxUpegCyzFaHn0s4huLqANyUFClCCCGEEMVkMlvx1Dv+Miq3EMrKliLF3cguH7ZxThePcCvZ2dlcuHCBzMxMV4ciSsBoNFK9enU8PDxcHYoQkk/cnOSTWzNZVHw8HV+k6HUKBp2CySLTvdyVoshYSklIkSLyuXDhAv7+/oSHh8s/LDejqipxcXFcuHCB2rVruzocISSfuDHJJwUzma14OGEkBbS+FBlJcUcylGILme4l8snMzCQ4OFguKNyQoigEBwfLXWtRakg+cV+STwpmMlvymtodzctDLz0pbkyyX8lIkSJuSi4o3Jf83YnSRn4m3Zf83d2asxrnATz1Olndyw1JT4ptpEgRpZKiKDzwwAN5X5vNZkJDQ7njjjuKdZzw8HBiY2Ntfo4Qwn1JPhGO4KzGeQAvDx0mGUlxW1Lrl4wUKaJU8vX1JSoqioyMDABWr15NtWrVXByVEMIdST4RjpBtUZ02kuJl0Ml0LzckAym2kSJFlFoDBgxg+fLlAERGRjJ69Oi8x+Lj4xk2bBjNmzenY8eOHDhwAIC4uDj69u1Lq1atmDBhAup1Y63fffcd7du3p2XLlkyYMAGLrJQiRLkh+UTYm3Mb56UnxZ0p0pVSIlKkiFJr1KhRLFy4kMzMTA4cOECHDh3yHnv11Vdp1aoVBw4c4I033uDBBx8E4LXXXqNLly7s3buXIUOGcO7cOQCOHDnCokWL2LJlC/v27UOv1/P999+75H0JIZxP8omwN2ftOA/aXinSk+J+pCfFNrIEsSjQa0sPcfhSsl2P2aRqAK8Oblro85o3b050dDSRkZEMHDjwhsc2b97Mzz//DECvXr2Ii4sjKSmJjRs38ssvvwAwaNAgKlasCMDatWvZvXs37dq1AyAjI4OwsDB7vi0hRCEkn4iyQlVVpzbOexmkJ8WdSU9KyUiRIkq1IUOGMG3aNNavX09cXFze99Wb3J7IXYXmZqvRqKrKmDFjePPNNx0XrBCiVJN8IuzFZNEKBqctQWzQkZpldsq5hP3cLLeIopMiRRSoKHcoHemRRx4hMDCQiIgI1q9fn/f9bt268f333/Pyyy+zfv16QkJCCAgIyPv+Sy+9xIoVK0hISACgd+/eDB06lKlTpxIWFkZ8fDwpKSnUqlXLRe9MiPJH8okoK3JHNZy2updBL5s5ujEZSCkZKVJEqVa9enWmTJmS7/vTp0/n4Ycfpnnz5vj4+LBgwQJAm1s+evRoWrduTffu3alZsyYATZo0YdasWfTt2xer1YqHhwcff/yxXFQIUY5IPhH2km3R7pB76J1z+enlIT0p7kjGUWwjRYoolVJTU/N9r0ePHvTo0QOAoKAgfv/993zPCQ4OZtWqVXlfz5kzJ+/PI0eOZOTIkfleEx0dbXvAQohSS/KJsLe8kRSD3inn89RLT4pbk6GUEpHVvYQQQgghiuGfIsV5mznKEsTuR1pSbCNFihBCCCFEMZhy9sVx3upesk+KO5N9UkpGihQhhBBCiGLIcnrjvPSkuCNVulJsIkWKEEIIIUQx5DbOexqcc4fc06Aj26JitcpFrzuSfVJKRooUIYQQQohi+GcJYuc0znvlNOjn7s8i3ITUlDaRIkUIIYQQohic3jifcx7ZK8U9yUBKyUiRIkqduLg4WrZsScuWLalcuTLVqlXL+9pkMjnknHv27GHlypU3fUxVVUaMGEHz5s358MMP7XbOdevWsX379ryvP/74Y77//nu7HV8IIflEOIbTG+c9cooU6UtxKzKQYhvZJ0WUOsHBwezbtw/QNlnz8/Nj2rRpRX69xWJBX8wh+D179hAVFUX//v3zPXbx4kV2797NqVOninXMwqxbt46QkBA6duwIwOOPP27X4wshJJ8Ix3D2jvO555EVvtyTIk0pJSIjKcKtDB48mDZt2tC0aVPmzZsHgNlspkKFCrz00ku0b9+eHTt2sGTJEho2bEjXrl2ZPHkyw4YNA7RN3R566CHat29Pq1atWLp0KRkZGcyYMYPvv/+eli1bsnjx4hvO2bdvXy5dukTLli3ZunUrXbp0ybvouXLlCvXq1QNg3rx53H333fTr14/69evzwgsv5B1j+fLltG7dmhYtWtC3b19OnTrFvHnzeOedd/KO+9JLL/H+++8D2kVOhw4daN68OcOHDycpKQmALl268Pzzz9O+fXsaNmzI1q1bHfuBC1GGST6RfFJSJic3znt5aIWyFCnuRfZJsY2MpAi3smDBAoKCgkhPT6dt27YMHz4cf39/kpKSaN26NbNmzSI9PZ0GDRqwZcsWatasyYgRI/JeP2PGDPr378/8+fNJSEigQ4cOHDhwgFdeeYWoqKi8/9Svt2TJEu6+++68C4mC7N+/nz179mAwGGjQoAGTJ09Gp9MxadIkNm3aRK1atYiPjycoKIixY8cSEhLCU089BcAff/yRd5z777+fzz//nC5duvDiiy8yc+ZM3n33XUCbLpJ74TRjxoxbTisRQhRM8onkk5JyfuO8TPdyZzKQUjJSpIiCrXgerhy07zErR8CA/5XopXPmzGHJkiUAXLhwgVOnTtGyZUs8PT258847ATh8+DANGzakVq1aAIwePZpvvvkGgFWrVrFixQr+9z/t/JmZmZw7d87Wd5SnT58++Pv7A9CoUSPOnTvH5cuX6dmzZ148QUFBBR4jLi6OzMxMunTpAsCYMWN44IEH8h6/6667AGjTpg3R0dF2i10Ih5N8UiyST0ovlzXOy0iKW5F9UmwjRYpwG2vWrGHjxo1s374db29vunTpQmZmJgDe3t55cz7VAsZXVVXlt99+o27dujd8f+PGjUWOw2AwYLVq/1Hknj+Xl5dX3p/1ej1msxlVVYs1H7Wg+K8/R+7xhRDFJ/nkxnNIPikek9m5jfO55zFJkeKWZCClZKRIEQUr4R1KR0hKSiIoKAhvb28OHTrEzp07b/q8pk2bcuzYMc6fP0/16tVZtGhR3mP9+vXjww8/5IMPPgBg7969tGrVCn9/f1JSUooUR3h4OLt376Z169b55pvfTOfOnXnqqac4e/bsDdMzbnXOkJAQvL292bp1K7fddhvffvst3bt3L1JsQpRqkk/ykXzinnL3K3HeSIr0pIjyRxrnhdsYNGgQ6enptGjRghkzZtChQ4ebPs/Hx4ePPvqIPn360LVrV6pWrUpgYCAAr776Kunp6URERNC0aVOmT58OQK9evdi/fz+tWrUq9ELhmWee4YMPPuC2224jISGh0LgrVarEp59+ytChQ2nRogX33XcfAEOHDuXHH3+kVatW+RpWv/32W6ZOnUrz5s05fPgwL730UqHnEUIUneQTYYvcEQ0PvZMa5/P2SZGeFHcijfO2UQobCi6Jtm3bqrt27bL7cYVzHDlyhMaNG7s6DJukpqbi5+eHqqpMmDCBiIgIJk+e7OqwnOZmf4eKouxWVbWti0IqMckn7k3yifuTfJLf7NXH+XDtCc68OdApy8uevJZCn9kb+b/RrRjcoqrDzyfsY+PxGB78agc/T+pEm1oF94/Zy7OL97PpRCzbXuh9y+c0+O8KHu1am+f6N8r3mNWqUufFP5japwFT+tR3ZKhAwblERlJEmfTpp5/SsmVLmjRpQkZGBuPGjXN1SEIINyX5RPybyWzFU69z2v4XudO9pCfFvchAim2kJ0WUSc888wzPPPOMq8MQQpQBkk/Ev5nMVqf1o8A/vS/Sk+KupHW+JGQkRQghhBCiGEwWi1OLFNknxT05oqWiPJEiRQghhBCiGHKnezmLrO7l3mQzx5KRIkUIIYQQohiyLSoeBuddeco+Ke5JxlFsI0WKEEIIIUQxOHskRa9TMOgUme7lpmQgpWSkSBGl1uuvv07Tpk1p3rw5LVu25O+//3bYuXr06IEscytE2SX5RNhTltmKZ84ULGfxMujIypaRFLciQyk2kdW9RKm0bds2li1bxp49e/Dy8iI2NhaTyeTqsIQQbkjyibA3k8W5q3sBeHnopSfFTTlrqeqyRkZShF3ExMDOndrv9nD58mVCQkLw8vICICQkhKpVqzJjxgzatWtHs2bNGD9+fN7KGT169GDq1Kl069aNxo0bs3PnTu666y7q16+ft7tydHQ0jRo1YsyYMTRv3py7776b9PT0fOdetWoVnTp1onXr1txzzz2kpqYC8Pzzz9OkSROaN2/OtGnT7PNGhRD5SD4RpZ3JbMHLidO9IGckRaZ7uRVVhlJsIkWKsFlkpEp4PQsD7kkjvJ6FyEjb/1H27duX8+fP06BBAx577DE2bNgAwBNPPMHOnTuJiooiIyODZcuW5b3G09OTjRs3MnHiRIYOHcrHH39MVFQU8+fPJy4uDoBjx44xfvx4Dhw4QEBAAJ988skN542NjWXWrFmsWbOGPXv20LZtW2bPnk18fDy//vorhw4d4sCBA3kXKkII+5J8ItyBsxvnQWuel8Z59yTjKCUjRYqwSUwMjJ1oJXD4FvxGrSdw+BbGTrTafAfUz8+P3bt38/nnnxMaGsrIkSOZP38+f/31Fx06dCAiIoJ169Zx6NChvNcMGTIEgIiICJo2bUqVKlXw8vKiTp06nD9/HoAaNWrQuXNnAO6//342b958w3m3b9/O4cOH6dy5My1btmTBggWcPXuWgIAAjEYjY8eO5ZdffsHHx8e2NyiEyEfyiXAXzm6ch9yRFClS3Ilsk2Ib6UkRNomOBu+KmXiGpQDgGZaCd4VMoqN9CQ217dh6vZ4ePXrQo0cPIiIi+Oyzzzhw4AC7du2iRo0aTJ8+nczMzLzn507l0Ol0eX/O/dpsNgP554X++2tVVbn99tuJjIzMF8+OHTtYu3YtCxcu5KOPPmLdunW2vUEhxA0knwh34ewd50HbK0WKFPckLSklIyMpwibh4ZCRYMR0zR8A0zV/MhKNhIfbdtxjx45x4sSJvK/37dtHw4YNAW0+eWpqKosXLy72cc+dO8e2bdsAiIyMpEuXLjc83rFjR7Zs2cLJkycBSE9P5/jx46SmppKUlMTAgQN5//332bdvX0nfmhDiFiSfCHehNc67YHUv6UlxKzKSYhsZSRE2CQ2FeXN1jJ3YGe8KmWQkGpk3V2fzXc/U1FQmT55MYmIiBoOBevXq8fnnn1OhQgUiIiIIDw+nXbt2xT5u48aNWbBgARMmTKB+/fpMmjTpX+8nlPnz5zN69GiysrIAmDVrFv7+/gwdOpTMzExUVWXOnDm2vUEhRD6ST4S7cMV0L+lJcT+WnCpFr5OhlJKQIkXYbPRohT599ERH+xIejs0XFABt2rRh69at+b4/a9YsZs2ale/769evz/tz7pSOfz8WHR2NTqdj7ty5Bb6+V69e7Ny5M99zduzYUfQ3IIQoEcknwh1o+6Q498LTy6AjNcvs1HMK21isWpFi0MnEpZKQIkXYRWiofS4mhBBC8oko7bItrmic18tmjm7GbJWRFFtIaSfKjfDwcKKiolwdhhCiDJB8Ur65pHHeQ3pS3I3FqhWVUqSUjBQpQgghhBDF4JId56Unxe1Ycv66DFKklIgUKeKmVFmSwm3J350obeRn0n3J311+FquKxariqXfu6l6esk+K25GRFNtIkSLyMRqNxMXFyX9ObkhVVeLi4jAaja4ORQhA8ok7k3xyc7mjGc7ecV72SXE/5rzGeSlSSkIa50U+1atX58KFC8TYus2zcAmj0Uj16tVdHYYQgOQTdyf5JD9Tzhwe1+w4Lz0p7iR3dS+dFCklIkWKyMfDw4PatWu7OgwhRBkg+USUNbkjKV4u2HE+26Jitapy0esmLDKSYhOZ7iWEEEIIUUR5IylOLlJyz5d7flH6WWQJYptIkSKEEEIIUUS5IymuWN0LkL1S3IhZNnO0iXxqQgghhBBFlNc47+yeFI+cIsUOfSmHLyVzOSnD5uOIgv3Tk+LiQNyU9KQ4UUwMREdDeLjspiyEsI3kEyFcI9tFjfNGg7bkcaYNIym7ouN5f80JNp+MpU6ILyue6kpygl5yiYOYLTKSYgv51JwkMlIlvJ6FAfekEV7PQmSkLMcphCgZySdCuE6Wi6Z7+Xpp95VTs8wlev3pmFRGfr6do1eSub9jTU7HpvH4R6cklziQJWfpdWlJKRkZSXGCI0fgkXFWKo7YgmdYCp7X/Bk7sTN9+ujlroUQolgknwjhWq7qSfGzsUhZdfgqFqvK7090oVoFb67EZ7P68CkCR1zCMzhNcokDWKxWDDoFRZEqpSRkJMXBIiNVWrWzYPHKwDMsBQDPsBS8K2QSHe3a2IQQ7kXyiRCul7u6lrOXIPYzakVKWgmLlLVHrtK0agDVKngDEHqhMapVwTM4DZBc4ghmqyore9lAihQHiomBsROtBA7ZgSXdE9M1fwBM1/zJSDQSHu7a+IQQ7kPyiRClg6sa5/28tJ6UlBIUKXGpWew+m0CfxpUALZ988I4nSX/XyXuO6Zo/GQmSS+zJYpEixRYy3cuBoqPBu2Im3jXjCeobxdXIjui9s9GbjHz1hU6GU4UoB+JSs4hNNZGQbkJVwd9oINjPk8oBxmJNAZB8IkTp4LrpXh5AyUZS/joWg1Ulr0iJjgaf4EyUSkmoFoW0w1WJX92MGa9KLrEniypFii2kSHGg8HDISDDiec0f38aX0flmkbSkPXt26mnc2NXRCSEcISHNxMpDV1h39BoHLyRxJTnzps8LMBpoVCWAznVD6N04jKZVAwosWiSfCFE6uGp1L9+ckZTUzOIXKWsOX6VSgBfNqgUA/+STwAoZoIJPo8ukb2rGhPFyQW1PFqsqu83bQIqUm7DX0p6hoTBvro6xEzvjXSGTjEQjX3+hkwsKIcqgqItJfPzXSVYfvorZqlIjyJuOdYIIDwxEl+lN3RoeVKygTdW4lpLF0cvJHLyYxPtrjzNnzXHqhPhyX8da3N26OoE+HvmOL/lEiNLBVSMpvp7aJdvxM2ZiGhf9+iQz28LGEzHc2apa3o2Q6/OJX8Q5vLsc5vl34wkNDXNU+OWS1pMinRUlJUXKv0RGqoydaMW7YiYZCUbmzdUxenTJq+DRoxX69NETHe0ra5ALUQZFx6Yxc9lh1h69hr/RwKNdajO4RVWaVg1g4UIYO6rgfBKTksVfR6+xaNd5Zi47zHurjvFol9qM7VqHQO8bixXJJ0K4XpbFNUXKokVgNen5/tcMPnnSUuTrk+2n40g3WejTpNIN38/NJ8dP1WTiiuMkBF4CpEixJ60nxdVRuC8pUq6T15g6/MalPfu2v0BwyhaIPQ5xpyAjAcxZoCjgGwp+laBSU6jaCio3B/2NH2toqFxMCFHWZFusfLHpNB+sOYGnXscz/RryQKdaBBi1wkLLJ5Yb8sn4SR3p3/wQFVN3QtxJSIgm1JTKCHMWI4xGkppUYFeCD7+vD+O+rQ0Y2a8793Wohe666QKST4RwrdyRFC+93mnnzL0+CRlvxbPRBQKDkpj8RCsGNNhBhYwDWj5JugjmTLCYwBgIvmEQVJu4KyGE6PzoVCc433G1fKJn8OWq/LrnIjOHmfOWOha2s6iqbORoA/lJvE5uY6pnWAp1lYvcU3UjfR+dTPC30doTdB5QMVwrTDx9QLVC/GmI3gy7vtSe410RGg6EpndB3V4gP5xClDnXUjJ54oe97DgTT/+mlZk+pCmVA403PCc3n3iFJdFJd5i7q22kx8RDVPwpQXuCpx8E1QavQDAGQHYGgUlH6Z1ygd6eWh/L6RWV+W1TV1rfMYHwJu2c/C6FEDeTt7qXwXm9Brn5xEefwUDdDoZV20z7x47jtdSkPcE3DCrUBA9v7VdmEsSehAOLGI7KnZ4Kum8XQOPB0HwE+IbccPy7WlXjh7/P8WfUFYa3qe6091XWWWQJYptIkXKd8FoqHf238yJvcJtXFGZVx9q4XoQNnIRfRDcIaZBvlAQAVYXEc3BxFxz/E44sg33fQ3A96DARWj0AHsb8rxNCuJ095xKY9N1ukjKymTOyBXe2uvl/6OE1shhdfSHTDG9Ry3CVJKsvS08NYciUXgREdIaAqtpo7L9ZzBBzBPXcdnx3/sKQmJ8x/PgTl4M7Urn/NJR6fW7+OiGEU7iicb522DVebvY+j3h9hb+SwZnsKszdP44x03tRoVk7beTkZrJSmfL+Am73PsYdmbvgzxdg7WsQcQ/c9iSENgCgTa2KVKvgzbIDl6RIsSOzNM7bRIqUXJf2ErrieZYN386l1Cq8tvO/LNhzP6/ProJfn0J+wBQFKtbSfjUbDmYTHFkC2z6GP6bBlg+h9yvaYzKyIoTb2nA8hgnf7iLM38gvk9rTpGpA/iepKhz4kdC1r/F/fS6y43IbXj/2Cj8fGMZHH/sS0KWQfKI3QOUIlMoRVGo/jtirl1j3wzt0i/0V5fu7MdfsgqH/LG16qRDC6UxmKzoFDM4oUkxpsPEdQrbPZXLbLH5L6c736f1Y+cM45s3VU6FtwfkkU+fN0sRwarXqA30bwrUj8PdnsH8h7PsB2oyBHi+g+IVxR/MqfLn5DInpJir4eDr+vZUDFqv1hum6onikSMlMgrUzYOeX2jSuge/iUeMBBl4w8lh4Ced+Gzwh4m6tKDm9Hla/DL+MhV1fwdCPILiund+EEMLRVkZdZnLkXuqF+fPNI+0J9ffK/6SYY7DsaTi7Gaq2hiH/R+2AXow7q/B6eMnySUilqtw9ZTZfb5zIubK/ZwgAACAASURBVDWfMvX8L1T4vAe0Gwd9poOXn21vTAhRLCaL1TlN80eWwYrnIPkCRIxA1/05lvyWSMy1dKJPGoqUT07HpGFVoX4lbfNXwhrD4Peh10uw4S3tuuTgYug7izsihvLZxtOsjLrCqPY1HfveygmzRUZSbFG+b+uf2w6fdtH+kbYfD5N3QftxhFYx0q6dHZpTFQXq9oTxG2HIR3DtEHzaGbbP1e62CiHcwrqjV3n8h71EVAtk4biO+QsUVYUdX8Bn3eBqFAz+AMauhXq9CQ1TbM4nOp3Coz0a0vehlxiofsj3DETdOQ8+7QTRW2x7c0KIYknNMuctB+wQWSnw22Ow6D5tGtcjf8LwLyCkHkEBBqx6c5HzyYlrKQA0yC1ScvmGwMB34LG/oUoLWPokzf56iLYVM1h24LKd31D5ZZXNHG3ikCLlclImKZnZjji0fVitsPEd+HqAVkg8sgoGvn3rOZ220umg9QNaMqjTHVY+Bz8+oI3iCCFKtd1nE3js+z00ruLPN492yL+HSUYiLLxXm9oZ3gWe2AltHnLI1M7O9UL44Ynbmec3gXvN00k3K7BgMGz5QG58COEkKZlmArzz72VkF1eitJsd+36Abs/AhA1Qs2Pew35eBlKLseP8iaup6HUK4SE+N39CSD14cAkMmo1yfgcLzM+gnt5AbGqWre9EID0ptnJIkRKbmkWv9zbwy54LqKXtP05TGix+CNbN0lbgmrgZajhp1ZyAKjB6IfR9HY7+gfnTnhz86wQxMc45vRCieE7FpPLogp1UDjAy/+H2+ZfmjDsF8/rAiVXQ70249yfwc+w+A+Ehviye2Im0ym3pGP8qFyr3htWvkPXt/ezelib5RAgHS87Ixt/ogJGUI0vhy76QnQEPLdemZOlvLIb8vAykZZmLfG11/GoK4cE+eBkKWC5Zp4N2j8L4vzD4BvGNxxscX/w/du5QJZ/YyGJVpSfFBg4pUuqF+lG1gjdP/7ife+ZuI+piKRkxSLmqjZ4cXgJ9Z8HwedrSn86kKHDbE6ypvpSEy0lU+/N2RnXbQmRkKSvmhCjnUrPMTPh2NzpF4ZtHOhDi968pXtFb4IuekBGv3Yns9JjTFsYI9vPih3EdiahTja7RY1ji+wyGk3+gfDeQdhFXJJ8I4UApmdl5+yHZharC5jmw6H4IawTj10N455s+1dfLQLZFJStnGeTCnLyWSv0w/8KfCBDaEK9J61mvdOK26NkceG8KdeqbJJ/YQHpSbOOQ/1G9PfX8Ouk23h7enDOxaQz+aDP//fUgCWkmR5yuaBKi4at+EHsC7l0Et0222zKeMTGwcydFvuMQEwNDn+7IkMQ3SPT0Yvk9Q/nlraUcOWKXcIQQNlJVlWcX7+d0TCofjW5FzeB/TZU4tgK+uwv8KsO4dbe8oCiJouYTPy8DX45pR/uaoUyObcW45GdpHHKY9Y/0YOa0U5JPhHCQ5Eyz/UZSVFVbXGfNdG2xnYf+AP/Kt3x67nnTijDl68JlC2di06jmX/TFNWKS/Rm19gPmmu/g0cYLWPTQMB6blC4jKiVkkZ4Umzjstp9OpzCiXQ3WTevBmE7hLNx5np7vrefb7WexWJ1clcccg6/6azvFP7gEGvSz26EjI1XC61kYcE8a4fUsRbrjkLsp0+UgPwZEzWXv5RZEDnmI1+9fLHcshCgFvtx8hj8OXuG5/o24rd6Nm55x4CdYeB+ENYGHV2gbvNpJcfOJ0UPP5FZtsVypyDpjc+668D/8zFmsGj2Ie24/LvlECAew20iK1QpLn4St/6ct3nPXvEL3VMtt2C+sLyUyUqXJbSmowJyZvkXOBdHRoF4L5c3se3nu9FQGVthC5OD7+fKz9CK9XtzIYpUd523h8E8u0NuD6UOasvzJLjSq7M/Lv0Ux5KPN7IqOd/SpNXGntMZSqwUe/sOu/ScxMTB2opXA4VvwG7WewOFbGDvRWugdh/BwyEgwknY8jBN/dOZB6zPsoR4L7hjHqjk/yh0LIVzo8KVk3lp5lH5NKzG+W50bH4z6GX4dD7VugzFLwDfYbuctaT5pWFdPwrK2WNI9OBxUhVHpL+Ptm8aah/rx9gtHJZ8IYWfJGWYCvG0cSbFaYdkU2PON1iA/4O0iTRf1MxZepOTmkoD+uwHwbn2iSLkEtOuT9EsBZJyoxMLK7Xkmezx96/xFh1OjiLkkhUpxmWXHeZs4rbxrVDmAyHEd+b/RrYhPM3H33G08vWgf15IzHXfShOh/CpQxS6FSU7sePndExDNMW+LPMywF7wqZREcX/LrQUHjkQYXYJa3R+5jIDrXwkOk5dlgb8UX/SSRuX2bXOIUQRZNltvD0j/sI9Pbkzbuao1w/JfTwEvh5HNToqE0Z9SriPO8isimf3Gvg4pfdyE7w4aRfJUaZXkavt7D8nju5dPicXeMUojzLtljJyLbgb8tIiqrCimf+KVB6vVTk6ee5i3ekZt66SMnNJYZA7frKIzi9SLkEtHxyxwCF9BOVUHQqP1l6MC17At1rbsTj1we1zapFkVmsVilSbODUMShFURjcoipr/9Odx3vWZdmBy/R6bwNfbDxNtqVoTWBFlnoNvhkK2enw4O9aM5qd5Y6ImK5pFyuma/5kJBoJDy/4dTEx8NU3KmEjdmDN8MR0zZ90jDxwaTp7rrak3t6H4cxGu8crhCjY7FXHOXolhXfubk6Q73U7Lp/eAIsfgWpt4L4fwdPX7ue2NZ+EDt1LzK9tUC0KJ9TqjLo2A299Ok33DNPyoRDCZik5xUGALT0p62bBznlw25PQ87/FemlukZJmunWRkptLrJnac4uaS0DLJ8tWqKSfDEO1aBfXC68MYvKa2VSIWQ2/TdJGgUSRWKxIkWIDl0yU8/E08Ey/Rqya2o124RV5/Y8j9H9/I5tO2GleQlYq/DBC+4/5vsVQuZl9jvsvoaEwb66OpJ87kxrZg6SfOzNvrq7QTZZy73J414wnqG8UVyM7cunzHlxc1IfzXX9CCa4DkaPh8gGHxC2EyO/AhUQ+33Sa0e1r0rPRdcsIXzmorboTXA/u+8nuIyi57JFPKnQ5wZXvOmHNNHDMGs7mut9jSLsE3w3X8qIQwia5e8CVeCRl55ew6V1o/SDcPqPYC/j45hQpKQWMpOTmEmu6EfOF4CLnEtDyiU9wJsF9D5G8ozYAcSuaU2Xgw9BnOkQthpXPFyvm8sxitcrqXjZwaTdPeIgvXz/cni/HtMVsVXngyx1M/HY3FxJsmPdoMcNPD8Hl/XD311C9rd3ivZnRoxWiT+pZsdiX6JN6Ro8u/Ifx+jumvo0vE3LnbnQmI3t26hl+fzA88Ku2seQPIyFZdn4VwtEsVpUXfz1IqJ8XLwy8btQ16QJ8fw94+sH9i8G7gkPjsEc+qdDzKAl/tMYYnEEkHpjvng9Xo+DnsdrUVyFEiSVn5IyklGQzx2MrtE1f6/eDQXNKtMLoP6t7FfxvedQo8A1LZ1jPgCLnEvgnn3gEp+LX4jyqRaFip9NMGK9Al6nQ8XHY8Rn8/VmxYy+PpCfFNqViyYHejSvx51PdeKZfQ9Yfv0bv9zbwwZoTZGaX4D/U1S/DydUwaDY07G//YG8iNBTataNIdylyn3/9HdPU5e35+gsdjRvnPCGgqjbnPSsZIkdqG1AKIRzm223RRF1M5pXBTf5ZtceUBpGjtN/vXwyB1Z0Siz3yyeevhfDW8Ai2nopj1vEaWlPu8RWw6iXHBi9EGffPSEoxp3tdPQSLH4UqLeCer0FfsuliuSMpqVnZBT4vJjWLLLOV9k18ipxL4MZ8kvF7ZzIOhOPd4DKpSs51SN+Z0HCgNppy/M8SvYfyxCJFik1KRZEC2lKaj/esx9r/9KBP40rMWXOcPrM3sOrQlaLvWr/nW9j+CXR8DNo+7NiAbVToHdPKEXD3V9pUk98maY12Qgi7u5acyburjtOtQSiDIqpo31RV+O0xuBIFw7+0+6Ib9nazfDK8TXXGdqnN/K3RRNIPOkzS8uOeb1wdrhBuKzmnSCnWEsRpcdoNDy9/GBVpU0+bj4ceRYHUQkZSzsdrM1JqBPkU+LybuT6fbPysLp4GHZ/8dVJ7UKfXNsKu1Ezr04s5VuzjlydSpNim1BQpuapV8Obj+1rzw9gO+HjqGf/tbh76eienYwqZT31+ByybCnV6wu0znROsjQq9Y9qgH/R5DQ7/Dlved2psQpQX7646hslsZcaQpv+s5rXpXTj8G9z+GjTo69oAi+hm+eT5AY3o1iCUV36PYm/jaVC3Fyz/D1zY7bpAhXBjyTm9IEUeSbGY4acxkHIVRv0AAVVsOr9Op+DraShwdS+AczlFSs0SFCnwTz5pFO7F6PY1+XXvxbzCB09fGL0QPLy1PaMyk0t0jvJA2ydFipSSKnVFSq7b6oWw/MmuvHxHE/acTaDf+xt5c8WRm68NnhYLP47RpmPYMIxaKt02GZreBWtnwKl1ro5GiDLlyOVkftp9gTG31SI8JOfu5qm/YN3rEDFCW33HjRn0Oj4c1ZJKAUYei9xPfP9PtN2sf3wAUmUDFSGKKzkjZySlqD0p62ZA9CYY8iFUb2OXGPy8DIXuOH8uLgNF0W782mpi97oY9Ar/W3H0n28GVoN75kP8afh1oqz4dQtaT0qpvdQu9Ur1J+eh1/Fol9qsm9aDYS2r8dmG0/R+bz2/77v4zxQwqwV+fhQy4mHEN+Bd0bVB25uiwNCPILSRNp816aKrIxKizHjjjyMEGD14omd97RtJF7V8EtoIBr9fosbW0qaCjydz729DXJqJJ38/h2XEd5AeDz8/Io30QhRT7qpauUsBF+joctjyAbR9FFqMslsMvl76QnecPxefTuUAI0YPvc3nqxxo5LEe9Vh+8DJbT8X+80B4F+j3BhxbDls/sPk8ZZE23cvVUbgvt/joQv29eOeeFvz62G1UCjAyZeE+Rn62ncOXkmHD23B6PQx8F6o0d3WojuHpS3yfb7CYsjAtHKsNHwshbLLheAybTsTyZO/6BPp4aP+uFj8C5iwY+a1D9kJxlWbVApk5tCmbT8by0WEfUrq+A2c2krbyPVeHJoRbSc7Mxt/LUHifQfwZ+HUSVG0F/d+0awx+Ro9Ci5Tz8ekl6ke5lfHd6lC9ojevLTmM+bp97WLqTCC+yjDUtTPh3N92O19ZYbZYMchISom51SfXqmZFfnusM2/eFcGJaym89tHnWDe8janZSGj9gKvDc5jISJUa7erwxF//w/PyVqI+tm/CE6K8UVWVt1cepWaQDw90rKV9c8P/4Px2GPwBhNR3bYAOMKJtDe5qVY331xynxoR+/HTqLozb32TNl5tcHZoQbiMl01x4P4olW1vyG+CeBWDwsmsMfkUcSSlpP8rNGD30vDSoCceupvDt9rOAdm0SXt9Km3f/R3RiddK+eUQbpRV5rKps5mgLtypSQGsaG92+Jhseb8nnvp9x1hpGj0ODiNxxDou17K2AFRMDYydaCRy+hRXtg4lMvZ0mce+RuFd2pBeipFYfvsqhS8k82bs+ngYdnNkEG9+FlvdDxN2uDs8hFEVhSpdmZCf6EDh4F9OrDSLaUoUmR8YRe14uLIQoiuSM7ML7Uda/CRd3wZAPoGItu8dQWE9KZraFK8mZdi1SAPo1rUTX+iG8t+o4h89k5l2bqMP3MCljCh5ZV8n6abKsRnods2zmaBO3K1IAUFUC1vyHQEsC6vAvqV4pjBd+Ociwj7ew51yCq6Ozq9zdpD3DUgCYYRjN6eTa+Pw5ETLK1nsVwhlUVeX9NSeoHeLLsJZVtTt/v4yH4Low4C1Xh+dQ1y4ZMG1vgqJXScObJ62PE+Idi275U3JhIUQRFDqScmYTbJoNrR6Apnc6JAZfL0OBO85fSMgASr6y160oisLMoc0wWazMXHb4hmuToxUq8/qe5/E6swz2fmfX87ozi1VFJ0VKiblnkbI/Eo4sgd4vU6dFFxZN6MgHo1pyLSWTuz7ZyrSf9hOTkuXqKO3i+t2kAZKuhfLw8s/xMF2F5dNcG5wQbujPQ1c5fDmZyb3qYdDrtCV5065pa/97+bk6PIcKD4e0MyGYU7TpJ/uTGzJr2wsEXfkd9i90bXBCuIHkzOxb75GSmaStdOXgGx7+XgbSTLcuUmzZI6Uw4SG+PNmrHtsuXsbin5J3bWK65s97m6dgqtZV2+gx/rTdz+2OzLIEsU3cr0hJPA8rnoNanaHTE4BW3Q9tWY21/+nBxO51+X3fRXq9u54vN58h2+Ley+L9ezfppJ8789hrbVG6Pw9Ri+HgYleHKITbUFWVD9ZqoyhDWlSFqJ/h0C/Q43mtwbWMy80nCd/1wHwpCL13NsH3PgI1b4M/noGEs64OUYhSrcCRlJUvQMoluPMzhy684eul7ZNyq42ubd0jpTDju9WlXpgf4XceJum3jnnXJl/MNeA5Yi4oevhlQrlfPdBqVVGlJ8Um7lWkWK3w+2OgWmHYJ9rOp9fx8zLw/IBGrHyqG61rVWTmssMM/GATW0/G3uKA7uGmu9N3mQrV28Ef07RNooQQhdpwPIYjl5N5vGc9DOnXtFGUam2h81RXh+Y0Wj4x8NW4VgT5G1iZEoVpyKeACktkPrkomMWqMum78rsZaHLmLXpSji6Hfd9D1/9A9bYOjcHPaMBsVcky3/wm7Ln4dLw99IT4eTrk/J4GHTOHNiPBlMGLX0ffeG0SWB0GvQcXdsC2jx1yfndhycmlMpJScu5VpOz6Es5s1Nblrhh+y6fVDfVj/sPt+PyBNmSaLdw7728e/34PlxIznBerneXbTVpvgKGfgCkdlk2VCwshiuCzDaepHGBkSPMqsHQKZGfCnXPL1gawRRAaCn27GnnnnuYcuZzMB7szoe9MOLMBdn3l6vBEKXbsSgoroq64OgyXUFX15iMp6fFaPqncHLo96/A4cvdoudUKX7kreykO3OepU91gBreoyoKdp6hUJ/2faxPQFh9pdAesmwUxxx0WQ2mXu5iTbOZYcu7zySWegzXToW4vaP1goU9XFIW+TSuzemp3nr69AWuPXqX3exv4aN0JMrPLyBBkaAPo9ZK2kdLBn1wdjRCl2oELiWw7HccjXcLxPPILHF8JvV8uk8sNF1WfJpUY0bY6n64/xe6QYVCnB6x+RaZ9iVvafbb8rgSXbrJgsar5e1JWPq8tZDPsEzA4ZvTienlFyi2a5+29R8qt/HdgYww6hRnLDt/4gKLAoNng6QO/TSq3e7uZ84oUFwfixtzjo1NV7S4FaHsYFOPugNFDz5O967Pm6e70aBjKu6uO0+/9jaw9UkamSHV6HKq3hxXPQmqMq6MRotT6bONp/L0M3NvUCCue0aZLdpjo6rBc7uU7mlAl0Jv//LSfjAHvA4qWb2V0VtzEzugEKgXYd98Pd5G7opb/9UXK8T/hwCLo8jRUjnBKHL4FjKSoqmr3PVJupXKgUbu+OnKVv45du/FB/0raJtsXd8Hfcx0eS2kkIym2c49Pbt8PcGod9JkOFWqW6BDVK/rw6f1t+O7RDnjodTy6YBcPf72DM7Fpdg3V6XR6GPJ/YErT7uZcJyYGdu7UfheiPDsXl86Kg5e5t2NN/Nb9V/v3MuSjfH1t5ZG/0YN372lBdFw6b21Lhz6vwum/tAuv60g+EQC7zybQNjzI1WG4RHJmNgAB3jnTvTKTYelTENoYujlvtU3/AoqU2FQT6SYLNYO8nRLLI51rUyfEl5lLD2P6d49Ms+HQYAD89TokROd9u7zkktwiRXpSSq70FympMfDni9rqM20ftflwXeqHsGJKV14a1Jid0Qn0m7ORt1YeLXBjpFIvrJHWrBe1GI6vAnJ2gq1nYcA9aYTXsxAZKXdFRfn17fZodIrCxMon4NCv2rzxsEauDqvU6FQ3mDGdajF/azR/Bw+DGh20mx45o7OSTwTApcQMLiZm0LZWRVeH4hIpOUVK3kjK2hmQchmGfmT3XeULkjuScrPrltMxqQDUDnXOcuqeBh0vD27C6dg05m89c+ODigKD3gVFl9c7W55yidmqFW2yulfJlf4i5c8Xtbueg98HOw2Zeeh1jO1ah3XTunNH8yp8uv4Uvd/bwNL9l265pF+p12UqhDaCZVOJuZiatxOs36j1BA7fwtiJ1jJ/10KIm8kwWVi08zxDGgdScf2L2l3PzlNcHVap89yARtQM8uHZX6LI6D8bslLhzxeIiUHyiQBg11ltA+F25XUkJUMrCgKMBji/E3bOg/bjHb6a17/5GW89knIqRpsdUjfUcUsg/1vPhmH0ahTGh2tPci0l88YHA6trs2BOrSN5y6JylUv+me4lRUpJle4i5dQ6OPhjzgV4Q7sfPszfyOyRLfl5UieC/TyZHLmX0V9s5+iVZLufy+EMXjD4Q0i+gGXNmzfsBOsZloJ3hUyio10bohCusGT/RZIzzTzj9QsknddueDihudXd+HgaeGt4c87GpfPOHh10fRoO/kT8rr8knwgAdkXH4+Opp1Flf1eH4hK50738PYBlT4F/FW3xGicraHWv0zGpGD10VA10znSvXC/f0YQss4XXlx/J/2DbR6F6O7w3/pcqlS6Xm1xitkiRYqvSW6RkZ8CypyGorjaVyYHa1ApiyRNdmDWsGUevpDDow81MX3KIpIxsh563uAqdx1mzA7QeQ6XoT6mrP3XDTrAZiUbCw50WqhClgqqqLNh6loEhMVQ+8jW0eQhqdnR1WC53q1zSqW4wD3Ssxddbz7C31sMQVIe6R/+DNRnJJ4Jd0Qm0rlkRQzldrig5p3G+8pGv4GoUDHwbjAFOjyMwZ5+WuFRTvsdOxaQSHuyLzokXxjExEHvGl4fb1+P3fZdYc/hfCxPpdHDHHAzZCbzQ7O1yk0ussk+KzUpvptn8PiScgTtmg4fR4afT6xTu71iLv/7Tg9Hta/DNtmh6vbueRTvPYbW6fgpYkedx9pmO4l2BZeP+Q/LPnfJ2gp03V3fjOuZClAO7zyZw5HIiMwxfovgEadMOyrnCcsmz/RtSJcDIs78dx9T/HQxJp9jw2ock/dxZ8kk5lpyZzdErybQNL5/9KKD1pFQhDt9t72oN4Y0HuyQOo4eeKoFGouPyL/xzOjaNumHO6UeBG/PJ6w/VprLRn//+djD/Td7KESgdJ/Fo8wU03OpRLnKJWaZ72ax0Finxp2HzHG1liDo9nHrqir6ezBoWwZInulA7xJfnfj7InZ9uZd/5RKfGcb1izQn3CYK+swjN3MHlJT/cuBOsEOXMd9vP8oBxMyGJB+D2meBdfi+woGi5xN/owet3RnDiWiofn60JzYYTkTiHc7uiJZ+UY3vPJWJVoW2t8tmPAlpPyiue34FqhQFvuTSW2iG++VYnzTJbOB+fTt0Q5/Sj5Msnd27jyLfNiE0x8dqSQ/l7fHs8DwHVWP34NFb86FXmc4n0pNiu9BUpqgp/PAt6T+j7usvCaFYtkJ8mdmLOyBZcSsxg2MdbeHbxfmJTs5weS3Q0xZsT3mI01LwN/+3Tadc0vszepRCiIEnp2WyNOsFz+kio2QlajHJ1SC5X1FzSs1EYw1pW5ZP1Jznd+kXQexK84wXatUPySTm180w8ep1Cq5oVXB2Ky1SK2cIA3d8oXadBxVoujeVmRcrZuHSsKk4bSblZPvEwe3Fno3r8svci7646duMLvPyh3xsY4qJox1dlPpfk9qTIdK+SK31FytHlcHI19HwBAqq4NBRFUbizVXXW/ac747vV4Zc9F+n57nrmbzmD2WIt/AB2Eh4OGQnGos/jVBQY+A5kJmnrkwtRDv2+/yJTWIiPNRUGvVesTWDLquLkkpfvaIKvl4Fn/7yGtftzcOJPOLbCqfGK0mP76TiaVQvMW/623DFnMej8bM4rVaHzk66OhtohviSmZ5OQ9k9fSu7yw3VCnFOk3CqfPDugPqPb1+Tjv07x4doTN46oNBlKdq3umFa/xhNf/MmDX+1g3De72Hoy1ikxO1NuT4ps5lhypeuTy86AP1/QlghtP97V0eTxN3rw4sDGrHyqKy1rVGD60sPc8X+b2X46zinnDw2FeXN1xZsTXrkZtB8Hu76Cy/udEqcQpcmOresZbViH0n48VGrq6nBKheLkkmA/L14a1IRdZxNYqBugLXG+4jnIzsz/ZFGmZZgs7L+QSMfa5XeqF9s/ISz7Al/6T3Dqnii3UjtnSteZ6/pScpcfru2k5YdvlU/CwhReH9aMu1pVY/bq4/R6bwMfrj3Bh2tP8MiCXQw5NQQlO4NB1z4nKSOb/ecTeXj+TqddUzmLWTZztFnpKlK2/h8kntPmeuo9XB1NPvXC/PnmkfbMvb81KZlmRn2+ncmRe7mclOHwc48erRB9Ul+8OeE9XgCfYPjjGW0anRDlRNSFRMYkfUyWZwVtHrTIU5xcMrx1NW6rG8ybf54iofssSDwLWz90YrSiNNh7LoFsi0qHOuW0SEm+DBveYYdnR04GdHJ1NMB1RUrM9UVKKpUDjHlLFDvDrfKJTqfwzj0teOfu5oT5ezF79XHmrDnOufh02rbtSGKLsQzIXsPvQ71YMaUrNYJ8GLtgl0v7f+3NkrOZozNXWitrSk+RkngeNs2GJkOhTndXR3NLiqLQv1kV1jzdnSm967Pq0BV6v7eBT9afJMtscei5Q0Mp3pxw7wrQ+1U4/zccXHzDQ4UuZyyEGzuy6kva6Y6j9npV+3cgblDUXKIoCq/fGUGW2corB0O0/LxpNiRduOF5kk/Ktu1n4tEp0LacbuLI6lfAauYjz4fxN5aO6W41gnzQ65Qb+lJOxaRRx4mbOOa6VT7R6xTuaVuDRRM6sePF3ux/tS9rnu7OzGHNCB30MvhVhhXPEezjwXePdqCirweTI/dw5apaJvKJ9KTYrvQUKatf1n7vO8u1cRSRt6eeqbc3YM3T3elSL4S3Vx6j//ub+OvoNVeHdqOW90GVllqSNWnJrMjLGQvhJAxpewAAIABJREFUhjLTkuh27v84Z2yIT/sxrg7H7dUO8eXxHvVYuv8Sf9ebCqiw+tW8xyWflH1/n46jSdUAAoylb4aDw53bDgd/JLn1RLbEB+SNYLiah15HjYreeUWKqqqcjkl1SZFSFGEBxht/frz8tSXhL+6CA4uoHGjkhQGNOR+fQeM+V8pEPrGosrqXrUpHkXJ2Gxz6FTpPgQo1XR1NsdQI8uHzB9uy4JH2KAo8PH8nYxfs5OxN1i93hZg4HUfC34aUS7B5TvGWMxbCDV1Y+iaVSCCu60xtEzFhs4k96lAnxJen/0zkXK0nIGoxnN0m+aQcyMy2sPd8Ih1qB7s6FOezWmHl8+BfhU/MQ1CA+zu6dlWv612/wldsqomUTDN1Q523R4otYmJgp2kk2WFtYM10yEqhVWglLGle+PfZXybyiUV6Umzm+v/Bc5KAxa8aazKnsGqVew7xdW8Qysop3Xh+QCO2nYrj9jkbeffPY6SbzC6LKfcOZ9cJzVh05G4smz7k0uGzxVvOWAh3kniOmse+ZIXSlSspt7ttPiltvAx6egU242JSOj0X9+dCSlXif3iO6DNWySdl3P7ziZjMVjqUx6b5Awvh0l5iWr/M/B1xtA6tgke2t6ujylM7xI8zsWmoqsqp3JW93KBIyRt9HZFBnw/ehNQrsGk2F87rsJypjM5Lmzrv7vkkt3FeelJKzvVFyv5IuLyPRxa+Qt+hngwalUaNcPcc4vM06JjYvS7rpvVgYLPKfPTXSfq8t4HlBy7n39TIwf59h/PtgEFkmXSEn5xevOWMhXAjppUvY7HCc7umMmiI1a3zSWkSEwNvTquIJdOApfE13sweSVDWfqrFRko+KeP+PhOPokD7clakbD18lsSlL3HJswWNXmpHpsXMsk+qlKp8UjvUl4xsC1eTs9if03Bet5RO98r172uTE52ziDwyAnXbx9QJOkvKntp56/y4ez6xSE+KzVxbpGSlYln9Gn9fasu3UXdS+f6tVBu/nqCRWxg7wX2H+CoFGHl/VCt+nNCJAG8PHv9hD/fN+5vjV1OcFsO/N1mKC/bi46iJBJ7/hd/m7CzecsZCuINz2/E8+hufWe7g2L4WZSaflAa5+URv1EaG//Bpw+6YVoTsn8H8T9Iln5Rh20/H0bCSPxV8PF0dilMlrnmbCpY4Hku+C98Op7Ga9IQN312q8kntYK0gibqYxBebTtOxThDVKpSekZ6budkGkG8ceBarqiN4z3Q+f88H05lKqJke/8/efYdHUXUBHP7N7mbTewUChBJ67xBK6KBgoSgg6IdiV4pgQymKBRUBRREVRRCNICAWegsltFCktwABQktCQkjdOt8fQ0KRkkB2Zze57/PwIHF35rBkTubOvfccMha3cOp8YhYd5++buoOUuC/Q5lzknW3v4eJ3U9dSH+ed4svXrFIA/7zamgkP1+bAuSt0/2Ij7/99kCt5Jpuf+1ZNlj7d9BoWzzJ0towm8ZhUtHLGguDIrFZYMZoUKZDplx4Dk67E5RM13ZxPZDS8c3gkesMF+pb5sujl0QWnkGeysONUOq2qBKkdil3Jl8/QIW0+G106sPVAB5BAo7+2BMlR8kl+P5QPlx4iNcvIm91qIDl409pb3ZucOFeZvEZD4cAi+kdtZ/rI8khuJn5bkeHU+SS/maNO7I28Z+p9chlnYfM08iJ7s/lES0yX3W/4pjVdcd4pvuvptBoGtYxg3ahoHmtSnlmbT9JhUiy/7ziD1Wq7KeNbNVn68itvtJ3HwtmdBF9YULRyxoLgyPYvhLM7mWjoS9rBKpgue5TIfKKWG/LJb+0wJftwtqU7ppqPwuYvCdYniXxSAu08lY7RbCWqaunaNJ+9bBySLHOq/htcia1N0tftHTKflPFxw1Wn4WRqNl1rh9Kwgr/aId3V7RpAenYZppQkXvE23RoEotdq2JvsANNV90HMpNw/1QYpeUsnYLVayW4xjh++0+Ci0XJhbivOfhtN2rwoZn7rvFN8txLgqefjXnX56+XWVAjw4PUFe+k9YzN7k2zXuOiWTZbq9YOwerDmfTDZvgmlINicKRfLyvc461qDRdbWvDe4XInPJ2ooyCe/e/HL8LpkWQx86zJIaRS79kO1wxNsIC4hFa1GKlX7UdIP7MLryEJ+sHSnaZM6V+9PdA6ZTzQaiYhATzQSvN61utrhFNot7030ntBReYjqfnQxTSv5s+Foqtqh3pf8Zo5ikHLvVBmkLJ+1G7cjMUzbM4QKjcIBSDqlZfk/OpbM8+RMYsldMlA33JcFL7RiUt/6nEnL5eGv43h70V7Sso02Od9/mixpNND1Q8g4A9tm2OScgmBPe76djjbrDMOTnsR40Z8AV49Sk0/sLT+fRNfzo1/T8kzZYeBS3aevFkDZo3Z4QjHbfPwS9cN98S4l/VFifrVyYMpoUs1+fJXVmx1rPenfX3LofDKoZUXe7FaDqiHeaodSJLdsAFm/H4TWhdXv0b6yD0cuZnLxSp5qMd4v0czx/tl9kJKSLOO+6V0uWXyYGdmwoA42QJcuyi9HeEJhSxqNRJ/G4awd1Y6noyoxf0cS7SfF8vOWxIK62jZVqS1U6650js527icVQumWeiqFSmcnsyK3GfHeEUgeeaUun6jl9a418HbTMep8R2R3f1jxDti5iqFgO1fyTOxNukxU1dKxHyUlBf74bAmtw7cwVX6ULMmN516USUlRcoij5pOBLSryfLsqaodRPDRa6PoBZJymp+EfADYec957FKto5njf7D5ISd++gnblN/GF9VEy8XD6Otj3w8fNhTE9arFsWBtql/VhzJ8H6DFtE/GJabY/eef3lQ70sR/b/lyCYCOWtZ/g4ZLLp5q+AOi8DaU2n9hbgKeeN7rWYN0pI/siX4TEjXB0hdphCcVk24k0rDKlZtP8qRMmPoweR4K1LDGWDmhcLSKXqKFyNER2IeTfaVTxNLDxmPPuSzGLZo73zb6DFIuZysfHciy9Cj+dfwRw/jrYxaFaqDe/DGnO9CcakZFjpO+MLQz7bTcXMmw4zRlcDRr/D3b+BKkJtjuPINhKagIhZ2bx476nOGYsD4h8Ym+PNy1P3XK+vHCoHlb/KrB6HFjUa2ArFJ+4hFTcXDQ0quindih2UT37JyJ9j/NBxpNY0GJK8xC5RC2dJyAZsxjrs4SNx1JtWmTIliyimeN9s+8gZffP6NKPcKH2eC4tbCfq6l9HkiQeqFuGNSOjebVDVZbtv0CHz2OZsf44RrPVNieNfgt0brBmvG2OLwi2tHocks4NXY9haFysGHZXEfnEzrQaifcfrs25TAt/BA6BlMPw7y9qhyUUg83HU2kaEYCrTqt2KLZnyMR7x0Quukex9EhPZIvE5d9bi1yilpAa0HAQbS4vxivnDAfPX1E7onsi9qTcP/sNUgxZytKi8i1o82xPUVf/Ntz1WkZ2qc6qEW1pVSWQicsO023qBtYftcGUp1cIRA2DQ3/D6W3Ff3xBsJXTW+HwPxA1DEMVpX/BD29XEPlEBQ0r+PNYk3DePFiR3LAmsO4jZSmp4LSSr+Rx9GJWqVnqRdyXkJNK6MAJNO2WQa1QfxKPuohcoqb2o5G0Lryhm+e0+1LEnpT7Z79BypavIesidJkAknTryg5CgYqBnsx8qimzBjdFBp76cTvPztnBmbSc4j1Ry5fBKxRWjRGbXgXnIMuwcoxSU7/ly6w4cIF64b48EO0h8olK3uhWA3e9js+sT0DWBdj8ldohCfdhw9WbwrbVSsEgJfMCbPkKavfCENqAoykZtK3lJ3KJ2rzDkKKG0kO7lUtH4tSO5p5c25MimjneK/t8clkpsPlLqNkTyjezyylLivbVQ1g+vA1vdKtOXEIqHSevZ/Kqo+QaLcVzAr0ntB8NZ7YpT6YFwdEd/geStkP7t0nO0/Lvmct0qRWqdlSlWpCXKyM7V+PH06FcKNdFyfdZzrvhtbTbcDSFIC9Xaob5qB2K7cV+DBYTdBzDkQuZmCwy9cJLxz4ch9fqVTJ1/nQ7Px2rxUbL3m3o2p4UlQNxYnb56HJXfIJsyiWt0Xh7nK7EcdVpeSm6KmtGtqNr7TC+XHOMTpPXs3z/eeTimP1oMBCCqsHq98SmV8GhpVwwkfvPeMz+1aHBQFYdughAl9phKkcmDGxRkRph3oxIeQjZlAsbPlU7JOEeWKwyG4+l0DYyqMRv+E07cgR55xxy6jwDAZXZk5QBQL1wX5UjEwBw9Sah5ss04RBn4/9UO5oiu7YnRYxS7pXNP7m/Zyeg+3cWsw4NpHyTSsTEiCVF96qMrzvT+jfkt+da4O2m44W5uxj0w3YSkjPv78BaHXQaD5eOwe45xRGqIBS7mBiZCX1n456dQP+fxhIzX8vKAxeJCPQgMsRL7fBKPZ1Ww/iHarPlSgB7Qx+BHT/CpeNqhyUU0f6zGaTnmGhbrWSvd4qJkYmbMJ4sowc1X3qNmBiZvWcuE+CpJ9zfXe3whKuC2j7HSWsoXps+AGsxrSCxE8vVh8glfKxvUzYdpKSkgHHF+xglHdMjWhQ0bkwRqwDuS4vKgfzzamvG96zF3qTLdJu6kQ+XHCQzz3TvB63+AJRvAbETxaZXweGkpMCwV6/wbvQEtlurE9fQm2dfNRKXkEqX2mFIkvgp4AhaVA6kR70yvJTUGatWD2snqB2SUEQbrhZpaR1ZcvejpKTAzPc307PqUmZID2B64DBDXrCy61QG9cJ9RT5xIOFBPnyvH4R/VgLsiVE7nCKxWK3oNJL4froPth2k/LuT3tX+5HvrA6TiW6obNxY3nVbD/6IqsW5UNH0ahzNz00k6fL6eRbuS7m0JmCQpRQ2yLipFDgTBgSQmwohW0wjRXmaiqT/6kCy8ap7DbJXFfhQH886DNUmT/Fnq1QcO/AFnd6odklAEG46lULecL0FermqHYjOJJ2U+ih7LRdmPHy3d0Idk4R6YzYnUTLEfxcFIkkRm5QfZL0Uir/sITLlqh1RoZqtc4pdM2prtBimyTNXEcSRnB/P1+ccB0WjNFgK9XJnYux5/vBRFWV83Xpu/hz4ztrD/bEbRD1a+GdTooZRjzHbOkn9CyVQpNJVX6nzD0pyW7JKrYUz2hpB0/N31NKzgr3Z4wnXK+LrzSoeqvHm+HUbXAFg1TlQOdBJX8kzsOn25xFf1qiYvpXnoDiZnP04ubliNWsx6AzJQX+xHcTjNKgUwIa8f0pWzsP07tcMpNItFFj1S7pPtBikJq9Gf3cjpSq9zbkFn0bjRxhqU9+OPl6L4tHc9ElOz6fnVJt75Yx/p2caiHajjODBlw4ZJtglUEO5B0IHP8NTn8s78KUou+aMlftVT6Fw7RNSgd0DPtK5EYEAg30p9IHEjJKxROyShEDYnpGKxyrSNLMk/pGV849/jij6SOQcGACC5WHhwSDKAmElxQE0iAtgm1+R8SFvY+DnkpqsdUqGYrbL4+XSfbDdIWTUO/CvR5LnBBY0bd2zVUrWqJPak2IhGI/FY0/KsHRXN/1pF8Fv8Gdp/HsvcracKSuHdVXA1aDgQ4mdCeqJN4xWEQjEbIP4HNI0HsTK+JssWePLjoivkWc00KSuWejkiNxctY3vU4svLrbniHg6rx4HV+UqIljZrDyfj7aajUcUSPDuZkwapR/B5dBwDntSilSQkCbZeOkWotxvB3iV3mZuzqh7qjY+bjnk+gyHvCmyaonZIhWKVxUzK/bLNICU3DZIPQMcxoNMTHAwJCTJNWljo3jebiKoWUeXLhnzdXRjXszZLh7ahRpg37y7ez0NfbWLnqbTCHSD6bdBoYe0Htg1UEAoj8wJodNDurYJc8urEi8gWiacf8he5xEF1rBlCq2pl+CCnF1zcD/vmqx2ScAdWq8zawym0qxaMi7YEl0zNvADhTaFGD7QuVrx0rpjO+2ORZU7v8RX5xAFpNBJNIwJYfN4f6j0OW2dARpLaYd2VMpNSgq8lO7jrpydJ0iuSJBXtscqV81CmAdR6FFAqaQx5wYpv7zi8+sWKKl92Uj3Mm5hnWzCtf0MuZRnp/c0WXpv3L8lX8u78Rp+y0OJF2Pc7nN9jn2AF27LcR+W3YnRP+SQ3DVq+BD5lCnKJW91TSFoZn4e2iVzioCRJYmzPWiw2N+esWySs/VCZFRMc0r6zGaRmGehYM+TuL3bmfGIxQqf3QJLIyLKSlqpB8swBQBuSLvKJg2pVNYjESzlcaDISkJUmnA7OYpEpyeN9eyjMxxcGxEuSNF+SpG5SYWqpWYxK342rI8jERHD3z0MfovTzEFW+7EeSJHrWL8uake14KboK/+w9T4fP1/P9hhOY7tTBNWo4uPkpDR4F57fgabUjyFf0fKLRQdQwQMklHuHpaHTK967IJY6tSrAXg6Oq8OaVPpBxGuJ/UDsk4TbWHE5GI0G7ancZpBgyYVpj+wR1d0XPJ26+EBEFwOVMK8ig81EGzzovo8gnDiqqaiAAG5Ldodlz8O+vkHxI5ajuzGyVRSPHuzm1+Y7/+66fnizL7wKRwA/A/4BjkiR9JElSldu+ySMQqrQv+GNEBOSmuykVeRBVvtTg6arjjW41WDGiLU0j/Plw6SG6Td3ApmP/reKVkgLx+/3IajQKjq+BE+tViFgoNmfi4dBfakcB3GM+8S6j3Fig5BKr77XmpSKXOL5+9aqyX9+E3boGyBs+g7x7qDwo2NzawxdpVMGfAE/9nV+45Wu4fMo+Qd3FPeeTq1xcrVhNOnFv4gSqh3oT4KHnr22ppNYaCXovWPO+2mHdkVUWG+fvSJZh1dg7vqRQQzxZabxx4eovM+APLJAk6dNbvsGvwg1/DA6GmTM0ZCyMElW+VFYpyJNZg5vxw1NNMFtlBv6wjRd+3klSujLdHRMjE1FV2TtUod/TZOuubnoVJUSdkywr/36ejnOxFTmfeF4rhxocDE0eTMGS4UFWTDuRSxxcTIxMndoaLsRWYkx2H6TcNKXEueBQLmTksf/sFdrXuMssSlYKbJ4GNR+yT2CFUOR84nJdN3mNlaoVteLexAn89huc2xPIhqMpVKzjwx7v4XBkKZzaonZot6XMpIhBym0d/geS4u/4ksLsSRkqSdJO4FMgDqgry/KLQGOgd2Fj6d9fKqjylZigpX9/8Q+npo41Q1kxvC2julQj9mgyHT9fz0d/HWPIy6aCvUNuj+xkxJK34dxuOLhY7ZCFe3FsFZyKg3Zvqh0JcP/5JNdoITH3EgOiQ1i2wEvkEgd2/V5EtxaH2Wusyp+GKKxbpiublwWHse6IUn73rvtRNnymNNPreOenn/Zyv/nEYLZQNkwj7k0cXH4u0YWmo3E34ffYRjqOew6LR5hDP0S1WK2imePtWMzKdoKgand8WWFmUoKAXrIsd5Vl+XdZlk0AsixbgR5FiSk4GJo2RTylcBBuLlpe6RDJmpHRdKoZynebjxI4YOMNe4cWn3+IHO9ayrSqg2yWFArJaoHV48G/EjT+n9rR5LuvfLLt5CUMZisPNAwRucTB3bwXUeNiZZLcB9lsgPWfqBuccIM1hy5Szs+d6qHet39R2knY8SM0ehKCIu0X3J3dVz4xmq24umjEvYmDy88lOh+l6I9LYA54aThd5W04s02ZUXFAZtHM8fb+nQuXjim9+e6gMHtSxsqyfMsFqLIsO/auJaFQyvm58/UTjfimb3OsRl3B142XPMlO98TUdjyknYBds9ULUii6vfOvlQLXuqgdDXD/+WTjsVT0Og3NKwUUf3BCsbrVXsSjBxvzi6Uj8s7ZkJqgboACANkGMxuOpdK5Vih33He+9oOrpcAdY1YW7j+fGMxW9KL8ksO7OZdYDVpyL7vh1XogBEYqT+QtZnWDvAWxJ+U2jDmw7mMIbwY1HrzjS8XVKRTo3jiIidGtyYqrgWzU4uKXQ++xR9HW7QAVoyD2EzBkqR2mUBimPFj3IZRtWFAKvCTYcDSF5pUCcHPRqh2KcBe32ov4Qd/q/KjtSx565LWOvem1tFh/NAWj2UrX2mG3f9G53bB/AbR8GXzK3P51TsZotqLXidsgR3d9LjEdKwsSfPsNBIfqoNM4SD0Ce35VO8z/EHtSbmPrdMi6AJ3fh7sU5BNXp3CDgQO0HFpUhZmPtueheuXYkHKCjpPXs6HiK5CdrFR2ERxf/PeQcUbpB1BCSiCez8jlWHIWbSKD7v5iwSHcvBfx2UFuDO7SjBnGB5AO/glJO9QOsdRbceAC/h4uNI24Q7uR1ePBPQCihtotLnswmK246sQDD2eQn0vGPFUWjd5CUL2rzWxq9FCac677SHlCf5XVKrPuSDJP/xTP5JVHVInZYpXFnpSbZV+CuC+g+gNQseVdX14y7l6EYhUcDJ3buDJtUH0WvdSKUB83nlwps801CkvcF5CVopQpjkc0vXJEuemwYRJU7QSV26kdTbHZeLVcdptIsXDcmdy83v+J5hXYEPQYafhiWTEGZFnkE5UYzVbWHkqmU81QdLdb9pSwBk7EQrs3CkqBlxQGMZPiVIKDYciDwZTxdeOXbVdX+UmS8kQ+8zxsm0FKCqzaaKDL5I0MnhVPXEIqX65NYMvxS3aPV+xJuYWNk8CYdde9KPnE1SncUaMK/ix+KYqJveoy0fQYsjGXFVPfoFLtXLr3zSaiqoWYGMesrFFqbZqq9KLoNF7tSIrVxmOpBHm5UiPsDpt7BYen02p46+GmTDb1QntmM+tnLi8oey7yiX1tPp5KpsF8+6VeVqtSPcmvAjRxmIawxcZotuAqBilORafVMKBZBTYeS+VkarbyxYqtoFp3jOum0KhOMoNn7uTYhWz6lq9P/LudqBjowduL9pJrtNg1VotV7Em5QXoixM+EBgMgpEah3iKuTuGuNBqJfs0q8NOoJ4j3f4gOpj9pPPhXvPrF4tt7E0NesIonoI7i8hnY+g3UexzC6qodTbGxWmU2HUuhbWTQnTf3Ck6heeVAsus8wUk5jOD94/DvveFqPokT+cSOVhy4iIdeS+vbLaHc9ztc2AcdxoLO1b7B2YGy3EvcBjmbx5uVR6eR+GXrtZoJaY3GoTVnMeZ/I9CVSceS48L00WUwZLrwca+6JF7KYerqo3aN0yKLjvM3WPsBSFqIHl3ot4hPTyg0Xw8X3Bp/itHiyhv63wDQh2ThWeUiiYnqxiZcte4j5fcO76gbRzE7cO4K6Tkm2lQT+1FKircerMtUa39q+R9hQJklgFL23N0vT+QTO7BYZVYdvEj76iG3LkRhyoO1E6BMfahT6JZoTkOWZYwWMUhxRiHebnStHcbvO5PIMymzI8cza/Lb2Ud4ym054VIyOh9DQS5pVSWIx5uUZ+amk6RnG+0Wp1nsSbnm3L/KQ48WL4JvuUK/TVydQpGE1whlavwrPKjdTgMpAdki4dFlN7MP7yEl06B2eKXbhf2wJwaaP6cszyhBNhxTHq1HVRWDlJIi1MeNii0GsttalRGahbhhwJjsTe5lNyIi1I6u5Nt+Mo3ULAPd6txmqVd+8Y3O75eY4hvXM1lkZBmxJ8VJDWxRkYxcE28t3Et6tpHTlot8GdARKxpG6eb/J5c83qw8FqvMpoRUu8VosVrFnhRQmm2uGqsU32g9vEhvFVenUCTBwVDtyVe5mB3Cm+n/kPpjB9oFV2bF4bN0mBTLD5tOYrJY1Q6zdFo9TtnY2mak2pEUuy3HL1EjzJsQbze1QxGK0StdKzNdepoy2jQGHjxExsIoZs7QiKZ6dvDP3nO4u2hv3WU+J+264hvR9g7NLoxXf06J6l7OqUXlAF6KrsLfe88TPSmWN/7aiYtXBF/tepFHtJupvMnjhlxSP9wPHzcdG4/Zby2p2SL2pABwfA2cXK/0WCpi8Q0xSBGKrO8TPng88BYtw7ZzbukaZo+syfLhbWlU0Z8J/xzkwS83stmOTysE4Pg6SFitDFDc71BK1AnJMsQnptGySqDaoQjFTK/T8NTAx1lpacyoxl9zal86/fuLH+q2ZrZYWb7/Ah1qhuCh1/33BRs/v1p84z37B2cnhqvLhMRMinOSJIk3utVg6dA21C3nS8caIax4qwWDZ4zCpA9k7evj6d/v2uu1GonWkUFsOJqKLNunOIdVFtW9sFpg5Rjwj7in4hvi6hTuiXfbpyAwEt/t48BiokqwFz8Nbsp3gxqTa7IwYOY2Xv5lF2cv56odasmXnwT8KkDz59WOptjlGM0YzFZaVRFLvUqi1pFBbK0yFJ01F9cdH6odTqmw5cQlLmUb6VnvFo0Z0xNh+3fQ4AkIq2P32OwlfyZFDFKcW/Uwb+YOac53TzbB01VHUDkfXDq+hf7sRji28obXto0M5sKVPI4l26cptVlU94J/f4Xkg0q1UZ2+yG8XV6dwb7Q66PweXDoGO38ClCcbXWqHsWpEO17rXI01hy/S8fNYpq05VrC5TbCBvfPg4j6l7ngJrMCTZTCjkaBZpQC1QxFs5NlHu7KAjrjvnQ2px9QOp8T7Z895PPVaoqvfYqnXmveVCjwlrPjGzQym/OVe4jaoxGkyGAKqKA/vLOaCL7eppqz92nDUPku+Sn0JYmM2rPtQabZZ65F7OoS4OoV7V/0BqNgaYj9WlgZc5eaiZWjHSFa/1o721UP4fNVRukzZwJpDF1UMtoQy5Spl/co2gtq91I7GJrINZuqW88XX3UXtUAQbKePrjqH1m+TKepIXval2OCWa0Wxl+YELdK4V+t+qXkk7Yf9CaPUK+JRVJ0A7ETMpJZjWRSn4kHoEds0u+HI5P3eqBHuy4Zh9lqOX+j0pW6YrTTa7fKA03bwH4uoU7p0kQZcJkHNJaSB4k3B/D74Z2Ji5zzRHr9PwzOwdDJ61/VoDJuH+bfkKrpxVkkAJrMADkGO00FIs9SrxBnRozO9uvQk5twbDsQ1qh1NixSWkkpFroke9mwYhsgwrRoNnMEQNUyc4OzKaxcb5Eq3Gg1Ch1dWHqFcKvty2WjDbTlyyy+pJ8dfmAAAgAElEQVSOUr0nJfMCbJoCNXpAhRb3fJiSeVcj2E+5RkrjwC1fK40Eb6F1ZBDLhrXh3QdrEp+YTtcpG/hk+WGyDeZbvl4opMyLyuCwRg+IiFI7GpuRgVZi03yJ56LVULv3aM7JAaQvfl3pdi4Uu0W7z+Lv4ULbajeVUDv0F5zZCu3fAVdvdYKzI4NZbJwv0SQJun4A2SkQd+0hatvIYAxmK/GJaTYPQdmTUkq/v9Z9CBajMqN1H0rppycUqw5jlISwevxtX+Ki1TCkTWXWjmpHj/pl+Cb2OB0/X89fe87ZrdJGibPuQzAb7jsJODoJaBJRsiqWCbfWvFo51pV7gbDsw1zcNPvubxCK5EqeiZUHLtCzftkbb87NBqWPQUgtaDhIvQDtyGAWe1JKvHKNoW5f2PwVXD4NQNNKAUgS7EhMt/nplT0pNj+N47mwH3b9DM2eg8Aq93Wo0vjxCcXNrzy0fAX2L4Az8Xd8aYi3G5Mfa8DCF1sS6KVnaMxu+n23lcMXrtzxfcJNLh6A3cWTBBydh1536zKpQonUpd9Q9lMFXewHyAb7VOEpLZbtO4/BbKVXo/Ab/8f275WqXl0+UIqilAL5gxQxk1LCdRx39SGqUk7by1VHtRBv/j1z2eanNlus6ErbTIosw8p3wd0P2r1+34crZZ+eYDOtR4BXqLKmuRAzI40rBvDXK6358NE6HLmYyYNfbmL8XwfIyDXZIVgnJ8uw/G2lKVLbUWpHY3MBXkUvWyg4r2Afd841H0OgNZVDiz5SO5wSZeGus1QO9qR++HUN1bJTYf2nSuPGqh3VC87OjGImpXTwKw+tXr3hIWqD8n7sSbps81UcVpnSt3H+6Ao4sU5p3FgMPdvE1SkUD1cvZdlX0nalOkwhaDUSTzSvyLqR0fRvVp45WxLpMCmWefGnsVrFErDbOrJU6d4aPRo8Sn5ZXj9R1avU6dT1EeJcW1PpyEwyLiaqHU6JcCYth+0n0+jVsBzS9ZV21n0IxizoWroGhGK5VykSNRy8wmD5WyDLNKjgx+UcE4mXcmx6WrPVWro2zpuNyoPqwEhoOqRYDimuTqH4NBgAYfVg1TgwFv7i9/fU88EjdfnrldZEBHny5sJ9PDo9zi7TsU7HbIAV70BQdaUWvCCUQBqNRMijn6CRrZyIKfmzhfawePdZAB5pWO7aFy8eUPpcNR0CwdXVCUwl+TMpeq2o7lXiuXpBx7FwdgfsnU+D8n4A/HvGtvtSLFYZTWkapGz/DtKOKw88tMXzcFEMUoTio9FC90/gStIN1TQKq045Xxa80JLJj9XnXEYej3wdx5sL9pKaZbBBsE5q27eQfhK6FV8SEARHFFmjDjvKPUHDy6s4uG2V2uE4NatVZv7OM7SsHEi4v4fyxeuXjUa/pW6AKsiv7uXqIm6DSoX6/aFsQ1g9jmr+Eh56Lf+etu2DULO1FJUgvn7ZaLUuxXZYcXUKxatiK6WpYNwXBdU0ikKSJHo1CmftyHY817YyC3cl0X5SLLPiTmK2lPKSpJkXlSQQ2VVJBIJQwjUc8B4pBKBZ8RYGk9ivdq/ijqdyJi2Xfs3KX/vi4X9K1bLRm12bSRG3QaWCRgPdP4XM82jjplK3nK9NV2tYrTJyadqTsuZ9MGUX+7JRcXUKxa/z+4AEK8fc8yG83VwY/UBNlg9vS4Pyfrz390Ee/HITW45fKr44nc3q8WDOg24fqx2JINiFh5cfKS1HU8OawPr5X6gdjtP6bfsZ/D1c6Fo7TPmCKVdZOx5SC5o8rW5wKinYOC9mUkqP8s2Uvm6bpxEdks3B81ds1tTRcnVTfqmYSTm3G3bNgWbPF/uyUXF1CsXPrzy0Hg4HF8OJ9fd1qKohXsx5uhkzBjYmy2Cm//dbeTVmN+czcospWCdxJh72/AotXy7xJYcF4Xq1ugzhhFttGh39guOnk9QOx+mkZhlYefACvRqF4+Zydf9F3JfKTHf3T0tNyeGbGcRMSunUaTxodPRKmY7JInPwvG3aH1iuFv8p8XtSrFZY+gZ4BkH0m8V+eHF1CrYRNQz8KsKyN8Byf8s0JEmiW50w1oxsx7COkaw8cIGOn69nemxCwbriEs1qgWWvg3cZaHv/dccFwalIEv59vyBAyuRQzNui8l8RLdyZhMki0z9/qdfl07BpMtR+FCq1UTc4FRnNVrQaCZ0YpJQuPmWh3euEnltDtOZfm+1LMVtLyUzKvvlKVddO45X9bcVMXJ2Cbbi4Q7eJkHJY2exdDNxctIzoXI3Vr7WjTWQQny4/QrepG1l3JLlYju+wds1WplM7v69UKRGEUsa/SlMSIx6je87f/LNKbKIvLKtV5rf4MzSp6E/VEG/li8vfBkkDnSeoG5zKDGaLmEUprVq8DIFVmaD/mX2nbHP/YLEogxRtSW7mmHtZWdZfrgnUH2CTU5TgT09QXfXuENkFYidC5oViO2z5AA++HdSE2U83QwIGz4rnmZ/iOXUpu9jO4TCyU5VOuRFtoG5ftaMRBNVUemwiOVpvym4ew5mSeK3bwMaEVE6mZvNEiwrKF46uVDbMt31dWZZbihnNVrEfpbTS6aH7p5TnPNVOzLJJU8dSsSdl3UeQkwoPfq4UJrABcYUKtiNJymyK5WqDn2LWrlowy4e35e3uNdh64hKdp2zg85VHyDWWoCVgq8cpjdYemKR8noJQSkkeAZjaj6WJdJilcyfbvFt0SfBT3EmCvV15sG5ZZbP8stchqBq0fEXt0FRnMFvFTEppVrUjp0M7Mdi8gKSTh4v98GarsuepxFb3Or8H4r+HJs9A2QY2O424QgXbCqwCbUYqXegT1hT74fU6Dc+3q8LaUdE8UCeMaWsT6Ph5LEv2nnf+m5jT22D3XGjxEoTUUDsaQVBdQNQzpPjVp0/atyyK26d2OA7tZGo2646k8ETzCuh1Gtg0BdITlQceOr3a4anOaLYqn4tQalm7fYwZLZplryt9g4pR/sb5EjlIsVphyUhwD4AO79j0VOIKFWyv9XDMflXJWzSKlPN5NjlFqI8bU/s15PcXWuLroeflX3fxxMxtHL2YaZPz2ZzFBP8MB59waFf8FTMEwSlpNAQ+/jV+UjbWlePYd7yUVfkrgtmbE3HRSgxoXgFSjiqDlDp9oHI7tUNzCAazFVcxSCnVKkZE8q2mH+VSNpKx7e9iPbbZUoIHKTt/hKR46PIBuPvb9FTiChVsLuZ3PQ/P/Ay3nBPMHPwZMTG2m+FoGhHAP6+2ZsLDtTlw7grdv9jI+38f5EqekzWC2/wlJB+EByeJzfKCcJ15sXX4au+z9NWsZcxHv/Drr6W8yestZOaZWLAziR71yhLi5Qr/jLhazET0WMpnMFvR67RqhyGo6LffYPLeJzhgjiDr9zf4fW5GsR3bWlL3pFw5r+yTrdQO6vez+enEIEWwqZQUGPKCld3NdCyytGZk86lMfmc/KSm2O6dWIzGoZQTrRkXzWJPyzNp8kg6T1rNgZ5JzlC9NO6F0lq/5kFJ8QBAE4Fo+mebfkSQ5iEnlPmfk18dsmk+c0bz4M2QZzPyvVQT8+wuc2qRU8/IKUTs0h2EwW8RMSimWn0uk0Cu8Y3maMl4XuLTg/WLLJeaSutxr+VtgNkCPKXbZJyuuUMGmEhPB3T8PfUgmE0wDycSd6d2GkXjS9pvbAzz1fNyrLn+93JryAe6M+n0PvWdsZm+SbeqiFwtZVp56alyg+ydqRyMIDiU/n5iDzbxrepqqmnO82e4TtuwX1b7yGcwWvt94gpaVA6nvb4SV70KFltBwkNqhORSxJ6V0y88lOh8D/8pV+cnSlefq/0Dqrq3FcvwSuSflyDKlSXfb1+3WVFpcoYJNRURAbrobxmRv0vFh3KVnaRqykxqZP9gthrrhvix8oRWT+tbnTFouD38dx9uL9pKWbbRbDIW2+2c4EQud31OaTgmCUOD6fBJrbcCinLa85LKYFbv/xGwRy74A/tx9jotXDLwYXQWWjgJjDvT80mYlQp2V0SL2pJRm13KJspz6k6wBJGWWI/LwUGWm4D7l70kpMcu9ci8rD1BDaivNuu1EXKGCTQUHw8wZGjIWRpEVE82sn9/hvEdHvLeOVyrN2IlGI9GncThrR7Xj6ahKzN+RRPtJsfy8JbHgiYcaUlIgPl75nSvnYMW7ULE1NB6sWkyC4Khuzicj5szEpPHm2cuTmL72iNrhqe7CRZmpK45TPcSHNqY4OPgnRL8JwdXUDs3hGExikFKaXcslrTElhpBj8eR4zcno0o8oy63v07WZFOf9Hjt7jmvL31aNgayL8PA0u1YHdN5PT3Aa/ftLJCZoWbbAk8QEHWWe+0LpePznK0opOzvycXNhTI9aLB/WhjrlfBjz5wF6TNvE9pNpdo0DICZGJqKqhe59s4moaibp2xFKT5mHxFNPQbid6/PJjn2hePb6nAaa4+Su/4Jdp9PVDk81MTEyNbue41xWNsfm+ZO36HUIqwethqodmkNSZlLExvnSLD+XvN0vHK2ngStN6yid0zdNgbO77uvYztzMMeY3Jfaf5xmJqGph7cx1sGuO0l+pXGO7xiLuhAS7CA6Gpk2V3/ErD10/gMSNsENZ9nXDjIIdRIZ6M/eZ5kx/ohEZOUYe+3YLw37bzcUrtimRfLP8TXu+vePw6hfLkCcnEJ69nKzm79ptracgOKsb8kmd3piq9WCE7nemxvxNZp7J7vlEbUo+seDdcS8Ak9u+hc6YRlrbr0HronJ0jslgtog9KQLBwfDSQ2FUD/XmyzXHsHT9SCkw8efLYDbccy6xXH0Aq3GyQUpKCjz3ohK7a91EwvusJPLgy5j9qkL74m/KfTfiChXU0egpqNIBVo3j79knCmYUKla28MEHsl1uLiRJ4oG6ZVgzMppXO1Rl2f4LtJ8UyzexxzGabTvDc31BgbKkMiHgW7ZcaM4h3xdtel5BKHEkCZeHpiC5ejMyewpPTfmXiKpmu+cTNSUmgle9M2hcrPTQbOERz418tuc1jmfXUzs0h2UUfVKEqzQaiWGdIjmeks0/R6/u4Uo+yIHpE69b7WDh22/lQg9YnHVPSv69Sb4PykynrNd5jtaaoZQxtzNxhQrqkCR4aBpWSUdw3HME9N6AVP8IeSYL703OIaKKxab9VK7nrtcyskt1Vo1oS6sqgXyy/DDdpm5g/VHb3dnkb9ozJXvymcu3aGQYsvQbIiqJ5QeCUGReIbg8NJX6mhO0y56O/xPrVcsnaihfQcal5imCLBlMcJnFLkM1Jm4YRUSE2pE5LoOo7iVcp1vta7MpeZU6kltzIDVSp9J5wHd49YtF3/AIL75ipVsfZcByt3zirNW98u9NADppdtJXt4FJ8SMIbthUlXjEFSqoxzeck7Um06JsPMOCYkhbWYewAVsp91wsvn3iGPKC1a5PQCsGejLzqabMGtwUGXjqx+08O2cHZ9Jyiv1c+Zv2Bp7YR5T2AG+u+Yh3P6msLF8RBKHoaj9CSrl+vKJdTFPfvVyJj1A1n9jTlnNncQnIZGLWL7hazDwd8zPffqMX+eQOjGYreq24BRIU18+m1By7nC6HupMkBzMt+HO8rLlk7o4gbFAc3v1j8e1993zirHtSgoPhu280BHOZidIP7EmuS8SgN1XLJeIKFVTlE9WHeYf6MNz9d1pGbkYfkgmAPiQTd788EhPtH1P76iEsH96GN7pVJy4hlY6T1zN51VFyjcXb26V/9F4mdniP9NAHGP/HYPr3d65kJgiORnrgM85klmOqy3QqPrzFIfKJrRnMFqasOsbb/rF0Ct1EcoMPWRVfQ+STuzCYrbi6iFsg4ZrudcL44akmDOsYSd3K4bx0YixlpVTe08/Cr93hIuWT/GaOzrYnBaD/4zKfu3yDrz6X8kO/5/EBrqrFIq5QQVXBwaDtOYlzmWWZ034orinKt6Qx2Zvcy26qLVdw1Wl5Kboqa0a2o2vtML5cc4xOk9ezbN95ZLkYlo0Ys2HBM0ieQfg/+RXBIc6XyATB0QSV8+FY7e8IJ5VPA7/Ckqcsn1Q7n9jSnM2n8E4/yBDDbKjRg4jez4gZlLuwWGUsVhm9ViyvFa6RJImONUMZ3qka059swMAeTzBx6+v01m1iYJ35mNI8gMLlE4uT7kkBYOt02mr3sbHyCAJq1FQ1FDFIEVTXZ6A/3oNnUdH/LB/n/kpWTDsyFkYxc4ZG9R+2ZXzdmda/ITHPtsDLVceLv+xi0A/bSUjOvL8DL3sDLiXAo9+CZ2DxBCsIAp2faUVey9E8ottMf8/VGHZEOkw+KW6XsgzMXLuXmZ7T0XgGwUPTlP1+wh3lF0YRMynCnfTvL/HcnDe57N+Kj1xmUVV3jqx5bQuVT8xOuieFs7uQ1rzHCksT9oX1VjsaMUgRHIN/3WZoO4+lV7W/2DEphsQErUMtV2hZJZAlQ1szvmct9iZdptvUjXy45CCZeaaiH2z3XOVXm5FQuV3xBysIpZxn19eQK7Vjgn4ODZqtY1HsZYfKJ8VlyqojjLV8Q5jlHPSeCR4BaofkFAxmZemu2JMi3E1wqA6/wT+gc3VjRtBExn6WXKj7E2vBnhQn+h7LSYP5T4F3KG+annWIBx5O9OkJJV7LVyGyCxGHRxNs2FHot9mrJ4JOq+F/UZVYNyqaPo3DmbnpJB0+X8+iXUmFXwJ2YR8sGQmV2qpSc1wQSgWNFqnX97h4+PKd61Q++nsj5y7n3vVtztRf5dD5K+h2zORB7VakjmMhorXaITkNMZMiFIlPWXS9v6em5jRNT71NcNDdf97nz6QcOiA5RT7BaoU/nofM88h9ZnMZb7UjAsQgRXAkGo2y/Mm7DMwbBFnJd33LjV3b7VNmNNDLlYm967H4pSjK+rnz2vw99J2xhf1nM+78xpw05e/l7g+9fwSNWA8tCDbjHYqm70+ES8mMt3zJ83O2k2M03/blauSSe2Wxyvw07zfecfkFU9Vu0GqY2iE5FcPVQYqYSREKS1O9C2tDB9MycwWmrd/d9fWb4pTvsWeH5Tp8PgFgw2dwbCV0+9juXeXvRFyhgkNJyQ5gf91fkHPTlWlHy+2XU93ctb0wZQGLU/3yfvzxYis+7V2Pk6nZPPTVJt5dvI/0bGNBfAVPZS1mWDAYMpKg72zwKmGL4wXBAaV4RnG6xoe0l3bSIXkOw377t6B/wQ2vUzmXFMb1+WTR2i2MSp+AwbMcLr1nKA94hEIzFMykiAdFQuHlNBnNGktDtCtHw6ktt31dSgrM/ElZUujZY7tD5hMZOH/+6v3Job8h9iOo1w+aDlE7tBuIzCY4jPwnmdHPVObpv76A05vhnxFwm6VU13dtB3XKjGo0Eo81Lc/aUdE81SqCmO1naP95LMOnJRIRaS54Knvkq3fgRCz0nAoVmtsvQEEopfLzSeM3BvLrwccZoVuA/vCffLT00H9e6wi55E6un+Wp0ziZ2htexFNjxvN/vyszs0KRiD0pQlHFxMg83sWf4VlDOZkXRt6cJyDt5C1fm5gIei9jwZ8dMZ8YjTLz/zDRo8VeTPOfh3JNoOcXDrEP5XriChUcws1PMlfUKcPErSNh988QN/WW78nvjGpMVtZOqllm1NfdhXE9a7N0aBuqBHqz+OwBAp9ZhVe/WF596h2qp88gp/7L0HCg/YMThFLmxnyynne8HmPz2eZMdZ3BrriV/LDpxpsLR8olN7v+7+LTby2zBw2ghnSK89HfIQVXVzs8p1SwJ0V0nBcKoeAafHQLma6uPGMdQU62BfPPj0Hu5f+8PiICTLm6gj87Yj5BIxNeaw8L+/ciJdOHS51/ARc3tcP7D3GFCg7hVk8yJx16jUtle8Pq8bB/4X/ek9+1PWNhFFkx0Q5RZrR6mDdvNm1BXlwtNC5Wumji+ch/BivOduBA2ffVC0wQSpGb84kcYmDw+plY3Msy22Mqs5esY/HuswWvd8Rcku/a3+UK43Wz6eYSz5gTr5Hm0VPt0JyWQQxShCK4OZ8kSmV4es8UNJdPwvxBYDbc8PrgYHhigPLf2YtaOWQ+8ZZy+En/KX7aTAauncWJlDC1Q7slcYUKDuHWTzLdsfacDhVawqLnIWH1f97Xv79EYoKWZQs8HaZscaVKEpl7K1D/8mm+dPmKPXJlhno/xaa005gsVrXDE4QS71b5JOliOXIe+R1vPfzu8Qmf/r6OdYevFedwxFwC1/4uzxuX8aRuFTMMDzH1r9EO8VTWWeXPpOjFIEUohJvziWyFrW6VyOr4FZzcAAuHgNVyw3saNFR+nzfL8fKJ5YrENy5TiZSSeDblbbYdb+aw+URcoYJDuO2TzDJu0P83CK6hVMY6s/2W723aFId4SgFKHIum7mGO/4ecvVKeXjELiQgsyxfrD9Ft6gY2HnOg3XOCUALdLp8EVKuG9MRCQrSZxLh/yhtz17PhaMoN73OkXAJKLGs+nMHbPnP5w9Ca13+cwcwZWoeK0dlcW+4lNs4Ld3dzPsndVxH36ue4VKMndP0YDv0Ffw+7Yf9sfoGOpo0lh7pWgwNM7Bv9DG20+xl16i3++uVVh5nluRUxSBEcxu2eZKZk+bG77iLMHmEwtzeciVc50rs4v5euFx/FK8ifrEf/ZvvmSqx4pxk/PNUEk0Vm0A/bef7nHZxJy1E7UkEosW6bT1wbc6RhDBXk88S4fszrc9ax6ViqytHenjV+Fi1SR7PS0oSMVjM5sd/TYZ7KOiuDmEkRiuj6fLLhm0hcXTR8tvQY8bqXSKo5FHb/TPJvryj9RgCLIzZztJhh4RDCs5Yy3jIYqd1DDjXLcysO9OkJwn+fZOZXtek80JNakxaTaQ2Cnx+F01vVDfR2zu6EOQ+D3gvt039Tv214wd+lY81QVo5oy6gu1dhwNJVOk9fzxepj5Jksdz6mIAj35Hb5pPWLTei18GcirEnE6D9k1Ow1rDl0Ud1gb2X792iWDGetpQEXu37D/3qWcdgnns7EaFFyrtiTIhRFfj6pEeFKU7+KLNl/lp4jT9BqS2u+Mfck5Mhc/v3mKQwmU8FMisOMUcwGWPg0HFwMXT7kV2tXypRxrFnjW3GUj08Q/uPmil9ZXU/R8Ou/MHuEIs/pxdElqxyq7jgnN8Dsh8DVC/73N/hH/Oclbi5aXukQyZqR7ehUK5Qpq4/Secp6Vh64UPiu9YIgFNnN+WRbYw8eXfArEdIF5usnMG7OcuZsOHv3A9mDLMP6z2DpKFZZGrOm7iQGRkWqHVWJYTCJmRTh3qWkwB+fRYAEbq0OIblYePOXaawIeIoGKX+xb8oTZGUom+kdYibFkAW/Pg4H/4QuH0KrV9SOqNAc4NMThFu7VcWvyy4BTMtawt5zlam87XHGPTLLMTq57v0d5vYB3/Lw9EoIqHzHl5f1c+frAY349dnmuLtoee7nnTw1K57jKVl2ClgQSpdb5ZMtV1oylwX45GawUDuOecsW8/zUBHUfGFhMsHQUrPuARZbWzK34AWN7NUJysP4FzsxoEdW9hHuXmAhubnJBSxFJK6O36onP/IKPUp+mSc4KomJfwYdsNGpftpkXYHZPOLkeHv7aqQYoIAYpggO7VYWenDQ33v00mD6Gd4mTazO983BSfh1NyoXbd6a3KatVKZG8aAiEN4HBS8GnTKHf3qpKEEuGtmFsj1rsPpVOt6kbmLjsMNkGs+1iFoRS6Hb55MXPWvFI2keYtRrm699DmzybV+buLWj4Z1c5acpy1viZfG/pwa9l3uabJ5uJDd7FTMykCPfjdrnk40+sTOcBhhlfopnvHhbpx5F+9Kh6gZ7dBd+1h5TD8Phcp+zTJq5QwWHdqkLP6Dc1uAfkYQq28IxpFLPNnRnaeDpu83rClfP2DTDzIvzSBzZNgUZPwaDF4BFQ5MO4aDU83boSa0dF80iDcsxYf5wOn8fy579nxRIwQSgmd8onif4BPGp4nyNyeabrv6ThkY/p/81G+xa3OLMdvmuH5fRWXje/wJ8hL/DD4OZ46HV3f69QJPkzKWKQItyLO+USnZeRP62tGWQcja+Uhe/89rfs82ZTssyjluUwqztodPDMSqjxoH1jKCbiChUc2s0Vep5/Xip4gmFGx9vnhjJ46Xd4Ze6Bb1rBvgU3lAG0mSPLsU5vhfVkHJntJkPPL0Cnv69DBnu78lnf+ix6qRWhPm4M++1fHv92KwfPXSmmoAWhdLtTPknGn0eTPmX67ucYolvGB5eGM/zLuSzbZ+OHHxYTxH6C/GM30rIs9M4dw7mIR4l5tgW+7i62PXcpZbharESvFbdAwr25fS7xAmCbXJMHsz/BGlwHFjyt9HrLSbN9YFkpDEoczRvm7zCWjYJn10JYXduf10bEFSo4vOsr9NzqCUaXEY8hPb9e2Qey8BmYNxDSTtommIwkpV9LzOMcOB1K64XLCXvkf8T8VnynaFTBn8UvRTGxV10SUrLoMW0jYxbv53KOsfhOIgil1J3ySerCaPz7fQqP/0J1jyx+422OzXuL1+bGkZpluPvBiyoxDma0gdiPWHQ5mrZZ77H1aGu6uTfB200MUGzFYLGi12nEPh/hvtw6l7TGdFopmXVFH4jLs0ug7RuwfwFMb6E8SLXaoKmz1QLxMzFObky1zO28lz2YgBHziPk7qPjPZUdikCI4neufYOzYqqVqVYkUqilTmp3fh+Nr4aumsHw0ZCXf/YCFkZUCq8bCV02Rj61i3KYx9MydwLnu5/DtHceQF6zFWmlMo5Ho16wC60ZGM6hFRX7Zdor2k2KJ2X66oLShIAj3Lz+f/Pq9J3/8rqVTJwlq9kD7yna0dXsxVLeYt44N4JvP32XOhsMFjQDvy4V9ysOUnx4gO/MKT2e8xWv6Z8jCA41vFi+8JDlW5cISxmCyik3zQrHLzyUvPxwGgItOIiXNBTq8A8+uA+8w5UHqzI5wIrZ4Vn1YrXDwL/i2HSwZyebE+nTN/IxZ2s749N5S7Pcm9iauUsEpBQdDQoJMkxYWupWq/IYAABXiSURBVPfNJqKqhZh5GogaBq/ugvr9YOt0mFIbFr+kNIAsakKQZUjaAX8Phy/qweZpUP0B9rbZxtcJL0JIHqBUCXL3yyMxsfj/nr4eLrz3cB3+ebUNkSHevL1oH498Hceu0+nFfzJBKKVWr5Z5tK+FAc9dzSUxMngEoOn9PTyzGu8ykYyRv6P7ms78MvEF/l6/pej9jcxGpQTo3D4wozWW47H86TuIJpcnsCqvRUGlIH1Ils3yiaAwWsQgRbCN4GDwzQ0EICODa/mkTD14NhYemQFZF5V+ajPawK6fIS+j6CfKvgRbZyjL3OcPAlMOxxv+QJ+1MSS6hgC2vTexF7EjT3BK1/c80Idkok/2ZsgLUXTqpCU4uAw8/BVEDYdtM+DfX5RfPuWgencIbwplGyrlgvUe1w5qylWWiaUchsSNcHwdpJ8EnTvU6QWtR0BQJGVTIDfdgj7ZG31IJsZkb3IvuxERYbu/b62yPsx7vgV/7TnHR0sP0Wv6Zvo0DufNbjUI9na13YkFoYS7cy4ByjfF/flVyCdi0ayZyuBz82DdPA6tq0xa2XaUqx1FxdrNkbzLgPa6JVq56XDpBFzYAyfWK09O8y6T5x7CEp8neS+5DZY8H55vV413BpTH99EtdssnpZ3RbBUV0wSbSEmBN4a5EfSchMbddHWlRX4+0UCD/lD7Edj3O2yZDn+9Akteg8rRENEGyjWC4BrgHnCtE6QsQ+Z5uJQASfFKLjm1BawmKFMfen0PdXrjc0lLbroF91wXtO6mEpFLxCBFcEq36nmgPDHwvNZBNagqPDgJOrwLR5crU6L//grxM68dyMVTubGwGMF0XSUfvRdEtIbWw6F2L3DzKfhf+WtPh7wQhbtfHrmX3Zg5Q2Pzzq2SJPFwg3J0rBnKV2sT+GHTCVbsv8CwTpE81SoCF7EJVBCKrFC5RJKQqrQnsEp75LSTnNoYg/7gX7Q4+xPac7NglfIyg84bjQRaiwGN9doeskx9CHv1zZhtacSa9NoEeHvwfNcInmheAT8PPaHTZbvnk9LMYLaKyl6CTeTnE0mrrNy4ZT5xcYdGT0LDQUpVv0N/weElcGzltQNJWuW+w2IGcy5Yr2tLEFoXWrwA9fpBWJ2CL+ffmwz/LQTXCqlkLIxy+lwiBimCU8qvU16o2Qx3P2X5V/1+yuaylCNwYa/yZCIrRXkaodWDqw8EVlF+hda58anoTfr3l+jUSUtioicREdg1CXi56nirew36Ngnn/b8P8sGSQ8yLP8P4h2oTVdW5N8kJgr0VKZcAUkAlIh4eDQ+PJv3yZXZtW8/F47vJSTuPLi8NGQkDLqTJ3pyQy5Igl+WUIYyIQC+iGgfyY60wWlYOvOEmWc18UhoZzRax3EuwiWv5xAt9SNad84kkQYXmyq+uHyr3I+d2Q9oJyE6BvMvKvYnOFXzDIbAqhNQGr9sniP79JbZYJOKOS+xM0Dp9LhGDFMEp3fNshkYLobWUX8UQg5oJoEqwFz8NbsqqgxeZsOQgT8zcxoN1yzD6wZqU83NXLzBBcCL3MzPq7+dHx64PAw8DkJZt5EJGHqlZBsxWK+1ddPi466gc5IW7/s7Li9TOJ6WJmEkRbOVaPmld9JlRr2Co1uW+Y3BzA72+ZOQTMUgRnJZ4+qgsAetSO4y21YL5bsMJvl6XwJrDF3k5uirPtq2Mm4tYdy0Id1NcuSTAU0+A5/31SxJsz2i2ih4pgs2Ie5PiIwYpglMTTx8Vbi5ahnaMpFejcnzwzyE+X3WU33cmMbZHLTrWDBH9AAThLkQuKT0MZituLmKQItiOyCfFQ1ylglCChPt7MGNQY+Y+0xy9TsOQOTsY/FM8J1Oz1Q5NEATBIZgsVlFoRBCcgLhKBaEEah0ZxLJhbXjngZrsSEyn65QNaockCILgEKyyjEbMLguCwxODFEEooVy0Gp5tW5m1I9vRo34ZtcMRBEFwGGKIIgiOTwxSBKGEC/FxY/JjDdQOQxAEwSHIstoRCIJQGGKQIgiCIAhCqSHLSosKQRAcmxikCIIgCIJQaigTKWKUIgiOTgxSBEEQBEEoNWRZFjMpguAExCBFEARBEIRSRYxRBMHxiUGKIAiCIAiCIAgORQxSBEEQBEEoNcTGeUFwDmKQIgiCIAhCqSEjI4kFX4Lg8HRqByAIgiAIgmBP18+kmEwmkpKSyMvLUy8g4Z64ubkRHh6Oi4uL2qEINiAGKYIgCIIglBo3N3NMSkrC29ubiIgIJLEOzGnIssylS5dISkqiUqVKaocj2IBY7iUIgiAIQqkhc+NMSl5eHoGBgWKA4mQkSSIwMFDMgJVgYpAiCIIgCEKpIcv/3ZMiBijOSfy7lWxikCIIgiAIQuniYPe2kiQxaNCggj+bzWaCg4Pp0aNHkY4TEfH/9u4+psr6/+P48wIVUDFDKPMOKhSVOBxQwRtUNEKTElPTWE7Lkajlyk03a06NcP22WlrTLOIPmPklm5mZdmNKhqImQkhgo3TDm26WHpRvfIGcev3+4Ov5SuAdB+TA9Xr807nu35dun/nufb0/nyDOnTvn8jki7kBJioiIiFiGefNT7rguXbpQUlJCTU0NAN988w29e/du5ahEWpeSFBEREbEO0+0KKQA8+uij7Ny5E4Ds7GySkpKcxyoqKpgyZQo2m43hw4dTXFwMgMPhID4+noiICFJSUjCvmRXgww8/JCoqCrvdTkpKCpcvX76zLyTiIs3uJSIiIpZR1zjfeJry6uelHPvt3836vMG9urHy8dCbnvfUU0+RmprKY489RnFxMXPnzmXfvn0ArFy5koiICLZt20ZOTg6zZ8+mqKiIV199lZiYGFasWMHOnTtJT08H4KeffmLz5s3k5eXRsWNHFi5cyKZNm5g9e3azvptIS1KSIiIiIpZR1zjvfmw2G+Xl5WRnZzNp0qR6x/bv388nn3wCwPjx43E4HFRWVpKbm8vWrVsBSEhI4O677wZgz549FBQUMGzYMABqamq455577uDbiLhOSYqIiIhYyvUmhbqVikdLmjx5MkuWLGHv3r04HA7nfvOfi7vwv2pQY1Uh0zSZM2cOr7/+essFK9LC1JMiIiIiluGOjfNXzZ07lxUrVhAWFlZv/5gxY9i0aRMAe/fuxd/fn27dutXb/+WXX3L+/HkAHn74YbZs2cKff/4J1PW0nDx58g6+iYjrVEkRERERyzDdtHEeoE+fPrz44osN9q9atYpnn30Wm81G586dycrKAup6VZKSkoiMjGTs2LH069cPgMGDB5OWlkZ8fDxXrlyhY8eOrF+/nsDAwDv6PiKuUJIiIiIilmFiut0igFVVVQ32xcbGEhsbC4Cfnx+fffZZg3N69OjBrl27nNtr1qxx/p45cyYzZ85scE15ebnrAYvcAfrcS0RERCzFvVIUEWmMkhQRERGxjEZ60EXEDSlJEREREcswTVRKEWkDlKSIiIiIpRjKUkTcnpIUERERsQzTNK+7ToqIuA8lKSIiImIpylFE3J+SFBEREbEMd+ubdzgc2O127HY7PXv2pHfv3s7tixcvtsgzCwsL+eqrrxo9ZpomM2bMwGaz8c477zTbM3Nycjh06JBze/369c6FKEUao3VSRERExDJME7f63KtHjx4UFRUBdYs2du3alSVLltzy9ZcvX8bT0/O2nllYWEhJSQkTJ05scOzXX3+loKCAEydO3NY9byYnJwd/f3+GDx8OwPPPP9+s95f2R5UUERERsQwTs800zj/++OMMGTKE0NBQMjIyALh06RLdu3dn+fLlREVFcfjwYbZv305ISAijR49m0aJFTJkyBahbJPKZZ54hKiqKiIgIPv/8c2pqakhNTWXTpk3Y7Xa2bNlS75nx8fH89ttv2O12Dhw4QExMjDOJ+uOPPwgODgYgIyOD6dOnM2HCBPr378/LL7/svMfOnTuJjIwkPDyc+Ph4Tpw4QUZGBm+88YbzvsuXL2ft2rVAXdIUHR2NzWZj2rRpVFZWAhATE8OyZcuIiooiJCSEAwcOtOwfuLgVVVJERETEUq5bSflyGfzxY/M+rGcYPPp/Tbo0KysLPz8/qqurGTp0KNOmTcPX15fKykoiIyNJS0ujurqaAQMGkJeXR79+/ZgxY4bz+tTUVCZOnEhmZibnz58nOjqa4uJiVqxYQUlJiTNJuNb27duZPn26MzG5kaNHj1JYWEiHDh0YMGAAixYtwsPDgwULFrBv3z4CAwOpqKjAz8+P5ORk/P39eemllwD44osvnPeZNWsW6enpxMTE8Morr/Daa6/x5ptvAnWfn11NxFJTU6/7mZq0P6qkiIiIiGW0pcUc16xZQ3h4OCNGjODMmTPOT7A6derEE088AcCxY8cICQkhMDAQwzBISkpyXr9r1y5Wr16N3W5n3Lhx1NbWcurUqWaLLy4uDl9fX3x8fBg4cCCnTp3i4MGDjBs3jsDAQAD8/PxueA+Hw0FtbS0xMTEAzJkzh9zcXOfxqVOnAjBkyBDKy8ubLXZxf6qkiIiIiGWY3KCS0sSKR0vYvXs3ubm5HDp0CB8fH2JiYqitrQXAx8cH478vYd4g6zJNk23btvHggw/W239tEnAzHTp04MqVKwDO51/l5eXl/O3p6cmlS5f+O8XzrX9Od6P4r33G1fuLdaiSIiIiIpZR929i9+9JqaysxM/PDx8fH0pLS8nPz2/0vNDQUMrKyjh9+jSmabJ582bnsQkTJtSboeuHH34AwNfXl7/++uuW4ggKCqKgoACgQf9KY0aNGkVOTg4nT54EoKKi4obP9Pf3x8fHx9lvsnHjRsaOHXtLsUn7piRFRERELMWdZve6noSEBKqrqwkPDyc1NZXo6OhGz+vcuTPr1q0jLi6O0aNH06tXL+666y4AVq5cSXV1NWFhYYSGhrJq1SoAxo8fz9GjR4mIiLhp4rF06VLefvttRo4cyfnz528a97333suGDRtITEwkPDycp59+GoDExEQ+/vhjIiIiGjTAb9y4kcWLF2Oz2Th27BjLly+/6XOk/dPnXiIiImIh7tuUcjWJAPD29ubrr79u9LwLFy7U246Li6OsrAzTNElJSWHo0KEAdOnShQ8++KDB9QEBARw5cqTRewcHB9drmg8NDaWkpMS5vXr1agCSk5PrXXdtQ3tCQgIJCQn1jg8cOJAff/zfpAQjR450/o6MjOT7779vEMv+/fudv3v27Mnx48cbjVnaJ1VSRERExDJMsy187HV7NmzYgN1uZ/DgwdTU1PDcc8+1dkgiLlMlRURERCzjho3zbdTSpUtZunRpa4ch0qxUSRERERHLMM22s5ijiJUpSRERERFLaW+VFJH2SEmKiIiIWIb7ts2LyLWUpIiIiIhltMfGeZH2SEmKiIiIWMbtroh+J6xevZrQ0FBsNht2u73R6XibS2xs7HWnHxZxJ5rdS0RERKSVHDx4kB07dlBYWIiXlxfnzp3j4sWLrR2WSKtTJUVEREQsozl6Us6ehfz8uv+66vfff8ff3x8vLy8A/P396dWrF6mpqQwbNoyHHnqIefPmYZp1kcfGxrJ48WLGjBnDoEGDyM/PZ+rUqfTv39+5Unt5eTkDBw5kzpw52Gw2pk+fTnV1dYNn79q1ixEjRhAZGcmTTz5JVVUVAMuWLWPw4MHYbDaWLFni+kuKNIGSFBEREbEO07XZvbKzTYKCL/Pok/8hKPgy2dmupT3x8fGcPn2aAQMGsHDhQr777jsAXnjhBfLz8ykpKaGmpoYdO3Y4r+nUqRO5ubnMnz+fxMRE1q9fT0lJCZmZmTgcDgDKysqYN28excXFdOvWjXfffbfec8+dO0daWhq7d++msLCQoUOH8tZbb1FRUcGnn35KaWkpxcXFzsRH5E5TkiIiIiKWYUKT10k5exaS51/hrml5dH1qL3dNyyN5/hWXKipdu3aloKCA9PR0AgICmDlzJpmZmXz77bdER0cTFhZGTk4OpaWlzmsmT54MQFhYGKGhodx33314eXnxwAMPcPr0aQD69u3LqFGjAJg1axb79++v99xDhw5x7NgxRo0ahd1uJysri5MnT9KtWze8vb1JTk5m69atdO7cuekvJ+IC9aSIiIiIpTS1klJeDj5319Lpnr8A6HTPX/h0r6W8vAsBAU2Px9PTk9jYWGJjYwkLC+P999+nuLiYI0eO0LdvX1atWkVtba3z/Kufhnl4eDh/X92+dOkSQIPJAf65bZomjzzyCNnZ2Q3iOXz4MHv27OGjjz5i3bp15OTkNP3lRJpIlRQRERGxjKu9HU0RFAQ15725+KcvABf/9KXmgjdBQU2Pp6ysjF9++cW5XVRUREhICFDXn1JVVcWWLVtu+76nTp3i4MGDAGRnZxMTE1Pv+PDhw8nLy+P48eMAVFdX8/PPP1NVVUVlZSWTJk1i7dq1FBUVNfXVRFyiSoqIiIhYRt3nXk0TEAAZ73mQPH8UPt1rqbngTcZ7Hi5VUaqqqli0aBEXLlygQ4cOBAcHk56eTvfu3QkLCyMoKIhhw4bd9n0HDRpEVlYWKSkp9O/fnwULFvzjXQLIzMwkKSmJv//+G4C0tDR8fX1JTEyktrYW0zRZs2ZN019OxAVKUkRERMQyTBcb55OSDOLiPCkv70JQEC4lKABDhgzhwIEDDfanpaWRlpbWYP/evXudv69+IvbPY+Xl5Xh4ePDee+/d8Prx48eTn5/f4JzDhw/f+guItBAlKSIiImIZJq4v5hgQ4HpyIiI3pp4UERERsRT3Wm+++QUFBVFSUtLaYYi4REmKiIiIWIYLffMicgcpSRERERHLMKFBKcWVGb+k9ejvrX1TkiIiIiLWYdZfzNHb2xuHw6F/8LYxpmnicDjw9vZu7VCkhahxXkRERCzl2r75Pn36cObMGc66smy8tApvb2/69OnT2mFIC1GSIiIiIpZhUr9i0rFjR+6///5WikZErkefe4mIiIhlmGb7n91LpD1QkiIiIiKWYeLaYo4icmcoSRERERFLMVRLkXasvcwBYbTEbBaGYZwFTjb7jUXEFYGmaba5NZI1noi4JY0nItIcrjuWtEiSIiIiIiIi0lT63EtERERERNyKkhQREREREXErSlJERERERMStKEkRERERERG3oiRFGjAMY5hhGMWGYXgbhtHFMIxSwzAeau24RKTt0XgiIs1F44m1aHYvaZRhGGmAN+ADnDFN8/VWDklE2iiNJyLSXDSeWIeSFGmUYRidgHygFhhpmublVg5JRNoojSci0lw0nliHPveS6/EDugK+1P0fCxGRptJ4IiLNReOJRaiSIo0yDGM78BFwP3CfaZovtHJIItJGaTwRkeai8cQ6OrR2AOJ+DMOYDVwyTfNfhmF4AgcMwxhvmmZOa8cmIm2LxhMRaS4aT6xFlRQREREREXEr6kkRERERERG3oiRFRERERETcipIUERERERFxK0pSRERERETErShJERERERERt6IkRURERERE3IqSFBERERERcStKUkRERERExK38P5p/O92FmQf6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1008x360 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(14, 5))\n",
    "for i in range(len(degrees)):\n",
    "    ax = plt.subplot(1, len(degrees), i + 1)\n",
    "    plt.setp(ax, xticks=(), yticks=())\n",
    "\n",
    "    polynomial_features = PolynomialFeatures(degree=degrees[i],   #use polynomial features of given degree\n",
    "                                             include_bias=False)\n",
    "    linear_regression = LinearRegression()                             #use linear regression\n",
    "    pipeline = Pipeline([(\"polynomial_features\", polynomial_features),    #put all togheter\n",
    "                         (\"linear_regression\", linear_regression)])\n",
    "    pipeline.fit(X[:, np.newaxis], y)                               #fit the model\n",
    "\n",
    "    \n",
    "    plt.plot(X_test, pipeline.predict(X_test[:, np.newaxis]), label=\"Model\")   #predicted function on the test set\n",
    "    plt.plot(X_test, target(X_test), label=\"Target function\")  #target function\n",
    "    plt.scatter(X, y, edgecolor='b', s=20, label=\"Samples\")   #plot data points\n",
    "    plt.xlabel(\"x\")\n",
    "    plt.ylabel(\"y\")\n",
    "    plt.xlim((0, 1))\n",
    "    plt.ylim((-2, 2))\n",
    "    plt.legend(loc=\"best\")\n",
    "    plt.title(\"Degree: {}\".format(degrees[i]))\n",
    "\n",
    "plt.show() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the picture above we can visualize the results of our model's predictions.\n",
    "\n",
    "What we see is that a too simple model leads to underfitting: it can't capture underlined relationships and we will get poor results. \n",
    "\n",
    "Instead a complex model causes overfitting: in fact, the fitted curve passes exactly through each data point, but it oscillates wildly and gives a very poor representation of our target function. \n",
    "\n",
    "The compromise is a simple model of degree 3: it doesn't fit perfectly the target function, but it performs significantly better than the complex model of degree 50, that fits noise and is so far from our target function.\n",
    "\n",
    "But does this mean that a simple model is always preferred to a complex model? Cleary not, in fact if you are dealing with a large dataset without noise, there is no reason to avoid a complex model.\n",
    "\n",
    "Let's repeat the same procedure, but this time we make use of a dataset _without noise_."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAykAAAE8CAYAAADE7v5rAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gU1dvG8e/ZTe+EJLQAoRMCSejSQZogRekREFCKigioKChKURBFBRHFgoK0IAIqCAgI0mtC70VCh/RASM/O+0cIL/5AWnYzu+H5XFcuSXZ35tlk93aePWfOKE3TEEIIIYQQQghrYdC7ACGEEEIIIYS4nTQpQgghhBBCCKsiTYoQQgghhBDCqkiTIoQQQgghhLAq0qQIIYQQQgghrIo0KUIIIYQQQgirIk2KEEIIIYQQwqpIk2JDlFJRSqlUpdR1pVSiUmqbUuolpZRN/h2VUh8opQ4qpbKUUmP1rkeIx0kBzJO/lVIxSqlrSqn9SqmOetckREFVAPMj9/kk3/xa8z+3D1dKXVFKJSmlflRKOepV6+PEJl9Mj7n2mqa5A6WBScDbwA+W2JFSymiJ7d7mFPAWsMLC+xFC3F1BypOhQDFN0zyAgcA8pVQxC+9TiMdZQcoPyHk+bje/Wt2279bASKA5EACUBcblQz2PPWlSbJSmaUmapi0DugN9lFJVAZRSjkqpT5VS55RSV5VS3yilnHMfp5R6Syl1WSl1SSnVXymlKaXK37xttlJqhlJqpVLqBtDsAbbXTim177ZPUoIf4jn8pGnaKuC6uX4vQoiHV0Dy5ICmaVm53wL2QMm8/3aEEPdSEPLjPvoAP2iadljTtATgA6CvmbYt7kGaFBunadou4ALQ6OaPPgYqAqFAeaAE8D6AUuop4HWgxc3bmtxlk88BEwB3YMt9tlcD+BEYBBQGvgWW5Q6DKqW+Vkp9bdYnLISwGFvPE6XUH0qpNGAnsAGIeMhfgRDiEdl6fgDzVc6U0TVKqZDbfh4E7L/t+/1AEaVU4ftsT+SRNCkFwyXAWymlgAHAcE3T4jVNuw5MBHrcvF83YNbNTwNSuPtw5e+apm3VNM0EpN9newOAbzVN26lpWramaT/dfMwTAJqmvaJp2iuWecpCCAux2TzRNK0dOQc0bYHVN/crhMg/tpofPcmZylUa+BtYrZTyunmbG5B0231z/+3+IL8Q8ejs9C5AmEUJIB7wBVyAyJx8AEABuXM5i/PvTxbP32Vbt//sftsrTc7Q7pDbHuNwcz9CCNtk03miaVomsEopNVQpdfrmNBQhRP6wyfzQNG3rbd9+pJTqQ86I0HIgGfC47fbcf8tUdQuTJsXGKaVqkxMKW4BYIBUI0jTt4l3ufhnwv+37u83X1m779/22dx6YoGnahEepXQhhXQpYntgB5cy0LSHEfRSw/NDIaYIADgMhwKKb34cAVzVNizPTvsR/kOleNkop5aGUagcsBOZpmnbw5pDo98AUpZTfzfuVUDkrU0DOG6yfUipQKeXCzbmc/+UBtvc98JJSqq7K4aqUelop9UBDoEope6WUEzmvQzullJPKnxU8hBC3sfU8UUpVVkq1UUo538yVXkBjYOND/zKEEA+lAORHKaVUA6WUw83jkBGAD5A7ujIHeFEpVUUpVQgYDcx+0N+PeHTSpNie5Uqp6+R8avAu8DnQ77bb3yZnad8dSqlrwF9AJYCbK2lNI2e+5Slg+83HpN9jf/faXgQ580CnAwk379c394E3V9745h7b/p6cT0bCbj6XVKD3PZ+9EMKcCkqeKGAsEA3EkLMccXdN0/bc7xcghHhkBSU/3IEZNx93EXgKaJM7UqJp2p/AJzdrPXvza8w96hRmojRNu/+9RIGklAoEDgGOty3dKYQQD03yRAjxqCQ/xN3ISMpjRin17M0hzULkLOe3XAJBCPEoJE+EEI9K8kPcjzQpj59B5EyHOA1kAy/rW44QwoZJngghHpXkh7gnme4lhBBCCCGEsCoykiKEEEIIIYSwKtKkCCGEEEIIIayKRS7m6OPjowUEBFhi00KIRxQZGRmraZqv3nU8LMkTIayP5IkQwhzulSUWaVICAgKIiIiwxKaFEI9IKXVW7xoeheSJENZH8kQIYQ73yhKZ7iWEEEIIIYSwKtKkCCGEEEIIIayKNClCCCGEEEIIq2KRc1KEbcvMzOTChQukpaXpXYp4BE5OTvj7+2Nvb693KUJIntg4yRNhTSRPbNejZIk0KeIOFy5cwN3dnYCAAJRSepcjHoKmacTFxXHhwgXKlCmjdzlCSJ7YMMkTYW0kT2zTo2aJTPcSd0hLS6Nw4cISADZIKUXhwoXlUyZhNSRPbJfkibA2kie26VGzRJoUcVcSALZL/nbC2shr0nbJ305YG3lN2qZH+btJkyKsklKK3r173/o+KysLX19f2rVr91DbCQgIIDY2Ns/3EULYLskTIYS5SJ7kH2lShFVydXXl0KFDpKamArB27VpKlCihc1VCCFskeSKEMBfJk/wjTYqwWm3atGHFihUAhIeHExYWduu2+Ph4nnnmGYKDg3niiSc4cOAAAHFxcbRq1Yrq1aszaNAgNE279Zh58+ZRp04dQkNDGTRoENnZ2fn7hIQQupE8EUKYi+RJ/pAmRVitHj16sHDhQtLS0jhw4AB169a9dduYMWOoXr06Bw4cYOLEiTz//PMAjBs3joYNG7J37146dOjAuXPnADh69Cg///wzW7duZd++fRiNRubPn6/L8xJC5D/JEyGEuUie5A9Zgljc07jlhzly6ZpZt1mluAdj2gfd937BwcFERUURHh5O27Zt/3Xbli1bWLJkCQBPPvkkcXFxJCUlsWnTJpYuXQrA008/TaFChQBYt24dkZGR1K5dG4DU1FT8/PzM+bSEEPcheSKEMBfJk4JPmhRh1Tp06MCbb77Jhg0biIuLu/Xz24dJc+WuHHG3FSQ0TaNPnz589NFHlitWCGHVJE+EEOYieWJ50qSIe3qQTxQs6YUXXsDT05Nq1aqxYcOGWz9v3Lgx8+fP57333mPDhg34+Pjg4eFx6+ejR49m1apVJCQkANC8eXM6duzI8OHD8fPzIz4+nuvXr1O6dGmdnpkQjx/JEyGEuUieFHzSpAir5u/vz9ChQ+/4+dixY+nXrx/BwcG4uLjw008/ATlzQcPCwqhRowZNmjShVKlSAFSpUoUPP/yQVq1aYTKZsLe356uvvpIQEOIxInkihDAXyRPLU3cblsqrWrVqaREREWbfrsgfR48eJTAwUO8yRB7c7W+olIrUNK2WTiU9MskT2yZ5YvskT4S1kDyxbQ+bJbK6lxBCCCGEEMKqSJMihBBCCCGEsCrSpAghhBBCCCGsijQpQgghhBBCCKsiTYoQQgghhBDCqkiTIoQQQgghhLAq0qQIqxMXF0doaCihoaEULVqUEiVK3Po+IyPDIvvcs2cPf/75511v0zSNbt26ERwczLRp08y2z/Xr17Njx45b33/11VfMnz/fbNsXQkieCCHMR/Ikf8nFHIXVKVy4MPv27QNyLork5ubGm2+++cCPz87Oxmg0PtQ+9+zZw6FDh3jqqafuuO3ixYtERkZy+vTph9rm/axfvx4fHx+eeOIJAAYPHmzW7QshJE+EEOYjeZK/ZCRF2JT27dtTs2ZNgoKCmDlzJgBZWVl4eXkxevRo6tSpw65du1i2bBmVKlWiUaNGDBkyhGeeeQaA5ORk+vbtS506dahevTrLly8nNTWV8ePHM3/+fEJDQ1m8ePG/9tmqVSsuXbpEaGgo27Zto2HDhrdC6sqVK5QvXx6AmTNn0qVLF1q3bk2FChUYNWrUrW2sWLGCGjVqEBISQqtWrTh9+jQzZ85k8uTJt7Y7evRopk6dCuSEUt26dQkODqZz584kJSUB0LBhQ0aOHEmdOnWoVKkS27Zts+wvXIgCTPJE8kQIc5E8sUCeaJpm9q+aNWtqwnYdOXJE7xJuGTNmjDZ58uRb38fFxWmapmk3btzQAgMDtfj4eC0zM1MDtCVLlty6rUSJElpUVJRmMpm0Ll26aB07dtQ0TdNGjBihhYeHa5qmafHx8VqFChW01NRU7fvvv9eGDh161xpOnjyphYSE3Pq+QYMG2t69ezVN07TLly9r5cqV0zRN077//nutfPny2rVr17SUlBTN399fu3jxonb58mWtZMmSWlRU1L+ew7vvvqtNmTLl1nZv/z4wMFDbvHmzpmmaNmrUKO2NN964te+33npL0zRN+/3337XWrVvftea7/Q2BCM0C73dLf0me2DbJk3+TPJE8EY9O8uTfbC1PHjZLZLqXuLdVI+HKQfNus2g1aDPpkR46ZcoUli1bBsCFCxc4ffo0oaGhODg48OyzzwJw5MgRKlWqROnSpQEICwtjzpw5AKxZs4ZVq1YxaVLO/tPS0jh37lxen9EtLVq0wN3dHYDKlStz7tw5Ll++TLNmzW7V4+3tfc9txMXFkZaWRsOGDQHo06cPvXv3vnV7p06dAKhZsyZRUVFmq10Ii5M8eSiSJ0Lcg+TJQ7HFPJEmRdiMv/76i02bNrFjxw6cnZ1p2LAhaWlpADg7O6OUAnJGB/+Lpmn89ttvlCtX7l8/37Rp0wPXYWdnh8lkAri1/1yOjo63/m00GsnKykLTtFu1PYh71X/7PnK3L4R4eJIn/96H5IkQj07y5N/7MFeeSJMi7u0RP1GwhKSkJLy9vXF2dubw4cPs3r37rvcLCgri+PHjnD9/Hn9/f37++edbt7Vu3Zpp06bxxRdfALB3716qV6+Ou7s7169ff6A6AgICiIyMpEaNGnfMD72bBg0aMGzYMM6ePUvp0qWJj4/H29v7P/fp4+ODs7Mz27Zto379+sydO5cmTZo8UG1CWDXJkztIngjxiCRP7lDQ8kROnBc24+mnnyYlJYWQkBDGjx9P3bp173o/FxcXpk+fTosWLWjUqBHFixfH09MTgDFjxpCSkkK1atUICgpi7NixADz55JPs37+f6tWr3/eNPWLECL744gvq169PQkLCfesuUqQIM2bMoGPHjoSEhNCzZ08AOnbsyKJFi6hevfodJ5jNnTuX4cOHExwczJEjRxg9evR99yOEeHCSJ0IIc5E8sQx1v6GbR1GrVi0tIiLC7NsV+ePo0aMEBgbqXUaeJCcn4+bmhqZpDBo0iGrVqjFkyBC9y8o3d/sbKqUiNU2rpVNJj0zyxLZJntg+yRNhLSRPbNvDZomMpIgCacaMGYSGhlKlShVSU1MZMGCA3iUJIWyU5IkQwlwkTx6cnJMiCqQRI0YwYsQIvcsQQhQAkidCCHORPHlwMpIihBBCCCGEsCrSpAghhBBCCCGsijQpQgghhBBCCKsiTYoQQgghhBDCqkiTIqzWhAkTCAoKIjg4mNDQUHbu3GmxfTVt2hRZllKIgkvyRAhhDpIl+UdW9xJWafv27fzxxx/s2bMHR0dHYmNjycjI0LssIYQNkjwRQpiDZEn+kpEUYRYxMbB7d85/zeHy5cv4+Pjg6OgIgI+PD8WLF2f8+PHUrl2bqlWrMnDgQHIvRtq0aVOGDx9O48aNCQwMZPfu3XTq1IkKFSrcuhpqVFQUlStXpk+fPgQHB9OlSxdSUlLu2PeaNWuoV68eNWrUoGvXriQnJwMwcuRIqlSpQnBwMG+++aZ5nqgQ4g6SJ0IIczFnnkiW5DNN08z+VbNmTU3YriNHjjzU/RcsMGkuHlla4dLJmotHlrZggSnPNVy/fl0LCQnRKlSooL388svahg0bNE3TtLi4uFv36dWrl7Zs2TJN0zStSZMm2ltvvaVpmqZNnTpVK1asmHbp0iUtLS1NK1GihBYbG6udOXNGA7QtW7ZomqZp/fr10yZPnnzr8bt379ZiYmK0Ro0aacnJyZqmadqkSZO0cePGaXFxcVrFihU1kynnuSUkJOT5OVrS3f6GQIRmgfe7pb8kT2yb5InkiTV9SZ7YNr3zRLIkbx42S2QkReRJTAz0f8mEZ+etuPXYgGfnrfR/yZTnTyzc3NyIjIzku+++w9fXl+7duzN79mz+/vtv6tatS7Vq1Vi/fj2HDx++9ZgOHToAUK1aNYKCgihWrBiOjo6ULVuW8+fPA1CyZEkaNGgAQK9evdiyZcu/9rtjxw6OHDlCgwYNCA0N5aeffuLs2bN4eHjg5ORE//79Wbp0KS4uLnl7gkKIO0ieCCHMxRJ5IlmSv+ScFJEnUVHgXCgNB7/rADj4XcfZK42oKFd8ffO2baPRSNOmTWnatCnVqlXj22+/5cCBA0RERFCyZEnGjh1LWlrarfvnDr8aDIZb/879PisrCwCl1L/28b/fa5pGy5YtCQ8Pv6OeXbt2sW7dOhYuXMj06dNZv3593p6gEOJfJE+EEOZiqTyRLMk/MpIi8iQgAFITnMiIdgcgI9qd1EQnAgLytt3jx49z8uTJW9/v27ePSpUqATlzQJOTk1m8ePFDb/fcuXNs374dgPDwcBo2bPiv25944gm2bt3KqVOnAEhJSeHEiRMkJyeTlJRE27ZtmTp1Kvv27XvUpyaE+A+SJ0IIc7FEnkiW5C8ZSRF54usLM78x0P+lBjh7pZGa6MTMbwx5/tQzOTmZIUOGkJiYiJ2dHeXLl+e7777Dy8uLatWqERAQQO3atR96u4GBgfz0008MGjSIChUq8PLLL//P8/Fl9uzZhIWFkZ6eDsCHH36Iu7s7HTt2JC0tDU3TmDJlSt6eoBDiDpInQghzsUSeSJbkL5Vzzop51apVS3uc13W2dUePHiUwMPChHhMTkzO0GhBAng8oLCUqKop27dpx6NAhvUuxuLv9DZVSkZqm1dKppEcmeWLbJE9sn+SJsBYFMU8kS/47S2QkRZiFr691vvmFELZH8kQIYS6SJ7ZLzkkRj42AgIDH4pMKIYTlSZ4IIcxBsuS/SZMihBBCCCGEsCrSpIi7ssS5SiJ/yN9OWBt5Tdou+dsJayOvSdv0KH83aVLEHZycnIiLi5MgsEGaphEXF4eTk5PepQgBSJ7YMskTYW0kT2zTo2aJnDgv7uDv78+FCxeIyetlnoUunJyc8Pf317sMIQDJE1sneSKsieSJ7XqULJEmRdzB3t6eMmXK6F2GEKIAkDwRQpiL5MnjRaZ7CSGEEEIIIayKNClCCCGEEEIIqyJNihBCCCGEEMKqWKRJuZaaaYnNCiGEEEIIIR4DFmlSzsanMHj+HmKT0y2xeSGEEEIIIUQBZpEmpYiHE2uPXKXl5xv5fd9FWc9aCCGEEEII8cAs0qT4uTuy4rWGlC7sytCF+xgwJ5Kr19IssSshhBBCCCFEAWOxE+crFHFnycv1Gf10IJtPxtDi840sijgvoypCCCGEEEKIe7Lo6l5Gg6J/o7L8OawxgUU9eGvxAfrM2s3FxFRL7lYIIYQQQghhw/JlCeIyPq4sHPgE4zoEEREVT6vPNzJvx1lMJhlVEUIIIYQQQvxbvl0nxWBQ9KkfwOphjaleqhCjfztEz5k7OReXkl8lWExMDOzenfNfIYT4LyaTRlpmNulZ2f95H8kTIYQ5SJYIW2eX3zss6e3C3BfrsHD3eSauOErrqZsY0boSfesHYDCo/C4nz8LDNfq/ZMK5UBqpCU7M/MZAWJjtPQ8hhHklpmSw+WQsW0/Fcvzqdc7E3iAx5f+vIeXuZEcRDycCi3kQ4u9J/XI+7NvgxoCXNckTIUSeyLGJKAiUJU5kr1WrlhYREXHf+11OSuWdpQf5+3gMtUoX4uMuwZTzdTN7PXmWcBbOboXYExB3GlITICudzCzFqi0+xBcxcdKpGJFxVdm2sBenTzri66t30UL8m1IqUtO0WnrX8bAeNE+sgcmkseVULHO2R7H+WDQmDTyd7alSzIOyvq74udlRPO0Ufon7cbp2Bucb58lOvQ7ZGaRjT3S6N5fx5rCxNBGx1dj/cxeiTtlJngirI3miv9Qb1zkVsZbks3sxxp/COfUKdqZ0DKZMLl9zJ9bOlfMGX/YmV2bjb88REeEvWSKszr2yRNcmBUDTNH7de5Fxy4+QmpnN6y0r0r9hGeyM+TYT7e5iTsC+eXBkGSScyfmZwR4KBYCrL9g5cC3JxIVT0RT3Oo+XugFAfFohssu3xbdJJyj3JBgMxMRAVBQEBCABIXQjBxWWteVkLBNWHuXo5Wv4uDnQtVZJWlYpQkgJD4xnN8O+BXByTc6HHAAObuBdBhw9SceO6OhrZFy7Sgm7aJxUzojL6esBuIQ+Q7FW3aFIFQDJE2EVJE/0kZKcxOG/5uJ0dDGV0g7ioLIAiMWLOLsiZBmcyMi2g9Tr+NrHU5w4DErDpCkOG4O4Uf4pKrZ4AW+/EpIlwipYdZOSK/p6Gu//dpg/D18h2N+TyV1CqFTU3ey13ZOmwT8bYNOncHYLKGNOo1G+BZRpDD4Vwfj/M+RiYiCgfDaenbdQtsgZql6/SJOYKLqHrMaQkQSFyxNhHESr0T0xeEBKnBPvvG1g0CAlgSDynRxUWMaVpDTe/fUg645FU9LbmWHNK9IupBiOZMG++bBlKiSeBUdPqPw0lGsGpRuAR3FQ/z/9IjdPvLtuoKrvcWqqE7QyRtDAeAQj2VC2KRsyXuXpN5rhXChd8kToSvIkf8VeOc+pXydQ9cpvuKlULqiiXPBrhkvlFgRUb4aHV+Fb9/3/Y5OtePnFUPnGJeomn6VrmbWUNUWRptmzxa4Vb//yBmevVZYsEbqyiSYFckZVVh68wvu/H+JaWiavNqvAy03L4WCXD6Mql/bCqpFwfge4F4O6L0FIGLgXuefDbs379EojNfHmvM+umXB0GZmbv8I+eg/ns/z48MIAZi58DYNLNg5ZTsz8VuaHivwlBxXmt/LgZUYtPUhGlolhLSrQt0EAjkYDHFgE68bBtYtQohY88XJOg2LvfM/t3Z4naakGnn37DCfiD9LDuJ4BTutwy4hha1o13o8azN+/dcbokil5InQheZI/UpKT2L9gNKEXF+JAJns9m+NSfwCBdVqhDP99bHTXY5MwRdTRCC79OY3qiauwJ4sFKa0YueAjYlKLSJYIXdhMk5Ir/kYGY5cdZtn+SwQW82Byl2CqlvA0Y4X/P2WiTLEkfPaPh90/5EzjavIWVO8N9k4Pva3/HTLdvUvjo6F/Mr75SKraRbEjLYi31YucvFqBpCUNiDpllE8tRL6RgwrzyTZpfLjiCLO2RhHi78l7LUPRrrlRzuM43ttezxmJLV4DnhydMxqrHvx/+v+bJ+fiUnj3t4PsOHmF7qnbGOa5AB/DNWZdf5rJ9p1JjPaVPBH5TvLEcnIzIOPiPMrtG0NRYonwaEGR9u9TskLIQ2/njmOT3dDrxVOMbPcOvez/IhUHJmT1Yu7ldiQtaSRZIvLVvbJE5xM/7s7b1YFpYdX5rndNYpPT6fjVViavPnbPZTsfRni4RkD5bMYM/psbnzXAtPtHqDMQhkRAnQEP1aBAzpu/du0753QGlFGsPtKKZvt/YODfkwl0PMOfDiMZWOwXHN1TWblSlgYUwtbcSM9i0NwIZm2Nol+DALp616NJHSd+eW86znMbk37+ELT/Avqvg/LNH6pBgTvzpFRhF+a8UId3n6rJHMOTNEmdwtdRYfRxW8lqh7dpWGQXju5pkidCFADh4RrVguP4Z343GuwbTDJuHGvzC7VeX/JQDQrc49gkAC6cLcOIgyOp+ccKDpnK8LH99ywsMZqyZY+zd68sXSysg1U2KblaBRXlr+FNeCa0BF/9fZqnp21h77mEPG0zJgYGvJTFe/1eY3mbZ8E1gyfDVxFT+xNwMu9oja8vzPzGwI0N1Zm5/QWaXvqaraYgxtrP4fuGgxjzwWUCymcTHi4XtRTCFlxLy6TnzJ2sPxbNBx2DeOWJIN4alsTSQW355Il32Z5ZhcozdhBTqi/cYyrGw1JK0bepP2+HNCQxtiifFG1P5+vjycLAAoeJ9C/zHa+/nyx5IoQNi4mBaZN3sH1Afbp6ruGLlM7U/HI7hcu2Mut+co9NUjaEcOhgbbpenMi7mS9Q03CC9e2fYvqE32nT9YbkidCdVTcpAJ4u9nzWLYTZ/WqTkp5F5xnbmLDiCKkZjzaqcu7UDeY/25eRXvP4w1SPp7MmcCitGlFR5q07V1iY4uw/RsaPMXLi5w50m/UHb6z9gI6VVrL+uabU6P4z/V8yyScWQli562mZ9PlxF4cuJjGjV0161wvg8uHTbOzZgiedIxif2ZsX1etcd/CwWJ4M7uvK7o/qU8W+NHsdytE6aga/nWjLJ83HMv/5ZyjaZZ3kiRA2aveKufzVviPOKp0eGaOZYuiMvZtmkTy5/dgkcXFjvp33IU2WriQx251fm/ZheM838ey8WfJE6Mrqm5RcTSv5sXp4Y8LqlOL7zWdo88Umdv4T93AbuX6V4D1taF96BeMSXmBo5mDio4uQmuhEQIBFygZyPrUY/a4i6rSRKRPcmH3hRcIyRuOhUlhWbASNy2+SqRpCWLHUjGz6ztrNwQtJTH+uBq2DikLUVqpub4aXfSLdoj/kx+w2pEd7WjxPihc1sPKDqoxqUZWs4ukM93yRDzJ70tIQwZISb1PC94LkiRA2RDOZ2D7nPdpGvcrxjNI8dflzdmmBZES7WzRPbj82WbXYlYmf1aLlkjX8aarNu/YLmFR8Os6eyZInQjc206QAuDvZM+HZaiwYUJdsTaP7dzsY8/shbqRn3f/BCVHwY2vsE0+y2X8hn8z+lOTwZiQtacDMbwz5cpKYry+0bQtpiU5svVqbThnjiEstxK9tuvPXzMUytCqEFTKZNF5ftI895xKYFladp6oWheOrYF4nDO5F2R60jjULBpEc3jRf82RQi9J82aUOyiWdmRnteClzOBW1i6x6ugPTJh+WPBHCBmgmEzu/e5V6/0wj0v1JjlT5m5M/d8zXPMk9d6V6dYi5XJSBF9/hm6x2PG+3lrldejBqfLTkidCFVa7u9SBSMrL45M/j/LQ9ihJezkzqFEzDCj53v3PMcZjTETJToediKFlb14sY5S4N6OiehvHaDVYNbkeow0kGx73JvLkjZWUNYRGyGs+jmbTqGN9sPM3opwPp36gsHPgFfh0ExUJy8sS1sK558ukPiUw7sBuURqm/7VnWoUl/G3IAACAASURBVDfpBnu6XP6IfT93lTwRFiF5knem7Gwipj9PnYQ/2OnTmdovf4/BaLSa45MXnviCT6t+zMbsYPpdeo+ri5tLngizs7nVvR6Ei4MdYzsEsWhQPRyMBnr9sJORSw5wLS3z33eMOw0/tQdTNvRbCSVrA/+96kV+CAtTRJ0yMmWCK6ZCrvTR3mKPVoGvC0+mV40FFpvPLoR4OL/uvcA3G0/Ts24pXmxYBg4tgV8HQun60GcZuOZcQE3PPHnzRS+WvlIPTzcD55pl8lzWuziQydJib1Ot9EHJEyGs0O0Nynb/F6jzykwMRiNgPccns3YPZUTmQBoZDjKn+Fi8/WIlT0S+stkmJVftAG9WDm3EoCZlWRRxnlafb2L9savExMD+jVFkz7rZoPRZDkWC9C73ltunfiVE+9E34212pFflq0ZDqWj6g5gYWQJQCD2dir7OO0sPUaeMN6/WD+L08uVoSwZAySfguZ/B0V3vEm+pVcmNn56vh5Zl5Gh2aXpkvIcywZKWPSjrfU7yRAgrEn3VxKbP+t9qUOr1n3LPCzPmt9uPT+Zfbs+bmYOobzjC7I49cXJMkywR+cZ63hV54GRvZFSbQH59pQEezna8MDuCZq+sw2NZexJjUljl+xv4Vda7zDvkLgOYtKQB0eFP8ezMFSS6VMftz34833SDLAEohE5SM7J5Zf4eXByMNHcO5cXmGym1+wV2XqzBL44/g4Or3iXeoUZFV96uVRdTigPHU8rS4fdF+HqmYL/wGeoEX5Y8EcIKhIdrzHtnOE1TljL9Yk/OOH+qd0l3dfvxyZz57/P64fdo7rabkz+G0bbrNckSkS8KRJOSK6SkF7N6NIQ9xZgf+CZF3K7SJ+VtugyrYrVdf+7Q6qrFrhw+6gk9f+F4bBl+6dyDOmGz8ey8VZYAFCKfjf/jCCejkxn7VCizxp1gYafnOKOK0Df7dfq+4mq178dX+7qx+NW6eHpqGHrc4HyredilXGJZv7b49fhT8kQIHcXEQMTSybxecjbhWc2YmN2NAa9Y76jE7ccngz54g3GnX6WT13o+6dVfskTkiwLVpABcOqfxfakPqKrO8GrmEA65lsKrbST7j6XrXdp/un3+6Zkr3nRbt4BrypkfHD7F3+8czl5pMg9UiHyy6UQM4bvOMbBRWUqlpvNrp27cUE70zXibNF/N6t+PdSu7M39gHa6lZ9J3PbywZQZV7M/whf10nPySrL5+IQqq3avC+aTKRNZlV2d01gs4+N2w+vdj7vFJcjJ8ufkdZma1oZ/dagYUW2z1tQvbV+CalKBL79HSfz0j419hvakGWckO2PnH8sa6jfy+7yKWWM3MnAIC4MzFMjx/ZSzupDDT8DkqOdui110QQuS4lpbJyCUHKOfryvAmJahyoAcudjfoeWUclyls8esWmEuwvxff9q7JpeRkNpUsxuiEQbQ07mFkxiKbqF+IgubM4Z00OT2UgxnlGXhpFNkYbSZPIOfYJDXBibGXBrI2uybv282hecVlNlG7sF0Fq0nZMxeX/V9zrNDLfD3nA5LDm5Iw90mGV2lIgK8rQxfuY8CcSK5eS9O70v+UOw90+8JeDPjrO4LszhA56lV8fay7uRKiIJi44ihXrqXxaZdgnFYMwT72EHvL/sDOhT3z/TooedWgvA+fdA7GwT+OWed78M3hFxnosYz1k+bZRP1CFBQJMZdxXNyLFOXCwQq/cGVxC5vLk9xjk4QlTXjx5wUcySzLD/WHkRK3R+/SRAFmp3cBZnN+F/wxHMo2o3LPD4l6zkhUlOvNdcY9GGKqz49bzvDpmuO0/Hwj77WrQpea/iil9K78DmFhihYtjERFdSQ1YRyltr0HW6dCw+F6lyZEgbU7Kp6Fu88zqHFZqp/9AY78Bi3H07RBa6Ke4bY80bvSB9e5pj8XE1P5nBOcDxpERsYZ6l56Ey5UBf+aepcnRIGXlZnBpe+7Ud6UwNmOi+ldowxPtbfNPPn/Y5Oi2Dv9QvqSZqhFvbg+dAvunt56lycKIJsfSYmJgb1bYsle2Ac8/aHrLDDa3bHOuNGgGNC4LH8Oa0zloh6MWHyAvrN2czExVd8n8B9y63drOQSCOsG68XB6vd5lCVEgZWWbeO+3QxRxd6Kd6SLa+glQrRvUfw3Q97oFeTXkyfJ0CCnOgiOn2Fp9IrgXhUW9IVnOeBXCkmJiYMO04QRlHOBAjfFUrNEUsO08ya29crVyXGn1DcWzL3Pq256YsrP1Lk0UQDbdpISHa5StkEHSnL5kJMWzyuMncC50z8eU8XFl4cAnGNu+Cruj4mk9ZRPzd5613nNVlIKO08G3Mix+EZIu6l2REAXO/J3nOHblOsl/eFFsywCOxlZiUdqUnPefjVNK8UmXYIJLeDL4t7NEtfgeUuJhyQs515ASQphdeLjG0N5zaXF9HrOin+FUyst6l2R2QfXbElH5TaqnbGPn/LF6lyMKIJttUo4ehRcGmBjb7zWaFt/Mu0kD6TKs6gMth2cwKPo2KMPqYY0JKenJu78eoufMnZyLS7F84Y/CwRW6zYGsdFjSH7Kz9K5IiAIjNjmdT1YdJ+ucF7OaDMbJMYVBGUPp94pzgVle08neyHfP18LV0Y5+f6aR2nISnNkEmz/TuzQhCpyjR2HC2MPMqPsW+01lGZfWu8Au11u3+yj2uDWh9unpHNu1Vu9yRAFjk01KeLhG9drZ1CvzN8M8FrI4uzFLXZ946OXwSnq7MO/FukzqVI2DF5JoPXUTs7aewWSywlEVnwrQbgqc2wYbPpIrSAthJq/OOElyejavpv1BHaejjMrsz3lvzwK3vGYRDye+7lmD8/EpDDkahFatG2z4iMR9WyRLhDCT8HCN2vVS+KnTCyhgcOZQ8EsvcHmSSxkMlO8/i2iDL14rX+Kfk1clT4TZ2FyTEhMD/V8yUfqZdfzUcghRWcV4P7PvIy/lp5SiR51SrB7emLplvRm3/Ajdvt3O6Zhki9SfJyHdoXovtM2fyRXphTCDPSdusD36HHUyTvBWtRksTG7BclN9m1oa9GHUDvDmvXZV+OtYNN+5D+aaXVlS5vUnrOcFyRIh8ij3+GRi/1ep6XicEUmvcEHzLbB5ksvDqzA3OszEW0vg+Ix+tOl6XfJEmIXNNSlRUeBcKJUpZSZTxC2aHvPncOq7p0hYlLel/Ip7OTOrb20+7xbCyehk2nyxmW82niYr22TW+vMqpvYnnIwvxw/PvkjxHn/IVV+FyIPJa47jabrBNM8v+Cfdn4Hffc+l75rmOU+s2fP1StMxtDgf/32Rdn98io9rDF9074dn5y2SJULkQVQUtKr3O6+6LmVBSgtmfvt2gc+TXF7+jZlw6lXaeGylX8/xcmwizMLmmpSAAOhU7DfaGnfxaXZXztXPwpDhxJ7dRsLC8naSq1KKTjX8Wft6Y5pV8mXSqmN0nrGN41eum6d4M4i66MrgrdPwNSbwgf1sHPyuF9hhZCEsaf/5RLZfuMy4tIX4kMQwNQjndsfNlifWSinFxGerUdTVlag6WUzJ6sLTxl30KLpSskSIPPAtHMf02iM4oxXlA8Nz+DwbWeDzJFdUFHy99U22ZVdhjN0cyhc5JXki8szmmhRf+/NMbzOKzRfqM3X+JySvqMOs7w0EBppvH37uTnzTqybTn6vO+YRU2n25mWnrTpJpBaMqAQGw7dQTfJ4URkfjNtom7ynQw8hCWMqna47Tw3k3z/r+xYRtI9mx4AWL5Ik1cnW047NONVB2WcxI7chOU2XG2f2ErylaskSIRxSzbDBFDXEMWPMVMeFPPTZ5AjnHJikJrrwW/TomDHxu9zXp1+wkT0Se2EyTEhMDu3eZyPjlFRzsTFQZNYMVv3gQdcoyn1AopWgXXJy1wxvTpmoxPl97gg7Tt3LoYpLZ9/Uwcq/6OmH25+yOrskHrj8wb3pMgR5GFsLc1u9P4NjJk4wx/AglajF43uusWuxqsTyxRvWqeNC5TBA4aLy6fyJkGfl7yFB8fWQeuRAPIyYGVv40j9qJq9hVsh8/L2/+2OVJ7rHJ0YWdePvIaGoZTzJ75EQ5NhF5YhNNSni4RkD5bBa9NwOHi5vYWWgihcsF5MvFkAq7OTItrDrf9a5JbHI6Hb/ayqerj5Oepd/1BcLCFKdPOuLU9Ru83VJ41u51sNbrvAhhZcLDNXpOOsFHxh/Q0jL4w2EGvkXsbPbiannx2eCStKhYjLgq6VyqM4qiKRsh4ke9yxLCZoSHa9SscZHap9/hYEY5/nGaaNMXa8yLsDBF1Ckj/d58mUinBrRL+Yqzx/fpXZawYVbfpOSullGl26+MeWI8G1Kr8+TInvl+MlaroKKsHd6YZ0JLMP3vU7SbtoW95xLyt4jb+PpCtWYVUU+OhuMr4OAvutUihK2IiYGX3kuga5nfaWG/h4+Tn6P7a+Ue25M7lVJ8FlaNYp5O9D8eSFZAE1j7PiSc1bs0Iaxe7vHJlN6D8FLJvJ74KgMHOzy2eQI5xyZ16hoo3fcbUpUjab8MJCszQ++yhI2y+iYldzWvz4pOBeBd1Rdnr3RdTsbycnHgs24hzOpXm+T0LDrP2MaEFUdIy9Txqs31BoN/HVj1FiQ/xskoxAOIioIyDXYyzv4n9pjKM9e52WN/cqensz1f9AjlYmIaH9sPBhQsHyqjs0LcR1QUPNM4nM4Om/k6uwMnPYo89nmSy6doKU7VGkOlrONELPpI73KEjbL6JiUgADoX+5XGxoNMygrjTHRZ3U8Ub1bJjzXDG9OjTim+33yGNl9sZteZeH2KMRihw5eQcQP+HKlPDULYCJP7dT4sNRUX0ngrcyBp0Z6654k1qBXgzeBm5fn+YBaHqwyHf/6GAz/rXZYQVs23cDyfhr7H8Wx/vsp6psBfD+Vh1Wzbn30u9Qg+8RWXzhzTuxxhg6y+SfF1jmFam9FsuVCPb+d9QNIS61hv3N3JnonPVmN+/7pkZpvo/t12xi47zI30rPwvxq8yNHoDDi0mafcaudqrEP/hyLZ5tDPu4ONdb7BvQS+ryRNr8FrzCgT7e9JrfxAZxWvDnyOJPRsjeSLEf7i68g2KGBJ4ee004sNbSp78D2UwUDRsOhqK2J9fIfqqSfJEPBSrb1JY/Q6O3KDKiC9Yudjd6lbLaFDeh9XDGtOnXgA/bY+i9dRNbD0Vm/+FNBxOkkNlroUPo0tYtFztVYj/ERMXR6uzn3LVqQwvz3rzsVt9537sjQamdA8lNUtjrDaI7NRk1r01kjZdb0ieCPE/jkWso3bMr+z26/xYrub1oIqWLM+hKsMJTovk/UEzJE/EQ7HuJuX0eji4CBoOx7tSJatdLcPV0Y6xHYJYNKgeDkYDPWfuZNTSA1xLy8y3GmISHHl2zhRKelzk3R7D5GqvQvyPs0veo4SKJavtFHyLOlhtnuipnK8bI5+qzIIzLnx26gW6By6m9XNfSp4IcZvMjHQcVr5OrCpElV6TH9vVvB5UQNMRRKZV5sOQSZTo8YfkiXhg1tukZKbCH6+Dd7mcqUw2oHaANyuHNmJQ47L8vPs8rT7fxN/HovNl31FRcCAthAVZzXjBuIrgIoflBD4hbko7v4/QS+FsdH+aEsHN9C7Hqj1fL4Ag38LMLNKQM6YifGA3C3e/OMkTIW6K/HkCZU1RXKg3HndPb73LsXrnzhl5c9vHeJHM23YLcfC7LnkiHoj1NilbpkLCGWj3Odg76V3NA3OyNzKqbSBLX2mAh7Md/Wbv5vVF+0hMsewSfAEBkJrgxIdXBpCEKx8ym7REBzmBTwiTieQlQ0jQ3HBt+6He1Vg9g0ExsWMwaZoDo1P7U9ZwhQFpa+SEYCGAK+dPEXzqG/a51KNG6956l2MTAgJg1/5G/JjWlufs1hN8/ZzkiXgg1tmkxP8DW6ZA1c5Qtqne1TyS0JJeLB/SkCFPlmfZvku0+HwTfx66YrH95V7t9ewvT/H+lg+o5XiMdZMWyPCzeOxpe+fik3iA2a4vUrNyGb3LsQkh5V3oXDaQrcYgfo1uxatuS1g4LUryRDz2Lv08HAMm/Lp9oXcpNiP3+OT9OdO4lF2YSZ7f8N3XWZIn4r6sr0nRNFj5FhgdoNUEvavJE0c7I2+0qsRvgxvg5+7IS/MiGbxgD3HJ6RbZX+7VXvt+1JeMYvWpe20spOi0NLIQ1iAlnuw1Y9hlqkSJxv1QSk5qfVCfDy5FnVI+fO7dCXtne9rbjdK7JCF0dXDjUmokb2JvwIsUD6ikdzk2JSxMcfBAYfaVfY8qDlGUN36ud0nCBlhfk3JsBZxaC81GgUcxvasxi6olPPn91Qa82aoiaw5foeWUTSzbfwnNAhdL8/WF2nUUDh0nQ1oS/G3bjZ4QebJuPCr9GpPUADpWL6F3NTZFKcVnPaoRbyjEQtfn4ORqOL5K77KE0EV6WgpeG97lvCpOjbAxepdjk3x9oU2fPhx0rE7gsWnEXb2gd0nCyllXk5KZCqtHgW8g1BmodzVmZW808OqTFVjxWiNKervwWvheBs6NJPpammV2WLQq1BkAET/C5f2W2YcQ1uzyfrTI2czNbkVwzfq4OtrpXZHNKentwsi2gYy50pAkt3Kw6m3ItFBmCWHF9iz6iJLaJRIaf4Cjk4ve5dgsZTDg0Wkqzlo6p8NH6F2OsHLW1aRs+xISz0Gbj8For3c1FlGxiDtLXqrHO20rs+lEDC0+38jiyAsWGVWh6ShwKQwrR+RMoxPicXFz2miavRefZ3ai1xOl9K7IZvWsU4oaZfx4I7knJJ6FbdP0LkmIfBVzKYqQ09+y16U+wc266F2OzStdKZTIYj2ok7iS4xHr9S5HWDHraVISz8Pmz6FKRyjbRO9qLMrOaGBg43KsGtqISkXdefOX/fSdtZtLianm3ZGzFzQfA+d3wsHF5t22ENbs4C9wfgdfqucILFOS8n7ueldkswwGxaRO1diUVYVI18Y5OZ0k0zTE4+PswjcwYsKvy6d6l1JgVH1uAjEUgj9HYsrO1rscYaWsp0lZ+17Of1s9PkuElvV14+eB9Rjbvgq7zsTTasomFuw8Z95RldCeUCwU1r4PGTfMt10hrFV6Mqx9n+TCwcy4Vo/utUvqXZHNK+vrxrAWFRga35lskwnWypx88Xg4tnMNta79xR7/XpQoG6R3OQWGm0chokJHUCnrOJHLv9G7HGGlrKNJObsdDv8KDYaC1+M1LcNgUPRtUIbVwxoT7O/JO78epOfMnZyLSzHXDqDNJ3D9Us6yzkIUdFunwvXL/Oj+Eq6ODrSpWjAW4NDbgEZl8Shajlm0h0OLc3JbiALMlJ2Nce07RONNcI+xepdT4NRs/xIn7CoSsG8yydcS9C5HWCH9mxSTCf4cCR4lcpqUx1Spwi7M71+XCc9W5cCFJFpP3cTsrWcwmcwwqlKqLlTrClunQcLZvG9PCGuVeA62fUlmlc7MOF2Y9iHFcXYw6l1VgWBvNPBRp2pMSW1Lor0f/Pl2Tn6Lx4qmaczc/I/eZeSLyOUzqJB1krPVR+Dq7qV3OQWOwWhEe+pjfEng4EIZnRV30rVJiYmBf34Lh8v7oMVYcHi8V8xQStGzbmnWDG9MnTLejF1+hO7fbeefmOS8b7zFOFAG+Gts3rclhBWKiYG4n8egoVhV5CVSM7PpVstf77IKlJCSXnSvH8iYG11yVg3cH653SSKfnYtP4cMVR/Uuw6JiYmDL5kQC9n3KCbuK1Gw3SO+SCqxKtZ4kwqMlNS4u4FLUcb3LEVZGtyYlPFwjKDAJ1x1j2XmpFuGHZMWMXMW9nJndrzafdg3h+JXrtPliM99uPE1Wdh4+tfQsAQ1eg8NL4dxO8xUrhBUID9fo0XgrhS8vZeLmIXy6MZMKfm6ElpRPP83tjVYViXRvzlFDRbR143POARKPja2n4vQuwaLCwzUCymeze977+JLAzkKTMBhlNNaSSnb7GBOKy0ve1rsUYWV0aVJiYqD/SyZG9R5BEZdoPjB0p//LGjExelRjnZRSdKnpz1+vN6FxRV8+WnWMzjO2cfzK9UffaIOh4F4s51o0Mk1DFBAxMTDgpSwmPzOMy5o3P5auybmURNpU9pcrzFuAq6Md456pxrupz6GSr8DWL/QuSeSjradjKerhpHcZFpF7bBLYYykvF13AbxkNeOmdpnJsYmFF/Muxr9Tz1Lz+N8d2rdW7HGFFdGlSoqKgbIl/eMnjV5Zl1+OQVwmcvdKIitKjGuvm5+HEd71r8mVYdc4npNLuy818ue4kmY8yquLgCs3fh4uROSe+ClEAREVBzxoLqeF4gk8zu5HtnY2mQRXX4nqXVmA1DyxCkaDGrDDVw7R1mixJ/JgwmTS2nYqlfvnCepdiEVFR4FwojXf8ZqOAyaZucmyST0K6v08MhTCseQdNPkQVN+nSpAQEwMiqn6LQ+CSrBxnR7qQmOhEQoEc11k8pRfuQ4qwd3pjWQUX5bO0JOk7fyqGLSQ+/seAeUDQY1o2HTDNfl0UIHQT4p/Ju9UkcyCjHUlNDtCxF5iVvalZx1ru0Am1M+yC+UD3Jys5GW//4LB3/ODt65RoJKZk0KOejdykWERAAVf1208m4hR+y23Amuqwcm+QTFzdPzoS8QcWsE0Su+kHvcoSV0KVJ8c3cS88qC/kq4hWOLehK0pIGzPzGgK+vHtXYjsJujkx/rgbf9KpJ9PV0nvlqK5+uPk561kNcCMlggNYTIOk87JS1yYXt8z31NaU8LvDW71O4sboWyk6jR90SkicWVtTTiedaNeTHrKdg/8KcE+lFgbbt5vkoDcoXzCbFp7CJL9q+S6zJg08WfyTHJvmsZvuXOW0sQ4ndH5OWKtd1E3o0KZoGa94Dl8L0+fYNVi12JeqUkbAwmTv+oJ6qWpS/Xm9Mx9ASTP/7FO2mbWHf+cQH30CZxlCxTc6Vo2/EWq5QISwtOQY2T4FKbZm7oQmdX4nH3mhgzItybZT80LteABt8e5GEG1mr3snJd1FgbT0dS1lfV4p6FsxzUvb9tYAQDnGk/BB+mesvxyb5zGhnR2rTcRQjhn2LP9a7HGEF8r9JObEaojZDk5H4+HtSuzbyKcUj8HJx4LNuIczqV5vk9Cw6fb2Vj1YeJS3zAUdVWo7PuQL9ho8sW6gQlrTxY8hMgRbjKORtYvfVS7QI9MPT2V7vyh4LRoPinc5PMCWzE3bntuTkuyiQMrJM7DoTX2CnemVmpFN4x0TOGvypHzZMjk10UrVRR/Y716HK6e9JiLmsdzlCZ/nbpGRnwdr3wbsc1OqXr7suqJpV8mP18MZ0r12Kbzf9Q5svNrM7Kv7+D/StCDX7QuRsiD1l6TKFML/YUxA5K+d17FuR7f/EEZucQcdQOWE+PwX7e6Fqv8AZU1HSVo3OyXlR4Ow7n0hKRnaBneq159eplDJdJKHeu9jZO+hdzmPNq8MkXLVUjv/yvt6lCJ3lb5Oydy7EHoeW48Aon3Sai4eTPR91qsb8/nXJzDbR7dvtjF12mJSM+xwsNB0Jdk6wbmy+1CmEWf01Juf123QkACsOXMbVwUjTSn46F/b4ef2pIGbY98Yp8SSmvfP1LkdYwJaTMRgU1Ctb8Fb2Sr6WQIWj0znsUI2Q5j30LuexVzqwJpGF21Hj6hIu/nNY73KEjvKvSUlPzplaVPIJqNwu33b7OGlQ3ofVwxrTp14AP22PovXUTWw7dY9zTtz8cq6dcnS5XOBR2JZzO+DYHzmvXzc/MrNNrD58hRZViuBkLxdey28eTvY0aNeXCFNF0tZ+kDOVVBQoG0/GElLSC0+XgvcB46FFH+DNNeyfmoAy6HaNa3Gbsl0mkIUdV5a+o3cpQkf5927c/hUkX4VWH4BcYM1iXB3tGNshiEWD6mFnMPDczJ288+tBrqdl3v0B9QaDWxFY+56c9CpsQ+7iG25Fc16/wI5/4khIyaRtNTlhXi8dQkuwzO8lXNJjuLFhqt7lCDNKuJHBgQuJNKlY8E7SiL10luDz84h0b0bFGk30Lkfc5FO8NPtL9aZm8gaOR6zXuxyhk/xpUpJjYNs0CGwPJevkyy4fd7UDvFn5WiMGNi7Lwl3naDVlE38fj77zjg6u0OwdOL8z55NpIazdsT/gwi5oNirn9QusPJgz1asgHkTZCqUUz3frxp+mOhh3fJmT+6JA2HIqFk2DxgXw/XV68WjsyKLosxP0LkX8j2pdRxOHJ9lr3pcLPD6m8qdJ2fhxzoUDm4/Nl92JHM4ORt5pG8iSl+vj5mhHv1m7eWPRfpJS/mdUJbQX+FSEv8bJSa/CumVnwl9jwadSzusWyMo2sfrwVZoHylQvvZX3c+dc6JvYZadz9Y/xepcjzGTjiRg8ne0J8ffSuxSzOntsD7XilrPHrxMlygbpXY74H24ehTgVOJgqGQc5sGGR3uUIHVi+SYk7/f8r8PiUt/juxJ2qlyrEH681ZMiT5flt30VaTNnImsNX/v8ORjtoMRbiTsLeOXqVKcT97ZkDcadyXq9GOwB2/BNP/I0MmeplJXq1a84yY0sKH5tPVoysHGjrNE1j88kYGlbwwWgoWFO145e9SypOVOw6Tu9SxH+o8ewwLqhieGyZSHaWfIj6uLF8k7JuPBgdb63AI/ThaGfkjVaV+H1wA3zdHBk4N5Ih4XuJS07PuUOltjmLGmyYJCe9CuuUnpzz+ixVDyq1ufXjVYcu4+JgpGmlgjcVxRa5ONjh1XY06Zod5xaP0rsckUfHr17n6rV0mlQoWO+vYzvXUD1lGwfL9MXbr4Te5Yj/YO/gyNXab1PGdJbI5V/rXY7IZ5ZtUi5EwpHfoP6QnJWkhO6qlvDk91cb8EbLivx56DItp2xi+f5LaJCzqEHy1ZxFDoSwNju+hhvRORcivbn4hsmksfbIVZpW8pWpXlakWc1qrPXsStmra4g/sV3vckQebDqRc25Ro4oF5/oomskEf40hhkKEdJFG2trVeKoPJ+wqUnr/VNJSgz3SLwAAIABJREFUkvUuR+QjyzUpmpZzHQNXX6j/qsV2Ix6evdHAkOYVWPFaI0oWcmZI+P+1d5/RUVVvG8avM+k9QEIvoXdpoXekiAVUmliwIUWwgIoiSlEsiKLYAEX/KkqUIiIKIigQilJD72Ug1AykkJ5MeT/kRbFRk5xJcv/WcukKkzN3lsyT/Zy9z94xDJ61mbjQG3K2h177LqReYutikfyWehbWTs35+3nR5hvbTyQRl5xJlzqlTAwnf2cYBo3uGss5VzBnvxutnQMLsJX7bNQoFUiZED+zo+SarctnUyt7N4frPoZ/YIjZceQyDIuF7E4TKMU5ts5/w+w4ko/yrkk5uBysq6HdKPAJyrO3kWtXo1QQ84e2YnT3Wqzcb6PLlGh+Lj0YV3YqRL9pdjyRP0VP/v/NN8b95cs/7zqNh8WgU001Ke6mUtlSbKsyiBppMexZs8DsOHINkjOy2XAknk61Cs/ny+VyUfz31zlmKUeT2x8zO45cobqtbmabXzPqHJpJUrx2Diwq8q5JWTYOilXOeWBe3Janh4XB7auy5Im2VC8ZyKCfkokO6IZr40zOHbKycSPYVA/ETPZM2PgJNL4Pwmv85Y+W7T5D88rFC+UBc4VBy75Pc8IohfeKCfz2u121pIBZfeAsdqeLTrUKz3LttMQ4KjljOdt8NJ5e3mbHkasQdMtEAl1pbJ89XmOTIiJvmpT0eIjbBTe+CJ4qAgVB1fBA5gxuybjb6jA2qQeZDvh10mi690khopqDqCgt1xCTJJ8Giye0/+vmG4dtKRyIS6Grlnq5LT8/P9YGj6Kq08rs995VLSlgftkTR4ifF40rFp6th33S49jnWYtGXe4xO4pcpSr1mrPKswtNbd8wcMgO1ZMi4LJNimEYww3DKHZVVz1/Cso0hDp3XHMwyX8Wi8GDrSvzzoDb+TjxDvqUXUzz/p8T0mc1A4c4ddeiIHNkX/41+eCa6kl6PLR8FIL/usXwst1nAOisJsVt2Wzw0PgB7HBE8Gy1Dwjv86tqSQHhcLpYuS+ODjXD8fT421ChANcTT+w4bhyPYcmfY+Ik99hsMOzLlzFw8Xz3FwjptVb1pJC7kk9paWCjYRhzDMO4yTCMy2+U7sjKOcdARaBAsif68/aS8SS6Ahjl+TXeYakENzvI4SO6Y1FgzXvI7AQXXH09sXhC6yf+8eVlu89Qt2ww5Yv550FMyQ1WK/gWy2aSoz/ljbM8VGohfqEZWK1mJ5PL2XY8kXOpWf9c6pWZDO81MSfUP111PUmzBFKnZffLvUzckNUKcZkV+MLRld4eq6hbao/qSQG3+/efLvnnl+0iXC7XC0B14BPgAeCAYRivGoZR9T+/yb8EVO14VUHFfUREwJkzpZma2I/2HttpYd+DT5ODvLb+Nw7btP1fgRO7EfZ8b3YK4BrrSVAZ8P3rDjwJqVlsOZbAjbU1i+LOIiIgPcGXX0+3YrWjHsM9F+AfcpqICLOTyeWs2BuHh8WgfY2/nY/y2weQeNScUH9zLfXEI6RsPqWT3Hahnrxz5m5S8eUZyxzSE31VTwool9OJx/Jxl3zNFU11uFwuF3D6//+xA8WAeYZh/PtecKEVryqouJfwcJg53cLUWRM5nlKWZ1IW0bt8fQ6fS6b71NV8FH0Ih1OzKgXCxVuBu4mrricB/zyfYdV+G04XheqB3sLoQi1Jmt+a8UsmUdxIYfwdrxNa3Gl2NLmMX/bE0aRSMUL9L3quNMUG696D2j3MC/Y3V1tPfPwC8jGd5KYL9cQ652beP34/Xb02Mf2VZYS7z683uQoxy76ipn3vJV9zJc+kPG4YxmbgDWAtUN/lcg0FmgC9ciOouJ/+/Q327Qsgq/WLNCm9lTc7bmb5yPa0qxHOq4v3cue0dew/k2x2TLmcA8vg6Fpo/6zZSYDcqycr9sVRIsCbG8rpjAN317+/gfWgB69Pa0Ns6Vu4i4V88+tGs2PJJZxITGf3qfP/vAnwx1bgY80J9jcanxQ9F+pJhwdfwEYxmiZNyDmcUwoUe3YWYetf46il/CVfdyUzKWHAnS6Xq5vL5ZrrcrmyAVwulxO49fqjirsKD4cqPftByTrwy0uUDPDgo/ua8G7/Rhw7l8qt767h/V8PkO1QgXBLTgcsH+9uW4Ffdz1xOF2s2m+jfc1wLJbLP9Ii5gsPh6ZNoULfiXgbdjzXTObM+QyzY8l/+HnXaQC61S395xfjj8CmT6HxAAirblKyf9D4pAgKD4dWbYI4Uu9xamXvZuvy2WZHkqu0ZeH7VHSeIL7F6Eu+7kqeSRnrcrn+dQGqy+Xac435pKCweORsghB/GLZ8jmEY9GhQlmUj29Olbine/Hk/Pd9fy66TSWYnlb/bPufPrcA93OMckdyoJ1tjE0hMy6ZjTS31KnCKVyH9hvvpzS/MWPCz2WnkPyzddZoapQKpHHbR0qhfJ/7/VuDuMSsLGp8UdY17DueYpRzFf38de3aW2XHkCqWnJlN557vs9axNw853X/K12n5LLq96V6jUGlZOgsycB+fDAn344O7GTL+3MXHJmfR8fy1v/byPTLvD5LACQHYGrHgFyjYqdFuB//r/D/S2q66FyAVRUNfncXr60uTge6w7eNbsOPI38alZbDgS/9dZlJMxsHMetBz2j63ARczi6eXN2eajqeSMZcv3H5odR67QtrmvEU4Cri4TLrsVuJoUuTzDgM4TIDUuZ2eXi9xUrwzLR7ajR8OyvPfrQW57bw3bYhNNCip/2PgxJMXm/H8rZFuBr9hro0nFYjplvqAKDMdo9Ri3eGxg9rffkmXXclF3snzPGZyuvy31Wj4e/IpD68dNyyXybxp1uYd9nrWI2DGV9FQ9J+vuEmynqHvkU2L8W1G7ebfLvr5wjV4k71RoCrVvg3Xv5uzwcpFQf2+m9G3I/x5oyvl0O3d8uJbXFu8hI1uzKqZIT4DoN6FaZ6jS3uw0uep0Uga7T52no3b1KtC82jxGlk8J7kv5hE/XHDY7jlzk512nKRfqR92ywTlfOPgLHF4J7Uf9YytwEbMZFguOzhMoSTxb571udhy5jH1zx+FPBsVvm3hFr1eTIlfuxnE5O7usmvSvf9yxVkl+HtmOfk0rMCP6MDdPXc0ma3w+hxTWvAMZSTnPEhUy0QdyGuR/nN0gBYtPEN43jqa5ZS/bfvmGk4npZicSIDXTTvSBs3StWwrDMMDpzNnCPLQiRLrNgbAif1GnxU1s9W9J3SOfknj2tNlx5D+cPLKXxmfms7lYdyrVvrIDYdWkyJULqw5N7ofN/4Nzh/71JcG+Xrx25w18NbA5WQ4nfWb8xvjvd5GWZc/nsEVUYiz8Pg1u6Ael65udJtetPnCWsEAfapcJMjuKXK8mD5AdWoURltlMXLTD7DQCrNxnI8vu/HOp1465cHoHdBoLnj7mhhO5hNDbJhLgSmfv3EsfDijmOblgDA4sVOp9ZbMooCZFrlb758DDB36ZcMmXta4WxtIn23F/ywg+W2el2zvRekg2P6x4NeffncaYmyMPOJ0u1hyw0a56WM5dXinYPLzw6jKOGsZxAvfOYeW+OLMTFXk/7jhJWKAPTSOK52y+8evLUKYB1NORI+LeImpHsrlYdxqfnsfJI5c+IFDy38Fta4g8v5yt5fpTqnzVK/4+NSlydYJKQavHYPdCOL7pki8N8PFkfI+6zBncEk+Lhbtnruf5BTtIzsjOp7BFzOmdsC0Kmg/KWZ5RyOw6eZ6EtGza1vjnCfRSQNXpibNcE0Z5z+fVhVv0HJuJUjPt/Lo3jpvrl8bDYvy5+UaXlwrd5htSOFXqPREHFk4uKHw36Qoyl9NJxuIxJBBE3b5XN9OlyiNXr9VwCCgJP78ALtdlX96scnEWP96WQe2q8PWGY3R7O1p3TXOZzQaJ88fh9AmBtk+ZHSdPXHgepXU1NSmFhmFg6TqRMFc8nZPmM33Vvy8jlby3fM8ZMrKd3FK/DGdj47GveJOsip2hSgezo4lckVLlq7K1/N1Enl/OgZhos+MIOWOTn2YvoF7mVvbVHEpwaImr+n41KXL1fIKgw3Nw7DeSNixm48acv4iX4uftwfM312b+0Fb4+3jywP828vTcbSSlaVblekVFuXiw0y+E2pbzwrIRRH0XanakPLH6gI3aZYIpGeRrdhTJTZVaQc1beMz7B6J+jWHL/lSzExVJP24/RalgHw7+XozZj07GkpVE21fGERV1+RtRIu6iXt9xJBBM5uIxbFjvvOzYRPJOVJSLKjUyqbxnAkftpTjkGHnV11CTItem8f2c967O6a/GcVvfRCKqOa7ol1mjisX48fE2DO9YjQUxJ+j89ip+3qXdOK6VzQaDhmYz6baniHWGMzuiPgOHFL7CnJppZ/PRBNpV1yxKYfRD5ji8HJkMZT63jt/B7Nk6OyU/JWdks3K/jY7VyvDKs1YGN/6IeY52xLZPLJT1RAqvoJDiLPd/lHrZ25k6cfYVj00kd9lsMHCIk0EPTqCW11FeTbmPQcO8r7qWqEmRa2KL92Tg3HHULH6AgXePJ6TX2iv+Zebj6cHT3WqycFhrwgJ9GDRrM49FxRCfmpX3wQsZqxUGNPmKut5HmGzvByUz8AvNwGo1O1nuWn/kHNkOF211ynyhY7NBvyeq8VVqV+7xWk6tSjsY/uZJDYzz0bLdZ8iyO6kXXJaJHcfjxMJb9j54l0wulPVECi+bDQZOeoLDztKMb/QqxXutUqNtAqsVSpS28UzQbLY4q7HUt9E11RI1KXJNrFb4NaEjvztrM8JzHiVKnrnqv4D1yoWwcFhrRnapwU87T9Flyip+2H4S1xU85yI5IsqnM7rhG8RkVmeRswVZcUGkJ/oSEWF2sty1+sBZfDwtREYUMzuK5DKrFfyKZfCuVw8y8OY5zygC2+xh9wEtBc0vi7adpGyIL93LHeLOKguZfv4OzlC80NYTKbysVvAKcvG6vT/VLScYUGaRGm0TRETA0BZvU9pI4JXse8iKC76mWqImRa5JRASkJ/gxPu4RShjJPJK59Jr+Anp7Wnj8xur88FhbyhXzY/jsGIZ8uZm45Iy8iF3ohB94n/JBJ3l24dskR3UiaX5rZk63EF7IJhx+O3SOphHF8fXyMDuK5LKcWuLLqbjyTLPfRlePzbQI2Mbi2P1mRysSbMmZRB84S8+GZSmxYQwZHuG8PusNUqI6FNp6IoXXhXryw+kOrHfW4knPeXhlpavRzmdG9lEej/gfi8+3YdXswddcS9SkyDUJD4eZ0y2sjnqAOQd7MSjge2a/d+Kaf5nVLB3Et0Nb8Vz3WqzYZ6PLlGi+3XJcsyqXknwm53T5Wrfy1aq2LJkXgPWgB/37F64zROxOF3tPJ9Oy6tXtCiIFw4VakjS/Ne9Gvcbx5LK85jeHuVsPs/14otnxCr1F207icLoYELINYn/Ht/sYdu4JLbT1RAq3P+tJW15c/TLhxnk+euxlNdr57PDc5/HCTsW7X7+uWqImRa5Z//4G1oMe1Bw0AR8fg57+lz7g8XI8PSwMaV+VJU+0pVrJQEbO2cZDn23kVFJ6LiUuZFa8AvZM6PIS4eHQtCmFshCnZtoBaKUmpdC6UEsWfBNG8B3jqOrcxwD/nHOV7A49RJ+XFsScoGFZP8psfA1K1oFG9xXqeiKF34V68sobXVkf0JmumV9x6ug+s2MVGYd3ricy/ke2lO5Dvcj611VL1KTIdQkPhwbtKmC0Gg4750Hsxuu+ZtXwQOYMbsnYW+vw++F4uk6JJmrDMc2qXOzMLoiZBc0GQYkrP721IErJtBPo40n9ciFmR5E8dGFgHNyqL5RtxLNeczh0Io7PfztqdrRC68CZZHacSGJ0iTWQYIWuE8HD0+xYItftQj2p1O91XMCJeaPNjlQkuJxOUhc9x3kjgNr9Jl739dSkSO5oMwICS8HS56/ogMfL8bAYPNSmMkufbEe9ciGM/nYH936yntj4tFwIW8C5XPDTaPANgXZPm50mz6Vk2mleuTieHipXRYLFAt1ewy/jDK+VWslbP+/jRKJmU/PCtzEnCLck0/TYx1CtM1S70exIIrmqdMXqxFS4j8jkX9i76Rez4xR621bMoX7mFvbWGEpI8eufitVvfckdPoHQ6UU4vgF2zs+1y1Ys4c9XA5vzyh312BabRLd3ovl8nRWnswjPquxbDEdWQYfnwb+42WnyXJbdqedRippKLaHO7fRInUtJ1znGfrdTM6m5zOl0sTDmBJOKL8KSlQrdXjU7kkieuKHfOGwUw/hpNC6nlo/mlazMDEqsmcAxSzma9H4mV66pJkVyT8O7ofQNsGwcZOXejIfFYnBP80osHdGOyIjijPt+F3d99DtHzhbBk6ntmbB0DITVhMgHzU6Tb1pV1SGORU6XCVhcTmaW+4Ff9saxeIcOfc1Nqw+eJej8fjqmLIamAyG8ptmRRPJEQFAoRxo8RU37Pjb/MMPsOIXWlnmTqOA6SUKb8Xh5++TKNdWkSO6xeED3SXD+OKx9J9cvXy7Uj88fbMrk3jew9/R5bnonmo+jD+MoSrMq62dAwhG46VXw8DI7Tb7wsBjUKh1kdgzJb8UioNVjVDu9mN4lTzB+0S6S0nR2Sm75ZsNRXvL5CvxCoMNzZscRyVORPR7lgGd1Km55g9Rk7RqY2+LjTlDnwHS2+zalQae+uXZdNSmSuyq1grp3wtqpkHgs1y9vGAZ9IiuwbGR72lYP55XFe+g1bR0HziTn+nu5neQzsOoNqN4tZ/14ERHo44nFom1Qi6Q2IyCoDC95zyIhNYNXF+8xO1GhcDYlE/b+SHN2YBSRZaNStFk8PHB0e52SxLP96/Fmxyl0Dn79LH6uTEJufyNXr6smRXJfl5cAA35+Mc/eolSwLx8PaMLUuxpy9Fwqt7y7hvd/PUB2Yd6udPl4sGfATa+ZnSRflQ72NTuCmMUnELq8hP/Z7UyttYdvNsWy7uBZs1MVeAs3HmS0ZRaZxWtC5ENmxxHJF7WadmZTcBcaH/+SE4d1wyO3HNi6mshzP7C5dB8q1Wqcq9dWkyK5L7QCtHkSdn8Hh1fl2dsYhkHPhuVYNrI9XeqU4s2f93P7B2vZffJ8nr2naWI3wrbZ0HJYod9y+O+8PVWmirT6faB8M24+M4O6xZ2MXrCD9CyH2akKLJfLheW396hgseFz21vacliKlIr9JuPAQty8kWZHKRScDgeOH54hwQimTv/c33xDv/0lb7R+AkIrYV80ik3rs7HZ8u6twgJ9+OCexky7pzFnzmfQ4/01TFm2nyx7IZlVcTpgyTMQVAba5c6OGSIFhmHAzZMxUs/yQanlHD2XxssLdTDbtdq2cwf9M+cRW6YbVG5rdhyRfFWyXGW2VR1Mo7R1rPthLhs3kqfjk8Ju8w8zqGXfw6EGTxMcmvu7cKpJkbzh5Ud0wGt4Juxl4YT3iKjmICoqbx9w716/DMtGtKdHg7K8+8sBbntvDdtiC/YDcjYbWL/9HE7G5Cyj8wk0O5JI/ivbkAOhD1Hh0BdUi03nq01HmDQz3uxUBY7NBtmLRuPEQvidk82OI2KKJv3GcNRVljK/v0jP/mfzZXxS2NhssGblWSrHTGKfZ00iewzLk/dRkyJ5wmaD7k93Y3l6JKOavkH1PosYOMSZ53csigV4M6VfQz59IJKk9Gzu+HAtry3ZQ0Z2wVseEhXlIrL+GUI2T2BVbBuidvY2O5KIKWw26Pjq8yS5Anil6mRwwnsbt3P8VMH7XJslKsrFkFu/p2nWGt6KfZAFyyuaHUnEFEnnfXly9UQqe55iRL/RhPRamy/jk8IiKspFRDUH++Y9QzFXEutLvIXFwyNP3ktNiuQJqxX8imXysuVuvLHzUpnp+IVmYLXmz/t3qlWKn0e2o29kBWasOszNU1ezyVpw7rzabDBwiJPX7x1GoHcyYz36M3CoS0VUiiSrFTL8/ZjkvIvmlr309orGs1gqk5ccMDtagWCzwfBhqUzp+jQHnWWZaemsQZkUWVYrrD58E4sdzXjMcwFVSh3K1/FJQXZhbNL6ns94oMQCZmV2ZehzrfKslqhJkTwREQHpCb4cOFOdD+w96eHxG62C1xERkX8Zgn29eL3XDcx6uBmZdid9ZvzG+O93kZZlz78Q1ygmBtpUW03/wGV84ujO0eLFVESlyLpQT748dSubndV53mM2vvuDWHTgEFuOJZgdz+3FxMCotpOp5HGGF+0PYoRlqZ5IkXWhnrwYNxg7HkywfE56one+jk8KKqsV/EJTeT3sQxIIYgq98rSWqEmRPBEeDjOnW0ia35o3Z09mf3w1vrprFOGhGfmepW31cH4e0Y77WlTis3VWur0TzbpD7ruNaVSUi959Mniz+XOcsIfxrv1OsuKCSE/0VRGVIulCPUmc35YR308jxEhlYZtvKB3sy9NztxXI5Zz5JSrKxahH9vDkDR+w0NGK35x1VU+kSLtQT/Z+fQeTrUO40SuGj8bPJjzc7GTub8sWF/dFTqOx5SCvZt/N2bgyeVpLDJcr9x8WioyMdG3atCnXrysFj82W03lX91hJ6Pc9od0o6DTGtDzrD5/j2fnbsZ5L457mFXmuey2CfN3n5HabDSKqORjz4JM8H/oFPeZ9yZITnfB2+DJzhoX+/a/9UEPDMDa7XK7IXIybL1RP5IIL9aTuiRfw3/oe27t8Q49FDh5qXZmxt9UxO57byakndpYNbU8d7yO0j/uA3d90wdvhp3oiRZ7NBocOZhG+rBlBziT8RmwmKEQHm/4Xmw1atTjE5ntbszWtJp0//w57QjDTPrAweFDe1BLNpEieCg+Hpk0htHEHuKEfrHkbzuw2LU/zKiVY8kQ7BrapzOwNx+j2djQr98WZlufvrFaoXXEvI0OiWOxoRkwnXwJ8LCyY53FdAwqRwuBCPfHv/hyEVOSGmBd5oFkZPl17RIc8/gurFR5q+jmtfHfymuNuEvz8CPDxUD0RIaeetGjpjf3mKYS5Etg96ymzI7k1qxWm3P4kPtgZY3+IEt13EVoqg8aN8q6WqEmR/NPtNfANhkWP55z9YRI/bw9euLUO84e2wt/Hkwf+t5Gn524jKS3btEwXRFRy8UbkaLJdnkzIHoAjxYfsdG8aNTI7mYgb8QmEW6fA2f2MCV5MlbCAnM9wuvmfYXdSuWQc45u8zAZ7LeY42mNP8lM9EfmbmpGd2FCqD01tC9i7/mez47it7FNR3BYUzdS0XsT6hGF4OMlMzttlo2pSJP8ElMhpVI5vhI2fmJ2GxhWL8cNjbRjWsSoLYk7Q5e1VLNt9xtRM4cdn0anSKsasmMihqDtJmt+amdMtWisr8nfVu0D9Pnite4cPu/pzJjmT8d/vMjuVWwlb/zQhfmmMdjxMxtaqJM1vo3oi8i/q3zeZOCMMv6UjycxIMzuO20lKOEu1Lc9zkAhe/XQKKVEd8mV8oiZF8tcNfaHqjbB8PCRYzU6Dr5cHz3SrxXePtqZ4gDePfLGJx6NiiE/Nyv8w50/C0hegUhtenP8QS+YFYD2oZRki/+mm18EniFrrn+OxDhEsiDnBwq0nzE7lHnYvhN0LWRI+gNNelYh6sbrqich/CAgK5Uz716jkjGXLrOfNjuN29n3xBCVcCTh7vMuBvQH5Nj5RkyL5yzDgtqlgWGDhcHA6zU4EQP3yIXw/vA0jOtdgyc5TdJmyih+3n8q/AC4X/DASHFnQ413CS1po2hTd8RS5lIAwuHkynNjMY74/0bhiKC8s2ElsfBG/E5oWDz8+TWZ4PZ4+0Y57WlSkQ2sv1RORS2jQsQ8bQ26i6fHPORATbXYct7EjeiHNEn5gY5m7qdG4/R/PBuZHPVGTIvkvtAJ0mwjW1bDJ/GVfF3h7Wniic3UWPdaGsqF+DJu9hSGzNhOXnA/bJm+Lgv1LoNMLUKJq3r+fSGFRrxfUvg2PVa/xQRd/AJ78Zit2h3vcADHF4qchPZ73Ap/E8PDikbZVzE4kUiDUeOAD4o1QPBcN17Iv4HziOcJ/HUmsUZaGA97I9/dXkyLmaHw/VO0Ey8ZB/GGz0/xFrdLBLHi0FaNuqsmv++Lo+nY0C2KOkxfbddtssC06FufiZ6FiK2gxNNffQ6RQMwy4ZQp4B1JmxUhe6VmTzUcTmPpL0TuN3maDQwvnw875nGs6gg/3+jOgZQThQT5mRxMpEEKKhXGq3SQqO4+y5YvnzI5jKpsNtk0fRrjrHKm3vI+vf2C+Z1CTIuYwDOjxHlg84dvB4HCvU+A9PSw82qEaix9vS9XwQEZ8s42HP9/EqaT0XHuPqCgXlatlk/TlEFJTnHzv8SFYPHLt+iJFRmBJuPVtOLmFHkmz6dOkPO+vOMiaA0VnW+KoKBctGpyg2G9PsfFUYx7Z2g0fTw8GtdMsisjVaNCpLxtCb6bZiS/Ys36p2XFMERXl4ol7v6JtxhKmHr+fmAOdTMmhJkXME1I+ZxvR4xtgzRSz0/yraiUDmTO4JWNvrcO6Q2fpOiWarzccu+5ZFZsNBg5x8tQDz9Ku7BomJD9A/8crYrPlUnCRoqbu7dCgP0RP5uUmaVQLD+TJb2KIO58PyzVNZrPBI0PszLzvXvx8UhlpeZjNCXH0aViJsEDNoohcrToPfchpS0lCfhrO+cRzZsfJVzYbjHn+KO+2eI5dzkpMdd3GwCFOU8YnalLEXPV7Q/0+sPJ1OO6epwB7WAwealOZpU+2o265YJ77dgf3fbLhuh7OtVqhWdWNjAn9jGWOJswLaIlfaAZWa67FFil6ur8BIeXxXTSEaX1qkJJpZ3hUTKF/PsVqhSdbv09Hvy28Yr+H2JBiYPegfSk93yZyLQKDi5F884eUctrY978hZsfJV4cPO/i0z4MEks6T2cMwSmaaNj5RkyLmu/lNCC4L8x6CjCSz0/ynSiUCmD2wBRNvr0fMsQS6vRPNF79ZcTqvflYlolwq09sM55wjhFHZj5AVF0x6Yt4eiiRS6PkQthyIAAAYRklEQVQGwx0fQeIxqm14kdfuqMeGI/G8sXSf2cnyVLWAbYxt8gpL0lrwpaMzAKmbq3JDTW+Tk4kUXLWadmZDxYE0TfqZjQs/NDtOvsnYNZEO/lsYn/wgB1zlyYoLMm18oiZFzOcXCr0/hfMn4PvHcrbjdVMWi8G9LSrx88j2NKlUjLELd3HXx79jPZt6VdcJ3ziKGsUP8vB3nxIb1UOHNorklkotocPzsGMud7CC+1pU4qPowyzZkY9biuenzGSK/fIQdp8wBn0+C0d8EI5kX6YMqqx6InKdmg54ld3e9am7ZTxH9201O06eOxATTcujU1lntOa9T17Nt0Mb/4uaFHEPFZrBjWNzDiDbONPsNJdVLtSPLx5qxhu9bmDPqfPcNDWamasP47iSWZWYLyHmS4y2T/H5io46tFEkt7UdCZXbw+JRjG0GjSqG8vTcbew7nWx2stzlcuXc2Ik/jP+9M3n5myw8iqfwar9aDLjH0+x0IgWep5c3YffPItPwxvHN/WSkpZgdKc8knTtD4MKHiDeKU2fYp1gPepk+PlGTIu6j5WNQvSssfd5tn0+5mGEY9G1ageUj29OmWhgTf9xDr2nrOHDmEgOh0zvgx6egcjvo+Hy+HookUmRYPODOj8EnCK95A5jeuzoBPp488sUmElKzzE6XezZ8BLsWwI1jSSrVnPfX7KFBhVDuaVPW7GQihUbJcpU51v5tqjit7JjxMC43OYQ6NzkdDqwz76WEK57zPWYSGlbaLcYnalLEfVgscMcMCCoD39wHKXFmJ7oipYJ9+XhAJFPvasjRc6nc8u4aPlhx8J8P66bF5/xcfsWg16fablgkLwWVgj6fQeJRSi1/nOn3NuJ0UgbDZm8huzA8SH/sd1g6BmreDK2e4M2l+4hPzeKV2+thsWhWViQ3NejYh98qDKRp0k9smJv/hxrmtfWfj6ZB+gZi6jxLjcYdzI7zBzUp4l78i8NdX0F6Asy5HxzZZie6IoZh0LNhOZaNbE+XOqWYvHQft3+4lt0nz+e8wGGHeQ9C0nHo8zkEaupEJM9FtIaur8D+JTQ+MpNX76zPukPnGLtwZ54czppvEmPhm3shtCLc/iFbT5zny/VHGdAygnrlQsxOJ1IoNX/gDbb5Nafx7jcK1fkpW5bOouWxGWwM6UqzPs+YHecv1KSI+yldP+egx2Pr4IcRbv0g/d+FBfrwwT2NmXZPY04nZdDj/TW8vWw/jqVj4PBKuO0dqNjc7JgiRUfzwXDDXbDyVXr7bGBYx6pEbYhlRvRhs5Ndm6xU+Lo/2DOh/9dke4cwZsEOwgN9eKprDbPTiRRaFg8PIgbN5rSlFKWXDOTE4T1mR7puh3eup9a6p9jnWZP6Qz7DsLhXW+BeaUQuuKEPtHsGYmbB2nfMTnPVutcvw7IR7enRoCznVn6Ix4bp2Oo9DI3uNTuaSNFiGHDbVKjQAhYM4ana57n1hjK8vmQvi7adNDvd1XE64NtBcHpnzo6I4TV479eD7Dp5npd61iXI18vshCKFWkixMLj7ayw4sH/Zm6SEs2ZHuma2k1b8591DihFAiYfm4usXYHakf1CTIu6r4xio1wuWj4ed87HZYONGCsyp7PY0b4b6H+dlr89YY0TSanMHXl+yl4xsh9nRRIoWL9+cZaTBZbB8czdjGgdTO6w4I77ZyuoD7l9QbDbYuMFF+rejYO8PcNPrUL0LW2MT+WDFQe5sVI6b6pUxO6ZIkVChegOOd51JGccpYqf34sTxtAI1NrHZYE10POdn3k6QK5nzd8wirGwls2P9KzUp4r4MA3p+CBVb4pg3mIE3/ky3O1OpUNnBjBnuuwTMZoOJr7i4q+1ayq9/mE2nGnGk8jx6RUYwfdUhbn53NZuPxpsdU6RoCQiDu+eSmW4nY8adxH5RnPS4QB78ZDMrtieYne4/zZjhomJlBz+9NAm/nTPZU+xxaDGE9CwHI+dspWSQD+N61DU7pkiRUrfVzWxrPJF6mVuJeasv3Xsnuf3YBHLqSbUaKTh/vJOKjmMsKPEh1Rq0MTvWf1KTIu7Ny5ezXb9mV1xNZve8j9reW3EGpDB0uJMZH7lfMYiKclGpioOFMzaxoG9vTllCedAxkiGPBfNUuxuY9XAzMrOd9J7+Gy8t2k1alt3syCJFho0adPlyDuFBcczt3pesHyPIOO/F/Z9u5K1PEs2O9w8zZrgYOtzJUw89x4uRr/Ftanuajh1LXJyLMd/t4MjZVN7s04AQPy3zEslvEa0eZcz+p7g1ZDXv3DoEZ8B5tx2bQE49eezJdOY82pN2fjE8k/wog1+83a1ngNSkiNs7ciqUPsu+4mRSGZbc05tbHvof4X028PgTTva40XNrNhsMHOKkWff5LL27F0kEcE/WGFLCLPiFZmC1Qtvq4Swd0Y77WlTi07VHuOmd1aw7VHDXtIoUJFYr7M6qy8Mpo6jidZJfH+xJWLANPB1M3bGBH9eeNzviH2w2eOIpJ4Nbz2Bi8Y9Y6ohklMfDeIdk8fGKY3y75QRP3Fid1tXCzI4qUiRZrTBt9VNMPvoA94Yv5pNHBlKyz29uNzaBnHoy8pkM5g7tRzevTbyY/QDfebfEJyRnbOKu1KSI24uIgKOnKtDthyjOEcQXnpOoscGOyz+dxk0dREW5x10LqxVaVfudb2uNJCUrkN6nXuUUJciKCyI90ZeIiJzXBfp48lLPenwzqAWGAXd/vJ4xC3aQkqlZFZG8FBEB6Qm+/HKgA32WfkwNr1i+dE3CPrcGLruFoXN+5003mVGxWuGxVtOY1mE0y9MjeSz7MdLiipHtlcH/tu2iQ81wHu9U3eyYIkVWRASkJfgyZvF4ptlv4z7P5Uwu9iH4p7jV2ATg4IE0vnmkDz2Do5mQcj+zHF1zxibxf45N3JGaFHF74eEw9S0L1tga3G59i5OJZfjp7t7cPXgqxfquZeAQp1tMV1b3jObbbn1JcfrRK3YSG2fdzokZHUia15qZ0y3/OLW1eZUS/PREOx5uU5nZG47R7e1oove7wQ8iUkiFh8PM6RbSVjbgh223cu+JCVT1PMnKe3tSMfA4hredd3eu5+cYk58Zc7moc3Yyk1uNYfHZtvT4YAHWTzty9vtGVOi3hTKhvrzTr6EObRQx0YWxiT0+mPGHhzL56APcF/4j3zx6F6X6rnCbsUlqciL+K2/h1qDVjD3zCC998CYnP23D6VmteWfKP8cm7kRNihQIgwcbTPvAwt65N9P5u3lYKcVMrze5r8z3+ARlsHixyTtrbJ9L6A+9cQRWoOOnKziyoRO+Xh6Me8of6yEP+vf/98GEn7cHL95ah/lDW+HrZWHApxsYNW8bSekF4xBLkYKmf3+Do4c9eGmcB4uihnDr919S2vMcC3zGUdsRiyvdm2Fz17Ns9xlzAjqyYfHTBKyfyJHgftz15Xx8Qz3wcHpTd8gmfHxdfPZgM0L9vc3JJyJ/uDA2SVzQgjHfv8Kk7Lu43WMdUeVeJLzUcdPHJmdPHuXE1C7Uy4zh66CXeGv2JEJKZGNJC2Ta+xYGD3LvGx1GXpy6GxkZ6dq0aVOuX1dkzx5o3NRB+X7LmFn2Fdp7bGfK748yae+zpCQEMXO65T8bgjzhdMKvL8Gat6FSa+j3JbbU4litOVPBV3OHIiPbwbu/HGBG9GFKBHjzyh316VKnVK5FNQxjs8vlisy1C+YT1RPJKxfqSfN7ZvNliZcIJYVHVk0mukYE3qWSGHNLbR5uUxnDyKeakhYPcwaAdTW0ehw6T8B2zsLO/dm8sWE9h88lM/uRFjSpVCx/8lyC6onIny7UkmJ919Kn9BIme83gqL0M96/4mG27muf/2AQ4EBNNyML7CXSlsq/1FBp1vRebjWsan+SlS9USNSlS4ERFuRg4xElAcDLjGrzMsMiZbHDWZPCp0eyf04NNv3uQkpIPH8LkM/DdUDj0CzS+H25+Ezyv/+7mjuNJPDNvG3tPJ9OzYVnG3VaX4gHXf10NKkT+KSrKxcODnBTzOsnCwb2I9NnHjPM9mXj8UTyqnOPGiApM6FmX8mU88jZI7AaY/zAkn845fLLh3QCcS8nkvk82cDAuhWn3NubG2rl34+J6qJ6I/NWFsYlPUAZNyq9gdreh+JPJ00nD+OJ/L+Tb2MTldLJh3ps02PUGCUYoab2+pGr9Fnn3htfpUrVEy72kwOnf38B60IPJE0MZu+MlHs8aRj3Dyi9lh9OnzhwaN7XTvU8qEdXy8MG1fT/BtFZwdC3cMiVnUJELDQpA/fIhfD+8DU92rs7iHafo+vYqFu84lSvXFpG/6t/f4Lv5HqQFFKe/awz/s3djcPBCFoU/TaXdDn6xxtL8hXV88Flq3gRwZMPKSfDpTYABDy75o0E5kZhO/49/55AthY/vj3SbBkVE/unC2OTtVwLYcqojt2a+yh5XJaaFTuGzB/rR+caTeT42OXfmOFvfvIXmu19hv18DvIeudOsG5XI0kyIFls0GEdUchPRaS+1Se3nD+QlN/PayJK0Fr3r049CZaiTNb431oEfu3bVIOg4/jYY930OpetDrEyhZK5cu/k97T5/nmbnb2XEiie71SjOhZ11KBvle07V051Pk311cS7xLJtPh3F5eD5lO8YB4pjtuZVpWD1KzAxnfow4Pdiife8u/rGvhx6fAtgfq94Vb3gLfYAA2WuMZ+uVmMrKdfDSgCa2qutdWw6onIv/u4nriXzKBYfZFDA/4lgRXEBPt9zL/dGcS57fL1bGJw25n07dTqL37HXxdmWyp+STN+j2PxSOPZ4BzgZZ7SaF1YXrVLzSDtHgvnmozldHNJuOJnS8cXXlr7ut88WUETZtyfWsxU2zw23uw4WNwuaD9M9ByOHj65MFP9Vd2h5OPVh/mneUH8PPyYNxtdbijUbmrHihpUCHy3y6uJSnnfCld8iRT7hnCnR5riHOFMiXhXhb4R9KqZjlevr0ePnb/a68np3fAqkmwZxGEVITuk6DWzQA4nC7+t/YIk37aS/li/nw8oAnVSgbl9o973VRPRP7b3+tJswbRfHDjk9S3WNnqrMLEjWMYNf5OmjW3XNfYxOlwsG35lwRveJuqjiPs9GlI0J1TqVSzYV78WHlCTYoUahc+4IGBENnCQY2+3/Nsqc/o67GKbIcXrnp9iE57gDtGNMavWCbpCb7/+hDbPwqFywUnNkPMl7D9G7BnQN074caxUKxSvv+cB+NSGDVvG1uOJXJjrZK8ckd9Sodc+ayKBhUil/b3WhLSay3NS21htOtrmvnuIc2rOJ9mdmRudnt2b2qG60hZ0m2BV1ZP7FmwfwlsmQUHl4FPMLQYCq2fAO8AAA7ZUhg1bzubjybQuXYp3urrvqfJq56IXNrf60lor2j6lf6Jpz3nUNYSzyFLZTb5P8iIt/uT5RXwn2OTi691oZ4k2E6xb/mnlD7wNRHOY8QaZTkTOZIm3R/GsBSsJznUpEiRcfHdi3IcZ+6Ij6me8hWGPY2T9jCW04iNifVY+2Nflq6NILys/x/fN3xYKrUr7qOy1xHG3r2W6pYVkHAEPP2g3p3QZgSEmXt4msPp4rN1ViYv3YuXxcILt9amb2SFK5pV0aBC5MpdXEvSE31Y9NZqOvlNgwNLAdjhjGCFsyExaTX57bs+rF4XQXhprz++9+kn4qlTcQ+1fPfxzB3RVMxeCRmJEFQGmjwIzQeDXygAp5LSefeXg8zdFEuAjyfje9Th9oZXP1uan1RPRK7cxfXElWbnw2f/R7PUT6jsOkamy5O1znqsTb+B39d3492ZjahcrfQfS7Vmf+Vg/Lgj1Ku2jUYlN3FL1XXUc+7A23Bw0KMqiQ0G0ejmgXh4epr8U14bNSlSpPz9jsOWdYl8/MJC7mj9OW0sO/E3Mv98sVcATosX6SlZBHil/fHl5KxAvGu0wad+95zZk/9fJ+4ujp5LZdS87aw/Ek+bamG8dmd9KhT3v+T3aFAhcnX+bRnG9lVH+PaNefRo8jUNjEN4GH/+Ds32CsbDgKy0THw9/6wzJ5LLUKJpR3yb3AlVOoKHJ9kOJ2sPnmXupuMs230GFy76N6vI8E7Vrvm5s/ykeiJydf5eTzasdzJhzBJ6tviKrpZNRFj+PJvJ7rKQYvjj6XLgQxZehuOPP9uVVZn4sq0o1+F+qtRrnv8/SC5TkyJF2sUPsfmWTKJSQiJVt3nwzoQzBLpsnDmVzbwfDTJrnsPqKs1hV2k2Rd3HormhNG1qdvr/5nS6mL3hGK8t3oMLeK57Le5tXuk/T6HWoELk+l1cT4JLnqVm8mmqp56jSsQx/O2JgIX0NH8S/Hw47CrLQVdZ9qzqzNhRvoSUzuRkYjo7TiSxyZpAeraDYv5e3N6oHA+1rnzZGw3uRPVE5Pr8fcOOoHPZ1LZmMuyeg3hn27BkJpGa4cWug16cKeHFYVcZ9jkrcDSqJ0vmBbj1+ORqXKqWFMy5IZGrEB4OM6dbGDikNVmhGWxI9GXQdAuB3XIG8xYbjHrSQUhwTqHIigsiOSGIiAhzc1+OxWJwb4tKdKgZzuhvdzB24S5+2H6KN3rdQERYgNnxRAqlv9eT2ERf7ppuoVcfF+sOneXXnef45IdzeHomYfz/0nC/druY/HvOf3taDKqVDKRvZHlaVwujQ82SeHsWrDXkInL9Lq4lfqEZ2BJ9eX66hfYXPZNis8FtFzUyWXFBpCf6uv34JLdoJkWKjEvtoPHX9ef//fCau3K5XMzddJyXf9xNtsPJ011r8mDrynhcNKuiO58iuefy9cSBf6lkMu0wboxBt5sMwgJ9CA/y+cvnsqBSPRHJHZfb3augj08uR8u9RK7AdW1R7CZOJ2Xwwnc7WL4njkYVQ5nc+4Y/ti/VoEIk/xSGenIpqici+acw1xMt9xK5AuHhBf/DXzrEl48HRLJw60nGL9rFzVPX8ETn6gxuV8XsaCJFSmGoJyLiHopqPVGTIlLIGIbB7Y3K0bpaGGMX7mTy0n38tPO02bFERERErpie1hMppMKDfJh2bxM+vKcxp5LSzY4jIiIicsXUpIgUcjfXL8PPI9qbHUNERETkiqlJESkCigd4mx1BRERE5IqpSREREREREbeiJkVERERERNyKmhQREREREXEralJERERERMStqEkRERERERG3oiZFRERERETcipoUERERERFxK2pSRERERETErahJERERERERt6ImRURERERE3IqaFBERERERcStqUkRERERExK2oSREREREREbeiJkVERERERNyKmhQREREREXEralJERERERMStqEkRERERERG3oiZFRERERETcipoUERERERFxK2pSRERERETErahJERERERERt6ImRURERERE3IqaFBERERERcStqUkRERERExK2oSREREREREbeiJkVERERERNyKmhQREREREXEralJERERERMStqEkRERERERG3oiZFRERERETcipoUERERERFxK2pSRERERETErahJERERERERt6ImRURERERE3IqaFBERERERcStqUkRERERExK2oSREREREREbeiJkVERERERNyKmhQREREREXEralJERERERMStqEkRERERERG3oiZFRERERETcipoUERERERFxK2pSRERERETErahJERERERERt6ImRURERERE3IqaFBERERERcStqUkRERERExK2oSREREREREbeiJkVERERERNyKmhQREREREXEralJERERERMStqEkRERERERG3oiZFRERERETcipoUERERERFxK2pSRERERETErahJERERERERt6ImRURERERE3IqaFBERERERcSuGy+XK/Ysahg04musXFpHrUcnlcoWbHeJqqZ6IuCXVExHJDf9ZS/KkSREREREREblWWu4lIiIiIiJuRU2KiIiIiIi4FTUpIiIiIiLiVtSkiIiIiIiIW1GTIv9gGEZTwzC2G4bhaxhGgGEYuwzDqGd2LhEpeFRPRCS3qJ4ULdrdS/6VYRgTAV/ADzjucrleMzmSiBRQqicikltUT4oONSnyrwzD8AY2AhlAK5fL5TA5kogUUKonIpJbVE+KDi33kv9SHAgEgsi5YyEicq1UT0Qkt6ieFBGaSZF/ZRjG98DXQGWgjMvlGm5yJBEpoFRPRCS3qJ4UHZ5mBxD3YxjGAMDucrlmG4bhAawzDKOTy+X61exsIlKwqJ6ISG5RPSlaNJMiIiIiIiJuRc+kiIiIiIiIW1GTIiIiIiIibkVNioiIiIiIuBU1KSIiIiIi4lbUpIiIiIiIiFtRkyIiIiIiIm5FTYqIiIiIiLgVNSkiIiIiIuJW/g+6PmsDeOESSgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1008x360 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "degrees = [1, 3, 50]\n",
    "\n",
    "#generate the sample (the training set)\n",
    "\n",
    "n_samples = 30    #dataset size\n",
    "\n",
    "X = np.sort(np.random.rand(n_samples))    #generate random points\n",
    "y = target(X)    #generate samples WITHOUT NOISE\n",
    "\n",
    "X_test = np.linspace(0, 1, 100)   #generate random points for testing our model\n",
    "\n",
    "plt.figure(figsize=(14, 5))\n",
    "for i in range(len(degrees)):\n",
    "    ax = plt.subplot(1, len(degrees), i + 1)\n",
    "    plt.setp(ax, xticks=(), yticks=())\n",
    "\n",
    "    polynomial_features = PolynomialFeatures(degree=degrees[i],   #use polynomial features of given degree\n",
    "                                             include_bias=False)\n",
    "    linear_regression = LinearRegression()                             #use linear regression\n",
    "    pipeline = Pipeline([(\"polynomial_features\", polynomial_features),    #put all togheter\n",
    "                         (\"linear_regression\", linear_regression)])\n",
    "    pipeline.fit(X[:, np.newaxis], y)                               #fit the model\n",
    "\n",
    "    \n",
    "    plt.plot(X_test, pipeline.predict(X_test[:, np.newaxis]), label=\"Model\")   #predicted function on the test set\n",
    "    plt.plot(X_test, target(X_test), label=\"Target function\")  #target function\n",
    "    plt.scatter(X, y, edgecolor='b', s=20, label=\"Samples\")   #plot data points\n",
    "    plt.xlabel(\"x\")\n",
    "    plt.ylabel(\"y\")\n",
    "    plt.xlim((0, 1))\n",
    "    plt.ylim((-2, 2))\n",
    "    plt.legend(loc=\"best\")\n",
    "    plt.title(\"Degree: {}\".format(degrees[i]))\n",
    "\n",
    "plt.show() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, in this case a complex model performs better than a simple one!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='2-2'></a>\n",
    "## 2.2 How to combact overfitting?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are two ways to cure overfitting, often used together: _regularization_ and _validation_.\n",
    "\n",
    "- With **regularization** one can put constraints to the learning process to improve the generalization performances of the model, and therefore its quality. The need and the amount of regularization depends on the quality and the quantity of your data: too much regularization can lead to underfitting, because the model is not free enough to fit the data well. In most of the models you will use, you can find the regularization term as hyperparameter.\n",
    "\n",
    "- **Validation** allow you to keep an eye on the model performance, in order to prevent overfitting. This will be the argument of the next section of this notebook: [Validation](#3)\n",
    "\n",
    "> Notice that the setting of optimal regularization parameters is done through validation!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='3'></a>\n",
    "# 3. Validation\n",
    "\n",
    "To contrast overfitting we need to correctly understand the quality of our model, i.e. we _want to check if the model gives expected results on unseen data_.\n",
    "\n",
    "The problem is that the performance of our model can differ on train data and on the test data, because the learning algorithm could just memorize all examples from the train data and be completely useless on unknown data.\n",
    "\n",
    "So, usually, we divide the data we have available into two parts, train and validation part. We fit our model on the train part and check its quality on the validation part.\n",
    "\n",
    "Only in this way we can select the model which will be expected to get the best quality on the test data, and adjust its hyperparameters (and often regularization is represented by one of these parameters): this could mean the choice between a linear or non linear model, the choice of the degree in a polynomial model, the choice of the value of a regularization parameter. \n",
    "\n",
    "> This step is called **model selection** and it is of paramount importance when you design a ML model: the goal is to select the best model and the optimal parameters for that model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='3-1'></a>\n",
    "## 3.1 Train, Validation, Test sets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section we will show how validation works in a practical and interactive way. \n",
    "\n",
    "Let's introduce another useful dataset from [sklearn.datasets](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.datasets): the _breast cancer wisconsin dataset,_ a classic and very easy binary classification dataset that is a collection of medical images of breast cancer together with their cancer characteristics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean radius</th>\n",
       "      <th>mean texture</th>\n",
       "      <th>mean perimeter</th>\n",
       "      <th>mean area</th>\n",
       "      <th>mean smoothness</th>\n",
       "      <th>mean compactness</th>\n",
       "      <th>mean concavity</th>\n",
       "      <th>mean concave points</th>\n",
       "      <th>mean symmetry</th>\n",
       "      <th>mean fractal dimension</th>\n",
       "      <th>...</th>\n",
       "      <th>worst radius</th>\n",
       "      <th>worst texture</th>\n",
       "      <th>worst perimeter</th>\n",
       "      <th>worst area</th>\n",
       "      <th>worst smoothness</th>\n",
       "      <th>worst compactness</th>\n",
       "      <th>worst concavity</th>\n",
       "      <th>worst concave points</th>\n",
       "      <th>worst symmetry</th>\n",
       "      <th>worst fractal dimension</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>17.99</td>\n",
       "      <td>10.38</td>\n",
       "      <td>122.80</td>\n",
       "      <td>1001.0</td>\n",
       "      <td>0.11840</td>\n",
       "      <td>0.27760</td>\n",
       "      <td>0.30010</td>\n",
       "      <td>0.14710</td>\n",
       "      <td>0.2419</td>\n",
       "      <td>0.07871</td>\n",
       "      <td>...</td>\n",
       "      <td>25.380</td>\n",
       "      <td>17.33</td>\n",
       "      <td>184.60</td>\n",
       "      <td>2019.0</td>\n",
       "      <td>0.16220</td>\n",
       "      <td>0.66560</td>\n",
       "      <td>0.7119</td>\n",
       "      <td>0.2654</td>\n",
       "      <td>0.4601</td>\n",
       "      <td>0.11890</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>20.57</td>\n",
       "      <td>17.77</td>\n",
       "      <td>132.90</td>\n",
       "      <td>1326.0</td>\n",
       "      <td>0.08474</td>\n",
       "      <td>0.07864</td>\n",
       "      <td>0.08690</td>\n",
       "      <td>0.07017</td>\n",
       "      <td>0.1812</td>\n",
       "      <td>0.05667</td>\n",
       "      <td>...</td>\n",
       "      <td>24.990</td>\n",
       "      <td>23.41</td>\n",
       "      <td>158.80</td>\n",
       "      <td>1956.0</td>\n",
       "      <td>0.12380</td>\n",
       "      <td>0.18660</td>\n",
       "      <td>0.2416</td>\n",
       "      <td>0.1860</td>\n",
       "      <td>0.2750</td>\n",
       "      <td>0.08902</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>19.69</td>\n",
       "      <td>21.25</td>\n",
       "      <td>130.00</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>0.10960</td>\n",
       "      <td>0.15990</td>\n",
       "      <td>0.19740</td>\n",
       "      <td>0.12790</td>\n",
       "      <td>0.2069</td>\n",
       "      <td>0.05999</td>\n",
       "      <td>...</td>\n",
       "      <td>23.570</td>\n",
       "      <td>25.53</td>\n",
       "      <td>152.50</td>\n",
       "      <td>1709.0</td>\n",
       "      <td>0.14440</td>\n",
       "      <td>0.42450</td>\n",
       "      <td>0.4504</td>\n",
       "      <td>0.2430</td>\n",
       "      <td>0.3613</td>\n",
       "      <td>0.08758</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>11.42</td>\n",
       "      <td>20.38</td>\n",
       "      <td>77.58</td>\n",
       "      <td>386.1</td>\n",
       "      <td>0.14250</td>\n",
       "      <td>0.28390</td>\n",
       "      <td>0.24140</td>\n",
       "      <td>0.10520</td>\n",
       "      <td>0.2597</td>\n",
       "      <td>0.09744</td>\n",
       "      <td>...</td>\n",
       "      <td>14.910</td>\n",
       "      <td>26.50</td>\n",
       "      <td>98.87</td>\n",
       "      <td>567.7</td>\n",
       "      <td>0.20980</td>\n",
       "      <td>0.86630</td>\n",
       "      <td>0.6869</td>\n",
       "      <td>0.2575</td>\n",
       "      <td>0.6638</td>\n",
       "      <td>0.17300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>20.29</td>\n",
       "      <td>14.34</td>\n",
       "      <td>135.10</td>\n",
       "      <td>1297.0</td>\n",
       "      <td>0.10030</td>\n",
       "      <td>0.13280</td>\n",
       "      <td>0.19800</td>\n",
       "      <td>0.10430</td>\n",
       "      <td>0.1809</td>\n",
       "      <td>0.05883</td>\n",
       "      <td>...</td>\n",
       "      <td>22.540</td>\n",
       "      <td>16.67</td>\n",
       "      <td>152.20</td>\n",
       "      <td>1575.0</td>\n",
       "      <td>0.13740</td>\n",
       "      <td>0.20500</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>0.1625</td>\n",
       "      <td>0.2364</td>\n",
       "      <td>0.07678</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>564</td>\n",
       "      <td>21.56</td>\n",
       "      <td>22.39</td>\n",
       "      <td>142.00</td>\n",
       "      <td>1479.0</td>\n",
       "      <td>0.11100</td>\n",
       "      <td>0.11590</td>\n",
       "      <td>0.24390</td>\n",
       "      <td>0.13890</td>\n",
       "      <td>0.1726</td>\n",
       "      <td>0.05623</td>\n",
       "      <td>...</td>\n",
       "      <td>25.450</td>\n",
       "      <td>26.40</td>\n",
       "      <td>166.10</td>\n",
       "      <td>2027.0</td>\n",
       "      <td>0.14100</td>\n",
       "      <td>0.21130</td>\n",
       "      <td>0.4107</td>\n",
       "      <td>0.2216</td>\n",
       "      <td>0.2060</td>\n",
       "      <td>0.07115</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>565</td>\n",
       "      <td>20.13</td>\n",
       "      <td>28.25</td>\n",
       "      <td>131.20</td>\n",
       "      <td>1261.0</td>\n",
       "      <td>0.09780</td>\n",
       "      <td>0.10340</td>\n",
       "      <td>0.14400</td>\n",
       "      <td>0.09791</td>\n",
       "      <td>0.1752</td>\n",
       "      <td>0.05533</td>\n",
       "      <td>...</td>\n",
       "      <td>23.690</td>\n",
       "      <td>38.25</td>\n",
       "      <td>155.00</td>\n",
       "      <td>1731.0</td>\n",
       "      <td>0.11660</td>\n",
       "      <td>0.19220</td>\n",
       "      <td>0.3215</td>\n",
       "      <td>0.1628</td>\n",
       "      <td>0.2572</td>\n",
       "      <td>0.06637</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>566</td>\n",
       "      <td>16.60</td>\n",
       "      <td>28.08</td>\n",
       "      <td>108.30</td>\n",
       "      <td>858.1</td>\n",
       "      <td>0.08455</td>\n",
       "      <td>0.10230</td>\n",
       "      <td>0.09251</td>\n",
       "      <td>0.05302</td>\n",
       "      <td>0.1590</td>\n",
       "      <td>0.05648</td>\n",
       "      <td>...</td>\n",
       "      <td>18.980</td>\n",
       "      <td>34.12</td>\n",
       "      <td>126.70</td>\n",
       "      <td>1124.0</td>\n",
       "      <td>0.11390</td>\n",
       "      <td>0.30940</td>\n",
       "      <td>0.3403</td>\n",
       "      <td>0.1418</td>\n",
       "      <td>0.2218</td>\n",
       "      <td>0.07820</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>567</td>\n",
       "      <td>20.60</td>\n",
       "      <td>29.33</td>\n",
       "      <td>140.10</td>\n",
       "      <td>1265.0</td>\n",
       "      <td>0.11780</td>\n",
       "      <td>0.27700</td>\n",
       "      <td>0.35140</td>\n",
       "      <td>0.15200</td>\n",
       "      <td>0.2397</td>\n",
       "      <td>0.07016</td>\n",
       "      <td>...</td>\n",
       "      <td>25.740</td>\n",
       "      <td>39.42</td>\n",
       "      <td>184.60</td>\n",
       "      <td>1821.0</td>\n",
       "      <td>0.16500</td>\n",
       "      <td>0.86810</td>\n",
       "      <td>0.9387</td>\n",
       "      <td>0.2650</td>\n",
       "      <td>0.4087</td>\n",
       "      <td>0.12400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>568</td>\n",
       "      <td>7.76</td>\n",
       "      <td>24.54</td>\n",
       "      <td>47.92</td>\n",
       "      <td>181.0</td>\n",
       "      <td>0.05263</td>\n",
       "      <td>0.04362</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.1587</td>\n",
       "      <td>0.05884</td>\n",
       "      <td>...</td>\n",
       "      <td>9.456</td>\n",
       "      <td>30.37</td>\n",
       "      <td>59.16</td>\n",
       "      <td>268.6</td>\n",
       "      <td>0.08996</td>\n",
       "      <td>0.06444</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.2871</td>\n",
       "      <td>0.07039</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>569 rows × 30 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     mean radius  mean texture  mean perimeter  mean area  mean smoothness  \\\n",
       "0          17.99         10.38          122.80     1001.0          0.11840   \n",
       "1          20.57         17.77          132.90     1326.0          0.08474   \n",
       "2          19.69         21.25          130.00     1203.0          0.10960   \n",
       "3          11.42         20.38           77.58      386.1          0.14250   \n",
       "4          20.29         14.34          135.10     1297.0          0.10030   \n",
       "..           ...           ...             ...        ...              ...   \n",
       "564        21.56         22.39          142.00     1479.0          0.11100   \n",
       "565        20.13         28.25          131.20     1261.0          0.09780   \n",
       "566        16.60         28.08          108.30      858.1          0.08455   \n",
       "567        20.60         29.33          140.10     1265.0          0.11780   \n",
       "568         7.76         24.54           47.92      181.0          0.05263   \n",
       "\n",
       "     mean compactness  mean concavity  mean concave points  mean symmetry  \\\n",
       "0             0.27760         0.30010              0.14710         0.2419   \n",
       "1             0.07864         0.08690              0.07017         0.1812   \n",
       "2             0.15990         0.19740              0.12790         0.2069   \n",
       "3             0.28390         0.24140              0.10520         0.2597   \n",
       "4             0.13280         0.19800              0.10430         0.1809   \n",
       "..                ...             ...                  ...            ...   \n",
       "564           0.11590         0.24390              0.13890         0.1726   \n",
       "565           0.10340         0.14400              0.09791         0.1752   \n",
       "566           0.10230         0.09251              0.05302         0.1590   \n",
       "567           0.27700         0.35140              0.15200         0.2397   \n",
       "568           0.04362         0.00000              0.00000         0.1587   \n",
       "\n",
       "     mean fractal dimension  ...  worst radius  worst texture  \\\n",
       "0                   0.07871  ...        25.380          17.33   \n",
       "1                   0.05667  ...        24.990          23.41   \n",
       "2                   0.05999  ...        23.570          25.53   \n",
       "3                   0.09744  ...        14.910          26.50   \n",
       "4                   0.05883  ...        22.540          16.67   \n",
       "..                      ...  ...           ...            ...   \n",
       "564                 0.05623  ...        25.450          26.40   \n",
       "565                 0.05533  ...        23.690          38.25   \n",
       "566                 0.05648  ...        18.980          34.12   \n",
       "567                 0.07016  ...        25.740          39.42   \n",
       "568                 0.05884  ...         9.456          30.37   \n",
       "\n",
       "     worst perimeter  worst area  worst smoothness  worst compactness  \\\n",
       "0             184.60      2019.0           0.16220            0.66560   \n",
       "1             158.80      1956.0           0.12380            0.18660   \n",
       "2             152.50      1709.0           0.14440            0.42450   \n",
       "3              98.87       567.7           0.20980            0.86630   \n",
       "4             152.20      1575.0           0.13740            0.20500   \n",
       "..               ...         ...               ...                ...   \n",
       "564           166.10      2027.0           0.14100            0.21130   \n",
       "565           155.00      1731.0           0.11660            0.19220   \n",
       "566           126.70      1124.0           0.11390            0.30940   \n",
       "567           184.60      1821.0           0.16500            0.86810   \n",
       "568            59.16       268.6           0.08996            0.06444   \n",
       "\n",
       "     worst concavity  worst concave points  worst symmetry  \\\n",
       "0             0.7119                0.2654          0.4601   \n",
       "1             0.2416                0.1860          0.2750   \n",
       "2             0.4504                0.2430          0.3613   \n",
       "3             0.6869                0.2575          0.6638   \n",
       "4             0.4000                0.1625          0.2364   \n",
       "..               ...                   ...             ...   \n",
       "564           0.4107                0.2216          0.2060   \n",
       "565           0.3215                0.1628          0.2572   \n",
       "566           0.3403                0.1418          0.2218   \n",
       "567           0.9387                0.2650          0.4087   \n",
       "568           0.0000                0.0000          0.2871   \n",
       "\n",
       "     worst fractal dimension  \n",
       "0                    0.11890  \n",
       "1                    0.08902  \n",
       "2                    0.08758  \n",
       "3                    0.17300  \n",
       "4                    0.07678  \n",
       "..                       ...  \n",
       "564                  0.07115  \n",
       "565                  0.06637  \n",
       "566                  0.07820  \n",
       "567                  0.12400  \n",
       "568                  0.07039  \n",
       "\n",
       "[569 rows x 30 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset=datasets.load_breast_cancer()  # load dataset\n",
    "pd.DataFrame(dataset.data, columns=dataset.feature_names)  # take a look at the dataset (patients images X features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['malignant', 'benign']"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(dataset.target_names)   # take a look at the target (binary classification!)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_Here you can proceed with EDA to take a look more in deep at the dataset, learned in the previous notebook (link)_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = dataset.data, dataset.target  # create features and targets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create our training and test set.\n",
    "\n",
    "> Remember that the **training set** is used to learn the model, instead the **test set** is used to test the quality of your model, and in many ML competitions is hidden to participants"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "Xtrain, Xtest, ytrain, ytest = train_test_split(X, y, random_state=50)   # Split arrays into random train and test subsets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "#import the models we will use \n",
    "\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.neural_network import MLPClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SV=SVC(random_state=0).fit(Xtrain, ytrain)    #train the model\n",
    "SV.score(Xtrain, ytrain)   #calculate the score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our model has learned the training examples perfectly! But is that really the case? Have we settled on a perfect classification scheme?\n",
    "\n",
    "**_No!_**  To accurately measure the performance, we need to use a separate set of examples, which the model has not yet seen. Of course, this is the role of the test set... "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6293706293706294"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SV.score(Xtest, ytest)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model is not able to generalize to new examples: this is the well known **OVERFITTING**!\n",
    "\n",
    "In this case we learn two important things: first, the fact that the model gives accurate results on training examples, only means that it has learned perfectly the training examples, i.e. it fits perfectly our data, and it is okay. But it doesn't mean it will work equally well on new examples: it doesn't guarantee a good generalization. It's our job to find the right balance between approximation and generalization!\n",
    "\n",
    "Second, the main rule you should know: **never use training examples to measure the quality of your model!** It produces a false prediction of the goodness of our work.\n",
    "\n",
    "That’s what **validation** is for! \n",
    "\n",
    "With validation, one can immediately notice if your model is overfitting, and correct it in time.\n",
    "\n",
    "The basic idea is to split the data with labels into two parts, one is used for the training itself, the other for validation.\n",
    "Validation set try to reproduce the test set, but this time, you can use the resulting information to correct the model!\n",
    "(generally you can't do this with the real test set, often it is unknown, mostly in ML competition)\n",
    "\n",
    "\n",
    "Let's do it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(340, 86)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#apply another time train_test split, this time in order to split in training and validation\n",
    "train_x, valid_x, train_y, valid_y = train_test_split(Xtrain, ytrain, test_size=0.2, random_state=20) \n",
    "len(train_x), len(valid_x)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.627906976744186"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SV=SVC(random_state=0).fit(train_x, train_y)    #train the model with the training set\n",
    "SV.score(valid_x, valid_y)   #calculate the score with the validation set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can notice in time that this model is not too good, and avoid using it!\n",
    "\n",
    "Let's try with other models... (this is the _model selection_ step)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9767441860465116"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RF=RandomForestClassifier(random_state=0).fit(train_x, train_y)\n",
    "RF.score(valid_x, valid_y)   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9534883720930233"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf = LogisticRegression(random_state=0).fit(train_x, train_y) \n",
    "clf.score(valid_x, valid_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9418604651162791"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DT=DecisionTreeClassifier(random_state=0).fit(train_x, train_y)  \n",
    "DT.score(valid_x, valid_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9418604651162791"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "NB=GaussianNB().fit(train_x, train_y)   \n",
    "NB.score(valid_x, valid_y) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9302325581395349"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MLP=MLPClassifier(random_state=0).fit(train_x, train_y)   \n",
    "MLP.score(valid_x, valid_y) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the winner is... Random Forest Classifier! We can now stay safe and use this classifier to perform our binary classification on the breast cancer dataset.\n",
    "\n",
    "> Notice that these validation schemes are supposed to be used to estimate the quality of the model. When you've found the right model and hyper-parameters and want to get test predictions, ***don't forget to retrain your model using all training data.***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9300699300699301"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#retrain the model with the entire dataset\n",
    "final_model=RandomForestClassifier(random_state=0).fit(Xtrain, ytrain)\n",
    "final_model.score(Xtest, ytest)  #the final evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have correct overfitting (almost) by using a different model.\n",
    "\n",
    "Another crucial step in model selection is *hyperparameter tuning*, that consists in adjusting the intrinsic parameters that characterize each model. You will see in detail this theme in a future lesson."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's recap the basic concepts seen in this section:\n",
    "![valid%20test%20train.png](https://github.com/MLJCUnito/ProjectX2020/blob/master/HowToTackleAMLCompetition/img/Lecture3/valid%20test%20train.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> An important thing to keep in mind when splitting your dataset in train and validation parts is that samples between train and validation must not overlap: if we evaluate our model with an example yet seen, the performance of the model increases, but we well know that is not a good thing.\n",
    "This is sometimes the case when we have repeated samples in the data: be aware to exclude duplicates before starting the training and validation steps!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='3-2'></a>\n",
    "## 3.2 Validation strategies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The way to split our dataset we just saw is called **HOLD OUT** strategy:\n",
    "\n",
    "it divides data into two parts. Training part is used to fit our model, the validation part to evaluate its performance. Using the scores from this evaluation step, we can choose the best model and the best hyperparameters for the selected model.\n",
    "\n",
    "If we have enough data, using holdout is usually a good choice. In particular we can use this scheme if we get similar scores for the same model when we try different Hold-out splits. \n",
    "\n",
    "_But what is the optimal size of our sets?_\n",
    "\n",
    "> A common split is using 80% of data for training and the remaining 20% of the data for testing.\n",
    "\n",
    "Another useful command you can use is [sklearn.model_selection.ShuffleSplit](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.ShuffleSplit.html) (this method just splits the data into one training and one validation set)\n",
    "\n",
    "\n",
    "Let's now introduce some other useful strategies to split data, and focus on the reasons to use each one of them."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Leave-One-Out cross validation\n",
    "\n",
    "If your dataset is small, it makes sense the validation set is small, in order to save enough examples for the training. But this means that the validation error will be a bad estimation of the test error, because a too little validation set doesn't capture the behaviour of the test set.\n",
    "\n",
    "To measure the performance of your model with only one example is too constrictive.\n",
    "\n",
    "The solution is the **leave-one-out** stategy: in that scheme we iterate through every sample in our data, each time using one example for validation and the remaining examples for training. You will need to retrain the model N times (if N is the number of samples in the dataset).\n",
    "\n",
    "In the end you will get predictions for every sample in the training set and can calculate the loss by average the single losses  (this averaging process is called _cross validation_ ).\n",
    "\n",
    "![loo.png](https://github.com/MLJCUnito/ProjectX2020/blob/master/HowToTackleAMLCompetition/img/Lecture3/loo.png)\n",
    "\n",
    "This method can be helpful if we have too little data and just enough model to choose from.\n",
    "\n",
    "In scikit-learn there exist a function able to to this:  [sklearn.model_selection.LeaveOneOut](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.LeaveOneOut.html#sklearn.model_selection.LeaveOneOut)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import LeaveOneOut"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For example, let's consider a very small dataset, with only 10 examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10, 4) (10,)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import make_classification\n",
    "\n",
    "# define the dataset\n",
    "X_clf, y_clf = make_classification(n_samples=10, n_features=4, random_state=3)\n",
    "\n",
    "# summarize the dataset\n",
    "print(X_clf.shape, y_clf.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Data')"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEICAYAAABS0fM3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAZ1ElEQVR4nO3deZRU9Z338feHRhpEDQgtIIugooIbJi3GIYgbCS6PmDwm4DwqbqMSHY1mPMKjk+S4HDE5M84kYxJ51EgSI+6m3cYFMcYFpDFEFkVaRRtRaEQQZe/+Pn/UxSnbarrbqu6i+35e59Spe3/3d+t+qUPXp+5S96eIwMzM0qtDsQswM7PichCYmaWcg8DMLOUcBGZmKecgMDNLOQeBmVnKOQjMzFLOQWDWAElLJW2QtE7SGkkvSbpIUqN/N5IGSgpJHVujVrN8OAjMtu9/RcSuwF7AFOAq4PbilmRWWA4CsyaIiLURUQGMAyZIOkjSSZL+JukTSdWSfpa1yvPJ8xpJn0o6UtI+kp6V9JGkVZLuktSt1f8xZvU4CMyaISJeAZYBI4HPgLOAbsBJwERJpyZdj0qeu0XELhHxMiDgRmBPYAjQH/hZ61VvlpuDwKz5lgO7R8RzETE/Iuoi4jXgbmBUQytFRFVEPB0RmyKiBvj37fU3ay0+kWXWfH2B1ZKOIHPe4CCgE1AK3NfQSpL2AH5JZm9iVzJfxD5u8WrNGuE9ArNmkHQ4mSB4AfgTUAH0j4ivAb8lc/gHINdtfW9M2g+JiN2AM7L6mxWNg8CsCSTtJulkYDrwx4iYT+Zb/eqI2ChpOPCPWavUAHXA3lltuwKfkjmB3Be4snWqN9s+eTwCs9wkLQV6AVvJfKgvAv4I/DYiaiWdBvwbsDvwF2ApmZPDZyTrXwtMBHYCxgDrgN8D+wNVwB+AyyOiX+v9q8y+zEFgZpZyPjRkZpZyDgIzs5RzEJiZpZyDwMws5drkD8p69uwZAwcOLHYZZmZtyty5c1dFRFn99jYZBAMHDqSysrLYZZiZtSmS3s3V7kNDZmYp5yAwM0s5B4GZWcoVJAgkjZG0WFKVpEk5lt8saV7yeFPSmqxltVnLKgpRj5mZNV3eJ4sllQC3AKPJDNgxR1JFRCza1iciLs/q/8/AYVkvsSEihuVbh5mZfTWF2CMYDlRFxNsRsZnM3RnHbqf/6WQG8DAzsx1AIYKgL1CdNb8safsSSXsBg4Bns5o7S6qUNCtrmL9c616Q9KusqalpdpERwUsvvcSvfvUrnnvuOXyzPTOzjEL8jiDXwBoNfcqOB+6PiNqstgERsVzS3sCzkuZHxFtfesGIqcBUgPLy8mZ9ikcEZ/6fs3i84nG61ZbxScfVHDnyCB5+5GFKSkqa81JmZu1OIfYIlpEZhHubfmTGdM1lPPUOC0XE8uT5beA5vnj+oCBeeOEFnqj4b4Z9dhT7bDqYQz8dySt/reTRRx8t9KbMzNqcQgTBHGCwpEGSOpH5sP/S1T+S9ge6Ay9ntXWXVJpM9wRGkBn8o6Bmz55Nty09KVFmB6iDOrDLp7sza9asL/RbsGABk66axDXXXMOSJUsKXYaZ2Q4p7yCIiK3AJcCTwOvAvRGxUNK1kk7J6no6MD2+eHB+CFAp6e/ATGBK9tVGhXLggQfyaac11EXdtprZsMs6DjrooM/7PPjgg/zDESO4798e4e6fP8g3hn2DGTNmFLoUM7MdTpscoay8vDyac6+h2tpajj9mNG+8+ia7fNad9V3X0nvfPXhp9ouUlpYSEfTfcwC9P9yH7srcj2llvE/dkE+Zv+i1lvpnmJm1KklzI6K8fnubvOlcc5WUlPDUjCd54IEHmPXyLA77+mGMGzeO0tJSALZs2cLyFe9zAEd8vk53ypj99rxilWxm1mpSsUfQFEMGD2Gnqm70UmYc8fd5h+6Hl/Li7BcLuh0zs2JJ9R5BU9x2522cOOYkPqGGIFhX8jF3T51Z7LLMzFqcgyAxYsQI3l76FhUVFXTs2JGxY8ey2267FbssM7MW5yDI0qNHD84555xil2Fm1qp8G2ozs5RzEJiZpZyDwMws5RwEZmYp5yAwM0s5B4GZWco5CMzMUs5BYGaWcg4CM7OUcxCYmaWcg8DMLOUcBGZmKecgMDNLOQeBmVnKOQjMzFKuIEEgaYykxZKqJE3KsfxsSTWS5iWP87OWTZC0JHlMKEQ9ZmbWdHkPTCOpBLgFGA0sA+ZIqoiIRfW63hMRl9Rbd3fgp0A5EMDcZN2P863LzMyaphB7BMOBqoh4OyI2A9OBsU1c9zvA0xGxOvnwfxoYU4CazMysiQoRBH2B6qz5ZUlbff9b0muS7pfUv5nrIukCSZWSKmtqagpQtuWyatUqVq5cWewyzKwVFSIIlKMt6s0/AgyMiEOAZ4BpzVg30xgxNSLKI6K8rKzsKxdrua1du5Yxo8cwoN8ABvYfyKiRR7Nq1apil2VmraAQQbAM6J813w9Ynt0hIj6KiE3J7P8DvtHUda11XDzxYt54/i2+uWkMR24+gerZH3LuhHOLXZaZtYJCBMEcYLCkQZI6AeOBiuwOkvpkzZ4CvJ5MPwl8W1J3Sd2Bbydt1soeevhhBmzenxKV0EEd2GvLATz+5BPU1tYWuzQza2F5XzUUEVslXULmA7wEuCMiFkq6FqiMiArgUkmnAFuB1cDZybqrJV1HJkwAro2I1fnWZM3XtcvObNmwmVK6ALCFzXTu1JkOHfxTE7P2Lu8gAIiIx4HH67X9JGt6MjC5gXXvAO4oRB321V3+48u5+Yb/pP/6/elAB6p3fpNLLr4YKddpHDNrTwoSBNb2TZo8iV1325Vb/vPX1NbW8uOJP+KKK64odllm1goUkfMinR1aeXl5VFZWFrsMM7M2RdLciCiv3+4DwGZmKecgMDNLOQeBmVnKOQjMzFLOQWBmlnIOAjOzlHMQmJmlnIPAzCzlHARmZinnIDAzSzkHgZlZyjkIzMxSzkFgZpZyDgIzs5RzEJiZpZyDwMws5RwEZmYpV5AgkDRG0mJJVZIm5Vh+haRFkl6TNEPSXlnLaiXNSx4VhajHzMyaLu8xiyWVALcAo4FlwBxJFRGxKKvb34DyiFgvaSLwc2BcsmxDRAzLtw4zM/tqCrFHMByoioi3I2IzMB0Ym90hImZGxPpkdhbQrwDbNTOzAihEEPQFqrPmlyVtDTkPeCJrvrOkSkmzJJ1agHrMzKwZ8j40BChHW+TsKJ0BlAOjspoHRMRySXsDz0qaHxFv5Vj3AuACgAEDBuRftZmZAYXZI1gG9M+a7wcsr99J0vHA1cApEbFpW3tELE+e3waeAw7LtZGImBoR5RFRXlZWVoCyzcwMChMEc4DBkgZJ6gSMB75w9Y+kw4BbyYTAyqz27pJKk+mewAgg+ySzmZm1sLwPDUXEVkmXAE8CJcAdEbFQ0rVAZURUAL8AdgHukwTwXkScAgwBbpVURyaUptS72sjMzFqYInIezt+hlZeXR2VlZbHLMDNrUyTNjYjy+u3+ZbGZWco5CMzMUs5BYGaWcg4CM7OUK8QPysyswDZu3Mhjjz3G2rVrOfHEE+ndu3exS7J2zHsEZjuY6upqDhy6D7/55USeenQyQ4fuwyOPPPL58pUrV3L99ddy/vlnMn36dGpra4tYrbUHvnzUbAdzztmns2e3Z7huUncAXnxlA+MvWs+7761g5cqVHDH8UE44VhwyBH5/31aGHjyaO6fdU+SqrS1o6PJRHxoy28G8/PIL3Hvrzp/PjxjehQ4d1lJdXc2tt97Cd08Q/3FdNwDOGV/H4H94nNdff50hQ4YUq2Rr43xoyGwHM2TIUP46a+Pn81XvbGb9hjp69+7NW1ULOeLrJZ8v69KlA4cM7UpVVVUxSrV2wnsEZjuYn/7sJkYffxSL3wp6dg9u+9MmfvrT6+jSpQsjvvUd7npwDj84JSgpEUurtzD71U+YNnx4scu2Nsx7BGY7mGHDhlE5dz49+k1kzdbTmX7vf3PppZcDcOGFF7K5bn8OPXYV4y/6hPJvr+D666fQq1evIldtbZlPFpu1MRHBCy+8wLvvvstRRx3l8TmsyXyy2KydkMTIkSMZOXJksUuxdsKHhszMUs5BYGaWcg6CdqYtnvMxs+JyELQT06ZNo2/vvpSUdORbR45k8eLFxS7JzNoIB0E7MHPmTC774Y/ou2J/jo6xrHrlU44ZdSxbtmwpdmlm1gY4CNqBW38zld4bBvE19aBEJfSPfYkN4vnnny92aWbWBjgI2gFJQP1zAz5XYGZNU5AgkDRG0mJJVZIm5VheKumeZPlsSQOzlk1O2hdL+k4h6kmbi354IR92WcqaWEVtbOU9LaFkZzFq1Khil2ZmbUDeQSCpBLgFOAEYCpwuaWi9bucBH0fEvsDNwE3JukOB8cCBwBjg18nrWTOMGjWKX/32l3zYp4rnSx6l1ze/xsznZ9Kxo38vaGaNK8QewXCgKiLejojNwHRgbL0+Y4FpyfT9wHHKHM8YC0yPiE0R8Q5QlbyeNdOZZ55J9fJqtm7dwvMv/oXBgwcXuyQzayMKEQR9geqs+WVJW84+EbEVWAv0aOK6AEi6QFKlpMqampoClG1mZlCYIFCOtvpnKhvq05R1M40RUyOiPCLKy8rKmlmimZk1pBBBsAzonzXfD1jeUB9JHYGvAaubuK6ZmbWgQgTBHGCwpEGSOpE5+VtRr08FMCGZPg14NjL3QqgAxidXFQ0CBgOvFKAmMzNrorwvK4mIrZIuAZ4ESoA7ImKhpGuByoioAG4H/iCpisyewPhk3YWS7gUWAVuBiyOiNt+azMys6TwwjZlZSjQ0MI1/WWxmlnIOAjOzlHMQmJmlnIPAzCzlHARmZinnIDAzSzkHgVkj6urq+PnPb2S/wf3Ye1Bv/vVfJ3v0N2tXHARmjZgy5QYevv8X3PVfHXjojlJmvzCVK6+8rNhlmRWMf1Bm1ogB/ct49A87c9ABpQB8uHIr+49Yzpo1n1FS4uEzrO3wD8rMvqKNGzezS9f/+VPZuYvYsqWWurq6IlZlVjgOArNGjBs/nquuX8faT2r5bH0dV13/Cd899SR22mmnYpdmVhAOArNG3HTTzXTtdhz9v76M3gdX89Gnh/Pr3/yu2GWZFYzPEZg10fr166mrq2OXXXYpdilmX0lD5wg8urlZE+28887FLsGsRfjQkJlZyjkIzMxSzkFgZpZyDgIzs5RzEJiZpZyDwMws5fIKAkm7S3pa0pLkuXuOPsMkvSxpoaTXJI3LWnanpHckzUsew/Kpx8zMmi/fPYJJwIyIGAzMSObrWw+cFREHAmOA/5DULWv5lRExLHnMy7MeMzNrpnyDYCwwLZmeBpxav0NEvBkRS5Lp5cBKoCzP7ZqZpcrq1atZt25di7x2vkHQKyI+AEie99heZ0nDgU7AW1nNNySHjG6WVLqddS+QVCmpsqamJs+yzczahhUrVvCdb49k4MA92XPPMs6eMI4NGzYUdBuNBoGkZyQtyPEY25wNSeoD/AE4JyK23b93MnAAcDiwO3BVQ+tHxNSIKI+I8rIy71CYWTqcc/Y4Dt73DVbM70f1q31Zu2oGP/nJ5IJuI6+bzklaDBwdER8kH/TPRcT+OfrtBjwH3BgR9zXwWkcD/xIRJze2Xd90zszSYN26dfTu3YOahQPo3DnzvX3R4k2cfNYGlr67stmv11ID01QAE5LpCcCfc2y4E/AQ8Pv6IZCEB5JE5vzCgjzrMTNrNzp27EiHDh34bP3/fGFfu66Orl0LewPEfINgCjBa0hJgdDKPpHJJtyV9fgAcBZyd4zLRuyTNB+YDPYHr86zHzKzd6NKlC2ec8Y+cdeka/jZ/I3+dtYEfTlrHxB9eUdDteDwCM7Md2ObNm7nuup9yz/Tf06VLZy686HImTryYzIGU5mno0JCDwMwsJTx4vZmZ5eQgMDNLOQeBmVnKOQjMzFLOQWBmlnIOAjOzlHMQmJmlnIPAzCzlHARmZinnIDCzFvP+++/zT/90FgcfNIjvn3YSr732WrFLshwcBGbWIjZt2sTRo75J99InuPPmWr41bA7HHzeS6urqYpdm9TgIzKxFPPbYY/Tvs5Ep13TnsIM788/nd2Pc2FJ+97vbi12a1eMgMLMWsWbNGnrt8cWPmF49g48/XlWkiqwhDgIzaxEnnHACT878lNmvbgRgafUWbvvTJr73vXFFrszq61jsAsysferTpw+33/5HvnfuuXQuXcvadVu55pqfMXLkyGKXZvV4PAIza1GbN29m6dKl9O3bl65duxa7nFRraDwC7xGYWYvq1KkT++23X7HLsO3wOQIzs5TLKwgk7S7paUlLkufuDfSrzRq4viKrfZCk2cn690jqlE89ZmbWfPnuEUwCZkTEYGBGMp/LhogYljxOyWq/Cbg5Wf9j4Lw86zEzs2bKNwjGAtOS6WnAqU1dUZKAY4H7v8r6ZmZWGPkGQa+I+AAged6jgX6dJVVKmiVp24d9D2BNRGxN5pcBfRvakKQLkteorKmpybNsMzPbptGrhiQ9A/TOsejqZmxnQEQsl7Q38Kyk+cAnOfo1eC1rREwFpkLm8tFmbNvMzLaj0SCIiOMbWiZphaQ+EfGBpD7AygZeY3ny/Lak54DDgAeAbpI6JnsF/YDlX+HfYGZmecj30FAFMCGZngD8uX4HSd0llSbTPYERwKLI/JJtJnDa9tY3M7OWlW8QTAFGS1oCjE7mkVQu6bakzxCgUtLfyXzwT4mIRcmyq4ArJFWROWfg2xKambUy32LCzCwlGrrFhH9ZbGaWcg4CM7OUcxCYmaWcg8DMLOUcBGZmKecgMDNLOQeBmVnKOQjMzFLOQWBmlnIOAjOzlHMQmJmlnIPAzCzlHARmZinnIDAzSzkHgZlZyjkIzMxSzkFgZpZyDgIzs5RzEJiZpZyDwMws5fIKAkm7S3pa0pLkuXuOPsdImpf12Cjp1GTZnZLeyVo2LJ96zMys+fLdI5gEzIiIwcCMZP4LImJmRAyLiGHAscB64KmsLlduWx4R8/Ksx8zMminfIBgLTEumpwGnNtL/NOCJiFif53bNzKxA8g2CXhHxAUDyvEcj/ccDd9dru0HSa5JullTa0IqSLpBUKamypqYmv6rNzOxzjQaBpGckLcjxGNucDUnqAxwMPJnVPBk4ADgc2B24qqH1I2JqRJRHRHlZWVlzNm1mZtvRsbEOEXF8Q8skrZDUJyI+SD7oV27npX4APBQRW7Je+4NkcpOk3wH/0sS6zcysQPI9NFQBTEimJwB/3k7f06l3WCgJDySJzPmFBXnWY2ZmzZRvEEwBRktaAoxO5pFULum2bZ0kDQT6A3+pt/5dkuYD84GewPV51mNmZs3U6KGh7YmIj4DjcrRXAudnzS8F+ubod2w+2zczs/z5l8VmZinnIDAzSzkHgZlZyjkIzMxSzkFgZpZyDgIzs5RzEJiZpZyDwMws5RwEZmYp5yAwM0s5B4GZWco5CMzMUs5BYGaWcg4CM7OUcxCYmaWcg8DMLOUcBGZmKZfXCGVmVnh1dXU8++yzvPfeexxzzDEMGjSo2CVZO+cgMNuBfPbZZxw98hjeq6pm57pdubT2Mm648Xou+9FlxS7N2jEfGjLbgdxyyy18+EYNB68bwb7rD2XYxpH838lXs2LFimKXZu1YXkEg6fuSFkqqk1S+nX5jJC2WVCVpUlb7IEmzJS2RdI+kTvnUY9bWzXjqWbpv6IUkALqoKz06lTF37twiV2btWb57BAuA7wHPN9RBUglwC3ACMBQ4XdLQZPFNwM0RMRj4GDgvz3rM2rRDDj2YTzut/Xx+a2zl4y0fsd9++xWxKmvv8gqCiHg9IhY30m04UBURb0fEZmA6MFaZrzzHAvcn/aYBp+ZTj1lbd/mPL+ez3T6mqvTvvBtvsrDry5z2/dPYd999i12atWOtcbK4L1CdNb8MOALoAayJiK1Z7X0behFJFwAXAAwYMKBlKjUrsj333JMFi+Zz++2389aStzjx5BMZO3Zsscuydq7RIJD0DNA7x6KrI+LPTdiGcrTFdtpzioipwFSA8vLyBvuZtXVlZWVMmjSp8Y5mBdJoEETE8XluYxnQP2u+H7AcWAV0k9Qx2SvY1m5mZq2oNS4fnQMMTq4Q6gSMByoiIoCZwGlJvwlAU/YwzMysgPK9fPS7kpYBRwKPSXoyad9T0uMAybf9S4AngdeBeyNiYfISVwFXSKoic87g9nzqMTOz5lPmi3nbUl5eHpWVlcUuw8ysTZE0NyK+9Jsv/7LYzCzlHARmZinnIDAzS7k2eY5AUg3wbgu9fE8yl7amnd8Hvwfb+H3IaA/vw14RUVa/sU0GQUuSVJnrZEra+H3we7CN34eM9vw++NCQmVnKOQjMzFLOQfBlU4tdwA7C74Pfg238PmS02/fB5wjMzFLOewRmZinnIDAzSzkHQQ6SfiHpDUmvSXpIUrdi19TamjoedXvV0DjbaSLpDkkrJS0odi3FIqm/pJmSXk/+Hi4rdk0twUGQ29PAQRFxCPAmMLnI9RRDo+NRt1eNjLOdJncCY4pdRJFtBX4cEUOAbwIXt8f/Cw6CHCLiqawhNGeRGTQnVZo4HnV7lXOc7SLX1Ooi4nlgdbHrKKaI+CAiXk2m15G5lX6DQ+q2VQ6Cxp0LPFHsIqxV5Rpnu9398VvzSBoIHAbMLm4lhdcag9fvkJoyFrOkq8nsGt7VmrW1lgKMR91eNWs8bWv/JO0CPAD8KCI+KXY9hZbaIGhsLGZJE4CTgeOinf7YogDjUbdXDY2zbSkkaScyIXBXRDxY7Hpagg8N5SBpDJlhNE+JiPXFrsdaXc5xtotckxWBJJEZQvf1iPj3YtfTUhwEuf0XsCvwtKR5kn5b7IJaW0PjUadBI+Nsp4aku4GXgf0lLZN0XrFrKoIRwJnAsclnwTxJJxa7qELzLSbMzFLOewRmZinnIDAzSzkHgZlZyjkIzMxSzkFgZpZyDgIzs5RzEJiZpdz/B36iI2hbxY22AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X_clf[:, 0], X_clf[:, 1], marker='o', c=y_clf,\n",
    "            s=25, edgecolor='k')\n",
    "plt.title(\"Data\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loo = LeaveOneOut()\n",
    "loo.get_n_splits(X_clf)   #Returns the number of splitting iterations in the cross-validator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<generator object BaseCrossValidator.split at 0x000001869E136EC8>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loo.split(X_clf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TRAIN: [1 2 3 4 5 6 7 8 9] VALID: [0]\n",
      "1.0\n",
      "TRAIN: [0 2 3 4 5 6 7 8 9] VALID: [1]\n",
      "0.0\n",
      "TRAIN: [0 1 3 4 5 6 7 8 9] VALID: [2]\n",
      "1.0\n",
      "TRAIN: [0 1 2 4 5 6 7 8 9] VALID: [3]\n",
      "1.0\n",
      "TRAIN: [0 1 2 3 5 6 7 8 9] VALID: [4]\n",
      "1.0\n",
      "TRAIN: [0 1 2 3 4 6 7 8 9] VALID: [5]\n",
      "1.0\n",
      "TRAIN: [0 1 2 3 4 5 7 8 9] VALID: [6]\n",
      "0.0\n",
      "TRAIN: [0 1 2 3 4 5 6 8 9] VALID: [7]\n",
      "1.0\n",
      "TRAIN: [0 1 2 3 4 5 6 7 9] VALID: [8]\n",
      "1.0\n",
      "TRAIN: [0 1 2 3 4 5 6 7 8] VALID: [9]\n",
      "1.0\n",
      "Accuracy: 0.80 (+/- 0.80)\n"
     ]
    }
   ],
   "source": [
    "#try with a naive bayes classifier\n",
    "from  numpy import mean, std\n",
    "score=[]\n",
    "\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "NB2=GaussianNB()  \n",
    "\n",
    "for train_index, valid_index in loo.split(X_clf):     #Generate indices to split data into training and test set.\n",
    "     print(\"TRAIN:\", train_index, \"VALID:\", valid_index)\n",
    "     X_train, X_valid = X_clf[train_index], X_clf[valid_index]\n",
    "     y_train, y_valid = y_clf[train_index], y_clf[valid_index]\n",
    "     NB2.fit(X_train, y_train)\n",
    "     print(NB2.score(X_valid, y_valid))\n",
    "     score.append(NB2.score(X_valid, y_valid))\n",
    "print(\"Accuracy: %0.2f (+/- %0.2f)\" % (mean(score), std(score) * 2))   #Average accuracy with cross validation"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### K-fold cross validation\n",
    "\n",
    "K-Fold divides data in K groups of samples of equal size, called folds (if K=size of training, this is equivalent to the Leave One Out strategy). The prediction function is learned using K-folds, and the fold left out is used as a validation set _only once._ After this procedure, we average scores over these K-folds (cross validation step). \n",
    "\n",
    "This method is a good choice when we have enough data and we can get different scores and optimal parameters for different splits. \n",
    "\n",
    "![kfold.png](https://github.com/MLJCUnito/ProjectX2020/blob/master/HowToTackleAMLCompetition/img/Lecture3/kfold.png)\n",
    "\n",
    "You can also estimate mean and variance of the loss. This is very helpful in order to understand the significance of improvement.\n",
    "\n",
    "> Here it is important to understand the difference between K-fold and a K-repeated hold-out. In the first case it is possible to average the scores in order to obtain a mean score indicating the quality of the model. In the second case, you can't: some samples may never get in validation, while others can be there multiple times. An average of the score is no longer informative."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's apply this strategy to our cancer breast classification problem and select the best classifier in this way.\n",
    "\n",
    "Useful commands are [sklearn.model_selection.KFold](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.KFold.html), that provides train/test indices to split data in train/test sets, and [sklearn.model_selection.cross_val_score](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_score.html), for evaluating the scores by cross-validation. We will use this second one command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.63 (+/- 0.01)\n",
      "Accuracy: 0.95 (+/- 0.05)\n",
      "Accuracy: 0.93 (+/- 0.06)\n",
      "Accuracy: 0.92 (+/- 0.06)\n",
      "Accuracy: 0.94 (+/- 0.10)\n",
      "Accuracy: 0.94 (+/- 0.06)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "for clf in [SVC(random_state=0), RandomForestClassifier(random_state=0), GaussianNB(), DecisionTreeClassifier(random_state=0), \n",
    "             LogisticRegression(random_state=0), MLPClassifier(random_state=0)]:\n",
    "    model = clf\n",
    "    scores = cross_val_score(model, Xtrain, ytrain, cv=5)\n",
    "\n",
    "    print(\"Accuracy: %0.2f (+/- %0.2f)\" % (scores.mean(), scores.std() * 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time, the winner is random forest classifier, and it is a more conscious and precise choice."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='3-3'></a>\n",
    "## 3.3 Data Splitting strategies"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Validation strategies we have just seen are holdout, leave-one-out cross validation and K-fold cross validation, and the main difference between them is the number of splits being done. \n",
    "\n",
    "The main idea when you are dealing with a ML competition is that we absolutely want to identify train/test split made by organizers in order to reproduce it with the train/validation split.\n",
    "\n",
    "Let's now summarize some splitting strategies and focus on some concepts to keep in mind when splitting the dataset in training and validation sets.\n",
    "\n",
    "- ## Random split\n",
    "Let's start with the most common way of making a train-validation split: split data randomly. \n",
    "Because the objects in a dataset are represented by rows (vectors of features), this is equivalent to splitting randomly the dataset by rows, and this usually means that the rows are independent of each other. \n",
    "\n",
    "> ###  Random split vs stratification\n",
    "\n",
    "> Sometimes, especially if you do not have enough samples for some class, if you are involved with unbalanced data sets or if you are dealing with multiclass classification tasks, a random split can fail. In fact, obtaining different distributions in different splits can drastically change predictions of our model. \n",
    "What we need here to handle this problem is _stratification._ It is just the way to enquire similar distribution over different faults. This split is made by preserving the percentage of samples for each class and not totally random.\n",
    "You can perform this kind of validation with scikit learn commands [sklearn.model_selection.StratifiedKFold](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.StratifiedKFold.html)\n",
    "\n",
    "![stratified.png](https://github.com/MLJCUnito/ProjectX2020/blob/master/HowToTackleAMLCompetition/img/Lecture3/stratified.png)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- ## Time based split\n",
    "\n",
    "The goal of time series forecasting is to make accurate predictions about the future.\n",
    "\n",
    "When we are dealing with time series (or other data that shows temporal structure), we need to rely on the time trend instead of randomly picking up values. In fact, when performing classical hold-out or k-fold cross validation, the temporal components of the data are ignored, resulting in poor performances of the model.\n",
    "The strategies seen so far do not work in the case of time series data. \n",
    "\n",
    "Since our observations are not independent, what we need is to split observations along with the sequences we can make a _time-based split,_ take everything before some date as a train and everything out there as a validation (then it is similar to Hold-out strategy. But this time without ignoring the temporal correlation between data). \n",
    "\n",
    "A special case of validation for the time-based split is a moving window validation: we can move the date which divides train and validation. Successively using week after week, or year after year, as a validation set and at the end averaging the resulting scores. \n",
    "\n",
    "In scikit-learn you can find the usefull command [sklearn.model_selection.TimeSeriesSplit](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html)\n",
    "\n",
    "![timeseries.png](https://github.com/MLJCUnito/ProjectX2020/blob/master/HowToTackleAMLCompetition/img/Lecture3/timeseries.png)\n",
    "\n",
    "Let's try to implement a very simple example.\n",
    "\n",
    "- ### Heat index prediction \n",
    "\n",
    "The Heat Index is a measure of how hot is really feels when relative humidity is factored in with the actual air temperature. ([National Weather Service page](https://www.weather.gov/ama/heatindex#:~:text=The%20heat%20index%2C%20also%20known,sweat%20to%20cool%20itself%20off.))\n",
    "\n",
    "To determine the heat index using the chart above, you need to know the air temperature and the relative humidity.\n",
    "\n",
    "Suppose we are dealing with a multiclassification task: at each Heat Index corresponds a critical situation, summarized with a qualitative value: caution, extreme caution, danger, extreme danger.\n",
    "\n",
    "Labels for the multiclassification task: \n",
    "- caution - 1\n",
    "- extreme caution -2 \n",
    "- danger - 3\n",
    "- extreme danger - 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>relative humidity (%)</th>\n",
       "      <th>temperature (°F)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>60</td>\n",
       "      <td>80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>65</td>\n",
       "      <td>82</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>55</td>\n",
       "      <td>92</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>55</td>\n",
       "      <td>96</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>60</td>\n",
       "      <td>102</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>65</td>\n",
       "      <td>96</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>55</td>\n",
       "      <td>94</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   relative humidity (%)  temperature (°F)\n",
       "0                     60                80\n",
       "1                     65                82\n",
       "2                     55                92\n",
       "3                     55                96\n",
       "4                     60               102\n",
       "5                     65                96\n",
       "6                     55                94"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#dataset and labels creation\n",
    "\n",
    "X = np.array([[60, 80], [65, 82], [55, 92], [55, 96], [60, 102], [65, 96], [55,94]])  \n",
    "y = np.array([1, 1, 2, 3, 4, 3, 3])\n",
    "\n",
    "df=pd.DataFrame(X)\n",
    "df.columns=['relative humidity (%)', 'temperature (°F)']\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "#let's create the test set: the two days after..\n",
    "#dataset and label creation\n",
    "\n",
    "Xtest = np.array([[60, 100], [55, 112]])  \n",
    "ytest = np.array([4, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TRAIN: [0] VALID: [1]\n",
      "prediction: [1]\n",
      "ground truth: [1]\n",
      "accuracy: 1.0\n",
      "TRAIN: [0 1] VALID: [2]\n",
      "prediction: [1]\n",
      "ground truth: [2]\n",
      "accuracy: 0.0\n",
      "TRAIN: [0 1 2] VALID: [3]\n",
      "prediction: [2]\n",
      "ground truth: [3]\n",
      "accuracy: 0.0\n",
      "TRAIN: [0 1 2 3] VALID: [4]\n",
      "prediction: [3]\n",
      "ground truth: [4]\n",
      "accuracy: 0.0\n",
      "TRAIN: [0 1 2 3 4] VALID: [5]\n",
      "prediction: [3]\n",
      "ground truth: [3]\n",
      "accuracy: 1.0\n",
      "TRAIN: [0 1 2 3 4 5] VALID: [6]\n",
      "prediction: [2]\n",
      "ground truth: [3]\n",
      "accuracy: 0.0\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import TimeSeriesSplit\n",
    "\n",
    "for train_index, valid_index in TimeSeriesSplit(n_splits=6).split(X):  #max_train_size=None, \n",
    "    print(\"TRAIN:\", train_index, \"VALID:\", valid_index)\n",
    "    X_train, X_valid = X[train_index], X[valid_index]\n",
    "    y_train, y_valid = y[train_index], y[valid_index]\n",
    "    \n",
    "    RF3=RandomForestClassifier(random_state=0).fit(X_train, y_train)\n",
    "    print('prediction:', RF3.predict(X_valid))\n",
    "    print('ground truth:', y_valid)\n",
    "    print('accuracy:', RF3.score(X_valid, y_valid))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "prediction: [4 3]\n",
      "ground truth: [4 3]\n",
      "accuracy: 1.0\n"
     ]
    }
   ],
   "source": [
    "print('prediction:', RF3.predict(Xtest))\n",
    "print('ground truth:', ytest)\n",
    "print('accuracy:', RF3.score(Xtest, ytest))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- ## ID-based split\n",
    "\n",
    "ID can be a unique identifier of objects: patients, authors, users on a social network, shops or any other entity. That means, we have different sets of users in train and test; in this way one can also prevent train-validation overlap.\n",
    "\n",
    "For example, if you are dealing with a medical task involving the analysis of a set of images, it is important that the validation set doesn't contain images of a patient used for training: this could result in an unbiased evaluation of your model in the validation step.\n",
    "\n",
    "You can use the command [GroupShuffleSplit](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GroupShuffleSplit.html) from sklearn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GroupShuffleSplit\n",
    "X=np.random.rand(8,2)\n",
    "y=(X[:,0]+X[:,1]).astype(int)\n",
    "ID = np.array([1, 1, 2, 2, 2, 3, 3, 3])  #let's create ID"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X_1</th>\n",
       "      <th>X_2</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ID</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.705334</td>\n",
       "      <td>0.685209</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.216381</td>\n",
       "      <td>0.195379</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.697304</td>\n",
       "      <td>0.897512</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.206541</td>\n",
       "      <td>0.723181</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.889667</td>\n",
       "      <td>0.182290</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.753047</td>\n",
       "      <td>0.224250</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.155819</td>\n",
       "      <td>0.354998</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.389877</td>\n",
       "      <td>0.063310</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         X_1       X_2  label\n",
       "ID                           \n",
       "1   0.705334  0.685209      1\n",
       "1   0.216381  0.195379      0\n",
       "2   0.697304  0.897512      1\n",
       "2   0.206541  0.723181      0\n",
       "2   0.889667  0.182290      1\n",
       "3   0.753047  0.224250      0\n",
       "3   0.155819  0.354998      0\n",
       "3   0.389877  0.063310      0"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d = {'X_1': X[:,0], 'X_2': X[:,1], 'label': y, 'ID': ID}\n",
    "df = pd.DataFrame(data=d)\n",
    "df.set_index('ID')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TRAIN INDEX: [2 3 4 5 6 7] , which IDs are [2 2 2 3 3 3] ; VALID: [0 1] , which IDs are [1 1]\n",
      "TRAIN INDEX: [0 1 5 6 7] , which IDs are [1 1 3 3 3] ; VALID: [2 3 4] , which IDs are [2 2 2]\n"
     ]
    }
   ],
   "source": [
    "gss = GroupShuffleSplit(n_splits=2, train_size=.7, random_state=42)\n",
    "gss.get_n_splits()\n",
    "\n",
    "for train_idx, valid_idx in gss.split(X,y,ID):\n",
    "   print(\"TRAIN INDEX:\", train_idx, \", which IDs are\", ID[train_idx], \"; VALID:\", valid_idx, \", which IDs are\", ID[valid_idx],)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that train and validation sets cointain examples with IDs not overlapping."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='3-4'></a>\n",
    "## 3.4 Problems occurring during validation "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's summarize the problems you can meet when performing the validation step:\n",
    "\n",
    "_**Get different optimal parameters for different faults**_ and  _**obtain difference in scores for different train validation splits.**_\n",
    "\n",
    "We can identify two main reasons for this:\n",
    "\n",
    "1. _Having too little data:_ if we have a lot of patterns but we do not have enough samples to generalize these patterns well, the model will utilize only some general patterns. And for each train validation split, these patterns will partially differ, resulting in different scores. \n",
    "\n",
    "2. _Data is too diverse and inconsistent:_ if you have very similar samples with different target variance, a model can confuse them.\n",
    "\n",
    "> What can we do to contain these problems? \n",
    "- You can increase the number of folds (but usually 5 folds are enough);\n",
    "- You can perform the validation strategy you choose several times, with different random splits and average scores to get a more stable estimate of model's quality;\n",
    "- Also it is useful to use one set of splits to select parameters, and another to check the model's performance. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='4'></a>\n",
    "# 4. References and additional materials"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 'Learning From Data, a short course', Hsuan-Tien Lin, Yaser S. Abu-Mostafa, Malik Magdon-Ismail  (cap 4)\n",
    "\n",
    "- [How to Select Your Final Models in a Kaggle Competition](http://www.chioka.in/how-to-select-your-final-models-in-a-kaggle-competitio/)\n",
    "\n",
    "- [validation stategies: visualization in scikit-learn](https://scikit-learn.org/stable/auto_examples/model_selection/plot_cv_indices.html)\n",
    "\n",
    "- [How to Win a Data Science Competition: Learn from Top Kagglers](https://www.coursera.org/learn/competitive-data-science)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}