{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "i_f5u2x9nn6I",
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"# **Lecture 2: Supervised Machine Learning**"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"This lecture will dive deeper into supervised learning and introduce mathematical notation that will be useful throughout the course."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# 2.1. Elements of A Supervised Machine Learning Problem\n",
"\n",
"\n",
"Let's first look at a more detailed example of a supervised learning problem. We will use a simple running example---predicting the diabetes risk of a patient from their BMI."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Three Components of Supervised Machine Learning\n",
"\n",
"It is useful to think of supervised learning as involving three key elements: a dataset, a learning algorithm, and a predictive model.\n",
"\n",
"To apply supervised learning, we define a dataset and a learning algorithm.\n",
"\n",
"$$ \\text{Dataset} + \\text{Learning Algorithm} \\to \\text{Predictive Model} $$\n",
"\n",
"The result of running the learning algorithm on the dataset is a predictive model that maps inputs to targets. For instance, it can predict targets on new inputs.\n",
"Next, we will give examples of each of these three components."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 2.1.1. A Supervised Learning Dataset\n",
"\n",
"Let’s start with a simple dataset of medical patients.\n",
"\n",
"* For each patient we have a access to their BMI and an estimate of diabetes risk (from 0-400).\n",
"* We are interested in understanding how BMI affects an individual's diabetes risk."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"We are going to load a real diabetes dataset from `scikit-learn`, a popular machine learning library that we will use throughout the course."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
bmi
\n",
"
target
\n",
"
\n",
" \n",
" \n",
"
\n",
"
422
\n",
"
27.335902
\n",
"
233.0
\n",
"
\n",
"
\n",
"
423
\n",
"
23.811456
\n",
"
91.0
\n",
"
\n",
"
\n",
"
424
\n",
"
25.331171
\n",
"
111.0
\n",
"
\n",
"
\n",
"
425
\n",
"
23.779122
\n",
"
152.0
\n",
"
\n",
"
\n",
"
426
\n",
"
23.973128
\n",
"
120.0
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" bmi target\n",
"422 27.335902 233.0\n",
"423 23.811456 91.0\n",
"424 25.331171 111.0\n",
"425 23.779122 152.0\n",
"426 23.973128 120.0"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"from sklearn import datasets\n",
"\n",
"# We will use the UCI Diabetes Dataset\n",
"# It's a toy dataset often used to demo ML algorithms.\n",
"diabetes_X, diabetes_y = datasets.load_diabetes(return_X_y=True, as_frame=True)\n",
"\n",
"# Use only the BMI feature\n",
"diabetes_X = diabetes_X.loc[:, ['bmi']]\n",
"\n",
"# The BMI is zero-centered and normalized; we recenter it for ease of presentation\n",
"diabetes_X = diabetes_X * 30 + 25\n",
"\n",
"# Collect 20 data points\n",
"diabetes_X_train = diabetes_X.iloc[-20:]\n",
"diabetes_y_train = diabetes_y.iloc[-20:]\n",
"\n",
"# Display some of the data points\n",
"pd.concat([diabetes_X_train, diabetes_y_train], axis=1).head()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Let's also visualize this two-dimensional dataset."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"Text(0, 0.5, 'Diabetes Risk')"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtMAAAEGCAYAAACn7xkwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAfE0lEQVR4nO3df3ilZ13n8fcnbaENFCh06A5tk4NYZAtKKbGCFZcfggrutrAsFgNUZI16UQFx9QLiAiqRKr8UQdxgCwXOAlX5UbCrQkGQaxV2ppT+BKmYDC2lHaQUMFKg890/zjMlk0lOkjM5OSeT9+u6cuV57uc5z/M988yZfHPPfX/vVBWSJEmS1m9k0AFIkiRJW5XJtCRJktQjk2lJkiSpRybTkiRJUo9MpiVJkqQeHTnoAA7F8ccfX61Wa9BhSJIk6TC3e/fur1TVjqXtWzqZbrVa7Nq1a9BhSJIk6TCXZH65dod5SJIkST0ymZYkSZJ6ZDItSZIk9chkWpIkSeqRybQkSZLUI5NpSZIkDbV2u02r1WJkZIRWq0W73R50SHfa0qXxJEmSdHhrt9tMTU2xsLAAwPz8PFNTUwBMTk4OMjTAnmlJkiQNsenp6TsT6f0WFhaYnp4eUEQHMpmWJEnS0NqzZ8+62jebybQkSZKG1tjY2LraN5vJtCRJkobWzMwMo6OjB7SNjo4yMzMzoIgOZDItSZKkoTU5Ocns7Czj4+MkYXx8nNnZ2aGYfAiQqhp0DD2bmJioXbt2DToMSZIkHeaS7K6qiaXtfeuZTnJ0kk8l+UySa5L8dtN+/ySfTHJ9kncnuUvTftdm//rmeKtfsUmSJEkboZ/DPG4HHltVDwVOA34qySOA3wdeV1XfD9wKPKc5/znArU3765rzJEmSpKHVt2S6Or7Z7B7VfBXwWOAvmvaLgLOb7bOafZrjj0uSfsUnSZIkHaq+TkBMckSSK4BbgA8B/wx8raq+25xyA3Bis30i8EWA5vhtwH2WueZUkl1Jdu3du7ef4UuSJEld9TWZrqo7quo04CTgDOBBG3DN2aqaqKqJHTt2HOrlJEmSpJ5tSmm8qvoa8FHgkcC9khzZHDoJuLHZvhE4GaA5fk/gXzcjPkmSJKkX/azmsSPJvZrtY4DHA9fRSaqf2px2LvD+ZvuSZp/m+EdqK9ftkyRJ0mHvyNVP6dlO4KIkR9BJ2i+uqg8muRZ4V5JXAJ8GLmjOvwB4e5Lrga8C5/QxNkmSJOmQ9S2ZrqorgYct0/4FOuOnl7Z/C/hv/YpHkiRJ2mguJy5JkiT1yGRakiRJ6pHJtCRJktQjk2lJkiSpRybTkiRJUo9MpiVJkqQemUxLkiRJPTKZliRJknpkMi1JkiT1yGRakiRJ6pHJtCRJktQjk2lJkiSpRybTkiRJUo9MpiVJkqQemUxLkiRJPTKZliRJknpkMi1JkiT1yGRakiRJ6pHJtCRJktQjk2lJkiSpRybTkiRJUo9MpiVJkqQemUxLkiRJPTKZliRJknpkMi1JkiT1yGRakiRJ6pHJtCRJktQjk2lJkiSpR31LppOcnOSjSa5Nck2S5zftL09yY5Irmq8nLnrNi5Ncn+RzSX6yX7FJkiRJG+HIPl77u8CvV9XlSY4Fdif5UHPsdVX16sUnJzkVOAd4MHA/4MNJHlhVd/QxRkmSJKlnfeuZrqqbquryZvsbwHXAiV1echbwrqq6var+BbgeOKNf8UmSJEmHalPGTCdpAQ8DPtk0nZfkyiQXJjmuaTsR+OKil91A9+RbkiRJGqi+J9NJ7g78JfCCqvo68CbgAcBpwE3Aa9Z5vakku5Ls2rt370aHK0mSJK1ZX5PpJEfRSaTbVfUegKq6uaruqKp9wJv53lCOG4GTF738pKbtAFU1W1UTVTWxY8eOfoYvSZIkddXPah4BLgCuq6rXLmrfuei0JwNXN9uXAOckuWuS+wOnAJ/qV3ySJEnSoepnNY8zgWcCVyW5oml7CfD0JKcBBcwBvwRQVdckuRi4lk4lkOdayUOSJEnDrG/JdFV9Asgyhy7t8poZYKZfMUmSJEkbyRUQJUmSpB6ZTEuSJEk9MpmWJEmSemQyLUmSJPXIZFqSJGmba7fbtFotRkZGaLVatNvtQYe0ZfSzNJ4kSZKGXLvdZmpqioWFBQDm5+eZmpoCYHJycpChbQn2TEuSJG1j09PTdybS+y0sLDA9PT2giLYWk2lJkqRtbM+ePetq14FMpiVJkraxsbGxdbXrQCbTkiRJ29jMzAyjo6MHtI2OjjIz46LUa2EyLUmStI1NTk4yOzvL+Pg4SRgfH2d2dtbJh2uUqhp0DD2bmJioXbt2DToMSZIkHeaS7K6qiaXt9kxLkiRJPTKZliRJknpkMi1J0hbn6nXS4Ky6AmKS1JKB1UnuWlW39y8sSZK0Fq5eJw3WWnqmL1i8k+TuwKX9CUeSJK2Hq9dJg7WWZPqGJH8CkOQ44G+Bd/Q1KkmStCauXicN1qrJdFW9FPhmkj+lk0i/pqre0vfIJEnSqly9ThqsFZPpJE/Z/wV8EngE8GmgmjZJkjRgrl4nDVa3CYj/ecn+p4GjmvYC3tOvoCRJ0trsn2Q4PT3Nnj17GBsbY2ZmxsmH0iZxBURJkiRpFT2vgJjkD5LcI8lRSS5LsjfJM/oTpiRJkrR1rKWaxxOq6uvAzwBzwPcDv9HPoCRJkqStYC3J9P5x1U8C/ryqbutjPJIkSdKWseoKiMAHk3wW+HfgV5LsAL7V37AkSZKk4beWOtMvAn4UmKiq7wD/BpzV78AkSZKkYbdiz3SSx1bVRxbXlE6y+BRL40mSJGlb6zbM4z8BH+HgetNgnWlJkiRp5WS6ql7WfH92LxdOcjLwNuAEOsn3bFX9UZJ7A+8GWnSqgzytqm5Np9v7j4AnAgvAz1fV5b3cW5IkSdoMXcdMJzkiyfGL9u+SZCrJdWu49neBX6+qU+ksRf7cJKcCLwIuq6pTgMuafYCfBk5pvqaAN6373UiSJEmbaMVkOsk5wFeBK5N8LMkTgC/QSXpXXaO0qm7a37NcVd8ArgNOpDN58aLmtIuAs5vts4C3Vcc/AvdKsrOndyVJkiRtgm5jpn8LeHhVXZ/kdOAfgKdW1QfWe5MkLeBhwCeBE6rqpubQl+kMA4FOov3FRS+7oWm7aVEbSabo9FwzNja23lAkSZKkDdNtmMe3q+p6gKaH+fM9JtJ3B/4SeEGzkuKdqqrojKdes6qaraqJqprYsWPHesORJEmSNky3nun7Jnnhov17Ld6vqteudvEkR9FJpNtVtb/6x81JdlbVTc0wjlua9huBkxe9/KSmTZIkSRpK3Xqm3wwcu+hr6X5XTXWOC4DrliTelwDnNtvnAu9f1P6sdDwCuG3RcBBJkiRp6HQrjffbh3jtM4FnAlcluaJpewlwPnBxkucA88DTmmOX0imLdz2d0ng9leSTJEmSNku3YR6HpKo+AWSFw49b5vwCntuveCRJkqSN1rXOtCRJkqSVmUxLkiRJPVo1mU7y/CT3aCYGXpDk8mYBF0mSJGlbW0vP9C809aGfABxHZ1Lh+X2NSpIkSdoC1pJM759E+ETg7VV1DStPLNRhrt1u02q1GBkZodVq0W63Bx2SJEnSwKylmsfuJH8L3B94cZJjgX39DUvDqN1uMzU1xcLCAgDz8/NMTU0BMDk5OcjQJEmSBiKdinRdTkhGgNOAL1TV15LcBzixqq7chPi6mpiYqF27dg06jG2j1WoxPz9/UPv4+Dhzc3ObH5AkSdImSbK7qiaWtq9lmEcBpwLPa/bvBhy9gbFpi9izZ8+62iVJkg53a0mm/wR4JPD0Zv8bwBv7FpGG1tjY2LraJWkjOFdD0jBbSzL9I1X1XOBbAFV1K3CXvkaloTQzM8Po6OgBbaOjo8zMzAwoIkmHu/1zNebn56mqO+dqmFBLGhZrSaa/k+QIOsM9SLIDJyBuS5OTk8zOzjI+Pk4SxsfHmZ2ddfKhpL6Znp6+c9LzfgsLC0xPTw8oIkk60FomIE4CPwucDlwEPBX4n1V1cf/D684JiJJ0eBsZGWG5n1NJ2LfPfh1Jm6fnCYhV1QZ+E3glcBNw9jAk0pKkw59zNbQcx9FrmKxlOfG3V9Vnq+qNVfWGqrouyds3IzhJ0vbmXA0t5Th6DZu1jJl+8OKdZvz0w/sTjiRJ3+NcDS3lOHoNmxXHTCd5MfAS4Bhgge8tIf5tYLaqXrwpEXbhmGlJkrYXx9FrUNY9ZrqqXllVxwKvqqp7VNWxzdd9hiGRliRJ24/j6DVs1jLMYzrJM5L8T4AkJyc5o89xSZIkHcRx9Bo2a0mm30hnBcSfa/a/iSsgSpKkAXAcvYbNWupMX15Vpyf5dFU9rGn7TFU9dFMi7MIx05IkSdoMPdeZxhUQJUmSpGWtJZl+PfBe4IQkM8AngN/ra1SSJEnSFnDkaidUVTvJbuBxTdPZVXVdf8OSJEmSht+qyXRjFNg/1OOY/oUjSZIkbR1rWU78pcBFwL2B44G3JPmtfgcmSZIkDbu19ExPAg+tqm8BJDkfuAJ4RR/jkiRJkobeWiYgfgk4etH+XYEb+xOOJEmStHWsmEwn+eMkrwduA65J8tYkbwGuBr62SfFJA9dut2m1WoyMjNBqtWi324MOSZIkDYluwzz2r4aym05pvP3+rm/RSEOm3W4zNTXFwsICAPPz80xNTQG42pYkSVp9BcSeL5xcCPwMcEtVPaRpeznwi8De5rSXVNWlzbEXA88B7gCeV1V/s9o9XAFR/dZqtZifnz+ofXx8nLm5uc0PSJIkDcRKKyCuOgExySnAK4FTWTR2uqq+b5WXvhV4A/C2Je2vq6pXL7nHqcA5wIOB+wEfTvLAqrpjtfikftqzZ8+62iVJ0vaylgmIbwHeBHwXeAyd5Pgdq72oqj4OfHWNcZwFvKuqbq+qfwGuB85Y42ulvhkbG1tXuyRJ2l7WkkwfU1WX0RkSMl9VLweedAj3PC/JlUkuTHJc03Yi8MVF59zQtB0kyVSSXUl27d27d7lTpA0zMzPD6OjoAW2jo6PMzMwMKCJJkjRM1pJM355kBPh8kvOSPBm4e4/3exPwAOA04CbgNeu9QFXNVtVEVU3s2LGjxzCktZmcnGR2dpbx8XGSMD4+zuzsrJMPJUkSsLZFW55PZznx5wG/CzwWOLeXm1XVzfu3k7wZ+GCzeyNw8qJTT8Ja1hoSk5OTJs+SJGlZqybTVfX/ms1vAs8+lJsl2VlVNzW7T6ZTsxrgEuB/J3ktnQmIpwCfOpR7SZIkSf22YjKd5A+r6gVJPgAcVD+vqv5LtwsneSfwaOD4JDcALwMeneS05npzwC8117omycXAtXQmOj7XSh6SJEkadt16pt/efH91l3NWVFVPX6b5gi7nzwDO6pIkSdKWsWIyXVW7m+8fS7Kj2bZ8hiRJktToWs0jycuTfAX4HPBPSfYmeenmhCZJkiQNtxWT6SQvBM4Efriq7l1VxwE/ApyZ5Nc2K0BJkiRpWHXrmX4m8PRmRUIAquoLwDOAZ/U7MEmSJGnYdUumj6qqryxtbMZNH9W/kCRJkqStoVsy/e0ej0lSz9rtNq1Wi5GREVqtFu12e9AhSZK0om6l8R6a5OvLtAc4uk/xSNrG2u02U1NTLCwsADA/P8/U1BSAq1BKkoZSqg5aj2XLmJiYqF27dg06DEkbpNVqMT8/f1D7+Pg4c3Nzmx+QJEmNJLuramJpe9fSeJK0mfbs2bOudkmSBs1kWtLQGBsbW1e7JEmDZjItaWjMzMwwOjp6QNvo6CgzMzMDikiSpO5MpiUNjcnJSWZnZxkfHycJ4+PjzM7OOvlQkjS0nIAoSZIkrcIJiJIkSdIGM5mWBsxFSiRJ2rq6Ldoiqc9cpESSpK3NnmlpgKanp+9MpPdbWFhgenp6QBFJkqT1MJmWBshFSiRJ2tpMpqUBcpESSZK2NpNpaYBcpESSpK3NZFoaIBcp0VZgxRlJWpmLtkiSVrS04gx0/vfEX/okbTcu2iJJWjcrzkhSdybTkqQVWXFGkrozmZYkrciKM5LUncm0JGlFVpyRpO5MpiUNFStHDBcrzkhSd1bzkDQ0rBwhSRpWVvOQNPSsHCFJ2mr6lkwnuTDJLUmuXtR27yQfSvL55vtxTXuSvD7J9UmuTHJ6v+KSNLysHCFJ2mr62TP9VuCnlrS9CLisqk4BLmv2AX4aOKX5mgLe1Me4JA0pK0dIkraaviXTVfVx4KtLms8CLmq2LwLOXtT+tur4R+BeSXb2KzZJw8nKEZKkrWazx0yfUFU3NdtfBk5otk8EvrjovBuatoMkmUqyK8muvXv39i9SSZvOyhGSpK1mYBMQq1NGZN2lRKpqtqomqmpix44dfYhMGqztXhpucnKSubk59u3bx9zcnIm0JGmoHbnJ97s5yc6quqkZxnFL034jcPKi805q2qRtZWlpuPn5eaampgBMKiVJGkKb3TN9CXBus30u8P5F7c9qqno8Arht0XAQaaj0s+fY0nCSJG0tfeuZTvJO4NHA8UluAF4GnA9cnOQ5wDzwtOb0S4EnAtcDC8Cz+xWXdCj63XNsaThJkrYWV0CU1qHVajE/P39Q+/j4OHNzc0N/fUmS1BtXQJQ2QL97ji0NJ0nS1mIyLa1DvxcVsTScJElbi8m0tA6b0XNsaThJkrYOk2lpHew5liRJizkBUZIkSVqFExAlSZKkDWYyLUmSJPXIZFqSJEnqkcm0JEmS1COTaUmSJKlHJtOSJElSj0ymJUmSpB6ZTG9z7XabVqvFyMgIrVaLdrs96JAkSZK2jCMHHYAGp91uMzU1xcLCAgDz8/NMTU0BuKKfJEnSGtgzvY1NT0/fmUjvt7CwwPT09IAikiRJ2lpMprexPXv2rKtdkiRJBzKZ3sbGxsbW1a7h4Dh3rca/I5K0eUymt7GZmRlGR0cPaBsdHWVmZmZAEWk1+8e5z8/PU1V3jnM3WdJ+/h2RpM1lMr2NTU5OMjs7y/j4OEkYHx9ndnbWyYdDbBDj3O3l3FqcCyFJmytVNegYejYxMVG7du0adBjSphkZGWG5z2wS9u3bt+H3W1rxBTr/e+EvXcNrs/+OSNJ2kWR3VU0sbbdnWtpCNnucu72cW49zISRpc5lMS1vIZo9zt+LL1uNcCEnaXCbT0hay2ePc7eXcepwLIUmby2T6EDk5S5ttcnKSubk59u3bx9zcXF+TJHs5t6bN/DsiSdudyfQhsASVDnf2ckqS1J3VPA5Bq9Vifn7+oPbx8XHm5uY2PyBJkiT1hdU8+sDJWZIkSdubyfQhcHKWJEnS9jaQZDrJXJKrklyRZFfTdu8kH0ry+eb7cYOIbT2cnCVJkrS9DbJn+jFVddqisScvAi6rqlOAy5r9oebkLEn9YJUgSdo6hmmYx1nARc32RcDZgwtlZUt/yAGWoJK0YawSJElby0CqeST5F+BWoID/VVWzSb5WVfdqjge4df/+ktdOAVMAY2NjD1+umka/7P8ht3h55dHRUXujJW0YqwRJ0nBaqZrHoJLpE6vqxiT3BT4E/CpwyeLkOcmtVdV13PRml8bzh5ykfhsZGWG5f5eTsG/fvgFEJEmCISuNV1U3Nt9vAd4LnAHcnGQnQPP9lkHE1o2l8CT1m1WCJGlr2fRkOsndkhy7fxt4AnA1cAlwbnPaucD7Nzu21fhDTlK/WSVIkraWQfRMnwB8IslngE8Bf1VVfw2cDzw+yeeBn2j2h4o/5CT1m1WCJGlrcTnxdWq320xPT7Nnzx7GxsaYmZnxh5wkSdJhbqgmIG6UQSTTkiRJ2n6GagKiJEmSdDgwmZYkSZJ6ZDItSZIk9chkWpIkSeqRybQkSZLUoy1dzSPJXuDg9b0PP8cDXxl0ENpUPvPtx2e+vfi8tx+f+dY3XlU7ljZu6WR6u0iya7lSLDp8+cy3H5/59uLz3n585ocvh3lIkiRJPTKZliRJknpkMr01zA46AG06n/n24zPfXnze24/P/DDlmGlJkiSpR/ZMS5IkST0ymZYkSZJ6ZDI9ZJKcnOSjSa5Nck2S5zftv5vkyiRXJPnbJPcbdKzaGCs980XHfz1JJTl+UDFq43T5jL88yY3NZ/yKJE8cdKzaGN0+40l+Nclnm/Y/GGSc2jhdPufvXvQZn0tyxYBD1QZwzPSQSbIT2FlVlyc5FtgNnA3cUFVfb855HnBqVf3y4CLVRlnpmVfVtUlOBv4MeBDw8Kqy4P8W1+Uz/jTgm1X16kHGp43X5ZmfAEwDT6qq25Pct6puGWCo2iDd/l1fdM5rgNuq6ncGFac2hj3TQ6aqbqqqy5vtbwDXASfuT6QbdwP8LegwsdIzbw6/DvhNfN6HjVWetw5DXZ75rwDnV9XtzTET6cPEap/zJKHzC/Q7BxOhNpLJ9BBL0gIeBnyy2Z9J8kVgEnjpAENTnyx+5knOAm6sqs8MNir1y9LPOHBeM5zrwiTHDS4y9cuSZ/5A4FFJPpnkY0l+eKDBqS+W+ZwDPAq4uao+P5CgtKFMpodUkrsDfwm8YH+vdFVNV9XJQBs4b5DxaeMtfubAd4GX4C9Nh61lPuNvAh4AnAbcBLxmcNGpH5Z55kcC9wYeAfwGcHHTY6nDxHI/yxtPx17pw4bJ9BBKchSdD1+7qt6zzClt4L9ublTqp2We+QOA+wOfSTIHnARcnuQ/DC5KbZTlPuNVdXNV3VFV+4A3A2cMMkZtrBX+Xb8BeE91fArYBzjR+DCx0s/yJEcCTwHePajYtLFMpodM0ytxAXBdVb12Ufspi047C/jsZsem/ljumVfVVVV136pqVVWLzg/d06vqywMMVRugy2d856LTngxcvdmxqT9WeubA+4DHNOc8ELgL4CTjw0CXZw7wE8Bnq+qGzY9M/WA1jyGT5MeAvweuotNLAZ3/7n8O8ANN2zzwy1V140CC1IZa6ZlX1aWLzpkDJqzmsfV1+Yw/nc4QjwLmgF+qqpsGEKI2WJdn/mHgQjrP/dvA/6iqjwwiRm2sbv+uJ3kr8I9V9aeDik8by2RakiRJ6pHDPCRJkqQemUxLkiRJPTKZliRJknpkMi1JkiT1yGRakiRJ6pHJtCQtkeSOJFck+UySy5P86Dpf/9YkT13H+X+XZM/i1e+SvC/JN9dz33Xc7+eTvGGdr1nXe1rlWn+R5Pua7bkkVzV/3lclOWvReZXkHYv2j0yyN8kHl76PJOcl+YWNiE+S1sNkWpIO9u9VdVpVPRR4MfDKTbjn14AzAZLcC9jZ7eStKsmDgSOq6guLmh9TVacBTwVev6j934CHJDmm2X88sFJ9/QuBX93gcCVpVSbTktTdPYBbobOqWZJXJbm66UX92UXtb0jyuSQfBu7btD82yfv2XyjJ45O8d4X7vAs4p9l+CrB4+eG7J7ms6SW/s/c2yd2S/FXTg371onjOT3JtkiuTvLrbm2t6nF+f5P8m+cL+3ueV3lNz7OFJPpZkd5K/SbIzyT2bc3+gOeedSX5xmVtOAu9fIZw7/6wXuRR4UrP9dOCdy72wqhaAuSQuwy5pU5lMS9LBjmmGHXwW+DPgd5v2p9BZre6hdJYEflWzDPiT6axQeirwLGD/sJCPAg9KsqPZfzadHtTlXAb8eJIj6CTV71507FvAk6vqdDrLT7+mGRLyU8CXquqhVfUQ4K+T3KeJ58FV9UPAK9bwfncCPwb8DHB+07bse0pyFPDHwFOr6uHN+5mpqtuA84C3JjkHOK6q3rzMvc4Edi9p+2iSq4GPAb+15Ni7gHOSHA38EPDJLu9jF/Co1d+uJG0ck2lJOtj+YR4PopOwvq1JXn8MeGdV3VFVN9NJ/n4Y+PFF7V8CPgJQnSVm3w48oxm68Ujg/6xwzzuAT9BJpI+pqrlFxwL8XpIr6SxBfSJwAp2lih+f5PeTPKpJaG+jk3xfkOQpwMIa3u/7qmpfVV3bXJeV3hOdBPshwIeSXEEn+T2peb8famJ6I/DfV7jXTmDvkrbHNL8M/CDwhiR333+gqq4EWnR6pS9d5X3cAtxvlXMkaUMdOegAJGmYVdU/JDke2LHqyct7C/ABOgnun1fVd7uc+y7gvcDLl7RPNvd/eFV9J8kccHRV/VOS04EnAq9IcllV/U4z1OFxdMYgnwc8dpUYb1+0nRXP+t7xa6rqkQcdSEaA/0gngT8OuGGZ1/87cPRyF66qf05yM53e8E8tOnQJ8Grg0cB9usR2dHN9Sdo09kxLUhdJHgQcAfwr8PfAzyY5ohm68eN0kr6PL2rfSWcoBgBNr+6X6PTgvmWV2/09ncmOS8cF3xO4pUmkHwOMN7HdD1ioqncArwJOb3p171lVlwK/RmdISi9Wek+fA3YkeWQTw1HNpEKa+10H/BzwlmZIyFLXAd+/3A2T3Be4PzC/5NCFwG9X1VWrxPxA4OpVzpGkDWXPtCQd7JhmCAN0emLPrao7msmDjwQ+AxTwm1X15ab9scC1wB7gH5Zcrw3sqKrrut20GRay3ITBNvCBJFfRGRf82ab9B+mM294HfAf4FeBY4P3NGOMAL1z72z7Asu+pqr7dTFJ8fZJ70vk58odJvktnaMcZVfWNJB+n8wvEy5Zc96/o9DB/eFHbR5PcARwFvKgZQnOnqrqBA6t8rORMDu7Vl6S+SuffbklSvzS1kD9dVRcMOpZBa8rcfRQ4s6ru2MDrPgx4YVU9c6OuKUlrYTItSX2UZDedesmPr6rbVzt/O0jyk8B1VbVnA6/5eODzSyZuSlLfmUxLkiRJPXICoiRJktQjk2lJkiSpRybTkiRJUo9MpiVJkqQemUxLkiRJPfr/VJRspPlt7HoAAAAASUVORK5CYII=",
"text/plain": [
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"plt.rcParams['figure.figsize'] = [12, 4]\n",
"\n",
"plt.scatter(diabetes_X_train, diabetes_y_train, color='black')\n",
"plt.xlabel('Body Mass Index (BMI)')\n",
"plt.ylabel('Diabetes Risk')"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"We see from the above figure that diabetes risk grows as the patient's BMI increases."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 2.1.2. A Supervised Learning Algorithm\n",
"\n",
"Next, suppose we wanted to predict the risk of a new patient given their BMI.\n",
"It will be useful to think about a supervised learning algorithm as having two components: a model class and an optimizer.\n",
"\n",
"### The Model Class\n",
"\n",
"Intuitively, the model class represents the set of possible relationships between BMI and risk that we believe to be true. Let's assume in this example that risk is a linear function of BMI. In other words, for some unknown $\\theta_0, \\theta_1 \\in \\mathbb{R}$, we have\n",
"\n",
"$$ \n",
"y = \\theta_1 \\cdot x + \\theta_0, \n",
"$$\n",
"\n",
"where $x$ is the BMI (also called the independent variable), and $y$ is the diabetes risk score (the dependent variable). \n",
"The parameters $\\theta_1, \\theta_0$ are the slope and the intercept of the line relates $x$ to $y$."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"We can visualize this for a few values of $\\theta_1, \\theta_0$."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAskAAAD4CAYAAAAejHvMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmPklEQVR4nO3de3Sc9X3n8c9XtmV8wzfZMtiWNbJlbIUGCFMIF4OxLQKU3ex2sw1pk6ZpWidt2abddJuQ3dP00O6eZLftlnOSpnUJ27KbkuxJwymnh5MwaiCk20KR06YpI2yMZLB8GVm+G2zr4u/+8cxIo9GMrLlIzzMz79c5Ph7NPNL8ZMajrz98nt9j7i4AAAAA4xrCXgAAAAAQNQzJAAAAQA6GZAAAACAHQzIAAACQgyEZAAAAyDE37AXk09TU5K2trWEvAwAAADVs7969g+6+Kt9jkRySW1tb1d3dHfYyAAAAUMPM7M1Cj1G3AAAAAHIwJAMAAAA5GJIBAACAHAzJAAAAQA6GZAAAACAHQzIAAACQgyEZAAAAyMGQDAAAgFnn7no9dU5ffv6Aug+eDHs5k0TyYiIAAACoPSOjl7X3zVNKJFNK9KT05ol3JEmf7tyseOuKkFc3EUMyAAAAZszbl0b0/deP67lkSs+/NqBT7wyrcU6Dbtu4Ur+4rU27tjZrzdKrwl7mJAzJAAAAqKiBsxf1N68NKJFM6W8PDGpo5LKWLpinHVtWa9fWZt193Sotnh/tMTTaqwMAAEDkubsODJzXc8mUEsmU/unQaUnSuuUL9DO3tqizo1k/3rpC8+ZUz+lwDMkAAAAoWqF+8bvXLdWnOzer813Nuq55icws5JWWhiEZAAAA01Kt/eJSMCQDAACgoFroF5ei9r4jAAAAlKwW+8WlYEgGAACoc7XeLy4FQzIAAEAdqqd+cSkYkgEAAOpEvfaLS8GfAgAAQI2iX1w6hmQAAIAaQr+4Mq44JJvZE5IelDTg7ten7/uGpOvShyyTdNrdb8zzuQclnZM0KmnE3eMVWTUAAADG0C+uvOkkyX8m6UuSnszc4e4fzNw2s9+XdGaKz7/H3QdLXSAAAAAmo188s674J+fuL5pZa77HLMjpf0rSjgqvCwAAAFnoF8+ucv95sU1Syt1fL/C4S3rOzFzSn7j7nkJfyMx2S9otSS0tLWUuCwAAoPrRLw5PuUPyhyQ9NcXjd7r7YTNbLSlhZq+5+4v5DkwP0HskKR6Pe5nrAgAAqEr0i6Oh5CHZzOZK+klJNxc6xt0Pp38fMLOnJd0iKe+QDAAAUK/oF0dPOX/auyS95u79+R40s0WSGtz9XPr2vZIeLeP5AAAAagL94uibzhZwT0naLqnJzPolfd7dvyrpIeVULczsWkmPu/sDkpolPZ3uyMyV9Bfu/u3KLh8AAKA60C+uLuYevfpvPB737u7usJcBAABQlqn6xZ0dzfSLQ2Zmewtdx4NyCwAAQAUNnL2orp4BdfVM7hd3djTrrs30i6sB/4UAAADK4O56feB8UKPI6hevX7FAH751g3Z1rKZfXIUYkgEAAIpUqF98A/3imsGQDAAAMA3sX1xfGJIBAAAKyPSLE8lj+n9vnKBfXEf4rwoAAJCW3S9+LpnSD3P6xcH+xcs1l35xzWNIBgAAdW1k9LK63zylrjz94t+4d7N2ddAvrkcMyQAAoO68fWlEL+4/rkTPxH7x7ZvoFyPAkAwAAOpCoX7xzi2rtYt+MXLwSgAAADWJfjHKwZAMAABqRqZfnEim1JWnX9zZsUabmxfTL8YVMSQDAICqNtYvTqb03X0DOp3VL959V9Avbr6afjGKw5AMAACqzsDZi0r0pNSVTI31i5ctnKcd1wX7F2+jX4wy8eoBAACR5+7anzqvrp6J/eKWFQv1kfdu0K6t9ItRWQzJAAAgkrL7xYlkSm+dTPeL1y+jX4wZx5AMAAAi4/ylEX0/t188t0F3bFypT9xNvxizhyEZAACEKnX2orp6grT47w6c0NDoxH7xXZtXaRH9YsyyK77izOwJSQ9KGnD369P3/bakX5R0PH3Y59z92Tyfe5+kxyTNkfS4u3+hQusGAABVKtMvTiSPKdEzMLFffFuwf3F8A/1ihGs6/yz7M0lfkvRkzv3/091/r9AnmdkcSV+W1CmpX9IrZvaMuydLXCsAAKhSI6OX9crBU2OJcXa/+D+97zrt2tpMvxiRcsUh2d1fNLPWEr72LZIOuHuvJJnZ1yW9XxJDMgAAdWCqfvEn796onVtX0y9GZJVT8HnYzH5WUrekT7v7qZzH10o6lPVxv6RbC30xM9stabcktbS0lLEsAAAQFvrFqBWlvkq/Iul3JHn699+X9PPlLMTd90jaI0nxeNzL+VoAAGB2TOgXJ1P6Yf8ZSfSLUf1KGpLdPZW5bWZ/Kumv8xx2WNL6rI/Xpe8DAABVLNMvTiRT6uqZ3C/u7GhW+2r6xahuJQ3JZnaNux9Nf/hvJf1LnsNekdRuZjEFw/FDkn66pFUCAIBQnb80ohf3H1cX/WLUielsAfeUpO2SmsysX9LnJW03sxsV1C0OSvpE+thrFWz19oC7j5jZw5K+o2ALuCfc/dWZ+CYAAEDlFewXb1mtzq30i1HbzD169d94PO7d3d1hLwMAgLoyVb+4s6OZfjFqjpntdfd4vsf45x8AAHWMfjGQH0MyAAB1hn4xcGUMyQAA1IGp+sX3djRrWzv9YiAbfxsAAKhB7q59qXPqSqYm9Is3rFyon71tg3bRLwamxJAMAECNyO4XJ3qO6dDJC5LoFwOlYEgGAKCKZfrFiWRK331tQGcujPeLf+nuTdq1dbVW0y8GisaQDABAlUmdvTi2G0V2v3jnVvrFQKXwNwgAgIijXwzMPoZkAAAiiH4xEC6GZAAAIoJ+MRAdDMkAAISIfjEQTfytAwBgFl2pX9zZ0ayb6RcDoWNIBgBgho2MXtY/HDypruTAhH7xjfSLgchiSAYAYAbQLwaqG0MyAAAVkukXJ5Ip/f0b9IuBasbfVAAASpTpFydeDU68Y/9ioHYwJAMAUAT6xUB9uOKQbGZPSHpQ0oC7X5++739I+leShiS9Ielj7n46z+celHRO0qikEXePV2zlAADMEvrFQP2ZTpL8Z5K+JOnJrPsSkh5x9xEz+6KkRyR9psDn3+Pug2WtEgCAWUa/GKhvV/zb7e4vmllrzn3PZX34kqQPVHhdAADMqiv1i9m/GKgvlfgn8M9L+kaBx1zSc2bmkv7E3fcU+iJmtlvSbklqaWmpwLIAAJga/WIAhZQ1JJvZf5Y0IulrBQ65090Pm9lqSQkze83dX8x3YHqA3iNJ8Xjcy1kXAACFnL80ou/tO66unon94js3NemXt2/Szi30iwGUMSSb2c8pOKFvp7vnHWrd/XD69wEze1rSLZLyDskAAMwU+sUAilXSO4KZ3SfpNyXd7e7vFDhmkaQGdz+Xvn2vpEdLXikAANOU3S9O9KT0z/SLARRpOlvAPSVpu6QmM+uX9HkFu1nMV1ChkKSX3P2TZnatpMfd/QFJzZKeTj8+V9JfuPu3Z+S7AADUvUy/OJEMTrzL7Rff29GsTfSLAUyTFWhKhCoej3t3d3fYywAARNxU/eLOjmb6xQCmZGZ7C13HgwIWAKCq0C8GMBt4FwEARBr9YgBhYEgGAEQO/WIAYWNIBgBEAvsXA4gShmQAQGiOnbmorp6J/eLlC+dp19ZmdXY0a1t7E/1iAKHgnQcAMGvoFwOoFgzJAIAZRb8YQDViSAYAVFymX5xIHtPz+45P7hdvXa3VS+gXA4guhmQAQEXQLwZQFHfpfEoa3C9dvVZauTHsFU3AuxUAoCT0iwFMy/BF6eQb0uDrwa8TrweD8eABaehccMzdn5XueSTcdeZgSAYATBv9YgB5ZafCE4bh16XTb0ny8WOvXic1bZJueEhq2hzcbr4+tKUXwpAMAJgS/WIAY6aTCkvSvIVBfWJdXLrhQ1JTe/Br5SapcVF46y8CQzIAYJJjZy4q0ZNSF/1ioP6Umwo3bZaWXCs1VHfVinc4AIDcXa8dO6eu5MR+cevKhfro7RvU2bFG72lZRr8YqCV1lAqXgiEZAOrU8OhlvZLuFyeSKfWfCvrFN7XQLwZqBqlwyRiSAaCOTNUv/pV76BcDVYtUuOIYkgGgxmX6xYlkSi/l6RfftblJCxv5cQBEHqnwrJrWu6KZPSHpQUkD7n59+r4Vkr4hqVXSQUk/5e6n8nzuRyX9l/SHv+vuf17+sgEAhWT6xZlt2ugXA1WGVDgSzN2vfJDZXZLOS3oya0j+75JOuvsXzOyzkpa7+2dyPm+FpG5JcQX/vNkr6eZ8w3S2eDzu3d3dpXw/AFCXpuoX79raTL8YiJpSUuGV7aTCFWZme909nu+xaSXJ7v6imbXm3P1+SdvTt/9c0guSPpNzzPskJdz9ZHohCUn3SXpqOs8LACiMfjFQBUiFq1Y5JbRmdz+avn1MUnOeY9ZKOpT1cX/6vknMbLek3ZLU0tJSxrIAoHbRLwYiyF06d2w8CaYrXBMq8k7q7m5mV+5tTP019kjaIwV1i0qsCwCqHf1iIELGUuF0EjxlKryJVLjKlTMkp8zsGnc/ambXSBrIc8xhjVcyJGmdgloGAKCA4dHLeqXv5Fhi3H/qgsykG9cv02/eF+xfvHEV/WJgRhRMhfdLpw8pbyp844fSfWFS4VpSzpD8jKSPSvpC+ve/ynPMdyT9NzNbnv74XkmPlPGcAFCTzl0c1ov7B5VIHtN3XxvQ2YsjapzboG30i4GZMXxBOvFGegA+EAzBmdt5U+Efl274aVLhOjLdLeCeUpAIN5lZv6TPKxiO/6+ZfVzSm5J+Kn1sXNIn3f0X3P2kmf2OpFfSX+rRzEl8AFDvCvWLOzvW0C8GKmFCKpxTkbhiKpz+RSpct6a1BdxsYws4ALUou1+cSKb0o8Pj/eLOjmZ1dqzRzRuWa04DNQqgKMWmwk3prdQyt0mF61bZW8ABAEqT6Rc/lz7xjn4xUKJ8qXBmGM6bCreTCqMsDMkAUGH0i4EyTEiFs0+cm6IrfOPPkAqj4hiSAaAC6BcDRcikwtm1CFJhRAzv2ABQgqn6xZn9i+kXo+4VnQrfkpUKbw6uQEcqXJN8dFTDhw9rqK9Pl3r7tOCGG7TwPTeFvawJGJIBYJroFwN5lJoKZ584Rypcs0bPng0G4b4+DfUd1FBvr4YO9mno4Jvy4eGx45p++ZcZkgGgmpy7OKzv7T+uRDKl59P94vlzG3TnpiY9fM8m7aBfjHpBKowCclPhob6+scF4dHBw/MA5c9S4fr0a29q06K67ND8WU2NbmxpjMc1dvrzwE4SEIRkAchw9c0FdPQNKJFP6+zcGNTzqWrGoUfe+a412baVfjBpWTCq8dH0wAE9IhTdLV18r8X9TatJ0U+E5S5eqsa1Ni++6S/PbYmqMxdQYa1Pj+nWyefNC/A6Kw7s8gLo3Vb/4525vpV+M2jPtVHhRkABnUuGm9uDkOVLhmjXtVHju3CAVjsWCVDidCEc1FS4FQzKAulSoX3wT/WLUipJS4Z8e30aNVLimTTsVXrZMjbGYFt+drkdUaSpcCoZkAHWDfjFqUjGpcNMmUuE6QipcHoZkADXt6JkL6kqm9FwypZd6T0zoF3d2NGtbO/1iVAFSYUyhqFS4rW08FW5rU2NrrC5S4VLwkwFATXF39Rw9p66eif3iWNMifeyOmHZtbaZfjOgiFUYBpaTCi+++eywRrvdUuBQMyQCq3vDoZf1D30kl8vSLP3PfFnV2rKZfjOgoJxXODMOkwjWLVDg6GJIBVKVC/eJt7UG/eOfWZq1aMj/sZaKelZQKfzi4TSpc04pOhdvaslLhNjXGWkmFZwFDMoCqQb8YkVNsKtzUTipcRyakwplhOF8qvHz5+A4SmZPmSIVDx08TAJGV6Rcnkikleo7pXw6flTTeL+7saNZ7WugXYxaU3RXeJDUuDG/9mDGZVPhSb29Qj+jr01Bvry4dPEgqXOUYkgFESna/OJFM6fDp3H5xszauWkS/GJVHKowplJUKx2JqXEcqXG1KHpLN7DpJ38i6q03Sb7n7H2Yds13SX0nqS9/1LXd/tNTnBFCbzl0c1gv7jqurZ3K/+Fd3btKOLfSLUUGZVHhwv3TiQDoZ3h/cN62uMKlwrSIVRraSh2R33yfpRkkyszmSDkt6Os+h33f3B0t9HgC16cjpC/qbnsn94ve9a4120S9GuSalwlkViUKp8PpbSYXrRL5U+FJfr4bffItUGGMq9RNop6Q33P3NCn09ADXG3ZU8elZdyQH6xaicgqnwAWno/PhxpMJ1x0dGglQ4eyu1vr4pU+El27eTCmNMpYbkhyQ9VeCx28zsh5KOSPoNd38130FmtlvSbklqaWmp0LIAhKlQv/g9LcvH+sWbVi8Oe5mIOnfp3NGsk+Wyfp0plAr/DKlwnRhLhSdspUYqjPKZu1/5qKm+gFmjggH4Xe6eynnsakmX3f28mT0g6TF3b7/S14zH497d3V3WugCEI9MvTiRTen7fgM5l9Ys7O5rpF6OwSanw/vRgXCAVXtkeXG6ZfYVrXsFUuK9PoydOjB+YlQrPj7WSCuOKzGyvu8fzPVaJJPl+ST/IHZAlyd3PZt1+1sz+yMya3H0w91gA1evI6Qtjl4HO7hffl96/+E76xcjIToUnVCQKpMIrN41vpUYqXPMmp8K9utTXNzkVzlxtbvvd7CuMGVOJn1ofUoGqhZmtkZRydzezWyQ1SDqR71gA1SPTL85cBjrTL25rWqSfT/eLb6JfXN+GL4wPwNNJhdffIt3E1ebqwYRUOPdqcwVSYbrCCENZQ7KZLZLUKekTWfd9UpLc/Y8lfUDSL5nZiKQLkh7ycvsdAEIxPHpZL/eeHEuMs/vFn71/i3ZtpV9cd0iFMYXRM2fSw+9BUmFUpbI7yTOBTjIQDWcvDut79ItRTCq8cmO6J0xXuB6UkgrTFUaUzHQnGUANydcvXpnVL97WvkoLGueEvUxUWrGpMDtI1JVpp8KZHSRIhVEDGJKBOke/uM6U3RVmX+FaRVcYmIghGahD9ItrHKkwplBWKsy+wqgjDMlAnSjcL16lX925iX5xNSonFW7aLK3YSCpco0pPhTPDMKkwwJAM1LBC/eL7r1+jXVvpF1eFolLhlvQwTCpcL6a9r3AmFb5nu+ZnEmFSYWBKDMlADcnuFyeSKb16hH5x1Sj2anMtt0orSYXrQVFXm2tpUWMsRioMVABDMlDlMv3iRPKYunoGJvWLOzuatXEV/eJIKDcVbtosLbmGVLhGkQoD0cKQDFShsxeH9cK+4+rK6hdfNa9Bd25apU/tbNc9W1bTLw5TyalwehgmFa5ZpMJA9WBIBqoE/eKIKSUVbvlwsIUaqXDNKz0VDrZSIxUGwseQDEQU/eKIKGoHiXZS4TpS1A4SmVT4nnuCi2uQCgORx5AMRAj94pBMOxW29L7CpML1pKh9hdva6AoDNYIhGQgZ/eJZNPSOdPKN8QH4xOukwpBEKgxgMoZkIAT0i2dQ3lR4vzR4oEAq3C61vDedCm8OPiYVrlmFUuGhN9+SpkyF6QoD9YYhGZgFBfvFq+gXl2y6qXDj4mAAbrlVavrIeEWCVLhmjaXCvb3BDhJ9fbrUF9y+YircFtP8WExzli0Lbf0AooEhGZgh9IsrYCwV3p9z4twB6cxbWQeSCtejCalwb6+GDvblT4VXrMjaQWJ8KzVSYQBTYUgGKijTL04kU3ohT794x9bValpMv3iSvKnw/mCv4byp8HuzUuHN0sqN0rwF4a0fM2baqfC8eWpcv16NbaTCACqDIRko0+HTF9SVTKmrZ3K/uLNjje7c1ES/WJqcCo8Nw1OlwrcFv69sJxWucXlT4d4+Db115VR4fltM89atk83lRxqAyin7HcXMDko6J2lU0oi7x3MeN0mPSXpA0juSfs7df1Du8wJhcXe9euTs2Il3E/rFd8bUubXO+8VjqXC6FkEqjLTJqXDv2JXnJqXCLS1qjLVqyY57xk6aIxUGMJsq9c/ue9x9sMBj90tqT/+6VdJX0r8DVYN+cQ536eyR8ZPlpkqFl60PkuAJqfBmackaUuEaNZYKZ7ZSu0IqvGTH+FZqpMIAomI23oXeL+lJd3dJL5nZMjO7xt2PzsJzAyWjX6zSU+HMMEwqXLN8ZETD/f1jSTCpMIBaU4kh2SU9Z2Yu6U/cfU/O42slHcr6uD9934Qh2cx2S9otSS0tLRVYFlC8uuwXF0yFM1ebyyAVrkfFpsJjXeH0SXOkwgCqVSXeue5098NmtlpSwsxec/cXi/0i6eF6jyTF43G/wuFARdRVv3jonWALtUwtYnB/cLtgKnyb1PSzpMJ1YEIq3Ns3vpVab59GT54cP5BUGEAdKXtIdvfD6d8HzOxpSbdIyh6SD0tan/XxuvR9QCgK9YtvroV+cUmp8O1S0yZS4ToweuZM3q3UCnaFd+6YsJUaqTCAelLWu52ZLZLU4O7n0rfvlfRozmHPSHrYzL6u4IS9M/SRMdtqrl9cKBUePCANvz1+HKlw3SEVBoDKKDcSaJb0dLDLm+ZK+gt3/7aZfVKS3P2PJT2rYPu3Awq2gPtYmc8JTEumX5xIBv3ikctV1i/OlwoP7g+G40Kp8HtIhevF6OnTeU+aIxUGgMqwYNOJaInH497d3R32MlBlMv3iRPrEu+x+cWdHs+7taNaN6yPYLy42Fc5cbplUuOblTYXTJ9AVSoXnx2KkwgAwTWa2N/caHxnECKhqQyOX9XLfifSOFBP7xY/cv0W7otIvzqTCmSR4WqkwO0jUi5JTYfYVBoAZw7sqqk6k+8UTUuGci2xM1RXmanM1bywVzt1KbYpUmK4wAISHIRlVIVL9YlJhTCFvKtzbp6FDh0iFAaCK8E6MSJqqX/zxbbHZ6RfnS4UzV5sjFa5ruanw2FZqhVLhtvQwTCoMAFWDIRmREUq/uJRUeMPtpMJ1glQYAOoX794I1az1i0mFUQCpMAAgH4ZkzLoZ6xeXlArfwb7CdaLkVJh9hQGgLvGOjxlX8X5xMalwUztXm6sjpaXCO9UYi5EKAwAmYEjGjMj0ixPJlLqSKR05c7G4fjGpMKZQVCrcRioMACgePyVQMWcuDOuFfQPq6hnQC68N6NyloF+8rX2Vfq1zs3ZsydMvLmZfYVLhulL0vsKkwgCACmJIRlkK9ot/LKtfPK8hSIVTfyclp7uv8O1BKty0OfiYVLhmFXW1OVJhAMAs4ScLipLdL04kU0oeHe8Xf+L2a/QT176tLfNSajixV0q+Ln0/Xyq8JBiAW26Tmj46XpEgFa5ZxabC89tiWrJjx9hWao2xmOYsXRreNwAAqDsMybiiCf3iV4/p8tkj2tRwRD+58oy+uPGkNjYc0cKzfVJ3nlS4abPUQipcL0pKhbO2UiMVBgBEBT+NkNfZc2e0d+8rOrjvh3rnSI/WXT6sDzYc1ecajuqqqy4GB52TdClPKty0WVrRRipco0iFAQD1gCG5nuXsIHH+SI/OHnpVjad71TQ6oHvSh10206Wr16qx+TrNWXVfcNJc5sQ5UuGaNe1UeOVKNcZaJ6bCbW2at3YtqTAAoGrxE6weFNhX2E+8IcvqCrsv0HG/RgONHWpc92+0btO71XrdTZrTtFELSIVrEqkwAAD5MSTXikn7Cu9Pb6c2cQcJl+ni4nXqt7X6weXt+ufhZvX6NVp87VbFr9+qznet0Q1T7V+MqjTtfYVJhQEAkFTGkGxm6yU9KalZkkva4+6P5RyzXdJfSepL3/Utd3+01OeExlPhSRfZyL3aXLorvOF2XVzaph9dXKXEwFJ96+B8DQ42jO1f3NnRrP+4ZbVW5u5fjKrjIyMaOnRo7ApzV7ra3PyNbWP7CpMKAwAwUTnR0IikT7v7D8xsiaS9ZpZw92TOcd939wfLeJ764y6dPTyeBGdS4cHXpbP9WQeatKwl6AdvuGNCV/jw6NL0ZaAH9FJ3sH9x0+JG7fyxZnV2NOuOTU1a0DgntG8RpRtLhXvT9QhSYQAAKq7kn5TuflTS0fTtc2bWI2mtpNwhGYVMSoUzFYkCqXDrHelLLk/eQWLC/sXPHBjbv3jjqkX6+LaY7u1o1o3rl2tOAyfZVYOCqXBvr0ZPnRo/kFQYAIAZUZE4ycxaJd0k6eU8D99mZj+UdETSb7j7qwW+xm5JuyWppaWlEsuKhuxUeOySy8WlwoV2kBgauayXXz8eJMbJlI6cuSgz6eaW5Xrk/i3q7GhWG/3iSJuUCmdOoCuUCu/aSSoMAMAsMHcv7wuYLZb0PUn/1d2/lfPY1ZIuu/t5M3tA0mPu3n6lrxmPx727u7usdc26obfHO8K5J84NvzN+XCYVzlxYo8h9hc9cGNYL+waUSKb0vX3Hde7SyIR+8U76xZFTdCrcFgsuu0wqDADAjDKzve4ez/dYWRGUmc2T9JeSvpY7IEuSu5/Nuv2smf2RmTW5+2A5zxuavKnw/uCyy2WmwlPpP/WOujL94t7xfvEDP3aNOjuadWd7k66aR784bKOnT+fdSo1UGACA6lPO7hYm6auSetz9Dwocs0ZSyt3dzG6R1CDpRKnPOWuyU+HsikShVHiKrnApMv3i59I1iux+8S9sa1NnR7NuXL+MfnEIJqbCWRfYoCsMAEBNKSe2ukPSRyT9yMz+KX3f5yS1SJK7/7GkD0j6JTMbkXRB0kNebr9jprz4e9LB7894KlzI0Mhlvdx3YlK/OL6BfnEYSIUBAKhv5exu8beSppwQ3f1Lkr5U6nPMquP7pItnKp4KT2WqfvGvdW6mXzzDfHhYQ/39ky+w0ddHKgwAQJ0j6sr4d386K0+T6RcnelJ6ufck/eJZMHLq1NhJc8EwnL791lvSyMjYcaTCAAAgg5/8M4x+8ewYT4X70jtITDMVbmvT/FgrqTAAAJiAIXkGDI1c1ku9J9TVM94vbjDp5g3L9bkHtmjXVvrFpSo+Fd6lxliMVBgAABSFaaFCCvWL76JfXLSiUuENpMIAAKDyGJLLQL+4POOpcG96GA62Uhs6dIhUGAAAhIoJowjZ/eJEMqWePP3im9YvUwP94jE+PKyhQ/0aOphOhXt7x+oSeVPhTRuDYZhUGAAAhIgh+Qoy/eJEMqWunpSO0i/Oq+RUuC2m+bEYqTAAAIgUppI8puoX/3od94snpMK9OVebO3167DibN0/zMqlwZ2ewrzCpMAAAqCIMyWn5+8Xz9RPvvka7ttZXv3hSKpx9tbmcVHh+LDY2CJMKAwCAWmFRvEp0PB737u7uWX3OOx//lE4M9WlB4xwtX9io5QsbteSq2h30GkYva9ngJa04fkErUhe1YuCCVgwEvy98e3wQHpljOr3qKp1cfZVOrF6gk6sX6OTqq3Ry9QJdWli7fz4AAGD2bFmxRZ+55TOz/rxmttfd4/keY8pJuzW2QkfeOVVzafFVbw9rxcBFrUxdmDAILxu8pDmXx/+B9PbieTrZfJVef/eK9DAcDMJnVsyXz+FERAAAUF8YktN+f+fnw15CyYruCt8QS192Oab5bbGgK3z11eF9AwAAABHDkFxFSuoKt7UF+wrTFQYAAJg2JqaIKWsHCVJhAACAimBIDgmpMAAAQHQxZc0gUmEAAIDqxJBcAaTCAAAAtaWsyczM7pP0mKQ5kh539y/kPD5f0pOSbpZ0QtIH3f1gOc8ZFlJhAACA+lHykGxmcyR9WVKnpH5Jr5jZM+6ezDrs45JOufsmM3tI0hclfbCcBc80UmEAAACUM83dIumAu/dKkpl9XdL7JWUPye+X9Nvp29+U9CUzM4/gZf6OPPI5nX/+eVJhAAAAlDUkr5V0KOvjfkm3FjrG3UfM7IyklZIGc7+Yme2WtFuSWlpaylhWaeatXUsqDAAAAEkROnHP3fdI2iNJ8Xh81pPmVQ//ymw/JQAAACKqoYzPPSxpfdbH69L35T3GzOZKWqrgBD4AAAAgssoZkl+R1G5mMTNrlPSQpGdyjnlG0kfTtz8g6btR7CMDAAAA2UquW6Q7xg9L+o6CLeCecPdXzexRSd3u/oykr0r632Z2QNJJBYM0AAAAEGlldZLd/VlJz+bc91tZty9K+vflPAcAAAAw28qpWwAAAAA1iSEZAAAAyMGQDAAAAORgSAYAAAByWBR3ZDOz45LeDOGpm5TnaoCAeG1garw+UAivDRTCayMaNrj7qnwPRHJIDouZdbt7POx1IHp4bWAqvD5QCK8NFMJrI/qoWwAAAAA5GJIBAACAHAzJE+0JewGILF4bmAqvDxTCawOF8NqIODrJAAAAQA6SZAAAACAHQzIAAACQgyE5zczuM7N9ZnbAzD4b9noQDWa23syeN7Okmb1qZp8Ke02IFjObY2b/aGZ/HfZaEB1mtszMvmlmr5lZj5ndFvaaEB1m9uvpnyn/YmZPmdlVYa8JkzEkK/ghJ+nLku6X1CHpQ2bWEe6qEBEjkj7t7h2S3ivpV3htIMenJPWEvQhEzmOSvu3uWyTdIF4jSDOztZJ+VVLc3a+XNEfSQ+GuCvkwJAdukXTA3XvdfUjS1yW9P+Q1IQLc/ai7/yB9+5yCH3Rrw10VosLM1kn6CUmPh70WRIeZLZV0l6SvSpK7D7n76VAXhaiZK2mBmc2VtFDSkZDXgzwYkgNrJR3K+rhfDELIYWatkm6S9HLIS0F0/KGk35R0OeR1IFpiko5L+l/pKs7jZrYo7EUhGtz9sKTfk/SWpKOSzrj7c+GuCvkwJAPTYGaLJf2lpF9z97NhrwfhM7MHJQ24+96w14LImSvpPZK+4u43SXpbEue6QJJkZssV/N/qmKRrJS0ysw+Huyrkw5AcOCxpfdbH69L3ATKzeQoG5K+5+7fCXg8i4w5J/9rMDiqoaO0ws/8T7pIQEf2S+t0983+dvqlgaAYkaZekPnc/7u7Dkr4l6faQ14Q8GJIDr0hqN7OYmTUqKNA/E/KaEAFmZgp6hT3u/gdhrwfR4e6PuPs6d29V8J7xXXcnDYLc/ZikQ2Z2XfqunZKSIS4J0fKWpPea2cL0z5id4sTOSJob9gKiwN1HzOxhSd9RcJbpE+7+asjLQjTcIekjkn5kZv+Uvu9z7v5seEsCUAX+g6SvpYOXXkkfC3k9iAh3f9nMvinpBwp2UPpHcYnqSOKy1AAAAEAO6hYAAABADoZkAAAAIAdDMgAAAJCDIRkAAADIwZAMAAAA5GBIBgAAAHIwJAMAAAA5/j86K0SLkVq6pwAAAABJRU5ErkJggg==",
"text/plain": [
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"theta_list = [(1, 2), (2,1), (1,0), (0,1)]\n",
"for theta0, theta1 in theta_list:\n",
" x = np.arange(10)\n",
" y = theta1 * x + theta0\n",
" plt.plot(x,y)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Our supervised learning algorithm will attempt to choose the linear relationship fits the training data well."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### The Optimizer\n",
"\n",
"Given our assumption that $x,y$ follow a linear relationship, the goal of a supervised learning algorithm is to find a good set of parameters consistent with the data.\n",
"\n",
"This is an optimization problem---we want to maximize the fit between the model and the data over the space of all possible models. The component of a supervised learning algorithm that performs this search procedure is called the optimizer."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"We will soon dive deeper into optimization algorithms for machine learning, but for now, let's call the `sklearn.linear_model` library to find a $\\theta_1, \\theta_0$ that fit the data well."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Slope (theta1): \t 37.378842160517664\n",
"Intercept (theta0): \t -797.0817390342369\n"
]
}
],
"source": [
"from sklearn import linear_model\n",
"from sklearn.metrics import mean_squared_error\n",
"\n",
"# Create linear regression object\n",
"regr = linear_model.LinearRegression()\n",
"\n",
"# Train the model using the training sets\n",
"regr.fit(diabetes_X_train, diabetes_y_train.values)\n",
"\n",
"# Make predictions on the training set\n",
"diabetes_y_train_pred = regr.predict(diabetes_X_train)\n",
"\n",
"# The coefficients\n",
"print('Slope (theta1): \\t', regr.coef_[0])\n",
"print('Intercept (theta0): \\t', regr.intercept_)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Here, we used `scikit-learn` to find the best slope and intercept to the above dataset."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 2.1.3. A Supervised Learning Model\n",
"\n",
"The supervised learning algorithm gave us a pair of parameters $\\theta_1^*, \\theta_0^*$. These define the *predictive model* $f$, defined as\n",
"\n",
"$$ \n",
"f(x) = \\theta_1^* \\cdot x + \\theta_0^*, \n",
"$$\n",
"\n",
"where again $x$ is the BMI, and $y$ is the diabetes risk score."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "k59_taGfnn6K",
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"We can visualize the linear model that best fits our data."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"[]"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtMAAAEGCAYAAACn7xkwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAst0lEQVR4nO3deXyU5bn/8e9FCBJBCGq0JC5QpXiwKLFoFaqgHprTagtFa7HWgx57bN1Kq1BF29pFhQq0aq11F6WutIAL/g4WEK2tVdlkEVG0QQlRQI2gRpZw/f6YJziTZTbyzJL5vF+vvJi557nnuYbJJF9u7ue+zd0FAAAAIHUdsl0AAAAAkK8I0wAAAECaCNMAAABAmgjTAAAAQJoI0wAAAECaOma7gN2x7777eq9evbJdBgAAANq5RYsWbXL3sqbteR2me/XqpYULF2a7DAAAALRzZra2pXameQAAAABpIkwDAAAAaSJMAwAAAGkiTAMAAABpIkwDAAAAacrr1TwAAADQ/s1aUqNJc1ZrfV29yktLNK6qr0ZUVmS7LEmEaQAAAOSwWUtqNH7GctVvb5Ak1dTVa/yM5ZKUE4GaaR4AAADIWZPmrN4VpBvVb2/QpDmrs1RRLMI0AAAActb6uvqU2jONMA0AAICcVV5aklJ7phGmAQAAkLPGVfVVSXFRTFtJcZHGVfXNUkWxuAARAAAAOavxIkNW8wAAAADSMKKyImfCc1OhTfMws85m9qKZvWxmK83sV0F7bzN7wczWmNnDZtYpaN8juL8meLxXWLUBAAAAbSHMOdNbJZ3k7kdKGiDpv8zsWEm/lfR7dz9U0geSzguOP0/SB0H774PjAAAAgJwVWpj2iI+Cu8XBl0s6SdJfgvZ7JY0Ibg8P7it4/GQzs7DqAwAAAHZXqKt5mFmRmS2VtEHS3yS9IanO3XcEh6yT1DgBpkLS25IUPP6hpH1aeM7zzWyhmS3cuHFjmOUDAAAAcYUapt29wd0HSDpA0jGSDmuD57zd3Qe6+8CysrLdfToAAAAgbRlZZ9rd6yQ9Lek4SaVm1riKyAGSaoLbNZIOlKTg8e6S3stEfQAAAEA6wlzNo8zMSoPbJZKGSVqlSKg+PThstKRHg9uPBfcVPD7f3T2s+gAAAIDdFeY60z0l3WtmRYqE9kfc/Qkze0XSQ2Z2jaQlku4Kjr9L0jQzWyPpfUmjQqwNAAAA2G2hhWl3XyapsoX2NxWZP920/VNJ3w6rHgAAAKCtZWTONAAAANAeEaYBAACANBGmAQAAgDQRpgEAAIA0EaYBAACANBGmAQAAgDQRpgEAAIA0EaYBAACANBGmAQAAgDQRpgEAAIA0EaYBAACANBGmAQAAgDQRpgEAAIA0EaYBAACANBGmAQAAgDQRpgEAAIA0EaYBAACANBGmAQAAgDQRpgEAAIA0EaYBAACANBGmAQAAgDQRpgEAAIA0EaYBAACANBGmAQAAgDQRpgEAAIA0EaYBAACANBGmAQAAgDQRpgEAAIA0hRamzexAM3vazF4xs5VmNiZo/6WZ1ZjZ0uDr61F9xpvZGjNbbWZVYdUGAAAAtIWOIT73DkmXuftiM9tL0iIz+1vw2O/dfXL0wWbWT9IoSYdLKpc018y+4O4NIdYIAAAApC20kWl3r3X3xcHtLZJWSaqI02W4pIfcfau7/1vSGknHhFUfAAAAsLsyMmfazHpJqpT0QtB0sZktM7O7zaxH0FYh6e2obusUP3wDAAAAWRV6mDazrpL+KunH7r5Z0p8kHSJpgKRaSVNSfL7zzWyhmS3cuHFjW5cLAAAAJC3UMG1mxYoE6fvdfYYkufu77t7g7jsl3aHPpnLUSDowqvsBQVsMd7/d3Qe6+8CysrIwywcAAADiCnM1D5N0l6RV7v67qPaeUYd9S9KK4PZjkkaZ2R5m1ltSH0kvhlUfAAAAsLvCXM1jsKSzJS03s6VB25WSzjSzAZJcUrWkH0iSu680s0ckvaLISiAXsZIHAAAAclloYdrdn5NkLTz0ZJw+10q6NqyaAAAAgLbEDogAAABAmgjTAAAAQJoI0wAAAECaCNMAAADIWe6uBQsW6NRTT9WVV16p7du3Z7ukGGGu5gEAAIA8MGtJjSbNWa31dfUqLy3RuKq+GlGZnY2oN2/erDvvvFOTJ09WbW1tzGOzZ8/WiSeeqGHDhmWltpYQpgEAAArYrCU1Gj9jueq3R1Ykrqmr1/gZyyUpI4H65Zdf1pQpUzRt2rSEx37ta1/T0KFDQ68pFUzzAAAAKGCT5qzeFaQb1W9v0KQ5q9v8XNu2bdO9996rI444QmYmM9OAAQNaDdJDhgzR448/roaGBrm7nnzySRUXF7d5XbuDkWkAAIACtr6uPqX2VFRXV+uGG27QjTfemNTxY8aM0ZgxY9S7d+/dPnemEKYBAAAKWHlpiWpaCM7lpSUpPc/OnTs1e/ZsTZkyRc8880zC4/v376+xY8dq1KhR6tSpU0rnyiWEaQAAgAI2rqpvzJxpSSopLtK4qr5x+23atEm33HKLJk+erC1btiQ8z9lnn63LLrtMRx555G7XnEsI0wAAAAWs8SLDRKt5/POf/9TkyZM1c+bMhM9ZUVGhyy67TOedd566desWSt25gjANAABQ4EZUVsSE548//lg333yzpkyZourq6oT9TznlFI0dO1ZDhgyRmYVYae4hTAMAABS4Z599Vqeffro2btyY8NiSkhJddtlluuiii/S5z30uA9XlNsI0AABAAdm6dat+9atfacKECUkdf9xxx2ns2LEaPny4ioqKQq4u/xCmAQDIc7m0ex1yz6pVq/Td735XS5cuTer4008/Xdddd5369OkTbmHtRMIwbWbm7t6kbQ933xpeWQAAIBnZ3r0OucXddeutt+rCCy9Mus8FF1yg66+/Xl27dg2xsvYrmR0Q74q+Y2ZdJT0ZTjkAACAVmdy9DrmntrZWp5566q7dBDt06BA3SBcVFWnWrFly911ft9xyC0F6NyQTpteZ2S2SZGY9JD0l6c+hVgUAAJIS5u51yD2PPfaYOnbsuCs8l5eXa/bs2a0e/41vfEPr16/fFZx37Nih4cOHZ7Di9i9hmHb3X0j6yMxuVSRIT3H3e0KvDAAAJNTaLnWp7l6H3LNlyxb98Ic/3BWczUzDhw9XQ0NDq31uueUWNTQ07ArPjz32mHr27JnBqgtPq2HazEY2fkl6QdKxkpZI8qANAABk2biqviopjl1hIZnd65B7XnjhBR188MG7gnO3bt102223tXr8gAEDtHLlypgpGxdccIE6dEhm4gHaSrwLEL/R5P4SScVBu0uaEVZRAAAgOcnuXofcsmPHDv32t7/Vz372s6T7XHXVVfr5z3+uPfbYI8TKkKpWw7S7n5vJQgAAQHqa7l6H3PPGG2/o7LPP1vPPP5/U8WVlZZo+fbqGDBkScmXYXQn/H8DMrjezbmZWbGbzzGyjmX0vE8UBAADkG3fX1KlTY+Y6H3rooXGD9OjRo/XBBx/smq6xYcMGgnSeSGZSzVfdfbOkUyVVSzpU0rgwiwIAAMgXmzZt0re//e2Y5enOPTf+f/A/9NBDMXOdp06dqtLS0swUjDaVzA6IjcecImm6u39oZiGWBAAAkLueeuopnXbaafroo4+SOv6kk07S3XffrYMPPjjkypANyYxMP2Fmr0r6kqR5ZlYm6dNwywIAAMi++vp6XXrppTFTNqqqquIG6SlTpmjHjh27Rp3nzZtHkG7HEo5Mu/sVZna9pA/dvcHMPpbEat8AAKDdWbp0qUaNGqXVq5PbQbJv37568MEHVVlZGXJlyFWthmkzO8nd50evKd1kegdL4wEAgLzV0NCgG2+8UZdddlnSfS699FJdc801KilhUxxExBuZHiJpvpqvNy2xzjQAAMgzb731ls4991zNnz8/qeO7du2qv/zlL6qqqgq5MuSzeOtMXx38mdZ602Z2oKT7JO2vSPi+3d1vNLO9JT0sqZciq4Oc4e4fWGTY+0ZJX5f0iaRz3H1xOucGAAB4+OGHNWrUqKSPP/3003XLLbeorKwsxKrQ3sS9ANHMisxs36j7nczsfDNblcRz75B0mbv3U2Qr8ovMrJ+kKyTNc/c+kuYF9yXpa5L6BF/nS/pTyq8GAAAUpLq6Op1zzjkxFwomCtL33HOPdu7cuetCwenTpxOkkbJWw7SZjZL0vqRlZvaMmX1V0puKhN6zEj2xu9c2jiy7+xZJqyRVKHLx4r3BYfdKGhHcHi7pPo/4l6RSM+uZ1qsCAADt2m233RYTnHv06KF777231eOPO+44vf766zFrOzeGb2B3xJsz/TNJX3L3NWZ2lKTnJZ3u7o+nehIz6yWpUtILkvZ399rgoXcUmQYiRYL221Hd1gVttVFtMrPzFRm51kEHHZRqKQAAIM988sknOuKII/TGG28k3eeaa67RT3/6UxUXF4dYGRB/msc2d18jScEI8+tpBumukv4q6cfBToq7uLsrMp86ae5+u7sPdPeB/FcMAADtz7x582JGnbt06RI3SB900EF6/vnnY0adr7rqKoI0MiLeyPR+ZnZp1P3S6Pvu/rtET25mxYoE6fvdvXH1j3fNrKe71wbTODYE7TWSDozqfkDQBgAA2qmdO3fqtNNO06xZs5LuY2Z69913md+MnBBvZPoOSXtFfTW9H1ewOsddklY1Cd6PSRod3B4t6dGo9v+2iGMV2SQmZooHAADIb6+++mrMqHNRUVHCIH3llVfGjDrv3LmTII2cEW9pvF/t5nMPlnS2pOVmtjRou1LSREmPmNl5ktZKOiN47ElFlsVbo8jSeGktyQcAAHLHL37xC/3mN79Jqc/KlSvVr1+/kCoC2lbC7cTT5e7PSWrtEtmTWzjeJV0UVj0AACBcmzZtUs+ePbVjx46k+3zzm9/UjBkzVFRUFGJlQHjirjMNAADQmgceeCBmykZZWVnCID1nzpyYKRuPPvooQRp5LbSRaQAA0H5s3bpVgwYN0uLFyW9OfPDBB2vFihXq2rVriJUB2ZVwZNrMxphZt+DCwLvMbHGwgQsAAGinnn/++ZhR586dOycM0n/84x9jRp2rq6sJ0mj3kpnm8T/B+tBfldRDkYsKJ4ZaFQAAyJjo3QAbvwYNGpSw39tvvx0Tni+88MIMVAvklmSmeTReRPh1SdPcfaWx92bBmrWkRpPmrNb6unqVl5ZoXFVfjaisyHZZAIAUVFdXq3fv3in1ufjii3XTTTex/TbQRDIj04vM7ClFwvQcM9tL0s5wy0IumrWkRuNnLFdNXb1cUk1dvcbPWK5ZS9hbBwBy2eTJk2NGnZMJ0gsXLowZdf7DH/5AkAZakMzI9HmSBkh6090/MbN9xBrQBWnSnNWq394Q01a/vUGT5qxmdBoAcsTmzZt1yCGHaNOmTUn3OeGEEzR37ly23wbSkMzItEvqJ+lHwf0ukjqHVhFy1vq6+pTaAQDha5x60fjVvXv3hEF6xowZMaPOzzzzDEEaSFMyI9O3KDKt4yRJv5a0RdJfJR0dYl3IQeWlJappITiXl5ZkoRoAhYJrNT6zfft2derUKaU+3bt3V3V1tUpLS8MpCihwyYxMf9ndL5L0qSS5+weSUvsko10YV9VXJcWxC+uXFBdpXFXfLFUEoL0r9Gs15s6dGzPqnEyQnjhxYsyoc11dHUEaCFEyI9PbzaxIkekeMrMycQFiQWocCWKECECmFNq1Gsccc4xeeumllPosXrxYlZWVIVUEIJFkwvRNkmZK2s/MrpV0uqSfh1oVctaIyop2+QsMQG5qz9dqrFmzRn369EmpT3l5udatW8eqGkAOSTjNw93vl/RTSRMk1Uoa4e6PhF0YAACtXZORj9dqXHrppTFTNpIJ0o888kjMlI2amhqCtCLTfwZPnK/eV8zW4InzC2baD3JTwpFpM5vm7mdLerWFNgAAQjOuqq/Gz1geM9UjH67V2Lx5s7p3755yv48++khdunQJoaL2o3EefeP3ROM8ekn8zymyIpkLEA+PvhPMn/5SOOUAAPCZEZUVmjCyvypKS2SSKkpLNGFk/5wLTffdd1+z5ekSueqqq2JGnd2dIJ2EePPogWxodWTazMZLulJSiZlt1mfbim+TdHsGagMAIOeu1WhoaFC3bt30ySefpNRv7dq1Ouigg0KqqnC053n0yE+tjky7+wR330vSJHfv5u57BV/7uPv4DNYIAEDWvPDCCzGjzh07dkwYpE8++eRmo84E6bbRnubRo31IZprHVWb2PTP7uSSZ2YFmdkzIdQEAkBWnnHJKTHg+9thjE/b5+9//HhOc586dm4FKCxN7HiDXJLM03h/12Q6Iv5H0UdDGDogAgLxWU1OjAw44IKU+RUVF+vTTT9WxYzK/QtHW2PMAuSaZnwRfdvejzGyJFNkB0czYAREAkHd+/etf6+qrr06pzx133KHvf//7IVWEdOTaPHoUNnZABAC0S/X19dpzzz1T7vfBBx+w/TaApCUzZ7pxB8T9gx0Qn5N0XahVAQCQolmzZsXMdU4mSF9wwQXNLhQkSANIRcKRaXe/38wWSTo5aBrh7qvCLQsAgNa5u3r37q21a9em1G/VqlU67LDDQqoKQCFKZmRakvaUVBQcz9ozAICMev7552NGnTt06JAwSB9xxBHauXNnzKgzQRpAW0sYps3sF5LulbS3pH0l3WNmPwu7MABA4aqsrIwJz4MGDUrY58knn4wJzi+//LLMLGE/ANgdyVyAeJakI939U0kys4mSlkq6JsS6AAAFora2VuXl5Sn3+/TTT7XHHnuEUBEAJC+ZaR7rJXWOur+HpJpwygEAtHdjx46NGXVOJkhfeOGFzS4UJEgDyAWtjkyb2R8UWQ7vQ0krzexvwf1hkl7MTHlA9s1aUsPmAECatm3bllboXbt2LdtvA8gL8aZ5LAz+XKTI0niNFoRWDZBjZi2p0fgZy1W/vUGSVFNXr/EzlksSgRpowcMPP6xRo0al1KdPnz567bXXQqoIAMLVaph293t354nN7G5Jp0ra4O5fDNp+Kel/JW0MDrvS3Z8MHhsv6TxJDZJ+5O5zduf8QFuYNGf1riDdqH57gybNWU2YBqS0LvB7+umnNXTo0LYvBgCyIOEFiGbWR9IESf0UNXfa3T+foOtUSTdLuq9J++/dfXKTc/STNErS4ZLKJc01sy+4e4OALFpfV59SO9CeLV++XEcccUTK/RoaGtShQ7IrsQJAfknmp9s9kv4kaYekExUJx39O1Mndn5X0fpJ1DJf0kLtvdfd/S1oj6Zgk+wKhKS9teVn11tqB9uTzn/98zIWCyQTpG264odmFggRpAO1ZMj/hStx9niRz97Xu/ktJp+zGOS82s2VmdreZ9QjaKiS9HXXMuqCtGTM738wWmtnCjRs3tnQI0GbGVfVVSXFRTFtJcZHGVfXNUkVAODZu3BgTnM1M//73vxP2q6uriwnOY8aMyUC1AJA7kgnTW82sg6TXzexiM/uWpK5pnu9Pkg6RNEBSraQpqT6Bu9/u7gPdfWBZWVmaZQDJGVFZoQkj+6uitEQmqaK0RBNG9me+NPLemDFjYoLzfvvtl7DPcccd12zUuXv37hmoFgByVzKbtoxRZDvxH0n6jaSTJI1O52Tu/m7jbTO7Q9ITwd0aSQdGHXqAWMsaOWJEZQXhGXmtoaFBHTsm8+M+1uLFi1VZWRlCRQDQfiQcmXb3l9z9I3df5+7nuvtId/9XOiczs55Rd78laUVw+zFJo8xsDzPrLamPWMsaANIyffr0mFHnZIN001FngjQAJBZv05Yb3P3HZva4Ipu1xHD3b8Z7YjN7UNJQSfua2TpJV0saamYDguerlvSD4LlWmtkjkl5R5ELHi1jJAwCSk87ydFOnTtXo0Wn9JyMAIEq84YppwZ+T4xzTKnc/s4Xmu+Icf62ka9M5FwAUilWrVqlfv34p99u2bZuKi4tDqAgAClur0zzcfVHw5zOKjBi/4u7PNH5lqkAAKGTHH398zJSNZIL0Oeec02zKBkEaAMIRdyJdsGPhxYqEbjOzHZL+4O6/zkBtAFBQ6urq1KNHj8QHNlFTU6Py8vIQKgIAJNLqyLSZXSppsKSj3X1vd+8h6cuSBpvZTzJVIAC0V02Xp0smSPfo0aPZqDNBGgCyJ97I9NmShrn7psYGd3/TzL4n6SlJvw+7OABoL9LdCXDBggUaMmRICBUBANpCvJ/sxdFBupG7b5TE5DsAiGPmzJkxo87JBumdO3fGjDoTpAEgt8Ubmd6W5mMAkLZZS2o0ac5qra+rV3lpicZV9c2LTXPSWZ7u8ssv18SJE0OoBgCQKfHC9JFmtrmFdpPUOaR6ABSwWUtqNH7GctVvjywzX1NXr/EzlktSTgXqNWvWqE+fPin327x5s/baa68QKgIAZEu8pfGK3L1bC197uTvTPAC0uUlzVu8K0o3qtzdo0pzVWaooYtCgQTFTNpIJ0n369Gl2oSBBGgDan+T2mAWADFhfV59Sexjq6+u15557ptxvxYoVOvzww0OoCACQy1K/tBwAQlJeWpJSe1u47rrrYkadkw3STUedCdIAUJgI0wByxriqviopLoppKyku0riqvm12jujgbGa66qqrEvaZNm1as/AMAIDENA8AOaTxIsO2Ws1jwYIFOvHEE1Put2PHDhUVFSU+EABQ8AjTAHLKiMqKtMNzOsvTjR49WlOnTk3rfAAAEKYB5KXa2tq0ttHesGGDysrKQqgIAFCICNNAluXrJiWZdsYZZ2j69Okp9SkuLta2bewxBQAID2EayKJ82aQk0xoaGtSxY+o/nv7xj39o0KBBIVQEAEDLWM0DyKJc3aQk06ZPnx6zwkayQbrpChsEaQBAphGmgSzKhU1KsqHp8nRnnHFGwj6/+93vWJ4OAJBzmOYBZFF5aYlqWgjOYW5SkmmrVq1Sv379Uu63detWderUKYSKAABoO4xMA1mUiU1KMm3IkCExo87JBOnRo0c3G3UmSAMA8gEj00AWtfUmJZm2ZcsWdevWLeV+NTU1aS1rh+xgxRkAaB1hGsiy3dmkJNOuu+66pLbfjta9e3fV1dWFUxBCx4ozABAfYRpAi9xdHTqkPhPs6aef1tChQ9u+IGRFvBVnCNMAQJgGEJg7d66GDRuWcr+dO3emtY038kOhrjgDAMniAkSgQHXu3DnmQsFkgvT111/f7EJBgnT71trKMu1pxRkA2B2MTAMF4K233tLBBx+ccr+PPvpIXbp0CaEi5ItxVX1j5kxL+b/iDAC0JUamgXbozDPPjBl1TiZIDxs2rNmoczaC9KwlNRo8cb56XzFbgyfO16wlNRmvAZ8ZUVmhCSP7q6K0RCaporREE0b2Z740AAQYmQby3NatW9W5c+eU+7322mvq06dPCBWlj5UjclM+rTgDAJnGyDSQZ26//faYUedkg3TTUedcC9JS/JUjAADIRaGFaTO728w2mNmKqLa9zexvZvZ68GePoN3M7CYzW2Nmy8zsqLDqAvJNdHA2M/3gBz9I2GfmzJnNwnM+YOUIAEC+CXNkeqqk/2rSdoWkee7eR9K84L4kfU1Sn+DrfEl/CrEuIGe9+OKLzcJzMhoaGmKC84gRI8ItNCSsHAEAyDehhWl3f1bS+02ah0u6N7h9r6QRUe33ecS/JJWaWc+wagNyRZ8+fWKC85e//OWEfcaOHdts1DmdzVVy0biqviopLoppY+UIAEAuy/QFiPu7e21w+x1J+we3KyS9HXXcuqCtVk2Y2fmKjF7roIMOCq9SoI1t3LhR++23X8r93nvvPe29994hVJR7Gi9ymzRntdbX1au8tETjqvpy8RsAIGdlbTUPd3czS3kip7vfLul2SRo4cGB+TARFQfrJT36iG264IaU+/fr107V/fiomTD67tl4jCiNLS2LlCABAfsl0mH7XzHq6e20wjWND0F4j6cCo4w4I2oC80NDQoI4dU/84LVq0SEcd9dn1tiwNBwBAfsn0RMvHJI0Obo+W9GhU+38Hq3ocK+nDqOkgQE6ZtaRGfc+6Omauc7JBuulc5+ggLbE0HAAA+Sa0kWkze1DSUEn7mtk6SVdLmijpETM7T9JaSWcEhz8p6euS1kj6RNK5YdUFpOPQQw/VG2+8kVKfO++8U+edd15KfVgaDgCA/BJamHb3M1t56OQWjnVJF4VVC5CK2tpalZeXp9xv69at6tSp026du7y0RDUtBGeWhgMAIDexnTgK3oQJE3TllVem1Kfbcd9RjxPO3nXfpN0O0lJkabjoOdMSS8MBAJDLCNMoKJ9++qlKSlIf5X3//ffVo0cPDZ44P9SRY5aGAwAgvxCm0a49/vjj+uY3v5lSn/PPP1+33XZbi49lYuSYpeEAAMgfhGm0G+6uww47TK+99lpK/VauXKl+/foldSwjxwAAIBphGnnrlVde0eGHH55Sny9+8YtatmyZzCzt8zJyDAAAGmV6nWkgbd///vdj1nZOJkg/8cQTMes6L1++fLeCNAAAQDRGppGT3n//fe2zzz4p96uvr1fnzp1DqAgAAKA5RqaRE2699daYUedkgvTEiROb7ShIkAYAAJnEyDQybseOHSouLk653/r169WzZ88QKgIAAEgPI9MI3YIFC2JGnZMJ0iNHjmw26kyQBgAAuYaRabS5448/Xs8991xKfV566SUNHDgwpIoAAADCQZjGbqmurlbv3r1T6lNWVqZ33nlHHTrwHyMAACC/kWYK3KwlNRo8cb56XzFbgyfO16wlNXGPv/vuu2OmbCQTpB944IGY6RobNmwgSAMAgHaBkekCNmtJTczW2DV19Ro/Y7mkyMYk9fX1Ouqoo/Tqq6+m9LxbtmxR165d27xeAACAXMPwYAGbNGf1riAtSZ++tUyvXvM1feuoA2Rm2nPPPRMG6csvv7zZhYIEaQAAUCgYmS5Q7q7VC2bqvTk3p9SvtrZWn/vc50KqCgAAIL8wMl0gNmzYoJEjR+6a69yhQ4eEQXrcuHHNRp0J0tmX6jx3FB6+RwAgcxiZbqdmz56tkSNHatu2bUn36fk/N6t7+SGaMLK/RlRWhFgd0pVonjvA9wgAZBYj0+3Axx9/rEsuuSRmlY1TTz01bpC+8cYb1dDQoJmL12nQhHnqdfkT6t3nPwjSOa7pPHdJqt/eoElzVod2TkY580s2vkcAoJAxMp2HFi5cqO985zt68803kzq+f//+uv/++9W/f/9mj42orCA855H1dfUpte8uRjnzT6a/RwCg0DEyneN27NihiRMnxow6H3300XGD9Lhx41RfX79rnvOyZctaDNLIP+WlJSm17y5GOfNPpr9HAKDQEaZzTHV1tYYOHborOBcXF2v8+PGtHr/33ntr3rx5MRcJXn/99ercuXMGq0amjKvqq5Liopi2kuIijavqG8r5GOXMP5n+HgGAQkeYziJ317Rp05rtKPjMM8+02ue73/2u3nvvvV3B+b333tNJJ52UwaqRTSMqKzRhZH9VlJbIJFWUloQ6z51RzvyT6e8RACh05u7ZriFtAwcO9IULF2a1hllLajRpzmqtr6tXeWmJxlX1bfWX1vvvv69LLrlEDzzwQNLPP23aNJ111lkys7YqGUha0znTUmSUk3AGACg0ZrbI3Qc2becCxN2Q6OKs+fPn67TTTlNdXV1SzzdkyBDdc8896t27d1glAylpDMzJ/oMRAIBCw8j0bhg8cb5qgrmjvmOb6v7+Z21+cUbS/SdMmKCxY8eqY0f+TQMAAJDLGJluY2+//bZWzLhZm1+amdTxhx56qB588EENHNjsPQAAAECeIkynqKamRgcccEDC4y6++GJNnDhRXbp0yUBVAAAAyIashGkzq5a0RVKDpB3uPtDM9pb0sKRekqolneHuH2Sjvnhqa2ubtVnHTtp3xHjtecjRXJwFAABQQLI5Mn2iu2+Kun+FpHnuPtHMrgjuX56d0lo3cOBArVu3Tl27dlX37t1TWs0DAJLBzxUAyB+5NM1juKShwe17JS1QDobpln7J/eMK1nkG0DbYwh0A8ku2Nm1xSU+Z2SIzOz9o29/dG+dQvCNp/5Y6mtn5ZrbQzBZu3LgxE7Xu0vhLrqauXq7PfsnNWlKT0ToAtF9s4Q4A+SVbYfor7n6UpK9JusjMToh+0CPr9bW4Zp+73+7uA919YFlZWQZK/Qy/5ACEjS3cASC/ZCVMu3tN8OcGSTMlHSPpXTPrKUnBnxuyUVs8/JIDEDa2cAeA/JLxMG1mXcxsr8bbkr4qaYWkxySNDg4bLenRTNeWCL/kAIRtXFVflRQXxbSVFBdpXFXfLFUEAIgnGyPT+0t6zsxelvSipNnu/n+SJkoaZmavS/rP4H5O4ZccgLCNqKzQhJH9VVFaIpNUUVrCcpsAkMPYTjxFLFkFAABQeNhOvI2MqKwgPAMAAEBS9lbzAAAAAPIeYRoAAABIE2EaAAAASBNhGgAAAEgTYRoAAABIU14vjWdmGyWtzXYdGbCvpE3ZLgIZxXteeHjPCwvvd+HhPc9/B7t7WdPGvA7ThcLMFra0riHaL97zwsN7Xlh4vwsP73n7xTQPAAAAIE2EaQAAACBNhOn8cHu2C0DG8Z4XHt7zwsL7XXh4z9sp5kwDAAAAaWJkGgAAAEgTYRoAAABIE2E6x5jZgWb2tJm9YmYrzWxM0P4bM1tmZkvN7CkzK892rWgbrb3nUY9fZmZuZvtmq0a0nTif8V+aWU3wGV9qZl/Pdq1oG/E+42Z2iZm9GrRfn8060XbifM4fjvqMV5vZ0iyXijbAnOkcY2Y9JfV098VmtpekRZJGSFrn7puDY34kqZ+7/zB7laKttPaeu/srZnagpDslHSbpS+7Ogv95Ls5n/AxJH7n75GzWh7YX5z3fX9JVkk5x961mtp+7b8hiqWgj8X6uRx0zRdKH7v7rbNWJtsHIdI5x91p3Xxzc3iJplaSKxiAd6CKJfwW1E62958HDv5f0U/F+txsJ3m+0Q3He8wskTXT3rcFjBOl2ItHn3MxMkX9AP5idCtGWCNM5zMx6SaqU9EJw/1oze1vSWZJ+kcXSEJLo99zMhkuqcfeXs1sVwtL0My7p4mA6191m1iN7lSEsTd7zL0g63sxeMLNnzOzorBaHULTwOZek4yW96+6vZ6UotCnCdI4ys66S/irpx42j0u5+lbsfKOl+SRdnsz60vej3XNIOSVeKfzS1Wy18xv8k6RBJAyTVSpqSveoQhhbe846S9pZ0rKRxkh4JRizRTrT0uzxwphiVbjcI0znIzIoV+fDd7+4zWjjkfkmnZbYqhKmF9/wQSb0lvWxm1ZIOkLTYzD6XvSrRVlr6jLv7u+7e4O47Jd0h6Zhs1oi21crP9XWSZnjEi5J2SuJC43aitd/lZtZR0khJD2erNrQtwnSOCUYl7pK0yt1/F9XeJ+qw4ZJezXRtCEdL77m7L3f3/dy9l7v3UuSX7lHu/k4WS0UbiPMZ7xl12Lckrch0bQhHa++5pFmSTgyO+YKkTpK4yLgdiPOeS9J/SnrV3ddlvjKEgdU8coyZfUXS3yUtV2SUQor8d/95kvoGbWsl/dDda7JSJNpUa++5uz8ZdUy1pIGs5pH/4nzGz1RkiodLqpb0A3evzUKJaGNx3vO5ku5W5H3fJmmsu8/PRo1oW/F+rpvZVEn/cvdbs1Uf2hZhGgAAAEgT0zwAAACANBGmAQAAgDQRpgEAAIA0EaYBAACANBGmAQAAgDQRpgGgCTNrMLOlZvaymS02s0Ep9p9qZqencPwCM3srevc7M5tlZh+lct4UzneOmd2cYp+UXlOC5/qLmX0+uF1tZsuDv+/lZjY86jg3sz9H3e9oZhvN7Immr8PMLjaz/2mL+gAgFYRpAGiu3t0HuPuRksZLmpCBc9ZJGixJZlYqqWe8g/OVmR0uqcjd34xqPtHdB0g6XdJNUe0fS/qimZUE94dJam19/bslXdLG5QJAQoRpAIivm6QPpMiuZmY2ycxWBKOo34lqv9nMVpvZXEn7Be0nmdmsxicys2FmNrOV8zwkaVRwe6Sk6O2Hu5rZvGCUfNforZl1MbPZwQj6iqh6JprZK2a2zMwmx3txwYjzTWb2TzN7s3H0ubXXFDz2JTN7xswWmdkcM+tpZt2DY/sGxzxoZv/bwinPkvRoK+Xs+ruO8qSkU4LbZ0p6sKWO7v6JpGozYxt2ABlFmAaA5kqCaQevSrpT0m+C9pGK7FZ3pCJbAk8KtgH/liI7lPaT9N+SGqeFPC3pMDMrC+6fq8gIakvmSTrBzIoUCdUPRz32qaRvuftRimw/PSWYEvJfkta7+5Hu/kVJ/2dm+wT1HO7uR0i6JonX21PSVySdKmli0NbiazKzYkl/kHS6u38peD3XuvuHki6WNNXMRknq4e53tHCuwZIWNWl72sxWSHpG0s+aPPaQpFFm1lnSEZJeiPM6Fko6PvHLBYC2Q5gGgOYap3kcpkhgvS8Ir1+R9KC7N7j7u4qEv6MlnRDVvl7SfEnyyBaz0yR9L5i6cZyk/9fKORskPadIkC5x9+qox0zSdWa2TJEtqCsk7a/IVsXDzOy3ZnZ8EGg/VCR832VmIyV9ksTrneXuO939leB51dprUiRgf1HS38xsqSLh94Dg9f4tqOmPkr7fyrl6StrYpO3E4B8D/SXdbGZdGx9w92WSeikyKv1kgtexQVJ5gmMAoE11zHYBAJDL3P15M9tXUlnCg1t2j6THFQm40919R5xjH5I0U9Ivm7SfFZz/S+6+3cyqJXV299fM7ChJX5d0jZnNc/dfB1MdTlZkDvLFkk5KUOPWqNvW6lGfPb7S3Y9r9oBZB0n/oUiA7yFpXQv96yV1bumJ3f0NM3tXkdHwF6MeekzSZElDJe0Tp7bOwfMDQMYwMg0AcZjZYZKKJL0n6e+SvmNmRcHUjRMUCX3PRrX3VGQqhiQpGNVdr8gI7j0JTvd3RS52bDovuLukDUGQPlHSwUFt5ZI+cfc/S5ok6ahgVLe7uz8p6SeKTElJR2uvabWkMjM7LqihOLioUMH5Vkn6rqR7gikhTa2SdGhLJzSz/ST1lrS2yUN3S/qVuy9PUPMXJK1IcAwAtClGpgGguZJgCoMUGYkd7e4NwcWDx0l6WZJL+qm7vxO0nyTpFUlvSXq+yfPdL6nM3VfFO2kwLaSlCwbvl/S4mS1XZF7wq0F7f0Xmbe+UtF3SBZL2kvRoMMfYJF2a/MuO0eJrcvdtwUWKN5lZd0V+j9xgZjsUmdpxjLtvMbNnFfkHxNVNnne2IiPMc6PanjazBknFkq4IptDs4u7rFLvKR2sGq/moPgCEyiI/uwEAYQnWQl7i7ndlu5ZsC5a5e1rSYHdvaMPnrZR0qbuf3VbPCQDJIEwDQIjMbJEi6yUPc/etiY4vBGZWJWmVu7/Vhs85TNLrTS7cBIDQEaYBAACANHEBIgAAAJAmwjQAAACQJsI0AAAAkCbCNAAAAJAmwjQAAACQpv8PkM4v8KbFY1MAAAAASUVORK5CYII=",
"text/plain": [
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plt.xlabel('Body Mass Index (BMI)')\n",
"plt.ylabel('Diabetes Risk')\n",
"plt.scatter(diabetes_X_train, diabetes_y_train)\n",
"plt.plot(diabetes_X_train, diabetes_y_train_pred, color='black', linewidth=2)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Our visualization seems reasonable: we see that the linear model that we found is close to the observed data and captures the trend we noticed earlier---higher BMIs are associated with higher diabetes risk."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 2.1.4. Making New Predictions\n",
"\n",
"Recall that one of the goals of supervised learning is to predict diabetes risk for new patients.\n",
"Given a new dataset of patients with a known BMI, we can use the predictive model to estimate their risk.\n",
"\n",
"Formally, given an $x_\\text{new}$, we can output prediction $y_\\text{new}$ as\n",
"\n",
"$$ \n",
"y_\\text{new} = f(x_\\text{new}) = \\theta_1^* \\cdot x_\\text{new} + \\theta_0^*. \n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Let's illustrate this equation with a specific example from our diabetes dataset. \n",
"\n",
"First, we start by loading more data. We will load three new patients (shown in red below) that we haven't seen before."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtMAAAEGCAYAAACn7xkwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAqNklEQVR4nO3dfZiVZbnw/+8pTjK+hSn5EwSxXaEmyMvgS6j5ktIu95bY7raGqT0Vafm0a/vj+WnuZ+tjmrSh7LFteViWmuze3EqmtikVMy2rQVAUJF8CZTBFE5UcCuH8/bHuwWGY18WsWWuY7+c41jHrvu573fe55p7FnHNxXtcVmYkkSZKkntuh2gFIkiRJ/ZXJtCRJklQmk2lJkiSpTCbTkiRJUplMpiVJkqQy7VjtALbFXnvtlaNGjap2GJIkSdrOLVy48IXMHNq2vV8n06NGjaKxsbHaYUiSJGk7FxEr22u3zEOSJEkqk8m0JEmSVCaTaUmSJKlM/bpmuj0bNmxg1apVrF+/vtqhqAuDBw9m3333pa6urtqhSJIklWW7S6ZXrVrFbrvtxqhRo4iIaoejDmQmL774IqtWrWL//fevdjiSJEll2e6S6fXr15tI9wMRwZ577smaNWuqHYokSapx8xY1MXv+clavbWbYkHpmThnN1PHDqx0WsB0m04CJdD/hfZIkSV2Zt6iJC25eQvOGjQA0rW3mgpuXANREQu0AREmSJNWs2fOXb06kWzRv2Mjs+curFNGWTKYrYNddd+3ymI9//OMsXboUgC9+8Ytb7Hv3u9/dK9fojuuuu47Vq1e3G1dP3XPPPfzqV7/qlbgkSZIAVq9t7lF7XzOZrpJvfetbHHTQQcDWyXRfJqRtk+nWcfWUybQkSeptw4bU96i9rw34ZHreoiYmz7qb/c+/ncmz7mbeoqZeO/c999zDMcccwymnnMIBBxzA9OnTyUwAjjnmGBobGzn//PNpbm5m3LhxTJ8+HXij13ndunUcf/zxTJgwgTFjxvDjH/+40+utWLFi83UOPPBATjnlFF577TUALrnkEiZNmsTBBx/MjBkzyExuuukmGhsbmT59OuPGjaO5uXlzXAA/+9nPOOKII5gwYQL/+I//yLp164DSMu4XXXTR5rgee+wxVqxYwdVXX80VV1zBuHHj+OUvf8mPfvQjDj74YA455BCOPvroXvu+SpKkgWPmlNHU1w3aoq2+bhAzp4yuUkRbGtDJdEtBe9PaZpI3Ctp7M6FetGgRX/3qV1m6dClPPfUU999//xb7Z82aRX19PYsXL2bu3Llb7Bs8eDC33HILDz74IAsWLOC8887bnIx3ZPny5XzqU59i2bJl7L777nz9618H4Nxzz+V3v/sdjzzyCM3Nzdx2222ccsopNDQ0MHfuXBYvXkx9/Rt/4b3wwgtceuml3HnnnTz44IM0NDTwla98ZfP+vfbaiwcffJBzzjmHOXPmMGrUKM4++2w+97nPsXjxYo466iguueQS5s+fz0MPPcStt966rd9KSZI0AE0dP5zLp41h+JB6Ahg+pJ7Lp42picGHMMCT6b4oaD/00EPZd9992WGHHRg3bhwrVqzo9mszk89//vOMHTuW9773vTQ1NfHcc891+poRI0YwefJkAE4//XTuu+8+ABYsWMBhhx3GmDFjuPvuu3n00Uc7Pc8DDzzA0qVLmTx5MuPGjeP6669n5cqVm/dPmzYNgIkTJ3b4niZPnsxZZ53FN7/5TTZu3NjuMZIkSV2ZOn44959/HH+Y9QHuP/+4mkmkoYJT40XEYOBeYKfiOjdl5kURsT/wfWBPYCHwkcz8a0TsBNwATAReBP4pM1dUKj7om4L2nXbaafPzQYMG8frrr3f7tXPnzmXNmjUsXLiQuro6Ro0a1eXKjm2nm4sI1q9fz6c+9SkaGxsZMWIEF198cZfnyUxOOOEEvve977W7v+V9dfaerr76an7zm99w++23M3HiRBYuXMiee+7Z6XUlSZL6k0r2TP8FOC4zDwHGAe+LiMOBLwFXZObbgZeAjxXHfwx4qWi/ojiuomqloL2uro4NGzZs1f7yyy/z1re+lbq6OhYsWLBFz3BHnn76aX79618D8J//+Z8ceeSRmxPnvfbai3Xr1nHTTTdtPn633Xbj1Vdf3eo8hx9+OPfffz9PPPEEAH/+85/5/e9/3+m1257rySef5LDDDuOSSy5h6NChPPPMM13GL0mS1J9ULJnOknXFZl3xSOA4oCWbux6YWjw/udim2H98VHhVj1opaJ8xYwZjx47dPACxxfTp02lsbGTMmDHccMMNHHDAAV2ea/To0Vx11VUceOCBvPTSS5xzzjkMGTKET3ziExx88MFMmTKFSZMmbT7+rLPO4uyzz948ALHF0KFDue666zjttNMYO3YsRxxxBI899lin1/67v/s7brnlls0DEGfOnMmYMWM4+OCDefe7380hhxzSw++MJElSbYuuBrRt08kjBlEq5Xg7cBUwG3ig6H0mIkYAP83MgyPiEeB9mbmq2PckcFhmvtDmnDOAGQAjR46c2La3dtmyZRx44IHdjrGWl6fsqRUrVnDSSSfxyCOPVDuUbuvp/ZIkSaqGiFiYmQ1t2yu6nHhmbgTGRcQQ4Bag667Vrs95DXANQENDwzb/JTB1/PB+mzxLkiSpuvpkNo/MXAssAI4AhkRESxK/L9AyD10TMAKg2P9mSgMR1U2jRo3qV73SkiRJ/V3FkumIGFr0SBMR9cAJwDJKSfUpxWFnAi0rkdxabFPsvzsrWYMiSZIkbaNKlnnsA1xf1E3vAPwwM2+LiKXA9yPiUmARcG1x/LXAdyPiCeBPwKkVjE2SJEnaZhVLpjPzYWB8O+1PAYe2074e+MdKxSNJkiT1tgG9AqIkSZK0LUymKyAiOO+88zZvz5kzh4svvrh6AQFf/epXee211zZvv//972ft2rVlnWvevHksXbq0lyKTJEnqv0ymK2CnnXbi5ptv5oUXXuj64D7SNpm+4447GDJkSFnnMpmWJEkqMZmeOxdGjYIddih9nTt3m0+54447MmPGDK644oqt9q1Zs4Z/+Id/YNKkSUyaNIn7778fgDFjxrB27Voykz333JMbbrgBgDPOOIOf//znW5zjnnvu4eijj+YDH/gAo0eP5uyzz2bTpk0AnHPOOTQ0NPCud72Liy66CIArr7yS1atXc+yxx3LssccCpWn0WpL9G2+8kUMPPZRx48bxyU9+ko0bNwKw6667cuGFF3LIIYdw+OGH89xzz/GrX/2KW2+9lZkzZzJu3DiefPJJrrzySg466CDGjh3Lqac6blSSJA0gmdlvHxMnTsy2li5dulVbh268MXPnnTPhjcfOO5fat8Euu+ySL7/8cu633365du3anD17dl500UWZmXnaaaflL3/5y8zMXLlyZR5wwAGZmfnJT34yb7vttlyyZEk2NDTkxz/+8czMfPvb357r1q3b4vwLFizInXbaKZ988sl8/fXX873vfW/+6Ec/yszMF198MTMzX3/99XzPe96TDz30UGZm7rfffrlmzZrN52jZXrp0aZ500kn517/+NTMzzznnnLz++uszMxPIW2+9NTMzZ86cmV/4whcyM/PMM8/cfL3MzH322SfXr1+fmZkvvfRSj75XPbpfkiRJVQI0Zjv56MDumb7wQmhV+gCUti+8cJtPvfvuu3PGGWdw5ZVXbtF+5513cu655zJu3Dj+/u//nldeeYV169Zx1FFHce+993LvvfdyzjnnsGTJEpqamthjjz3YZZddtjr/oYceytve9jYGDRrEaaedxn333QfAD3/4QyZMmMD48eN59NFHuyzHuOuuu1i4cCGTJk1i3Lhx3HXXXTz11FMAvOlNb+Kkk04CYOLEiaxYsaLdc4wdO5bp06dz4403suOOFV1UU5IkqaYM7Mzn6ad71t5Dn/3sZ5kwYQIf/ehHN7dt2rSJBx54gMGDB29x7NFHH81VV13F008/zWWXXcYtt9zCTTfdxFFHHdXuuSNiq+0//OEPzJkzh9/97nfssccenHXWWaxfv77TGDOTM888k8svv3yrfXV1dZuvM2jQIF5//fV2z3H77bdz77338pOf/ITLLruMJUuWmFRLkqQBYWD3TI8c2bP2HnrLW97Chz70Ia699trNbSeeeCJf+9rXNm8vXrwYgBEjRvDCCy/w+OOP87a3vY0jjzySOXPmcPTRR7d77t/+9rf84Q9/YNOmTfzgBz/gyCOP5JVXXmGXXXbhzW9+M8899xw//elPNx+/22678eqrr251nuOPP56bbrqJ559/HoA//elPrFy5stP31fpcmzZt4plnnuHYY4/lS1/6Ei+//DLr1q3r3jdIkiSpnxvYyfRll8HOO2/ZtvPOpfZect55520xq8eVV15JY2MjY8eO5aCDDuLqq6/evO+www7jne98JwBHHXUUTU1NHHnkke2ed9KkSZx77rkceOCB7L///nzwgx/kkEMOYfz48RxwwAF8+MMfZvLkyZuPnzFjBu973/s2D0BscdBBB3HppZdy4oknMnbsWE444QSeffbZTt/TqaeeyuzZsxk/fjyPP/44p59+OmPGjGH8+PF85jOfKXuWEEmSpP4mSvXU/VNDQ0M2NjZu0bZs2TIOPPDA7p9k7txSjfTTT5d6pC+7DKZP7+VIe9c999zDnDlzuO2226odyjbr8f2SJEmqgohYmJkNbdstbJ0+veaTZ0mSJNUmk+l+6JhjjuGYY46pdhiSJEkD3nZZM92fS1cGEu+TJEnq77a7ZHrw4MG8+OKLJmo1LjN58cUXt5oiUJIkqT/Z7so89t13X1atWsWaNWuqHYq6MHjwYPbdd99qhyFJklS27S6ZrqurY//99692GJIkSRoAtrsyD0mSJKmvmExLkiRJZTKZliRJkspkMi1JkiSVyWRakiRJKpPJtCRJklQmk2lJkiSpTCbTkiRJUplMpiVJkqQymUxLkiRJZTKZliRJkspkMi1JkiSVqWLJdESMiIgFEbE0Ih6NiH8u2i+OiKaIWFw83t/qNRdExBMRsTwiplQqNkmSJKk37FjBc78OnJeZD0bEbsDCiPh5se+KzJzT+uCIOAg4FXgXMAy4MyLemZkbKxijJEmSVLaK9Uxn5rOZ+WDx/FVgGTC8k5ecDHw/M/+SmX8AngAOrVR8kiRJ0rbqk5rpiBgFjAd+UzSdGxEPR8S3I2KPom048Eyrl62i8+RbkiRJqqqKJ9MRsSvwX8BnM/MV4BvA3wDjgGeBL/fwfDMiojEiGtesWdPb4UqSJEndVtFkOiLqKCXSczPzZoDMfC4zN2bmJuCbvFHK0QSMaPXyfYu2LWTmNZnZkJkNQ4cOrWT4kiRJUqcqOZtHANcCyzLzK63a92l12AeBR4rntwKnRsROEbE/8A7gt5WKT5IkSdpWlZzNYzLwEWBJRCwu2j4PnBYR44AEVgCfBMjMRyPih8BSSjOBfNqZPCRJklTLKpZMZ+Z9QLSz645OXnMZcFmlYpIkSZJ6kysgSpIkSWUymZYkSZLKZDItSZIklclkWpIkSSpTJWfzkCRJUj8wb1ETs+cvZ/XaZoYNqWfmlNFMHe9C1N1hMi1JkjSAzVvUxAU3L6F5Q2lG4qa1zVxw8xIAE+pusMxDkiRpAJs9f/nmRLpF84aNzJ6/vEoR9S8m05IkSQPY6rXNPWrXlkymJUmSBrBhQ+p71K4tmUxLkiQNYDOnjKa+btAWbfV1g5g5ZXSVIupfHIAoSZI0gLUMMnQ2j/KYTEuSJA1wU8cPN3kuk2UekiRJUplMpiVJkqQyWeYhSVI/5+p1UvV0mUxHRGRmtmnbKTP/UrmwJElSd7h6nVRd3SnzuLb1RkTsCtxRmXAkSVJPuHqdVF3dSaZXRcTXASJiD+BnwI0VjUqSJHWLq9dJ1dVlMp2Z/wasi4irKSXSX87M71Q8MkmS1CVXr5Oqq8NkOiKmtTyA3wCHA4uALNokSVKVuXqdVF2dDUD8uzbbi4C6oj2BmysVlCRJ6h5Xr5Oqq8NkOjM/2peBSJKk8rh6nVQ9XdZMR8S/R8TuEVEXEXdFxJqIOL0vgpMkSZJqWXdm8zgxM18BTgJWAG8HZlYyKEmSJKk/6E4y3VIK8gHgR5n5cgXjkSRJkvqN7iwnfltEPAY0A+dExFBgfWXDkiRJkmpfd+aZPh94N9CQmRuAPwMnVzowSZIkqdZ12DMdEcdl5t2t55SOiNaHODWeJEmSBrTOyjzeA9zN1vNNg/NMS5IkSZ3OM31R8bWs+aYjYgRwA7A3peT7msz8vxHxFuAHwChKs4N8KDNfilK39/8F3g+8BpyVmQ+Wc21JkiSpL3RaMx0RgyJir1bbb4qIGRGxrBvnfh04LzMPorQU+acj4iDgfOCuzHwHcFexDfC3wDuKxwzgGz1+N5IkSVIf6jCZjohTgT8BD0fELyLiROApSknv9K5OnJnPtvQsZ+arwDJgOKXBi9cXh10PTC2enwzckCUPAEMiYp+y3pUkSZLUBzqrmf5XYGJmPhERE4BfA6dk5k96epGIGAWMB34D7J2Zzxa7/kipDARKifYzrV62qmh7tlUbETGDUs81I0eO7GkokiRJUq/prMzjr5n5BEDRw/x4mYn0rsB/AZ8tVlLcLDOTUj11t2XmNZnZkJkNQ4cO7Wk4kiRJUq/prGf6rRHxL622h7TezsyvdHXyiKijlEjPzcyW2T+ei4h9MvPZoozj+aK9CRjR6uX7Fm2SJElSTeqsZ/qbwG6tHm23O1XMznEtsKxN4n0rcGbx/Ezgx63az4iSw4GXW5WDSJIkSTWns6nx/s82nnsy8BFgSUQsLto+D8wCfhgRHwNWAh8q9t1BaVq8JyhNjVfWlHySJElSX+mszGObZOZ9QHSw+/h2jk/g05WKR5IkSeptnc4zLUmSJKljJtOSJElSmbpMpiPinyNi92Jg4LUR8WCxgIskSZI0oHWnZ/p/FPNDnwjsQWlQ4ayKRiVJkiT1A90ZgNgyiPD9wHcz89Fi2jsNQPMWNTF7/nJWr21m2JB6Zk4ZzdTxw6sdliRJUlV0J5leGBE/A/YHLoiI3YBNlQ1LtWjeoiYuuHkJzRs2AtC0tpkLbl4CYEItSZIGpO6UeXwMOB+YlJmvAW/COaAHpNnzl29OpFs0b9jI7PnLqxSRJElSdXUnmU7gIOAzxfYuwOCKRaSatXptc4/aJUmStnfdKfP4OqWyjuOAS4BXgf8CJlUwLtWgYUPqaWoncR42pL4K0UgaKByrIamWdadn+rDM/DSwHiAzX6JU6qEBZuaU0dTXDdqirb5uEDOnjK5SRJK2dy1jNZrWNpO8MVZj3qKmaocmSUD3kukNETGIUrkHETEUByAOSFPHD+fyaWMYPqSeAIYPqefyaWPsIZJUMY7VkFTrulPmcSVwC/DWiLgMOAX43xWNSjVr6vjhJs+S+oxjNSTVui6T6cycGxELgeMpzTk9NTOXVTwySdKA51gNtcc6etWS7iwn/t3MfCwzr8rM/8jMZRHx3b4ITpI0sDlWQ21ZR69a052a6Xe13ijqpydWJhxJkt7gWA21ZR29ak2HZR4RcQHweaA+Il7hjWXF/wpc0wexSZLkWA1twTp61ZoOe6Yz8/LM3A2YnZm7Z+ZuxWPPzLygD2OUJEkCOq6Xt45e1dKdMo8LI+L0iPjfABExIiIOrXBckiRJW7GOXrWmO8n0VcARwIeL7XVFmyRJUp+yjl61pjvzTB+WmRMiYhGUVkCMCFdAlCRJVWEdvWqJKyBKkiRJZepOMt2yAuLexQqI9wFfrGhUkiRJUj/Q0xUQwRUQJUmSJKB7NdMAOwMtpR7OPSNJkiTRveXE/w24HngLsBfwnYj410oHJkmSJNW67vRMTwcOycz1ABExC1gMXFrBuCRJkqSa150BiKuBwa22dwKaKhOOJEmS1H902DMdEV+jVCP9MvBoRPy82D4B+G3fhCdV37xFTcyev5zVa5sZNqSemVNGO7+pJEkCOu+ZbgQWUpoW7/PAAuAe4ELgxxWPTKoB8xY1ccHNS2ha20wCTWubueDmJcxb5H/OSNJ2be5cGDUKdtih9HXu3GpHpBrVYc90Zl6/LSeOiG8DJwHPZ+bBRdvFwCeANcVhn8/MO4p9FwAfAzYCn8nM+dtyfak3zJ6/nOYNG7doa96wkdnzl9s7LUnbq7lzYcYMeO210vbKlaVtgOnTqxeXalJ3ZvN4R0TcFBFLI+Kplkc3zn0d8L522q/IzHHFoyWRPgg4FXhX8ZqvF6suSlW1em1zj9olSduBCy98I5Fu8dprpXapje4MQPwO8A3gdeBY4Abgxq5elJn3An/qZhwnA9/PzL9k5h+AJ4BDu/laqWKGDWl/WvWO2iVJ24Gnn+5Zuwa07iTT9Zl5FxCZuTIzLwY+sA3XPDciHo6Ib0fEHkXbcOCZVsesKtq2EhEzIqIxIhrXrFnT3iFSr5k5ZTT1dVv+J0l93SBmThldpYgkSRU3cmTP2jWgdSeZ/ktE7AA8HhHnRsQHgV3LvN43gL8BxgHPAl/u6Qky85rMbMjMhqFDh5YZhtQ9U8cP5/JpYxg+pJ4Ahg+p5/JpY6yXlqTt2WWXwc47b9m2886ldqmN7iza8s+UlhP/DPAF4DjgzHIulpnPtTyPiG8CtxWbTcCIVofui3NZq0ZMHT/c5FmSBpKWQYYXXlgq7Rg5spRIO/hQ7egymc7M3xVP1wEf3ZaLRcQ+mflssflB4JHi+a3Af0bEV4BhwDtwLmtJklQt06ebPKtbOizziIivFl9/EhG3tn10deKI+B7wa2B0RKyKiI8B/x4RSyLiYUqDGT8HkJmPAj8ElgL/DXw6Mzd2cGpJklRJzrEsdVtkZvs7IiZm5sKIeE97+zPzFxWNrBsaGhqysbGx2mFIkrT9aDvHMpTqha+5xp5aDWgRsTAzG7Zq7yiZbvPioQCZWVPTZ5hMS5LUy0aNKi1S0tZ++8GKFX0djVQzOkqmO53NIyIujogXgOXA7yNiTUT8W6WClCRJVeYcy1KPdFYz/S/AZGBSZr4lM/cADgMmR8Tn+ipASZLUh5xjWeqRznqmPwKcVqxICEBmPgWcDpxR6cAkSVIVOMey1COdJdN1mflC28aibrquciFJkqSqmT69NNhwv/0govTVwYdShzqbZ/qvZe6TpLLNW9TE7PnLWb22mWFD6pk5ZbSL5kh9zTmWpW7rLJk+JCJeaac9gMEVikfSADZvURMX3LyE5g2laeab1jZzwc1LAEyoJUk1qcMyj8wclJm7t/PYLTMt85DU62bPX745kW7RvGEjs+cvr1JEkiR1rtOp8SSpL61e29yjdkmSqs1kWlLNGDakvkftkiRVm8m0pJoxc8po6usGbdFWXzeImVNGVykiSZI619kAREnqUy2DDJ3NQ5LUX5hMS6opU8cPN3mWJPUblnlIkiRJZbJnWqoyFymRJKn/MpmWqshFSiRJ6t8s85CqyEVKJEnq30ympSpykRJJkvo3k2mpilykRJKk/s1kWqoiFymRJKl/cwCiVEUuUqL+wBlnJKljJtNSlblIiWqZM85IUucs85AkdcgZZySpcybTkqQOOeOMJHXOZFqS1CFnnJGkzplMS5I65IwzktQ5ByBKqinOHFFbnHFGkjpnMi2pZjhzRG1yxhlJ6phlHpJqhjNHSJL6m4ol0xHx7Yh4PiIeadX2loj4eUQ8Xnzdo2iPiLgyIp6IiIcjYkKl4pJUu5w5QpLU31SyZ/o64H1t2s4H7srMdwB3FdsAfwu8o3jMAL5Rwbgk1ShnjpAk9TcVS6Yz817gT22aTwauL55fD0xt1X5DljwADImIfSoVm6Ta5MwRkqT+pq8HIO6dmc8Wz/8I7F08Hw480+q4VUXbs7QRETMo9V4zcuTIykUqqc85c4Qkqb+p2mwemZkRkWW87hrgGoCGhoYev16qdQN9ajhnjpAk9Sd9nUw/FxH7ZOazRRnH80V7EzCi1XH7Fm3SgOLUcJIk9S99PTXercCZxfMzgR+3aj+jmNXjcODlVuUgUk2Zt6iJybPuZv/zb2fyrLuZt6j3/u5zajhJkvqXivVMR8T3gGOAvSJiFXARMAv4YUR8DFgJfKg4/A7g/cATwGvARysVl7QtKt1z7NRwkiT1LxVLpjPztA52Hd/OsQl8ulKxSL2ls57j3kimhw2pp6mdxNmp4SRJqk2ugCj1QKV7jp0aTpKk/sVkWuqBSi8qMnX8cC6fNobhQ+oJYPiQei6fNsbBh5Ik1aiqTY0n9Uczp4zeomYaer/n2KnhJEnqP0ympR5wURFJktSaybTUQ/YcS5KkFtZMS5IkSWUymZYkSZLKZDItSZIklclkWpIkSSqTybQkSZJUJpNpVdfcuTBqFOywQ+nr3LnVjkiSJKnbnBpP1TN3LsyYAa+9VtpeubK0DTB9evXikiRJ6qbIzGrHULaGhoZsbGysdhj92rxFTdVbgGTUqFIC3dZ++8GKFX0TgyRJUjdExMLMbGjbbs/0ADZvUdMWS2M3rW3mgpuXAPRNQv300z1rlyRJqjHWTA9gs+cv35xIt2jesJHZ85f3TQAjR/asXZIkqcaYTA9gq9c296i91112Gey885ZtO+9capckSeoHLPMYwIYNqaepncR52JD6vgmgZZDhhReWSjtGjiwl0g4+7FRV69zVL/gzIkl9x57pAWzmlNHU1w3aoq2+bhAzp4zuuyCmTy8NNty0qfTVRLpTLXXuTWubSd6oc5+3qKnaoalG+DMiSX3LZHoAmzp+OJdPG8PwIfUEMHxIPZdPG2MPVg2rRp37vEVNTJ51N/uffzuTZ91tUlbjqj4WQpIGGMs8Brip44ebPPcjfV3nXvUZX9RjVR8LIUkDjD3TUj/SUT17perc7eXsf/r6Z0SSBjqTaakf6es6d3s5+5+aGAshSQOIZR5SP9JSWtFXMzVUfcYX9Vhf/4xI0kDncuLbyCmotD1rWzMNpV5OB6pKkgYalxOvAAdnaXtnL6ckSZ0zmd4GnQ3OMtnQ9sIZXyRJ6pgDELeBg7MkSZIGNpPpbeAUVJIkSQNbVZLpiFgREUsiYnFENBZtb4mIn0fE48XXPaoRW084BZUkSdLAVs2a6WMz84VW2+cDd2XmrIg4v9j+/6oTWvc4OEtSJThLkCT1H7U0APFk4Jji+fXAPdRgMt3eL7n7zz+u2mFJ2k44S5Ak9S/VqplO4GcRsTAiZhRte2fms8XzPwJ7t/fCiJgREY0R0bhmzZq+iHWzll9yTWubSd74JTdvUVOfxiFp++US7pLUv1QrmT4yMycAfwt8OiKObr0zSyvJtLuaTGZek5kNmdkwdOjQPgj1Df6Sk1RpzhIkSf1LVZLpzGwqvj4P3AIcCjwXEfsAFF+fr0ZsnfGXnKRKc5YgSepf+jyZjohdImK3lufAicAjwK3AmcVhZwI/7uvYuuIvOUmV5ixBktS/VKNnem/gvoh4CPgtcHtm/jcwCzghIh4H3lts1xR/yUmqtKnjh3P5tDEMH1JPAMOH1HP5tDEOPpSkGhWl8uT+qaGhIRsbG/v0mk5ZJUmSNPBExMLMbGjbXktT4/ULU8cPN3mWJEkS4HLikiRJUtlMpiVJkqQymUxLkiRJZTKZliRJkspkMi1JkiSVqV9PjRcRa4CV1Y6jD+wFvFDtINSnvOcDj/d8YPF+Dzze8/5vv8wc2raxXyfTA0VENLY3r6G2X97zgcd7PrB4vwce7/n2yzIPSZIkqUwm05IkSVKZTKb7h2uqHYD6nPd84PGeDyze74HHe76dsmZakiRJKpM905IkSVKZTKYlSZKkMplM15iIGBERCyJiaUQ8GhH/XLR/ISIejojFEfGziBhW7VjVOzq65632nxcRGRF7VStG9Z5OPuMXR0RT8RlfHBHvr3as6h2dfcYj4n9GxGNF+79XM071nk4+5z9o9RlfERGLqxyqeoE10zUmIvYB9snMByNiN2AhMBVYlZmvFMd8BjgoM8+uXqTqLR3d88xcGhEjgG8BBwATM9MJ//u5Tj7jHwLWZeacasan3tfJPd8buBD4QGb+JSLempnPVzFU9ZLO/l1vdcyXgZcz85JqxaneYc90jcnMZzPzweL5q8AyYHhLIl3YBfCvoO1ER/e82H0F8L/wfm83urjf2g51cs/PAWZl5l+KfSbS24muPucREZT+gP5edSJUbzKZrmERMQoYD/ym2L4sIp4BpgP/VsXQVCGt73lEnAw0ZeZD1Y1KldL2Mw6cW5RzfTsi9qheZKqUNvf8ncBREfGbiPhFREyqanCqiHY+5wBHAc9l5uNVCUq9ymS6RkXErsB/AZ9t6ZXOzAszcwQwFzi3mvGp97W+58DrwOfxj6btVjuf8W8AfwOMA54Fvly96FQJ7dzzHYG3AIcDM4EfFj2W2k6097u8cBr2Sm83TKZrUETUUfrwzc3Mm9s5ZC7wD30blSqpnXv+N8D+wEMRsQLYF3gwIv6f6kWp3tLeZzwzn8vMjZm5CfgmcGg1Y1Tv6uDf9VXAzVnyW2AT4EDj7URHv8sjYkdgGvCDasWm3mUyXWOKXolrgWWZ+ZVW7e9oddjJwGN9HZsqo717nplLMvOtmTkqM0dR+qU7ITP/WMVQ1Qs6+Yzv0+qwDwKP9HVsqoyO7jkwDzi2OOadwJsABxlvBzq55wDvBR7LzFV9H5kqwdk8akxEHAn8ElhCqZcCSv/d/zFgdNG2Ejg7M5uqEqR6VUf3PDPvaHXMCqDB2Tz6v04+46dRKvFIYAXwycx8tgohqpd1cs/vBL5N6b7/Ffh/M/PuasSo3tXZv+sRcR3wQGZeXa341LtMpiVJkqQyWeYhSZIklclkWpIkSSqTybQkSZJUJpNpSZIkqUwm05IkSVKZTKYlqY2I2BgRiyPioYh4MCLe3cPXXxcRp/Tg+Hsi4unWq99FxLyIWNeT6/bgemdFxH/08DU9ek9dnOumiHhb8XxFRCwpvt9LIuLkVsdlRNzYanvHiFgTEbe1fR8RcW5E/I/eiE+SesJkWpK21pyZ4zLzEOAC4PI+uOZaYDJARAwB9uns4P4qIt4FDMrMp1o1H5uZ44BTgCtbtf8ZODgi6ovtE4CO5tf/NvA/ezlcSeqSybQkdW534CUorWoWEbMj4pGiF/WfWrX/R0Qsj4g7gbcW7cdFxLyWE0XECRFxSwfX+T5wavF8GtB6+eFdI+Kuopd8c+9tROwSEbcXPeiPtIpnVkQsjYiHI2JOZ2+u6HG+MiJ+FRFPtfQ+d/Sein0TI+IXEbEwIuZHxD4R8ebi2NHFMd+LiE+0c8npwI87CGfz97qVO4APFM9PA77X3gsz8zVgRUS4DLukPmUyLUlbqy/KDh4DvgV8oWifRmm1ukMoLQk8u1gG/IOUVig9CDgDaCkLWQAcEBFDi+2PUupBbc9dwNERMYhSUv2DVvvWAx/MzAmUlp/+clES8j5gdWYekpkHA/8dEXsW8bwrM8cCl3bj/e4DHAmcBMwq2tp9TxFRB3wNOCUzJxbv57LMfBk4F7guIk4F9sjMb7ZzrcnAwjZtCyLiEeAXwL+22fd94NSIGAyMBX7TyftoBI7q+u1KUu8xmZakrbWUeRxAKWG9oUhejwS+l5kbM/M5SsnfJODoVu2rgbsBsrTE7HeB04vSjSOAn3ZwzY3AfZQS6frMXNFqXwBfjIiHKS1BPRzYm9JSxSdExJci4qgioX2ZUvJ9bURMA17rxvudl5mbMnNpcV46ek+UEuyDgZ9HxGJKye++xfv9eRHTVcDHO7jWPsCaNm3HFn8MjAH+IyJ2bdmRmQ8Doyj1St/Rxft4HhjWxTGS1Kt2rHYAklTLMvPXEbEXMLTLg9v3HeAnlBLcH2Xm650c+33gFuDiNu3Ti+tPzMwNEbECGJyZv4+ICcD7gUsj4q7MvKQodTieUg3yucBxXcT4l1bPo8Oj3tj/aGYesdWOiB2AAykl8HsAq9p5fTMwuL0TZ+aTEfEcpd7w37badSswBzgG2LOT2AYX55ekPmPPtCR1IiIOAAYBLwK/BP4pIgYVpRtHU0r67m3Vvg+lUgwAil7d1ZR6cL/TxeV+SWmwY9u64DcDzxeJ9LHAfkVsw4DXMvNGYDYwoejVfXNm3gF8jlJJSjk6ek/LgaERcUQRQ10xqJDiesuADwPfKUpC2loGvL29C0bEW4H9gZVtdn0b+D+ZuaSLmN8JPNLFMZLUq+yZlqSt1RclDFDqiT0zMzcWgwePAB4CEvhfmfnHov04YCnwNPDrNuebCwzNzGWdXbQoC2lvwOBc4CcRsYRSXfBjRfsYSnXbm4ANwDnAbsCPixrjAP6l+297C+2+p8z8azFI8cqIeDOl3yNfjYjXKZV2HJqZr0bEvZT+gLiozXlvp9TDfGertgURsRGoA84vSmg2y8xVbDnLR0cms3WvviRVVJT+7ZYkVUoxF/KizLy22rFUWzHN3QJgcmZu7MXzjgf+JTM/0lvnlKTuMJmWpAqKiIWU5ks+ITP/0tXxA0FETAGWZebTvXjOE4DH2wzclKSKM5mWJEmSyuQAREmSJKlMJtOSJElSmUymJUmSpDKZTEuSJEllMpmWJEmSyvT/A2Km2GItyUADAAAAAElFTkSuQmCC",
"text/plain": [
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# generate predictions on the new patients\n",
"diabetes_y_test_pred = regr.predict(diabetes_X_test)\n",
"\n",
"# visualize the results\n",
"plt.xlabel('Body Mass Index (BMI)')\n",
"plt.ylabel('Diabetes Risk')\n",
"plt.scatter(diabetes_X_train, diabetes_y_train)\n",
"plt.scatter(diabetes_X_test, diabetes_y_test, color='red', marker='o')\n",
"plt.plot(diabetes_X_train, diabetes_y_train_pred, color='black', linewidth=1)\n",
"plt.plot(diabetes_X_test, diabetes_y_test_pred, 'x', color='red', mew=3, markersize=8)\n",
"plt.legend(['Model', 'Prediction', 'Initial patients', 'New patients'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"For each patient, we look up their BMI $x$ and compute the value $f(x)$ of the linear model $f$ at $x$. On the above figure, $f(x)$ is denoted by a red cross.\n",
"\n",
"We can compare the predicted value of $f(x)$ to the known true risk $y$ (which the model didn't see, and which is denoted by a red circle). The model is especially accurate on the leftmost patient: the prediction $f(x)$ and the true $y$ almost overlap. The model is somewhat off on the other two points---however, it still correctly identifies them as being at a higher risk."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"## 2.1.5 Why Supervised Learning?\n",
"\n",
"We have just seen a simple example of a supervised learning algorithm. Again, we want to emphasize that supervised learning is a tool that is applicable on tasks such as:\n",
"\n",
"* Predictive modeling. As we gather more data to characterize a patient (their age, gender, historical blood pressure, medical notes, etc), supervised learning can often outperform even human experts.\n",
"* Understanding the mechanisms through which input variables affect targets. Instead of using the predictions from the model, we may investigate the model itself. In the above example, we inspected the slope of the model, and noted that it was positive. Thus, we have inferred from data that a high BMI tends to increase diabetes risk."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"More generally, supervised learning finds applications in many areas. In fact, many of the most important applications of machine learning are supervised:\n",
"\n",
"* Classifying medical images. Similarly to how we predicted risk of BMI we may, for example, predict the severity of a cancer tumor from its image.\n",
"* Translating between pairs of languages. Most machine translation systems these days are created by training a supervised learning model on large datasets consisting of pairs of sentences in different languages.\n",
"* Detecting objects in a self-driving car. Again, we can explain to an algorithm what defines a car by providing many examples of cars. This enables the algorithm to detect new cars.\n",
"\n",
"We will see many more examples in this course."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# 2.2. Anatomy of a Supervised Learning Problem: The Dataset\n",
"\n",
"Let's now examine more closely the components of a supervised learning problem, starting with the dataset."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
"## 2.2.1. What is a Supervised Learning Dataset?\n",
"\n",
"We will again use the UCI Diabetes Dataset as our running example."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"The UCI dataset contains many additional data columns besides `bmi`, including age, sex, and blood pressure. We can ask `sklearn` to give us more information about this dataset."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
".. _diabetes_dataset:\n",
"\n",
"Diabetes dataset\n",
"----------------\n",
"\n",
"Ten baseline variables, age, sex, body mass index, average blood\n",
"pressure, and six blood serum measurements were obtained for each of n =\n",
"442 diabetes patients, as well as the response of interest, a\n",
"quantitative measure of disease progression one year after baseline.\n",
"\n",
"**Data Set Characteristics:**\n",
"\n",
" :Number of Instances: 442\n",
"\n",
" :Number of Attributes: First 10 columns are numeric predictive values\n",
"\n",
" :Target: Column 11 is a quantitative measure of disease progression one year after baseline\n",
"\n",
" :Attribute Information:\n",
" - age age in years\n",
" - sex\n",
" - bmi body mass index\n",
" - bp average blood pressure\n",
" - s1 tc, T-Cells (a type of white blood cells)\n",
" - s2 ldl, low-density lipoproteins\n",
" - s3 hdl, high-density lipoproteins\n",
" - s4 tch, thyroid stimulating hormone\n",
" - s5 ltg, lamotrigine\n",
" - s6 glu, blood sugar level\n",
"\n",
"Note: Each of these 10 feature variables have been mean centered and scaled by the standard deviation times `n_samples` (i.e. the sum of squares of each column totals 1).\n",
"\n",
"Source URL:\n",
"https://www4.stat.ncsu.edu/~boos/var.select/diabetes.html\n",
"\n",
"For more information see:\n",
"Bradley Efron, Trevor Hastie, Iain Johnstone and Robert Tibshirani (2004) \"Least Angle Regression,\" Annals of Statistics (with discussion), 407-499.\n",
"(https://web.stanford.edu/~hastie/Papers/LARS/LeastAngle_2002.pdf)\n"
]
}
],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"plt.rcParams['figure.figsize'] = [12, 4]\n",
"from sklearn import datasets\n",
"\n",
"# Load the diabetes dataset\n",
"diabetes = datasets.load_diabetes(as_frame=True)\n",
"print(diabetes.DESCR)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 2.2.2. A Supervised Learning Dataset: Notation\n",
"\n",
"We say that a training dataset of size $n$ (e.g., $n$ patients) is a set\n",
"\n",
"$$\n",
"\\mathcal{D} = \\{(x^{(i)}, y^{(i)}) \\mid i = 1,2,...,n\\}$$\n",
"\n",
"Each $x^{(i)}$ denotes an input (e.g., the measurements for patient $i$), and each $y^{(i)} \\in \\mathcal{Y}$ is a target (e.g., the diabetes risk). You may think of $x^{(i)}$ as a column vector containing numbers useful for predicting $y^{(i)}$.\n",
"\n",
"Together, $(x^{(i)}, y^{(i)})$ form a *training example*."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"We can look at the diabetes dataset in this form."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
age
\n",
"
sex
\n",
"
bmi
\n",
"
bp
\n",
"
s1
\n",
"
s2
\n",
"
s3
\n",
"
s4
\n",
"
s5
\n",
"
s6
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
0.038076
\n",
"
0.050680
\n",
"
0.061696
\n",
"
0.021872
\n",
"
-0.044223
\n",
"
-0.034821
\n",
"
-0.043401
\n",
"
-0.002592
\n",
"
0.019908
\n",
"
-0.017646
\n",
"
\n",
"
\n",
"
1
\n",
"
-0.001882
\n",
"
-0.044642
\n",
"
-0.051474
\n",
"
-0.026328
\n",
"
-0.008449
\n",
"
-0.019163
\n",
"
0.074412
\n",
"
-0.039493
\n",
"
-0.068330
\n",
"
-0.092204
\n",
"
\n",
"
\n",
"
2
\n",
"
0.085299
\n",
"
0.050680
\n",
"
0.044451
\n",
"
-0.005671
\n",
"
-0.045599
\n",
"
-0.034194
\n",
"
-0.032356
\n",
"
-0.002592
\n",
"
0.002864
\n",
"
-0.025930
\n",
"
\n",
"
\n",
"
3
\n",
"
-0.089063
\n",
"
-0.044642
\n",
"
-0.011595
\n",
"
-0.036656
\n",
"
0.012191
\n",
"
0.024991
\n",
"
-0.036038
\n",
"
0.034309
\n",
"
0.022692
\n",
"
-0.009362
\n",
"
\n",
"
\n",
"
4
\n",
"
0.005383
\n",
"
-0.044642
\n",
"
-0.036385
\n",
"
0.021872
\n",
"
0.003935
\n",
"
0.015596
\n",
"
0.008142
\n",
"
-0.002592
\n",
"
-0.031991
\n",
"
-0.046641
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" age sex bmi bp s1 s2 s3 \\\n",
"0 0.038076 0.050680 0.061696 0.021872 -0.044223 -0.034821 -0.043401 \n",
"1 -0.001882 -0.044642 -0.051474 -0.026328 -0.008449 -0.019163 0.074412 \n",
"2 0.085299 0.050680 0.044451 -0.005671 -0.045599 -0.034194 -0.032356 \n",
"3 -0.089063 -0.044642 -0.011595 -0.036656 0.012191 0.024991 -0.036038 \n",
"4 0.005383 -0.044642 -0.036385 0.021872 0.003935 0.015596 0.008142 \n",
"\n",
" s4 s5 s6 \n",
"0 -0.002592 0.019908 -0.017646 \n",
"1 -0.039493 -0.068330 -0.092204 \n",
"2 -0.002592 0.002864 -0.025930 \n",
"3 0.034309 0.022692 -0.009362 \n",
"4 -0.002592 -0.031991 -0.046641 "
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Load the diabetes dataset\n",
"diabetes_X, diabetes_y = diabetes.data, diabetes.target\n",
"\n",
"# Print part of the dataset\n",
"diabetes_X.head()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"In this example, each row $i$ contains the vector $(x^{(i)})^\\top$.\n",
"\n",
"Note that some numbers are unusual (e.g., age is negative). This is because `scikit-learn` normalized this data to have mean zero and standard deviation one."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 2.2.3. Training Dataset: Inputs\n",
"\n",
"More precisely, an input $x^{(i)} \\in \\mathcal{X}$ is a $d$-dimensional vector of the form\n",
"\n",
"$$ \n",
"x^{(i)} = \\begin{bmatrix}\n",
"x^{(i)}_1 \\\\\n",
"x^{(i)}_2 \\\\\n",
"\\vdots \\\\\n",
"x^{(i)}_d\n",
"\\end{bmatrix}$$\n",
"\n",
"For example, $x^{(i)}$ could be vector of attributes describing patient $i$. One of these attributes could be their BMI.\n",
"\n",
"The set $\\mathcal{X}$ is called the feature space. Often, we have, $\\mathcal{X} = \\mathbb{R}^d$."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Let's look at data for one patient."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"age 0.038076\n",
"sex 0.050680\n",
"bmi 0.061696\n",
"bp 0.021872\n",
"s1 -0.044223\n",
"s2 -0.034821\n",
"s3 -0.043401\n",
"s4 -0.002592\n",
"s5 0.019908\n",
"s6 -0.017646\n",
"Name: 0, dtype: float64"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"diabetes_X.iloc[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This indeed looks like a column vector."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Attributes\n",
"\n",
"We refer to the numerical variables describing the patient as *attributes*. Examples of attributes include:\n",
"* The age of a patient.\n",
"* The patient's gender.\n",
"* The patient's BMI.\n",
"\n",
"In other applications of machine learning, a vector of attributes $x$ could be comprised of all the pixel values of an image or it could contain indicator variables for whether certain words are present or absent in an input sentence."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Features\n",
"\n",
"Often, an input object has many attributes, and we want to use these attributes to define more complex characteristics of the input.\n",
"\n",
"* Is the patient old and a man? (Useful if old men are at risk).\n",
"* Is the BMI above the obesity threshold?\n",
"\n",
"We call these custom attributes *features*."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Let's create an \"old man\" feature."
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
age
\n",
"
sex
\n",
"
bmi
\n",
"
bp
\n",
"
s1
\n",
"
s2
\n",
"
s3
\n",
"
s4
\n",
"
s5
\n",
"
s6
\n",
"
old_man
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
0.038076
\n",
"
0.050680
\n",
"
0.061696
\n",
"
0.021872
\n",
"
-0.044223
\n",
"
-0.034821
\n",
"
-0.043401
\n",
"
-0.002592
\n",
"
0.019908
\n",
"
-0.017646
\n",
"
False
\n",
"
\n",
"
\n",
"
1
\n",
"
-0.001882
\n",
"
-0.044642
\n",
"
-0.051474
\n",
"
-0.026328
\n",
"
-0.008449
\n",
"
-0.019163
\n",
"
0.074412
\n",
"
-0.039493
\n",
"
-0.068330
\n",
"
-0.092204
\n",
"
False
\n",
"
\n",
"
\n",
"
2
\n",
"
0.085299
\n",
"
0.050680
\n",
"
0.044451
\n",
"
-0.005671
\n",
"
-0.045599
\n",
"
-0.034194
\n",
"
-0.032356
\n",
"
-0.002592
\n",
"
0.002864
\n",
"
-0.025930
\n",
"
True
\n",
"
\n",
"
\n",
"
3
\n",
"
-0.089063
\n",
"
-0.044642
\n",
"
-0.011595
\n",
"
-0.036656
\n",
"
0.012191
\n",
"
0.024991
\n",
"
-0.036038
\n",
"
0.034309
\n",
"
0.022692
\n",
"
-0.009362
\n",
"
False
\n",
"
\n",
"
\n",
"
4
\n",
"
0.005383
\n",
"
-0.044642
\n",
"
-0.036385
\n",
"
0.021872
\n",
"
0.003935
\n",
"
0.015596
\n",
"
0.008142
\n",
"
-0.002592
\n",
"
-0.031991
\n",
"
-0.046641
\n",
"
False
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" age sex bmi bp s1 s2 s3 \\\n",
"0 0.038076 0.050680 0.061696 0.021872 -0.044223 -0.034821 -0.043401 \n",
"1 -0.001882 -0.044642 -0.051474 -0.026328 -0.008449 -0.019163 0.074412 \n",
"2 0.085299 0.050680 0.044451 -0.005671 -0.045599 -0.034194 -0.032356 \n",
"3 -0.089063 -0.044642 -0.011595 -0.036656 0.012191 0.024991 -0.036038 \n",
"4 0.005383 -0.044642 -0.036385 0.021872 0.003935 0.015596 0.008142 \n",
"\n",
" s4 s5 s6 old_man \n",
"0 -0.002592 0.019908 -0.017646 False \n",
"1 -0.039493 -0.068330 -0.092204 False \n",
"2 -0.002592 0.002864 -0.025930 True \n",
"3 0.034309 0.022692 -0.009362 False \n",
"4 -0.002592 -0.031991 -0.046641 False "
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"diabetes_X['old_man'] = (diabetes_X['sex'] > 0) & (diabetes_X['age'] > 0.05)\n",
"diabetes_X.head()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Formally, we may denote features via a function $\\phi : \\mathcal{X} \\to \\mathbb{R}^p$ that takes an input $x^{(i)} \\in \\mathcal{X}$ and outputs a $p$-dimensional vector\n",
"\n",
"$$ \n",
"\\phi(x^{(i)}) = \\left[\\begin{array}{@{}c@{}}\n",
"\\phi(x^{(i)})_1 \\\\\n",
"\\phi(x^{(i)})_2 \\\\\n",
"\\vdots \\\\\n",
"\\phi(x^{(i)})_p\n",
"\\end{array} \\right]\n",
"$$\n",
"\n",
"We say that $\\phi(x^{(i)})$ is a *featurized* input, and each $\\phi(x^{(i)})_j$ is a *feature*."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"#### Features vs Attributes\n",
"\n",
"In practice, the terms attribute and features are often used interchangeably. Most authors refer to $x^{(i)}$ as a vector of features. \n",
"We will follow this convention and use the term \"attribute\" only when there is ambiguity between features and attributes."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Features: Discrete vs. Continuous \n",
"\n",
"Features can be either discrete or continuous. We will see that some ML algorithms handle these differently."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"The BMI feature that we have seen earlier is an example of a continuous feature.\n",
"We can visualize its distribution."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAsUAAAD4CAYAAAAEsJtCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAASwUlEQVR4nO3de4xc53kf4N9bMbYVM9XFcreCpGZlWHXhmGgSbR23QoOlFcSylVoCahgKBId0VRAFklRoWMBMjUJAgaA0CtdwgV5AxGkZoAntqEEliEkchfbm8ofUkI5rWlZV0QqdiJClXCjFdIWkbL/+sUfoasUlV3PZWc73PMBg5nxzLu/i5Zn54fDMOdVaCwAA9OwvzboAAACYNaEYAIDuCcUAAHRPKAYAoHtCMQAA3dsx6wKS5LrrrmuLi4sbvv/tb387b37zm7euIKZOT+eLfs4fPZ0/ejpf9HN0J06c+OPW2lvXj2+LULy4uJjjx49v+P7KykqWl5e3riCmTk/ni37OHz2dP3o6X/RzdFX1jQuNO30CAIDuXTIUV9XPVdULVfXVNWPXVtWjVfX08HzNMF5V9W+q6lRVfaWqvn+axQMAwCRs5kjxf0pyx7qxA0mOtdZuSXJsmE6S9ye5ZXjsS/LvJ1MmAABMzyVDcWvtt5L86brhu5IcHl4fTnL3mvGfb6seS3J1VV0/oVoBAGAqqrV26ZmqFpM80lp71zD9Ymvt6uF1JTnbWru6qh5JcrC19jvDe8eSfKy19ppf0VXVvqweTc7CwsKtR44c2XD7586dy86dO1/nn8Z2pqfzRT/nj57OHz2dL/o5ut27d59orS2tHx/76hOttVZVl07Wr13uUJJDSbK0tNQu9gtKv7CcP3o6X/Rz/ujp/NHT+aKfkzfq1Seef+W0iOH5hWH8TJKb1sx34zAGAADb1qih+OEke4bXe5I8tGb8x4arULwnyUuttefGrBEAAKbqkqdPVNUvJllOcl1VPZvkgSQHk3yuqu5L8o0kHx5m/5UkH0hyKsn/SvLRKdQMAAATdclQ3Fr70Q3euv0C87YkPz5uUTBJiweOzrqELXf64J2zLgEALivuaAcAQPeEYgAAuicUAwDQPaEYAIDuCcUAAHRPKAYAoHtCMQAA3ROKAQDonlAMAED3hGIAALonFAMA0D2hGACA7gnFAAB0TygGAKB7QjEAAN0TigEA6N6OWRcATN7igaNbur39u85n7xZvc73TB++c6fYBuLw5UgwAQPeEYgAAuicUAwDQPaEYAIDuCcUAAHRPKAYAoHtCMQAA3ROKAQDonlAMAED3hGIAALonFAMA0D2hGACA7gnFAAB0TygGAKB7QjEAAN0TigEA6J5QDABA98YKxVX1T6rqiar6alX9YlW9qapurqrHq+pUVX22qt4wqWIBAGAaRg7FVXVDkn+cZKm19q4kVyS5J8knknyqtfb2JGeT3DeJQgEAYFrGPX1iR5Irq2pHku9M8lyS9yZ5cHj/cJK7x9wGAABMVbXWRl+46v4kP5Pk5SS/nuT+JI8NR4lTVTcl+dXhSPL6Zfcl2ZckCwsLtx45cmTD7Zw7dy47d+4cuU62n63s6ckzL23Jdnq2cGXy/MuzrWHXDVfNtoA543N3/ujpfNHP0e3evftEa21p/fiOUVdYVdckuSvJzUleTPJLSe7Y7PKttUNJDiXJ0tJSW15e3nDelZWVXOx9Lj9b2dO9B45uyXZ6tn/X+Xzy5MgfJxNx+t7lmW5/3vjcnT96Ol/0c/LGOX3ih5L8fmvtj1pr/zvJLye5LcnVw+kUSXJjkjNj1ggAAFM1Tij+gyTvqarvrKpKcnuSryX5YpIPDfPsSfLQeCUCAMB0jRyKW2uPZ/UHdV9KcnJY16EkH0vyU1V1KslbknxmAnUCAMDUjHUSYGvtgSQPrBt+Jsm7x1kvAABsJXe0AwCge0IxAADdE4oBAOieUAwAQPeEYgAAuicUAwDQPaEYAIDuCcUAAHRPKAYAoHtCMQAA3ROKAQDonlAMAED3hGIAALonFAMA0D2hGACA7gnFAAB0TygGAKB7QjEAAN0TigEA6J5QDABA94RiAAC6JxQDANA9oRgAgO7tmHUBbK3FA0dnXUKSZP+u89m7TWoBAHCkGACA7gnFAAB0TygGAKB7QjEAAN3zQztgLmyXH5FupdMH75x1CQBzw5FiAAC6JxQDANA9oRgAgO4JxQAAdG+sUFxVV1fVg1X1P6rqyar621V1bVU9WlVPD8/XTKpYAACYhnGPFH86ya+11v5Gkr+Z5MkkB5Ica63dkuTYMA0AANvWyKG4qq5K8oNJPpMkrbW/aK29mOSuJIeH2Q4nuXu8EgEAYLqqtTbaglXfm+RQkq9l9SjxiST3JznTWrt6mKeSnH1let3y+5LsS5KFhYVbjxw5suG2zp07l507d45UJ6928sxLsy4hSbJwZfL8y7OugknRz9nYdcNVU1u3z935o6fzRT9Ht3v37hOttaX14+OE4qUkjyW5rbX2eFV9OsmfJfnJtSG4qs621i56XvHS0lI7fvz4hu+vrKxkeXl5pDp5te1yg4P9u87nkyfdO2Ze6OdsTPPmHT5354+ezhf9HF1VXTAUj3NO8bNJnm2tPT5MP5jk+5M8X1XXDxu9PskLY2wDAACmbuRQ3Fr7ZpI/rKp3DEO3Z/VUioeT7BnG9iR5aKwKAQBgysb9/86fTPKfq+oNSZ5J8tGsBu3PVdV9Sb6R5MNjbgMAAKZqrFDcWvtykteck5HVo8YAAHBZcEc7AAC6JxQDANA9oRgAgO4JxQAAdE8oBgCge0IxAADdE4oBAOieUAwAQPeEYgAAuicUAwDQPaEYAIDuCcUAAHRPKAYAoHtCMQAA3ROKAQDo3o5ZFwDAaBYPHJ3auvfvOp+9U1z/qE4fvHPWJQBzypFiAAC6JxQDANA9oRgAgO4JxQAAdE8oBgCge0IxAADdE4oBAOieUAwAQPeEYgAAuicUAwDQPaEYAIDuCcUAAHRPKAYAoHtCMQAA3ROKAQDonlAMAED3hGIAALo3diiuqiuq6veq6pFh+uaqeryqTlXVZ6vqDeOXCQAA0zOJI8X3J3lyzfQnknyqtfb2JGeT3DeBbQAAwNSMFYqr6sYkdyb52WG6krw3yYPDLIeT3D3ONgAAYNqqtTb6wlUPJvmXSb4ryT9NsjfJY8NR4lTVTUl+tbX2rgssuy/JviRZWFi49ciRIxtu59y5c9m5c+fIdfL/nTzz0qxLSJIsXJk8//Ksq2BS9HP+bNee7rrhqlmXcNnyXTpf9HN0u3fvPtFaW1o/vmPUFVbVjyR5obV2oqqWX+/yrbVDSQ4lydLSUlte3ngVKysrudj7bN7eA0dnXUKSZP+u8/nkyZH/+bHN6Of82a49PX3v8qxLuGz5Lp0v+jl543zi3Zbkg1X1gSRvSvKXk3w6ydVVtaO1dj7JjUnOjF8mAABMz8jnFLfWfrq1dmNrbTHJPUm+0Fq7N8kXk3xomG1PkofGrhIAAKZoGtcp/liSn6qqU0nekuQzU9gGAABMzEROGGutrSRZGV4/k+Tdk1gvAABsBXe0AwCge0IxAADdE4oBAOieUAwAQPeEYgAAuicUAwDQPaEYAIDuCcUAAHRPKAYAoHtCMQAA3ROKAQDo3o5ZFzBriweOzroEAABmzJFiAAC6JxQDANA9oRgAgO4JxQAAdE8oBgCge0IxAADdE4oBAOieUAwAQPeEYgAAuicUAwDQPaEYAIDuCcUAAHRPKAYAoHtCMQAA3ROKAQDonlAMAED3hGIAALonFAMA0L0dsy4AADZr8cDRWZew5U4fvHPWJUAXHCkGAKB7QjEAAN0bORRX1U1V9cWq+lpVPVFV9w/j11bVo1X19PB8zeTKBQCAyRvnSPH5JPtba+9M8p4kP15V70xyIMmx1totSY4N0wAAsG2NHIpba8+11r40vP5WkieT3JDkriSHh9kOJ7l7zBoBAGCqJnJOcVUtJvm+JI8nWWitPTe89c0kC5PYBgAATEu11sZbQdXOJL+Z5Gdaa79cVS+21q5e8/7Z1tprziuuqn1J9iXJwsLCrUeOHNlwG+fOncvOnTvHqnMjJ8+8NJX1cnELVybPvzzrKpgU/Zw/erp97LrhqomsZ5rfpWw9/Rzd7t27T7TWltaPjxWKq+o7kjyS5POttX89jD2VZLm19lxVXZ9kpbX2joutZ2lpqR0/fnzD91dWVrK8vDxynRfT4zUvt4P9u87nkyddJnte6Of80dPtY1LXKZ7mdylbTz9HV1UXDMXjXH2iknwmyZOvBOLBw0n2DK/3JHlo1G0AAMBWGOcwwG1JPpLkZFV9eRj7Z0kOJvlcVd2X5BtJPjxWhQAAMGUjh+LW2u8kqQ3evn3U9QIAwFZzRzsAALonFAMA0D2hGACA7gnFAAB0TygGAKB7QjEAAN0TigEA6J5QDABA94RiAAC6JxQDANA9oRgAgO7tmHUBAMDGFg8cnch69u86n70TWtc0nT5456xLoFOOFAMA0D2hGACA7gnFAAB0TygGAKB7QjEAAN0TigEA6J5QDABA94RiAAC6JxQDANA9oRgAgO4JxQAAdE8oBgCge0IxAADdE4oBAOjejlkXAADwisUDR2ddwpY7ffDOWZdAHCkGAAChGAAAhGIAALonFAMA0D2hGACA7gnFAAB0zyXZAABmaJTL0O3fdT57L+PL123Hy9BN5UhxVd1RVU9V1amqOjCNbQAAwKRMPBRX1RVJ/m2S9yd5Z5Ifrap3Tno7AAAwKdM4UvzuJKdaa8+01v4iyZEkd01hOwAAMBHVWpvsCqs+lOSO1to/HKY/kuQHWms/sW6+fUn2DZPvSPLURVZ7XZI/nmihzJqezhf9nD96On/0dL7o5+i+u7X21vWDM/uhXWvtUJJDm5m3qo631pamXBJbSE/ni37OHz2dP3o6X/Rz8qZx+sSZJDetmb5xGAMAgG1pGqH4d5PcUlU3V9UbktyT5OEpbAcAACZi4qdPtNbOV9VPJPl8kiuS/Fxr7YkxV7up0yy4rOjpfNHP+aOn80dP54t+TtjEf2gHAACXG7d5BgCge0IxAADd2zahuKqurapHq+rp4fmaDeb7tap6saoeWTd+c1U9Ptxa+rPDj/yYkdfRzz3DPE9X1Z414yvDrcK/PDz+ytZVz1qXum17Vb1x2OdODfvg4pr3fnoYf6qq3relhXNBo/azqhar6uU1++R/2PLiuaBN9PQHq+pLVXV+uJfA2vcu+BnMbI3Z0/+zZj91oYPXYduE4iQHkhxrrd2S5NgwfSH/KslHLjD+iSSfaq29PcnZJPdNpUo265L9rKprkzyQ5AeyeifEB9aF53tba987PF7YiqJ5tU3etv2+JGeHfe9TWd0XM8x3T5LvSXJHkn83rI8ZGaefg6+v2Sf/0ZYUzUVtsqd/kGRvkl9Yt+ylPoOZgXF6Onh5zX76wakWO2e2Uyi+K8nh4fXhJHdfaKbW2rEk31o7VlWV5L1JHrzU8myZzfTzfUkeba39aWvtbJJHsxqe2D42c9v2tb1+MMntwz55V5IjrbU/b639fpJTw/qYnXH6yfZ0yZ621k631r6S5P+uW9Zn8PY0Tk8Zw3YKxQutteeG199MsvA6ln1Lkhdba+eH6WeT3DDJ4njdNtPPG5L84Zrp9X37j8N///xzX8ozc6kevWqeYR98Kav75GaWZWuN088kubmqfq+qfrOq/u60i2VTxtnP7KPb07h9eVNVHa+qx6rq7olWNue29DbPVfUbSf7qBd76+NqJ1lqrKteK2+am3M97W2tnquq7kvyXrJ4y8/OjVQpMwHNJ/lpr7U+q6tYk/7Wqvqe19mezLgx4le8evj/fluQLVXWytfb1WRd1OdjSUNxa+6GN3quq56vq+tbac1V1fZLXcw7pnyS5uqp2DEc23Fp6C0ygn2eSLK+ZvjHJyrDuM8Pzt6rqF7L630lC8dbbzG3bX5nn2arakeSqrO6Tbvm+/Yzcz7Z6Ufs/T5LW2omq+nqSv57k+NSr5mLG2c82/Axmpsb67Fzz/flMVa0k+b4kQvEmbKfTJx5O8sovX/ckeWizCw4f1l9M8sovMF/X8kzFZvr5+SQ/XFXXDD/u+OEkn6+qHVV1XZJU1Xck+ZEkX92Cmnmtzdy2fW2vP5TkC8M++XCSe4arGdyc5JYk/22L6ubCRu5nVb31lR9KDkegbknyzBbVzcY209ONXPAzeEp1snkj93To5RuH19cluS3J16ZW6bxprW2LR1bPWTuW5Okkv5Hk2mF8KcnPrpnvt5P8UZKXs3qezfuG8bdl9Qv3VJJfSvLGWf9NPT9eRz//wdCzU0k+Ooy9OcmJJF9J8kSSTye5YtZ/U6+PJB9I8j+zeqTh48PYv0jyweH1m4Z97tSwD75tzbIfH5Z7Ksn7Z/23eIzezyR/f9gfv5zkS0n+3qz/Fo9N9/RvDd+X387q/+I8sWbZ13wGe8z+MWpPk/ydJCeT/Pfh+b5Z/y2X08NtngEA6N52On0CAABmQigGAKB7QjEAAN0TigEA6J5QDABA94RiAAC6JxQDANC9/wfiF1LiZIwXkQAAAABJRU5ErkJggg==",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"diabetes_X.loc[:, 'bmi'].hist()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Other features take on one of a finite number of discrete values. The `sex` column is an example of a categorical feature."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ 0.05068012 -0.04464164]\n"
]
},
{
"data": {
"text/plain": [
""
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAD4CAYAAADvhyBBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAARLUlEQVR4nO3de6xl51nf8e/TuEVppoqdpkxdO+1ExVJlsBrIKFDRSjMNhAREnbYoSoSKQ1O5f4ReU4mhSA3lUpmqAYS4qG6DML0wRBQUK+ai1GWKqATETiMcQ9O4iSMyDbYgxmJISmX69o+zHR3csWdybnvc8/lIR3utd717v8+e58zMb9ass9estQIAgOPuD227AAAAuBYIxgAAkGAMAACVYAwAAJVgDAAAVV237QKqXvrSl65Tp05tu4wj87u/+7u96EUv2nYZbIn+H2/6f3zp/fGm/9eOBx988DfXWn/icseuiWB86tSpHnjggW2XcWQuXLjQmTNntl0GW6L/x5v+H196f7zp/7VjZj72bMdcSgEAAAnGAABQCcYAAFAJxgAAUAnGAABQCcYAAFAJxgAAUAnGAABQCcYAAFBdI3e+26ZT5+478jXfdttTvXkL6z7t0bu+amtrAwBcq5wxBgCABGMAAKgEYwAAqARjAACoBGMAAKgEYwAAqARjAACoBGMAAKgEYwAAqARjAACoBGMAAKgEYwAAqARjAACoBGMAAKgEYwAAqARjAACoBGMAAKgEYwAAqARjAACoBGMAAKgEYwAAqARjAACoBGMAAKjqum0XAABwnJ06d9+2Szhyj971Vdsu4bKcMQYAgARjAACoriIYz8zLZubnZuZXZ+bhmfl7m/GXzMx7Z+bDm8cbNuMzM987M4/MzK/MzBcd9psAAID9upozxk9Vb1tr3Vp9SfXWmbm1Olfdv9a6pbp/s1/1uuqWzded1Q8eeNUAAHDArhiM11qfWGu9f7P9O9WvVTdVt1f3bKbdU71+s3179SNrxy9W18/MjQddOAAAHKTP6hrjmTlVfWH1S9XJtdYnNod+ozq52b6p+vVdT/v4ZgwAAK5Zs9a6uokzJ6r/XH3HWusnZua311rX7zr+xFrrhpl5T3XXWusXNuP3V9+41nrgGa93ZzuXWnTy5MlXnj9//kDe0GfroYtPHvmaJ19Yj336yJf9jNtuevH2FqdLly514sSJbZfBluj/8aX3x9tz9X8bWWTbtplFzp49++Ba6/Tljl3V5xjPzB+u/kP179ZaP7EZfmxmblxrfWJzqcTjm/GL1ct2Pf3mzdgfsNa6u7q76vTp0+vMmTNXU8qBe/MWPjvwbbc91Tse2t5HSD/6tWe2tjZ14cKFtvX9zvbp//Gl98fbc/V/G1lk267VLHI1n0ox1TurX1trfdeuQ/dWd2y276jevWv86zafTvEl1ZO7LrkAAIBr0tWctvzS6m9UD83MBzZj/7i6q3rXzLyl+lj1hs2xn6q+snqk+lT19QdZMAAAHIYrBuPNtcLzLIdffZn5q3rrPusCAIAj5c53AACQYAwAAJVgDAAAlWAMAACVYAwAAJVgDAAAlWAMAACVYAwAAJVgDAAAlWAMAACVYAwAAJVgDAAAlWAMAACVYAwAAJVgDAAAlWAMAACVYAwAAJVgDAAAlWAMAACVYAwAAJVgDAAAlWAMAACVYAwAAJVgDAAAlWAMAACVYAwAAJVgDAAAlWAMAACVYAwAAJVgDAAAlWAMAACVYAwAAJVgDAAAlWAMAACVYAwAANVVBOOZ+aGZeXxmPrhr7Ftm5uLMfGDz9ZW7jn3TzDwyMx+ama84rMIBAOAgXc0Z4x+uXnuZ8e9ea71i8/VTVTNza/XG6vM3z/mBmXnBQRULAACH5YrBeK3189Unr/L1bq/Or7V+b6310eqR6lX7qA8AAI7Efq4x/oaZ+ZXNpRY3bMZuqn5915yPb8YAAOCaNmutK0+aOVW9Z631BZv9k9VvVqv6turGtdbfnJnvq35xrfVvN/PeWf30WuvHL/Oad1Z3Vp08efKV58+fP5h39Fl66OKTR77myRfWY58+8mU/47abXry9xenSpUudOHFi22WwJfp/fOn98fZc/d9GFtm2bWaRs2fPPrjWOn25Y9ft5QXXWo89vT0z/6p6z2b3YvWyXVNv3oxd7jXuru6uOn369Dpz5sxeStm3N5+778jXfNttT/WOh/b0S38gHv3aM1tbm7pw4ULb+n5n+/T/+NL74+25+r+NLLJt12oW2dOlFDNz467dv1o9/YkV91ZvnJnPmZmXV7dUv7y/EgEA4PBd8bTlzPxodaZ66cx8vHp7dWZmXtHOpRSPVn+7aq318My8q/rV6qnqrWut3z+UygEA4ABdMRivtd50meF3Psf876i+Yz9FAQDAUXPnOwAASDAGAIBKMAYAgEowBgCASjAGAIBKMAYAgEowBgCASjAGAIBKMAYAgEowBgCASjAGAIBKMAYAgEowBgCASjAGAIBKMAYAgEowBgCASjAGAIBKMAYAgEowBgCASjAGAIBKMAYAgEowBgCASjAGAIBKMAYAgEowBgCASjAGAIBKMAYAgEowBgCASjAGAIBKMAYAgEowBgCASjAGAIBKMAYAgEowBgCASjAGAIBKMAYAgOoqgvHM/NDMPD4zH9w19pKZee/MfHjzeMNmfGbme2fmkZn5lZn5osMsHgAADsrVnDH+4eq1zxg7V92/1rqlun+zX/W66pbN153VDx5MmQAAcLiuGIzXWj9fffIZw7dX92y276lev2v8R9aOX6yun5kbD6hWAAA4NLPWuvKkmVPVe9ZaX7DZ/+211vWb7ameWGtdPzPvqe5aa/3C5tj91TeutR64zGve2c5Z5U6ePPnK8+fPH8w7+iw9dPHJI1/z5AvrsU8f+bKfcdtNL97e4nTp0qVOnDix7TLYEv0/vvT+eHuu/m8ji2zbNrPI2bNnH1xrnb7csev2++JrrTUzV07X/+/z7q7urjp9+vQ6c+bMfkvZkzefu+/I13zbbU/1jof2/Uu/Z49+7ZmtrU1duHChbX2/s336f3zp/fH2XP3fRhbZtms1i+z1Uykee/oSic3j45vxi9XLds27eTMGAADXtL0G43urOzbbd1Tv3jX+dZtPp/iS6sm11if2WSMAABy6K/5//sz8aHWmeunMfLx6e3VX9a6ZeUv1seoNm+k/VX1l9Uj1qerrD6FmAAA4cFcMxmutNz3LoVdfZu6q3rrfogAA4Ki58x0AACQYAwBAJRgDAEAlGAMAQCUYAwBAJRgDAEAlGAMAQCUYAwBAJRgDAEAlGAMAQCUYAwBAJRgDAEAlGAMAQCUYAwBAJRgDAEAlGAMAQCUYAwBAJRgDAEAlGAMAQCUYAwBAJRgDAEAlGAMAQCUYAwBAJRgDAEAlGAMAQCUYAwBAJRgDAEAlGAMAQCUYAwBAJRgDAEAlGAMAQCUYAwBAJRgDAEAlGAMAQCUYAwBAVdft58kz82j1O9XvV0+ttU7PzEuqH6tOVY9Wb1hrPbG/MgEA4HAdxBnjs2utV6y1Tm/2z1X3r7Vuqe7f7AMAwDXtMC6luL26Z7N9T/X6Q1gDAAAO1Ky19v7kmY9WT1Sr+pdrrbtn5rfXWtdvjk/1xNP7z3jundWdVSdPnnzl+fPn91zHfjx08ckjX/PkC+uxTx/5sp9x200v3t7idOnSpU6cOLHtMtgS/T++9P54e67+byOLbNs2s8jZs2cf3HWlwx+w32B801rr4sx8bvXe6u9U9+4OwjPzxFrrhud6ndOnT68HHnhgz3Xsx6lz9x35mm+77ane8dC+Lu/el0fv+qqtrU1duHChM2fObLsMtkT/jy+9P96eq//byCLbts0sMjPPGoz3dSnFWuvi5vHx6ierV1WPzcyNm4VvrB7fzxoAAHAU9hyMZ+ZFM/PHnt6uXlN9sLq3umMz7Y7q3fstEgAADtt+/j//ZPWTO5cRd13179daPzMz76veNTNvqT5WvWH/ZQIAwOHaczBea32k+vOXGf+t6tX7KQoAAI6aO98BAECCMQAAVIIxAABUgjEAAFSCMQAAVIIxAABUgjEAAFSCMQAAVIIxAABUgjEAAFSCMQAAVIIxAABUgjEAAFSCMQAAVIIxAABUgjEAAFSCMQAAVIIxAABUgjEAAFSCMQAAVIIxAABUgjEAAFSCMQAAVIIxAABUgjEAAFSCMQAAVIIxAABUgjEAAFSCMQAAVIIxAABUgjEAAFSCMQAAVIIxAABUgjEAAFSCMQAAVIcYjGfmtTPzoZl5ZGbOHdY6AABwEA4lGM/MC6rvr15X3Vq9aWZuPYy1AADgIBzWGeNXVY+stT6y1vrf1fnq9kNaCwAA9u26Q3rdm6pf37X/8eqLd0+YmTurOze7l2bmQ4dUyzXn79ZLq9/c1vrzndtamY2t9p+t0//jS++PN/3fZctZ5M8824HDCsZXtNa6u7p7W+tv08w8sNY6ve062A79P970//jS++NN/58fDutSiovVy3bt37wZAwCAa9JhBeP3VbfMzMtn5o9Ub6zuPaS1AABg3w7lUoq11lMz8w3Vz1YvqH5orfXwYaz1PHUsLyHhM/T/eNP/40vvjzf9fx6Ytda2awAAgK1z5zsAAEgwBgCASjA+NDPzkpl578x8ePN4w7PMu2Mz58Mzc8dljt87Mx88/Io5SPvp/8z80Zm5b2b+28w8PDN3HW317MXMvHZmPjQzj8zMucsc/5yZ+bHN8V+amVO7jn3TZvxDM/MVR1o4B2Kv/Z+ZL5+ZB2fmoc3jXz7y4tm3/fz+3xz/0zNzaWb+0ZEVzWUJxofnXHX/WuuW6v7N/h8wMy+p3t7OzU9eVb19d4Camb9WXTqacjlg++3/v1hr/bnqC6svnZnXHU3Z7MXMvKD6/up11a3Vm2bm1mdMe0v1xFrr86rvrr5z89xb2/nkns+vXlv9wOb1eJ7YT//bueHDV6+1bqvuqP7N0VTNQdln/5/2XdVPH3atXJlgfHhur+7ZbN9Tvf4yc76ieu9a65NrrSeq97bzF2Mzc6L6h9W3H36pHII993+t9am11s9VbW6p/v52Pguca9erqkfWWh/Z9Ox8O98Du+3+nvjx6tUzM5vx82ut31trfbR6ZPN6PH/suf9rrf+61vqfm/GHqxfOzOccSdUclP38/m9mXl99tJ3+s2WC8eE5udb6xGb7N6qTl5lzuVtn37TZ/rbqHdWnDq1CDtN++1/VzFxffXU7Z525dl2xl7vnrLWeqp6s/vhVPpdr2376v9tfr96/1vq9Q6qTw7Hn/m9Ogn1j9U+PoE6uwtZuCf3/g5n5j9WfvMyhb969s9ZaM3PVn4s3M6+o/uxa6x888zokrh2H1f9dr39d9aPV9661PrK3KoHng5n5/Hb+e/01266FI/Ut1XevtS5tTiCzZYLxPqy1vuzZjs3MYzNz41rrEzNzY/X4ZaZdrM7s2r+5ulD9her0zDzaTo8+d2YurLXOxDXjEPv/tLurD6+1vmf/1XLILlYv27V/82bscnM+vvlHz4ur37rK53Jt20//m5mbq5+svm6t9T8Ov1wO2H76/8XV18zMP6+ur/7PzPyvtdb3HXrVXJZLKQ7Pve38IEWbx3dfZs7PVq+ZmRs2P3T1mupn11o/uNb6U2utU9VfrP67UPy8s+f+V83Mt7fzB+ffP/xSOQDvq26ZmZfPzB9p54fp7n3GnN3fE19T/ae1c4ele6s3bn5q/eXVLdUvH1HdHIw9939zudR91bm11n85qoI5UHvu/1rrL621Tm3+vv+e6p8JxdslGB+eu6ovn5kPV1+22W9mTs/Mv65aa32ynWuJ37f5+tbNGM9/e+7/5uzRN7fz083vn5kPzMzf2sab4Opsrhn8hnb+YfNr1bvWWg/PzLfOzF/ZTHtnO9cUPtLOD9ae2zz34epd1a9WP1O9da31+0f9Hti7/fR/87zPq/7J5vf6B2bmc4/4LbAP++w/1xi3hAYAgJwxBgCASjAGAIBKMAYAgEowBgCASjAGAIBKMAYAgEowBgCAqv4vrDPHe537qgEAAAAASUVORK5CYII=",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"print(diabetes_X.loc[:, 'sex'].unique())\n",
"diabetes_X.loc[:, 'sex'].hist()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"In this example, the dataset has been pre-processed such that the sex attribute takes one of two possible values (these happen to be the somewhat arbitrary numerical values `0.05068012` and `-0.04464164`)."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 2.2.4. Targets\n",
"\n",
"For each patient, we may be interested in predicting a quantity of interest, the *target*. In our example, this is the patient's diabetes risk.\n",
"\n",
"Formally, when $(x^{(i)}, y^{(i)})$ form a *training example*, each $y^{(i)} \\in \\mathcal{Y}$ is a target. We call $\\mathcal{Y}$ the target space."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"We plot our distribution of risk scores below."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAs0AAAEGCAYAAACeiKhrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgNUlEQVR4nO3dfZRddX3v8feHp4pGCCjOokgNVq4WyZLKFKVaOwGtD6hQL7V6qRcot9F11YuV3mV0qdxba4t1odXWp/Rqja06IkKholZKGbz2KpIoGgFdIMbWFIPV8BDEh+D3/nF26HScmXMymX32ycz7tdass/f+7Yfv+WZn8s0+v/P7paqQJEmSNLd9ug5AkiRJGnUWzZIkSVIfFs2SJElSHxbNkiRJUh8WzZIkSVIf+3UdwCAe+tCH1qpVq7oOY69zzz338KAHPajrMJYd894dc98dc98dc98dc9+dtnK/adOmf6uqw2Zr2yuK5lWrVrFx48auw9jrTE1NMTEx0XUYy4557465746574657465705buU/yrbna7J4hSZIk9WHRLEmSJPVh0SxJkiT1YdEsSZIk9WHRLEmSJPVh0SxJkiT10WrRnOT3k9yQ5KtJPpzkAUmOSnJtkluSfCTJAW3GIEmSJO2p1ormJEcA/wMYr6pjgX2BFwBvAt5aVY8CtgPntBWDJEmStBja7p6xH3Bgkv2ABwK3AScBFzftG4DTWo5BkiRJ2iOpqvZOnpwLvBG4F/g0cC7w+eYpM0mOBD7ZPImeeexaYC3A2NjY8ZOTk63FuVTt2LGDFStW7NYxm7fe2VI0o2v1EQcv6vkWknctDnPfHXPfHXPfHXPfnbZyv2bNmk1VNT5bW2vTaCc5BDgVOAq4A/go8IxBj6+q9cB6gPHx8XKayt23kCkmz1p3RTvBjLAtZ0ws6vmcVrU75r475r475r475r47XeS+ze4ZTwW+WVXfraqfAJcATwJWNt01AB4ObG0xBkmSJGmPtVk0/zPwxCQPTBLgZOBG4Grg9GafM4HLWoxBkiRJ2mOtFc1VdS29L/x9EdjcXGs98CrglUluAR4CvLetGCRJkqTF0FqfZoCqOh84f8bmW4ET2ryuJEmStJicEVCSJEnqw6JZkiRJ6sOiWZIkSerDolmSJEnqw6JZkiRJ6sOiWZIkSerDolmSJEnqw6JZkiRJ6sOiWZIkSerDolmSJEnqw6JZkiRJ6sOiWZIkSerDolmSJEnqw6JZkiRJ6sOiWZIkSerDolmSJEnqo7WiOcmjk1w/7eeuJK9IcmiSK5Pc3Lwe0lYMkiRJ0mJorWiuqq9X1XFVdRxwPPAD4FJgHXBVVR0NXNWsS5IkSSNrWN0zTga+UVXfAk4FNjTbNwCnDSkGSZIkaUGGVTS/APhwszxWVbc1y98BxoYUgyRJkrQgqap2L5AcAPwr8Niq2pbkjqpaOa19e1X9TL/mJGuBtQBjY2PHT05OthrnUrRjxw5WrFixW8ds3npnS9GMrtVHHLyo51tI3rU4zH13zH13zH13zH132sr9mjVrNlXV+Gxt+y361X7WM4EvVtW2Zn1bksOr6rYkhwO3z3ZQVa0H1gOMj4/XxMTEEEJdWqamptjdvJ217op2ghlhW86YWNTzLSTvWhzmvjvmvjvmvjvmvjtd5H4Y3TNeyL93zQC4HDizWT4TuGwIMUiSJEkL1mrRnORBwNOAS6ZtvgB4WpKbgac265IkSdLIarV7RlXdAzxkxrbv0RtNQ5IkSdorDKNPszTSVi1yP+7zVu8c+b7hWy44pesQJEnaqziNtiRJktSHRbMkSZLUh0WzJEmS1IdFsyRJktSHRbMkSZLUh0WzJEmS1IdFsyRJktSHRbMkSZLUh0WzJEmS1IdFsyRJktSHRbMkSZLUh0WzJEmS1IdFsyRJktSHRbMkSZLUx35dByBp+Fatu6LrEFpx3uqdnDXHe9tywSlDjkaStJT4pFmSJEnqo9WiOcnKJBcn+VqSm5KcmOTQJFcmubl5PaTNGCRJkqQ91faT5rcBn6qqxwCPA24C1gFXVdXRwFXNuiRJkjSyWiuakxwMPAV4L0BV/biq7gBOBTY0u20ATmsrBkmSJGkxpKoG3znZB1hRVXcNsO9xwHrgRnpPmTcB5wJbq2pls0+A7bvWZxy/FlgLMDY2dvzk5OTAcapnx44drFixYreO2bz1zpaiWT7GDoRt93YdxfI0X+5XH3HwcINZZhby+0aLw9x3x9x3p63cr1mzZlNVjc/W1rdoTvIh4CXAfcB1wEHA26rqzX2OGwc+Dzypqq5N8jbgLuDl04vkJNurat5+zePj47Vx48Z549TPmpqaYmJiYreOWaqjKgzTeat3cuFmB6bpwny5d/SMdi3k940Wh7nvjrnvTlu5TzJn0TxI94xjmifLpwGfBI4CXjTAcd8Gvl1V1zbrFwOPB7YlObwJ7HDg9gHOJUmSJHVmkKJ5/yT70yuaL6+qnwxy4qr6DvAvSR7dbDqZXleNy4Ezm21nApftVsSSJEnSkA3yGfJ7gC3Al4HPJHkEMGjH15cDH0xyAHArcDa9Qv2iJOcA3wKev7tBS5IkScM0SNH8d1X19l0rSf4Z+N1BTl5V1wOz9Qs5eaDoJEmSpBEwSPeMj01fqd43Bx3KQpIkScvGnE+akzwGeCxwcJLnTWs6CHhA24FJkiRJo2K+7hmPBp4NrASeM2373cDvtRiTJEmSNFLmLJqr6jLgsiQnVtXnhhiTJEmSNFIG+SLgLUleA6yavn9VDfRlQEmSJGlvN0jRfBnwf4F/oDcroCRJkrSsDFI0P7CqXtV6JJIkSdKIGmTIuY8neVbrkUiSJEkjapCi+Vx6hfMPk9yV5O4kd7UdmCRJkjQq+nbPqKoHDyMQSZIkaVT1fdKcnt9J8rpm/cgkJ7QfmiRJkjQaBume8U7gROC/NOs7gHe0FpEkSZI0YgYZPeMJVfX4JF8CqKrtSQ5oOS5JkiRpZAzypPknSfYFCiDJYcBPW41KkiRJGiGDFM1vBy4FHpbkjcBngT9uNSpJkiRphAwyesYHk2wCTgYCnFZVN7UemSRJkjQi5iyakxxUVXclORS4HfjwtLZDq+r7wwhQkiRJ6tp8T5o/BDwb2ETTn7mRZv2R/U6eZAtwN3AfsLOqxpsi/CPAKmAL8Pyq2r6A2CVJkqShmLNorqpnN69H7eE11lTVv01bXwdcVVUXJFnXrL9qD68hSZIktWaQyU2uGmTbbjgV2NAsbwBO24NzSZIkSa1LVc3ekDwAeCBwNTBBr1sGwEHAp6rqMX1PnnwT2E6vO8d7qmp9kjuqamXTHmD7rvUZx64F1gKMjY0dPzk5uVtvTLBjxw5WrFixW8ds3npnS9EsH2MHwrZ7u45ieTL3/9HqIw4e2rUW8vtGi8Pcd8fcd6et3K9Zs2ZTVY3P1jZfn+YXA68Afp5ev+ZdRfNdwF8MeO0nV9XWJA8DrkzytemNVVVJZq3aq2o9sB5gfHy8JiYmBrykdpmammJ383bWuivaCWYZOW/1Ti7cPMi8QVps5v4/2nLGxNCutZDfN1oc5r475r47XeR+vj7NbwPeluTlVfXnCzl5VW1tXm9PcilwArAtyeFVdVuSw+mNzCFJkiSNrEHGaf7zJMcCxwAPmLb9A/Mdl+RBwD5VdXez/BvAHwKXA2cCFzSvly08fEmSJKl9fYvmJOfT69N8DPAJ4Jn0ZgWct2gGxoBLe92W2Q/4UFV9Ksl1wEVJzgG+BTx/wdFLkiRJQzBI57/TgccBX6qqs5OMAX/T76CqurU5bub279GbXVCSJEnaK/Qdcg64t6p+CuxMchC9PshHthuWJEmSNDoGedK8MclK4C/pjaKxA/hcm0FJkiRJo2SQLwL+92bx3Uk+BRxUVV9pNyxJkiRpdAw0oGmS5wFPpjdJyWcBi2ZJkiQtG4NMo/1O4CXAZuCrwIuTvKPtwCRJkqRRMciT5pOAX6pmvu0kG4AbWo1KkiRJGiGDjJ5xC/AL09aPbLZJkiRJy8IgT5ofDNyU5Av0+jSfQG9EjcsBquq5LcYnSZIkdW6Qovn1rUchSZIkjbBBhpy7ZhiBSJIkSaNqkD7NkiRJ0rJm0SxJkiT1MWfRnOSq5vVNwwtHkiRJGj3z9Wk+PMmvAs9NMglkemNVfbHVyCRJkqQRMV/R/HrgdcDDgbfMaCt6k55IkiRJS96cRXNVXQxcnOR1VfWGIcYkSZIGsGrdFZy3eidnrbui61CGZssFp3QdgpapQYace0OS5wJPaTZNVdXH2w1LkiRJGh19R89I8ifAucCNzc+5Sf540Ask2TfJl5J8vFk/Ksm1SW5J8pEkByw0eEmSJGkYBpkR8BTguKr6KUCSDcCXgNcMeI1zgZuAg5r1NwFvrarJJO8GzgHetVtRS5L6WjXEj+xHoYuAH9tLatOg4zSvnLZ88KAnT/JwekX3/2nWQ+8LhBc3u2wAThv0fJIkSVIXUlXz75C8ELgAuJresHNPAdZV1Uf6njy5GPgT4MHAHwBnAZ+vqkc17UcCn6yqY2c5di2wFmBsbOz4ycnJwd+VANixYwcrVqzYrWM2b72zpWiWj7EDYdu9XUexPJn77oxC7lcfMfAznSVj89Y7RyL3wzRKf84L+XdWi6Ot3K9Zs2ZTVY3P1jbIFwE/nGQK+JVm06uq6jv9jkvybOD2qtqUZGLwcO+/7npgPcD4+HhNTOz2KZa9qakpdjdvXX+8uhSct3onF24epOeTFpu5784o5H7LGROdXr8LZzWjZ3Sd+2EapT/nhfw7q8XRRe4H+ltWVbcBl+/muZ9Eb2KUZwEPoNen+W3AyiT7VdVOemNAb93N80qSJElDNWif5t1WVa+uqodX1SrgBcA/VtUZ9Lp5nN7sdiZwWVsxSJIkSYuhtaJ5Hq8CXpnkFuAhwHs7iEGSJEka2LzdM5LsC9xQVY/Zk4tU1RQw1SzfCpywJ+cblmEO19SGURgCSpKGZW//nS1ptM37pLmq7gO+nuQXhhSPJEmSNHIG+SLgIcANSb4A3LNrY1U9t7WoJEmSpBEySNH8utajkCRJkkbYIOM0X5PkEcDRVfUPSR4I7Nt+aJIkSdJo6Dt6RpLfozft9XuaTUcAf9tiTJIkSdJIGWTIuZfSm6jkLoCquhl4WJtBSZIkSaNkkKL5R1X1410rSfYDqr2QJEmSpNEySNF8TZLXAAcmeRrwUeDv2g1LkiRJGh2DFM3rgO8Cm4EXA58AXttmUJIkSdIoGWT0jJ8m2QBcS69bxteryu4ZkiRJWjb6Fs1JTgHeDXwDCHBUkhdX1SfbDk6SJEkaBYNMbnIhsKaqbgFI8ovAFYBFsyRJkpaFQfo0372rYG7cCtzdUjySJEnSyJnzSXOS5zWLG5N8AriIXp/m3wKuG0JskiRJ0kiYr3vGc6YtbwN+vVn+LnBgaxFJkiRJI2bOormqzh5mIJIkSdKoGmT0jKOAlwOrpu9fVc9tLyxJkiRpdAwyesbfAu+lNwvgTwc9cZIHAJ8Bfq65zsVVdX5ThE8CDwE2AS+aPk23JEmSNGoGKZp/WFVvX8C5fwScVFU7kuwPfDbJJ4FXAm+tqskk7wbOAd61gPNLkiRJQzHIkHNvS3J+khOTPH7XT7+DqmdHs7p/81PAScDFzfYNwGkLiFuSJEkamvSbETvJnwAvojcj4K7uGVVVJ/U9ebIvvS4YjwLeAbwZ+HxVPappPxL4ZFUdO8uxa4G1AGNjY8dPTk4O+p4Wzeatdw79motp7EDYdm/XUSw/5r075r475r47yy33q484uOsQ7rdjxw5WrFjRdRjLUlu5X7NmzaaqGp+tbZDuGb8FPHIh/Y6r6j7guCQrgUuBx+zGseuB9QDj4+M1MTGxu5ffY2etu2Lo11xM563eyYWbB/kj1mIy790x990x991ZbrnfcsZE1yHcb2pqii7qE3WT+0G6Z3wVWLknF6mqO4CrgROBlUl2/e1+OLB1T84tSZIktW2Q/5quBL6W5Dp6X+4D+g85l+Qw4CdVdUeSA4GnAW+iVzyfTm8EjTOByxYWuiRJkjQcgxTN5y/w3IcDG5p+zfsAF1XVx5PcCEwm+SPgS/SGs5MkSZJGVt+iuaquWciJq+orwC/Psv1W4ISFnFOSJEnqwiAzAt5Nb6g4gAPoDR13T1Ud1GZgkiRJ0qgY5Enzg3ctJwlwKvDENoOSJEmSRskgo2fcr5mw5G+Bp7cTjiRJkjR6Bume8bxpq/sA48APW4tIkiRJGjGDjJ7xnGnLO4Et9LpoSJIkScvCIH2azx5GIJIkSdKomrNoTvL6eY6rqnpDC/FIkiRJI2e+J833zLLtQcA5wEMAi2ZJkiQtC3MWzVV14a7lJA8GzgXOpjf99YVzHSdJkiQtNfP2aU5yKPBK4AxgA/D4qto+jMAkSZKkUTFfn+Y3A88D1gOrq2rH0KKSJEmSRsh8k5ucB/w88FrgX5Pc1fzcneSu4YQnSZIkdW++Ps27NVugJEmStFRZGEuSJEl9WDRLkiRJfVg0S5IkSX1YNEuSJEl9tFY0JzkyydVJbkxyQ5Jzm+2HJrkyyc3N6yFtxSBJkiQthjafNO8EzquqY4AnAi9NcgywDriqqo4GrmrWJUmSpJHVWtFcVbdV1Reb5buBm4AjgFPpzS5I83paWzFIkiRJiyFV1f5FklXAZ4BjgX+uqpXN9gDbd63POGYtsBZgbGzs+MnJydbjnGnz1juHfs3FNHYgbLu36yiWH/PeHXPfHXPfneWW+9VHHNx1CPfbsWMHK1as6DqMZamt3K9Zs2ZTVY3P1tZ60ZxkBXAN8MaquiTJHdOL5CTbq2refs3j4+O1cePGVuOczap1Vwz9movpvNU7uXDznPPXqCXmvTvmvjvmvjvLLfdbLjil6xDuNzU1xcTERNdhLEtt5T7JnEVzq6NnJNkf+Bjwwaq6pNm8LcnhTfvhwO1txiBJkiTtqTZHzwjwXuCmqnrLtKbLgTOb5TOBy9qKQZIkSVoMbX6e8yTgRcDmJNc3214DXABclOQc4FvA81uMQZIkSdpjrRXNVfVZIHM0n9zWdSVJkqTF5oyAkiRJUh8WzZIkSVIfy2eMGkmStNcbpeFgz1u9k7OGEM8oDbO3nPmkWZIkSerDolmSJEnqw6JZkiRJ6sOiWZIkSerDolmSJEnqw6JZkiRJ6sOiWZIkSerDolmSJEnqw6JZkiRJ6sOiWZIkSerDolmSJEnqw6JZkiRJ6sOiWZIkSerDolmSJEnqo7WiOcn7ktye5KvTth2a5MokNzevh7R1fUmSJGmxtPmk+f3AM2ZsWwdcVVVHA1c165IkSdJIa61orqrPAN+fsflUYEOzvAE4ra3rS5IkSYslVdXeyZNVwMer6thm/Y6qWtksB9i+a32WY9cCawHGxsaOn5ycbC3OuWzeeufQr7mYxg6Ebfd2HcXyY967Y+67Y+67Y+67M6zcrz7i4PYvspfZsWMHK1asWPTzrlmzZlNVjc/Wtt+iX21AVVVJ5qzYq2o9sB5gfHy8JiYmhhXa/c5ad8XQr7mYzlu9kws3d/ZHvGyZ9+6Y++6Y++6Y++4MK/dbzpho/Rp7m6mpKYZdGw579IxtSQ4HaF5vH/L1JUmSpN027KL5cuDMZvlM4LIhX1+SJEnabW0OOfdh4HPAo5N8O8k5wAXA05LcDDy1WZckSZJGWmsdcarqhXM0ndzWNSVJkqQ2OCOgJEmS1IdFsyRJktSHRbMkSZLUh0WzJEmS1IejoUuSJI2wVXv5ZGsLseWCU7oO4Wf4pFmSJEnqw6JZkiRJ6sOiWZIkSerDolmSJEnqw6JZkiRJ6sOiWZIkSerDolmSJEnqw6JZkiRJ6sOiWZIkSerDolmSJEnqw6JZkiRJ6sOiWZIkSeqjk6I5yTOSfD3JLUnWdRGDJEmSNKihF81J9gXeATwTOAZ4YZJjhh2HJEmSNKgunjSfANxSVbdW1Y+BSeDUDuKQJEmSBpKqGu4Fk9OBZ1TVf2vWXwQ8oapeNmO/tcDaZvXRwNeHGujS8FDg37oOYhky790x990x990x990x991pK/ePqKrDZmvYr4WLLYqqWg+s7zqOvVmSjVU13nUcy4157465746574657465704Xue+ie8ZW4Mhp6w9vtkmSJEkjqYui+Trg6CRHJTkAeAFweQdxSJIkSQMZeveMqtqZ5GXA3wP7Au+rqhuGHccyYfeWbpj37pj77pj77pj77pj77gw990P/IqAkSZK0t3FGQEmSJKkPi2ZJkiSpD4vmJSLJliSbk1yfZGOz7dAkVya5uXk9pOs4l4Ik70tye5KvTts2a67T8/ZmyvivJHl8d5Hv/ebI/f9KsrW5969P8qxpba9ucv/1JE/vJuq9X5Ijk1yd5MYkNyQ5t9nufd+yeXLvfd+yJA9I8oUkX25y/7+b7UclubbJ8UeaQQ1I8nPN+i1N+6pO38BebJ7cvz/JN6fd98c124fyO8eieWlZU1XHTRu3cB1wVVUdDVzVrGvPvR94xoxtc+X6mcDRzc9a4F1DinGpej8/m3uAtzb3/nFV9QmAJMfQG53nsc0x70yy79AiXVp2AudV1THAE4GXNvn1vm/fXLkH7/u2/Qg4qaoeBxwHPCPJE4E30cv9o4DtwDnN/ucA25vtb23208LMlXuA/zntvr++2TaU3zkWzUvbqcCGZnkDcFp3oSwdVfUZ4PszNs+V61OBD1TP54GVSQ4fSqBL0By5n8upwGRV/aiqvgncApzQWnBLWFXdVlVfbJbvBm4CjsD7vnXz5H4u3veLpLl/dzSr+zc/BZwEXNxsn3nf7/r7cDFwcpIMJ9qlZZ7cz2Uov3MsmpeOAj6dZFN6U5ADjFXVbc3yd4CxbkJbFubK9RHAv0zb79vM/w+eFuZlzUdy75vWDcnct6D5yPmXgWvxvh+qGbkH7/vWJdk3yfXA7cCVwDeAO6pqZ7PL9Pzen/um/U7gIUMNeAmZmfuq2nXfv7G579+a5OeabUO57y2al44nV9Xj6X1E8dIkT5neWL2xBR1fcAjM9dC9C/hFeh/h3QZc2Gk0S1iSFcDHgFdU1V3T27zv2zVL7r3vh6Cq7quq4+jNXnwC8JhuI1o+ZuY+ybHAq+n9GfwKcCjwqmHGZNG8RFTV1ub1duBSen+5t+36eKJ5vb27CJe8uXLttPEtq6ptzS/XnwJ/yb9/FG3uF1GS/ekVbR+sqkuazd73QzBb7r3vh6uq7gCuBk6k99H/rsnhpuf3/tw37QcD3xtupEvPtNw/o+muVFX1I+CvGPJ9b9G8BCR5UJIH71oGfgP4Kr3pyc9sdjsTuKybCJeFuXJ9OfBfm2/2PhG4c9rH2VoEM/qt/Sa9ex96uX9B8432o+h9QeQLw45vKWj6Zb4XuKmq3jKtyfu+ZXPl3vu+fUkOS7KyWT4QeBq9PuVXA6c3u82873f9fTgd+MdyBrkFmSP3X5v2n/TQ60s+/b5v/XfO0KfRVivGgEub7xvsB3yoqj6V5DrgoiTnAN8Cnt9hjEtGkg8DE8BDk3wbOB+4gNlz/QngWfS+jPMD4OyhB7yEzJH7iWbYoQK2AC8GqKobklwE3EhvBIKXVtV9HYS9FDwJeBGwueljCPAavO+HYa7cv9D7vnWHAxua0Uf2AS6qqo8nuRGYTPJHwJfo/aeG5vWvk9xC7wvLL+gi6CVirtz/Y5LDgADXAy9p9h/K7xyn0ZYkSZL6sHuGJEmS1IdFsyRJktSHRbMkSZLUh0WzJEmS1IdFsyRJktSHRbMkzSPJfUmuT3JDki8nOS/JPk3beJK39zn+rCR/sZvXfM2exDzPeZ+bZN087bsdqyQtF47TLEnzu7eZypUkDwM+BBwEnF9VG4GNLVzzNcAfL+YJk+xXVZfTmwRgZDRx7ew6DknqxyfNkjSgZpr6tcDLmpmnJpJ8HCDJCUk+l+RLSf5fkkdPO/TIJFNJbk5y/q6NSX4nyReaJ9nvSbJvkguAA5ttH5xnv32TvD/JV5NsTvL7M+Nt2t+d5FrgT6c/SU7yW82xX07ymVmOPaV5Pw+dsf3Xmziub97rrtlIX9XE8eXmPZDkuCSfT/KVJJcmOaTZPpXkz5JsBM5NcnySa5JsSvL3M2a7k6SR4JNmSdoNVXVrM0vVw2Y0fQ34tarameSp9J4U/+em7QTgWHozVV2X5ArgHuC3gSdV1U+SvBM4o6rWJXnZtKfbvzTbfsANwBFVdWyz38o5Qn448KtVdV+Ss6Ztfz3w9KraOvPYJL8JvBJ4VlVtn3G+P6A3y9w/JVkB/DDJM4FTgSdU1Q+SHNrs+wHg5VV1TZI/pDeD4yuatgOqajzJ/sA1wKlV9d0kvw28EfjdOd6PJHXColmSFsfB9KZ9PZre1Mb7T2u7sqq+B5DkEuDJ9KY4Pp5eEQ1wIHD7LOc9eY79/g54ZJI/B64APj1HXB+dYxrlfwLe30y5fMm07ScB48BvVNVdcxz3luYp+CVV9e3mPwl/VVU/AKiq7yc5GFhZVdc0x20APjrtPB9pXh9N7z8UVzbvb1/gtjneiyR1xqJZknZDkkcC99ErXH9pWtMbgKur6jeTrAKmprXVjNMUEGBDVb263yXn2i/J44CnAy8Bns/sT2fvme2kVfWSJE8ATgE2JTm+afoG8EjgPzFLf+2quqB5Uv4s4J+SPL1P/HPZFVeAG6rqxAWeR5KGwj7NkjSgJIcB7wb+oqpmFsIHA1ub5bNmtD0tyaFJDgROo/e09irg9ObLhTTtj2j2/0nTbYG59mv6Gu9TVR8DXgs8fjffyy9W1bVV9Xrgu8CRTdO36HUr+UCSx85x3OaqehNwHfAY4Erg7CQP3BVjVd0JbE/ya82hL6LXDWOmrwOHJTmxOXb/2a4rSV3zSbMkze/AJNfT626xE/hr4C2z7Pen9LpnvJZed4npvgB8jF7/4r9pRt2g2ffT6Q1h9xPgpfSK1vXAV5J8sarOmGO/e4G/arYB9HtiPdObm64koVeYfxk4DqCqvpbkDOCjSZ5TVd+YdtwrkqwBfkqvX/Unq+pHSY4DNib5MfAJeiOAnAm8uymmbwXOnhlEVf04yenA25suHfsBf9acW5JGRn72YYkkSZKk6eyeIUmSJPVh0SxJkiT1YdEsSZIk9WHRLEmSJPVh0SxJkiT1YdEsSZIk9WHRLEmSJPXx/wHsH02OMjdjMQAAAABJRU5ErkJggg==",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plt.xlabel('Diabetes risk score')\n",
"plt.ylabel('Number of patients')\n",
"diabetes_y.hist()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Targets: Regression vs. Classification\n",
"\n",
"We distinguish between two types of supervised learning problems, depending on the form of the target variable.\n",
"\n",
"1. __Regression__: The target variable $y$ is continuous. We are fitting a curve in a high-dimensional feature space that approximates the shape of the dataset.\n",
"2. __Classification__: The target variable $y$ is discrete. Each discrete value corresponds to a *class* and we are looking for a hyperplane that separates the different classes."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 2.2.5. The Feature Matrix\n",
"\n",
"Machine learning algorithms are most easily defined using the language of linear algebra. Therefore, it will be useful to represent the entire dataset as one matrix $X \\in \\mathbb{R}^{n \\times d}$, of the form:\n",
"\n",
"$$ X = \\begin{bmatrix}\n",
"x^{(1)}_1 & x^{(2)}_1 & \\ldots & x^{(n)}_1 \\\\\n",
"x^{(1)}_2 & x^{(2)}_2 & \\ldots & x^{(n)}_2 \\\\\n",
"\\vdots \\\\\n",
"x^{(1)}_d & x^{(2)}_d & \\ldots & x^{(n)}_d\n",
"\\end{bmatrix}.$$\n",
"\n",
"Similarly, we can vectorize the target variables into a vector $y \\in \\mathbb{R}^n$ of the form\n",
"\n",
"$$ y = \\begin{bmatrix}\n",
"y^{(1)} \\\\\n",
"y^{(2)} \\\\\n",
"\\vdots \\\\\n",
"y^{(n)}\n",
"\\end{bmatrix}.$$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# 2.3. Anatomy of a Supervised Learning Problem: The Learning Algorithm\n",
"\n",
"Let's now turn our attention to the second component of a supervised learning problem---the learning algorithm."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 2.3.1. Three Components of a Supervised Machine Learning Algorithm\n",
"\n",
"We can also define the high-level structure of a supervised learning algorithm as consisting of three components:\n",
"* A __model class__: the set of possible models we consider.\n",
"* An __objective__ function, which defines how good a model is.\n",
"* An __optimizer__, which finds the best predictive model in the model class according to the objective function"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 2.3.2. The Model Class\n",
"\n",
"### Defining a Model\n",
"\n",
"We'll say that a model is a function\n",
"\n",
"$$\n",
"f : \\mathcal{X} \\to \\mathcal{Y} $$\n",
"\n",
"that maps inputs $x \\in \\mathcal{X}$ to targets $y \\in \\mathcal{Y}$.\n",
"Often, models have *parameters* $\\theta \\in \\Theta$ living in a set $\\Theta$. We will then write the model as\n",
"\n",
"$$ f_\\theta : \\mathcal{X} \\to \\mathcal{Y} $$\n",
"\n",
"to denote that it's parametrized by $\\theta$."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Defining a Model Class\n",
"\n",
"Formally, the model class is a set \n",
"\n",
"$$\n",
"\\mathcal{M} \\subseteq \\{f \\mid f : \\mathcal{X} \\to \\mathcal{Y} \\}$$\n",
"of possible models that map input features to targets. When the models $f_\\theta$ are paremetrized by *parameters* $\\theta \\in \\Theta$ living in some set $\\Theta$. Thus we can also write\n",
"\n",
"$$\n",
"\\mathcal{M} = \\{f_\\theta \\mid \\theta \\in \\Theta \\}.$$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### An Example\n",
"\n",
"One simple approach is to assume that $x$ and $y$ are related by a linear model of the form\n",
"\n",
"$$\n",
"y=\\theta_0+\\theta_1 \\cdot x_1+\\theta_2 \\cdot x_2+\\ldots+\\theta_d \\cdot x_d\n",
"$$\n",
"\n",
"where $x$ is a featurized input and $y$ is the target.\n",
"The $\\theta_j$ are the *parameters* of the model, $\\Theta = \\mathbb{R}^{d+1}$, and $\\mathcal{M} = \\{ \\theta_0 + \\theta_1 \\cdot x_1 + \\theta_2 \\cdot x_2 + ... + \\theta_d \\cdot x_d \\mid \\theta \\in \\mathbb{R}^{d+1} \\}$\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 2.3.3. The Objective \n",
"\n",
"Given a training set, how do we pick the parameters $\\theta$ for the model? A natural approach is to select $\\theta$ such that $f_\\theta(x^{(i)})$ is close to $y^{(i)}$ on a training dataset $\\mathcal{D} = \\{(x^{(i)}, y^{(i)}) \\mid i = 1,2,...,n\\}$\n",
"\n",
"### Notation\n",
"\n",
"To capture this intuition, we define an *objective function* (also called a *loss function*)\n",
"\n",
"$$\n",
"J(f) : \\mathcal{M} \\to [0, \\infty), $$\n",
"which describes the extent to which $f$ \"fits\" the data $\\mathcal{D} = \\{(x^{(i)}, y^{(i)}) \\mid i = 1,2,...,n\\}$."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"When $f$ is parametrized by $\\theta \\in \\Theta$, the objective becomes a function $J(\\theta) : \\Theta \\to [0, \\infty).$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Examples\n",
"\n",
"What would are some possible objective functions? We will see many, but here are a few examples:\n",
"* Mean squared error: \n",
" \n",
"$$\n",
"J(\\theta) = \\frac{1}{2n} \\sum_{i=1}^n \\left( f_\\theta(x^{(i)}) - y^{(i)} \\right)^2\n",
"$$\n",
"* Absolute (L1) error: \n",
" \n",
"$$\n",
"J(\\theta) = \\frac{1}{n} \\sum_{i=1}^n \\left| f_\\theta(x^{(i)}) - y^{(i)} \\right|\n",
"$$\n",
"\n",
"These are defined for a dataset $\\mathcal{D} = \\{(x^{(i)}, y^{(i)}) \\mid i = 1,2,...,n\\}$.\n",
"\n",
"\n",
"Both examples measure the difference between model predictions $f_\\theta(x^{(i)})$ and the correct targets $y^{(i)}$. By minimizing these objectives, we are trying to find $\\theta$ such that the predictions $\\theta^\\top x^{(i)}$ are as close as possible to the targets $y^{(i)}$."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 2.3.4. The Optimizer\n",
"\n",
"In order to find a good model that fits the data, we also need a procedure that will actually compute the parameters $\\theta$ that minimize our learning objective (i.e., out training loss). This is the goal of the optimizer.\n",
"\n",
"An optimizer takes an objective $J$ and a model class $\\mathcal{M}$ and finds a model $f \\in \\mathcal{M}$ with the smallest value of the objective $J$.\n",
"\n",
"\n",
"\n",
"$$\n",
"\\min _{f \\in \\mathcal{M}} J(f)\n",
"$$\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"When $f$ is parametrized by $\\theta \\in \\Theta$, the optimizer minimizes a function $J(\\theta)$ over all $\\theta \\in \\Theta$."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### An Example\n",
"\n",
"Underneath the hood, the `sklearn.linear_models.LinearRegression` algorithm optimizes the MSE loss.\n",
"\n",
"\n",
"\n",
"$$\n",
"\\min _{\\theta \\in \\mathbb{R}} \\frac{1}{2 n} \\sum_{i=1}^n\\left(f_\\theta\\left(x^{(i)}\\right)-y^{(i)}\\right)^2\n",
"$$\n",
"\n",
"We can easily measure the quality of the fit on the training set and the test set."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"In the next lecture, we will learn how to minimize this objective using a clever formula. For now, let's use `sklearn` to directly fit a linear model on our diabetes dataset."
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"# Collect 20 data points for training\n",
"diabetes_X_train = diabetes_X.iloc[-20:]\n",
"diabetes_y_train = diabetes_y.iloc[-20:]\n",
"\n",
"# Create linear regression object\n",
"regr = linear_model.LinearRegression()\n",
"\n",
"# Train the model using the training sets\n",
"regr.fit(diabetes_X_train, diabetes_y_train.values)\n",
"\n",
"# Make predictions on the training set\n",
"diabetes_y_train_pred = regr.predict(diabetes_X_train)\n",
"\n",
"# Collect 3 data points for testing\n",
"diabetes_X_test = diabetes_X.iloc[:3]\n",
"diabetes_y_test = diabetes_y.iloc[:3]\n",
"\n",
"# generate predictions on the new patients\n",
"diabetes_y_test_pred = regr.predict(diabetes_X_test)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"The algorithm returns a predictive model. We can visualize its predictions below."
]
},
{
"cell_type": "code",
"execution_count": 55,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 55,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtQAAAEGCAYAAABFMwJJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAxXklEQVR4nO3de3hV5Zn38e9NTCUoGtToK0EM7VgUAQNExOKBeopjpUTGzuhgRZQydurbmemUKdR5C7QqTGnHjo4jk2oRK1VbikFHW1QOg1ptmwgFBFFaIxKoIooVCTYk9/vHWjvsJDvJTnb2Kfv3ua597bWetfZa93rI4ebJczB3R0REREREuqdPugMQEREREclmSqhFRERERBKghFpEREREJAFKqEVEREREEqCEWkREREQkAUekO4BEnHDCCV5SUpLuMERERESkl6upqXnX3YtiHcvqhLqkpITq6up0hyEiIiIivZyZvdneMXX5EBERERFJgBJqEREREZEEKKEWEREREUlAVvehjqWhoYGdO3dy8ODBdIcinejbty+DBg0iPz8/3aGIiIiIdFuvS6h37txJ//79KSkpwczSHY60w93Zu3cvO3fuZMiQIekOR0RERKTbel1CffDgQSXTWcDMOP7449mzZ0+6QxEREZEMV7W+joUrt7FrXz0DCwuYWT6UilHF6Q6rWa9LqAEl01lC/04iIiLSmar1dcxevon6hkYA6vbVM3v5JoCMSapze1DiihXQ1NSyrKkpKBcRERGRtFu4cltzMh1R39DIwpXb0hRRW7mbUM+dCxUVMH364aS6qSnYr6gIjneTmXHdddc17x86dIiioiKuvPLKLl2npKSEd999N+FzRERERLLVrn31XSpPh9xMqFesgHnzgu3Fi4Mk+tCh4H3x4qB83rxut1QfddRRbN68mfr64B/6mWeeobg4M/4kISIiIpJNBhYWdKk8HXIzoZ44EaZNO7y/eDHk5x9OpiE4PnFit29xxRVX8OSTTwLw8MMPc+211zYfe++996ioqGDkyJGMGzeOjRs3ArB3714uu+wyzjzzTKZPn467N3/moYceYuzYsZSWlvJ3f/d3NDa2/NOHiIiISG80s3woBfl5LcoK8vOYWT40TRG1lZsJdZ8+cN99LZPqaNOmBcf7dL96rrnmGh555BEOHjzIxo0bOeecc5qPzZkzh1GjRrFx40buuOMOrr/+egDmzZvHeeedxyuvvMJVV13Fjh07ANi6dSuPPvooL7zwAhs2bCAvL4+lS5d2OzYRERGRbFExqpj5k0dQXFiAAcWFBcyfPCJjBiRCL53lIy59+kBlZctW6YjKyoSSaYCRI0dSW1vLww8/zBVXXNHi2PPPP8/Pf/5zAC666CL27t3Ln/70J9atW8fy5csB+NznPseAAQMAWLVqFTU1NZx99tkA1NfXc+KJJyYUn4iIiEi2qBhVnFEJdGtJS6jNrC+wDjgyvM8yd59jZkOAR4DjgRrgi+7+ZzM7EngQGAPsBf7G3WuTFR9NTTBjRuxjM2Yk3EIN8PnPf56vf/3rrF27lr1793b7Ou7O1KlTmT9/fkLxiIiIiEjPS2aXj4+Bi9z9LKAUuNzMxgH/Btzp7n8BvA/cFJ5/E/B+WH5neF5yRGbziNU6DYcHKraeUq+LbrzxRubMmcOIESNalJ9//vnNXTbWrl3LCSecwDHHHMMFF1zAT37yEwB+8Ytf8P777wNw8cUXs2zZMt555x0g6IP95ptvJhSbiIiIiPSMpCXUHtgf7uaHLwcuApaF5UuAinB7UrhPePxiS9bKH0880XYAYkND24GKTzyR0G0GDRrEV7/61Tblc+fOpaamhpEjRzJr1iyWLAkee86cOaxbt44zzzyT5cuXM3jwYACGDRvGbbfdxmWXXcbIkSO59NJL2b17d0KxiYiIiEjPsOiZJHr84mZ5BN06/gK4B1gIvBS2QmNmpwC/cPfhZrYZuNzdd4bHfg+c4+7vtrrmDGAGwODBg8e0bqndunUrZ5xxRufBzZ0bTI0XPQAxuuV6zpyE5qKW+MT97yUiIiKSRmZW4+5lsY4ldVCiuzcCpWZWCDwGnN4D16wEKgHKysq6/7+BuXNh1KhgarxIX+nI7B+TJgUvEREREZFOpGTaPHffB6wBzgUKzSySyA8C6sLtOuAUgPD4sQSDE5Nn0qS2Aw/79FEyLSIiIiJxS1pCbWZFYcs0ZlYAXApsJUisrw5PmwpEliN8PNwnPL7ak9kfRURERESkBySzy8fJwJKwH3Uf4Kfu/j9mtgV4xMxuA9YD94fn3w/82My2A+8B1yQxNhERERGRHpG0hNrdNwKjYpT/ARgbo/wg8IVkxSMiIiIikgy5ufS4iIiIiEgPUUKdBHl5eZSWljJ8+HC+8IUvcODAgW5f64YbbmDZsmDa7unTp7Nly5Z2z127di2/+tWvmvcXLVrEgw8+2O17i4iIiEjnlFAnQUFBARs2bGDz5s184hOfYNGiRS2OHzp0qFvXve+++xg2bFi7x1sn1DfffDPXX399t+4lIiIiIvHJ+YS6an0d4xesZsisJxm/YDVV6+s6/1AXnH/++Wzfvp21a9dy/vnn8/nPf55hw4bR2NjIzJkzOfvssxk5ciT//d//DYC7c8sttzB06FAuueSS5uXGASZMmEB1dTUAv/zlLxk9ejRnnXUWF198MbW1tSxatIg777yT0tJSnnvuOebOncv3vvc9ADZs2MC4ceMYOXIkV111VfOy5hMmTOAb3/gGY8eO5dOf/jTPPfdcjz6/iIiISG+X1IVdMl3V+jpmL99EfUMjAHX76pm9fBMAFaOKE77+oUOH+MUvfsHll18OwMsvv8zmzZsZMmQIlZWVHHvssfz2t7/l448/Zvz48Vx22WWsX7+ebdu2sWXLFt5++22GDRvGjTfe2OK6e/bs4Utf+hLr1q1jyJAhvPfeexx33HHcfPPNHH300Xz9618HYNWqVc2fuf7667n77ru58MIL+da3vsW8efP4wQ9+0Bznb37zG5566inmzZvHs88+m/Czi4iIiOSKnG6hXrhyW3MyHVHf0MjCldsSum59fT2lpaWUlZUxePBgbrrpJgDGjh3LkCFDAHj66ad58MEHKS0t5ZxzzmHv3r28/vrrrFu3jmuvvZa8vDwGDhzIRRdd1Ob6L730EhdccEHztY477rgO4/nggw/Yt28fF154IQBTp05l3bp1zccnT54MwJgxY6itrU3o2UVERERyTU63UO/aV9+l8nhF+lC3dtRRRzVvuzt333035eXlLc556qmnErp3dxx55JFAMJiyu/27RURERHJVTrdQDyws6FJ5TyovL+fee++loaEBgNdee42PPvqICy64gEcffZTGxkZ2797NmjVr2nx23LhxrFu3jjfeeAOA9957D4D+/fvz4Ycftjn/2GOPZcCAAc39o3/84x83t1aLiIiISGJyuoV6ZvnQFn2oAQry85hZPjTp954+fTq1tbWMHj0ad6eoqIiqqiquuuoqVq9ezbBhwxg8eDDnnntum88WFRVRWVnJ5MmTaWpq4sQTT+SZZ55h4sSJXH311axYsYK77767xWeWLFnCzTffzIEDB/jkJz/J4sWLk/6MIiIiIrnA3D3dMXRbWVmZR2a9iNi6dStnnHFG3NeoWl/HwpXb2LWvnoGFBcwsH9ojAxIlPl399xIRERFJBzOrcfeyWMdyuoUagtk8lECLiIiISHfldB9qEREREZFEKaEWEREREUmAEmoRERERkQQooRYRERERSYASahERERGRBCihToKjjz6603OmT5/Oli1bALjjjjtaHPvMZz7TI/eIxwMPPMCuXbtixtVVa9eu5Ve/+lWPxCUiIiKSLZRQp8l9993HsGHDgLYJdSqT0tYJdXRcXaWEWkRERHKREuqlS6GkBPr0Cd6XLu2xS69du5YJEyZw9dVXc/rppzNlyhQiC+lMmDCB6upqZs2aRX19PaWlpUyZMgU43Pq8f/9+Lr74YkaPHs2IESNYsWJFh/erra1tvs8ZZ5zB1VdfzYEDBwD49re/zdlnn83w4cOZMWMG7s6yZcuorq5mypQplJaWUl9f3xwXwNNPP825557L6NGj+cIXvsD+/fsBKCkpYc6cOc1xvfrqq9TW1rJo0SLuvPNOSktLee655/jZz37G8OHDOeuss7jgggt6rF5FREREMoq7Z+1rzJgx3tqWLVvalLXroYfc+/Vzh8Ovfv2C8gQcddRR7u6+Zs0aP+aYY/ytt97yxsZGHzdunD/33HPu7n7hhRf6b3/72xbnt/58Q0ODf/DBB+7uvmfPHv/Upz7lTU1NMT/j7v7GG2844M8//7y7u0+bNs0XLlzo7u579+5tPu+6667zxx9/vE0c0ft79uzx888/3/fv3+/u7gsWLPB58+a5u/upp57qd911l7u733PPPX7TTTe5u/ucOXOa7+fuPnz4cN+5c6e7u7///vsx66pL/14iIiIiaQJUezs5aW63UN96K4QtuM0OHAjKe8jYsWMZNGgQffr0obS0lNra2rg/6+5885vfZOTIkVxyySXU1dXx9ttvd/iZU045hfHjxwNw3XXX8fzzzwOwZs0azjnnHEaMGMHq1at55ZVXOrzOSy+9xJYtWxg/fjylpaUsWbKEN998s/n45MmTARgzZky7zzR+/HhuuOEGfvjDH9LY2BjvY4uIiIhkldxeenzHjq6Vd8ORRx7ZvJ2Xl8ehQ4fi/uzSpUvZs2cPNTU15OfnU1JSwsGDBzv8jJm12T948CB///d/T3V1Naeccgpz587t9DruzqWXXsrDDz8c83jkuTp6pkWLFvHrX/+aJ598kjFjxlBTU8Pxxx/f4X1FREREsk1ut1APHty18iTJz8+noaGhTfkHH3zAiSeeSH5+PmvWrGnRQtyeHTt28OKLLwLwk5/8hPPOO685eT7hhBPYv38/y5Ytaz6/f//+fPjhh22uM27cOF544QW2b98OwEcffcRrr73W4b1bX+v3v/8955xzDt/+9rcpKirirbfe6jR+ERERkWyT2wn17bdDv34ty/r1C8pTaMaMGYwcObJ5UGLElClTqK6uZsSIETz44IOcfvrpnV5r6NCh3HPPPZxxxhm8//77fPnLX6awsJAvfelLDB8+nPLycs4+++zm82+44QZuvvnm5kGJEUVFRTzwwANce+21jBw5knPPPZdXX321w3tPnDiRxx57rHlQ4syZMxkxYgTDhw/nM5/5DGeddVYXa0ZEREQk85mHs05ko7KyMo/MSBGxdetWzjjjjPgvsnRp0Gd6x46gZfr226FVYpstamtrufLKK9m8eXO6Q4lbl/+9RERERNLAzGrcvSzWsdzuQw1B8pylCbSIiIiIpF9ud/noZUpKSrKqdVpERESkN+iVCXU2d2PJJfp3EhERkd4gaQm1mZ1iZmvMbIuZvWJm/xCWzzWzOjPbEL6uiPrMbDPbbmbbzKy8O/ft27cve/fuVbKW4dydvXv30rdv33SHIiIiIpKQZPahPgT8s7u/bGb9gRozeyY8dqe7fy/6ZDMbBlwDnAkMBJ41s0+7e5dWBBk0aBA7d+5kz549PfAIkkx9+/Zl0KBB6Q5DREREJCFJS6jdfTewO9z+0My2AsUdfGQS8Ii7fwy8YWbbgbHAi125b35+PkOGDOlm1CIiIiIiXZOSPtRmVgKMAn4dFt1iZhvN7EdmNiAsKwaiV/7YSccJuIiIiIhI2iU9oTazo4GfA//o7n8C7gU+BZQStGB/v4vXm2Fm1WZWrW4dIiIiIpJuSU2ozSyfIJle6u7LAdz9bXdvdPcm4IcE3ToA6oBToj4+KCxrwd0r3b3M3cuKioqSGb6IiIiISKeSOcuHAfcDW93936PKT4467SogMnHy48A1ZnakmQ0BTgN+k6z4RERERER6QjJn+RgPfBHYZGYbwrJvAteaWSngQC3wdwDu/oqZ/RTYQjBDyFe6OsOHiIiIiEiqJXOWj+cBi3HoqQ4+cztwe7JiEhERERHpab1ypUQRERERkVRRQi0iIiIikgAl1CIiIiIiCVBCLSIiIiKSgGTO8iEiIiIiWaBqfR0LV25j1756BhYWMLN8KBWjtGB1vJRQi4iIiOSwqvV1zF6+ifqGYLbiun31zF6+CUBJdZzU5UNEREQkhy1cua05mY6ob2hk4cptaYoo+yihFhEREclhu/bVd6lc2lJCLSIiIpLDBhYWdKlc2lJCLSIiIpLDZpYPpSA/r0VZQX4eM8uHpimi7KNBiSIiIiI5LDLwULN8dJ8SahEREZEcVzGqWAl0AtTlQ0REREQkAUqoRUREREQSoC4fIiIiWU6r3ImkV6cJtZmZu3ursiPd/ePkhSUiIiLx0Cp3IukXT5eP+6N3zOxo4KnkhCMiIiJdoVXuRNIvnoR6p5n9F4CZDQCeBh5KalQiIiISF61yJ5J+nSbU7v4tYL+ZLSJIpr/v7ouTHpmIiIh0SqvciaRfuwm1mU2OvIBfA+OA9YCHZSIiIpJmWuVOJP06GpQ4sdX+eiA/LHdgebKCEhERkfholTuR9Gs3oXb3aakMRERERLpHq9yJpFenfajN7LtmdoyZ5ZvZKjPbY2bXpSI4EREREZFMF88sH5e5+5+AK4Fa4C+AmckMSkREREQkW8STUEe6hXwO+Jm7f5DEeEREREREsko8S4//j5m9CtQDXzazIuBgcsMSEREREckO8cxDPQv4DFDm7g3AR8CkZAcmIiIiIpIN2m2hNrOL3H119JzTZhZ9iqbNExEREZGc11GXjwuB1bSdjxo0D7WIiIiICNDxPNRzwvduzUdtZqcADwInESTgle7+H2Z2HPAoUEIwa8hfu/v7FjR//wdwBXAAuMHdX+7OvUVEREREUqXDPtRmlmdmJ0Ttf8LMZpjZ1jiufQj4Z3cfRrBs+VfMbBgwC1jl7qcBq8J9gL8ETgtfM4B7u/w0IiIiIiIp1m5CbWbXAO8BG83sf83sMuAPBInvlM4u7O67Iy3M7v4hsBUoJhjQuCQ8bQlQEW5PAh70wEtAoZmd3K2nEhERERFJkY76UP8rMMbdt5vZaOBF4Gp3f6KrNzGzEmAU8GvgJHffHR76I0GXEAiS7beiPrYzLNsdVYaZzSBowWbw4MFdDUVEREREpEd11OXjz+6+HSBsaX69m8n00cDPgX8MV1xs5u5O0L86bu5e6e5l7l5WVFTU1XBERERERHpURy3UJ5rZ16L2C6P33f3fO7u4meUTJNNL3T0yK8jbZnayu+8Ou3S8E5bXAadEfXxQWCYiIiIikrE6aqH+IdA/6tV6v0PhrB33A1tbJd+PA1PD7anAiqjy6y0wDvggqmuIiIiIiEhG6mjavHkJXns88EVgk5ltCMu+CSwAfmpmNwFvAn8dHnuKYMq87QTT5nVruj4RERERkVTqqMtHQtz9ecDaOXxxjPMd+Eqy4hERERERSYYO56EWEREREZGOKaEWEREREUlApwm1mf2DmR0TDha838xeDhd5ERERERHJefG0UN8Yzh99GTCAYKDhgqRGJSIiIiKSJeIZlBgZWHgF8GN3fyWcEk9yUNX6Ohau3MauffUMLCxgZvlQKkYVpzssERERkbSJJ6GuMbOngSHAbDPrDzQlNyzJRFXr65i9fBP1DY0A1O2rZ/byTQBKqkVERCRnxdPl4yZgFnC2ux8APoHmiM5JC1dua06mI+obGlm4cluaIhIRERFJv3gSageGAV8N948C+iYtIslYu/bVd6lcREQkK61YAU2t/hjf1BSUi8QQT0L9X8C5wLXh/ofAPUmLSDLWwMKCLpWLiPSEqvV1jF+wmiGznmT8gtVUra9Ld0jSm82dCxUVMH364aS6qSnYr6gIjou0Ek9CfY67fwU4CODu7xN0+5AcM7N8KAX5eS3KCvLzmFk+NE0RiUhvFxm7UbevHufw2A0l1ZIUK1bAvHnB9uLFQRJ96FDwvnhxUD5vnlqqpY14EuoGM8sj6PqBmRWhQYk5qWJUMfMnj6C4sAADigsLmD95hAYkikjSaOyGpNTEiTAtapjY4sWQn384mYbg+MSJqY9NMlo8s3zcBTwGnGhmtwNXA/8vqVFJxqoYVawEWkRSRmM3JKX69IH77gu2o5PoiGnTguN9tNC0tNRpQu3uS82sBriYYE7qCnffmvTIREQk5w0sLKAuRvKssRu5LalrIvTpA5WVsRPqykol0xJTPEuP/9jdX3X3e9z9P919q5n9OBXBiYhIbtPYDWkt6f3qm5pgxozYx2bMaDv7hwjx9aE+M3on7E89JjnhiIiIHKaxG9JaUvvVR2bziNU6DYcHKiqpllba7fJhZrOBbwIFZvYnDi9B/megMgWxiYiIaOyGtJDUfvVPPNF2AGJlZdAyHSlfvBgmTQpeIqF2W6jdfb679wcWuvsx7t4/fB3v7rNTGKOIiIgIkOQ1ESZNgjlzgu3IAMQjjgjeI7N/zJmjZFraiKfLx61mdp2Z/T8AMzvFzMYmOS4RERGRNpLer37uXKiqajmbR2T2j6oqLewiMcWTUN9DsFLi34b7+9FKiSIiIpIGKelXP2lS29k8+vRRy7S0K555qM9x99Fmth6ClRLNTCslioiISFqoX71kGq2UKCIiIiKSgHgS6shKiSeFKyU+D9yR1KhERERERLJEV1dKBK2UKCIiIiLSLJ4+1AD9gEi3D633KiIiIiISimfp8W8BS4DjgBOAxWb2r8kOTEREREQkG8TTQj0FOMvdDwKY2QJgA3BbEuMSEREREckK8QxK3AX0jdo/EqhLTjgiIiIiItml3RZqM7uboM/0B8ArZvZMuH8p8JvUhCeSflXr61i4chu79tUzsLCAmeVDNf+piIiINOuohboaqCGYMu+bwBpgLXArsCLpkYlkgKr1dcxevom6ffU4ULevntnLN1G1Xn+kERHp1ZYuhZKSYIXEkpJgX6Qd7bZQu/uSRC5sZj8CrgTecffhYdlc4EvAnvC0b7r7U+Gx2cBNQCPwVXdfmcj9RXrCwpXbqG9obFFW39DIwpXb1EotItJbLV0KM2bAgQPB/ptvBvsAU6akLy7JWPHM8nGamS0zsy1m9ofIK45rPwBcHqP8TncvDV+RZHoYcA1wZviZ/wpXZxRJq1376rtULiIivcCttx5OpiMOHAjKRWKIZ1DiYuBe4BDwWeBB4KHOPuTu64D34oxjEvCIu3/s7m8A24GxcX5WJGkGFsaedr29chER6QV27OhaueS8eBLqAndfBZi7v+nuc4HPJXDPW8xso5n9yMwGhGXFwFtR5+wMy9owsxlmVm1m1Xv27Il1ikiPmVk+lIL8ln8sKcjPY2b50DRFJCIiSTd4cNfKJefFk1B/bGZ9gNfN7BYzuwo4upv3uxf4FFAK7Aa+39ULuHulu5e5e1lRUVE3wxCJT8WoYuZPHkFxYQEGFBcWMH/yCPWfFhHpzW6/Hfr1a1nWr19QLhJDPAu7/APB0uNfBb4DXARM7c7N3P3tyLaZ/RD4n3C3Djgl6tRBaK5ryRAVo4qVQIuI5JLIwMNbbw26eQweHCTTGpAo7eg0oXb334ab+4FpidzMzE52993h7lXA5nD7ceAnZvbvwEDgNDTXtYiIiKTLlClKoCVu7Xb5MLMfhO9PmNnjrV+dXdjMHgZeBIaa2U4zuwn4rpltMrONBAMc/wnA3V8BfgpsAX4JfMXdG9u5tIiIiCST5mAW6RJz99gHzMa4e42ZXRjruLv/b1Iji0NZWZlXV1enOwwREZHeo/UczBD0H66sVIut5DQzq3H3spjH2kuoW12gCMDdM2paDSXUIiIiPaykJFjIpLVTT4Xa2lRHI5IxOkqoO5zlw8zmmtm7wDbgNTPbY2bfSkaQIiIikgE0B7NIl3XUh/prwHjgbHc/zt0HAOcA483sn1IVoIiIiKSQ5mAW6bKOWqi/CFwbrlwIgLv/AbgOuD7ZgYmIiEgaaA5mkS7rKKHOd/d3WxeG/ajzkxeSiIiIpM2UKcEAxFNPBbPgXQMSRTrU0TzUf+7mMRGRbqtaX8fCldvYta+egYUFzCwfqoV1RFJNczCLdElHCfVZZvanGOUG9E1SPCKSw6rW1zF7+SbqG4Jp6Ov21TN7+SYAJdUiIpKx2u3y4e557n5MjFd/d1eXDxHpcQtXbmtOpiPqGxpZuHJbmiISERHpXIfT5omIpNKuffVdKhcREckESqhFJGMMLCzoUrmIiEgmUEItIhljZvlQCvLzWpQV5Ocxs3xomiISERHpXEeDEkVEUioy8FCzfIiISDZRQi0iGaViVLESaBERySrq8iEiIiIikgC1UIukmRYyERERyW5qoRZJo8hCJnX76nEOL2RStb4u3aGJSK5asQKamlqWNTUF5SISkxJqkTTSQiYiklHmzoWKCpg+/XBS3dQU7FdUBMdFpA0l1CJppIVMRCRjrFgB8+YF24sXB0n0oUPB++LFQfm8eWqpFolBCbVIGmkhExHJGBMnwrRph/cXL4b8/MPJNATHJ05MfWwiGU4JtUgaaSETEckYffrAffe1TKqjTZsWHO+j1EGkNXP3dMfQbWVlZV5dXZ3uMEQSolk+JNPpazTHHDoUtEy31tAAR2hyMMldZlbj7mWxjuk7QyTNtJCJZLLITDSRwbORmWgAfd32Rk1NMGNG7GMzZqiFWqQd+q4QEZF2aSaaHBKZzSO6z3S0yEDF1lPqiYgSahERaZ9moskhTzzRdgBiQ0PbgYpPPJH62EQynBJqERFpl2aiySGTJsGcOcF2ZADiEUe0HKg4Z05wnoi0oIRa0kercYlkPM1Ek2PmzoWqqpZ9pSOzf1RVaWEXkXYooZb00Gpc0o6q9XWMX7CaIbOeZPyC1VqGPc0qRhUzf/IIigsLMKC4sID5k0doQGJvNmlS24GHffqoZVqkA5o2T1JvxYogaY6YNg0qK4MR5NH996qq9AM8x7SeUQKC1lAlcCIikm4dTZunFmpJPa3GJe3QjBIiIpKNkpZQm9mPzOwdM9scVXacmT1jZq+H7wPCcjOzu8xsu5ltNLPRyYpLMoBW45J2aEYJERHJRsnMWB4ALm9VNgtY5e6nAavCfYC/BE4LXzOAe5MYl2SCPn2Cbh6xVFYqmc5RmlFCRESyUdKyFndfB7zXqngSsCTcXgJURJU/6IGXgEIzOzlZsUkG6Gw1Li0ckJM0o4SIiGSjVC89fpK77w63/wicFG4XA29FnbczLNtNK2Y2g6AVm8GDBycvUkmeeFbjAnX7yEGRgYcLV25j1756BhYWMLN8qAYkiohIRkt1Qt3M3d3MujzFiLtXApUQzPLR44FJ8sVajav1LB+LFwczfOTgLB9V6+tyOqGsGFWcU88rIiLZL9XNf29HunKE7++E5XXAKVHnDQrLpDfSalztikwbV7evHgfq9tUze/kmzcUsIiKSwVKdUD8OTA23pwIrosqvD2f7GAd8ENU1RHqjLF6NK5kLj2jaOBERkeyTtC4fZvYwMAE4wcx2AnOABcBPzewm4E3gr8PTnwKuALYDB4B25lOTXiVWC3SGr8bVeuGRSAsy0CPdFDRtnIiISPZJWkLt7te2c+jiGOc68JVkxSLSUzpqQe6JhHpgYQF1MZJnTRsnIiKSuTSFgkgXJLsFWdPGiYiIZB8l1CJdkOyFRypGFTN/8giKCwswoLiwgPmTR2jWCxERkQyWtmnzRLLRzPKhLfpQQ8+3IGvaOBERkeyihFqkC7TwiIiIiLSmhFqki9SCLCIiItHUh1pEREREJAFKqEVEREREEqCEWkREREQkAUqoRUREREQSoIRaRERERCQBSqglvZYuhZIS6NMneF+6NN0RiYiIiHSJps2T9Fm6FGbMgAMHgv033wz2AaZMSV9cIiIiIl1g7p7uGLqtrKzMq6ur0x1GVqtaX5e+RUpKSoIkurVTT4Xa2tTEICIiIhIHM6tx97JYx9RCncOq1te1WEa7bl89s5dvAkhNUr1jR9fKRURERDKQ+lDnsIUrtzUn0xH1DY0sXLktNQEMHty1chEREZEMpIQ6h+3aV9+l8h53++3Qr1/Lsn79gnIRERGRLKEuHzlsYGEBdTGS54GFBakJIDLw8NZbg24egwcHybQGJHYorf3eJSvoa0REJLXUQp3DZpYPpSA/r0VZQX4eM8uHpi6IKVOCAYhNTcG7kukORfq91+2rxznc771qfV26Q5MMoa8REZHUU0KdwypGFTN/8giKCwswoLiwgPmTR6glK4Olo9971fo6xi9YzZBZTzJ+wWolZhku7WMjRERykLp85LiKUcVKoLNIqvu9p30mGOmytI+NEBHJQWqhFski7fVvT1a/d7V2Zp9Uf42IiIgSapGskup+72rtzD4ZMTZCRCTHqMuHSBaJdLNI1QwOaZ8JRros1V8jIiKipccTpumppDdr3YcagtZODV4VEZFco6XHk0QDtqS3U2uniIhI55RQJ6CjAVtKOKS30EwwIiIiHdOgxARowJaIiIiIKKFOgKanEhEREZG0JNRmVmtmm8xsg5lVh2XHmdkzZvZ6+D4gHbF1haanEhEREZF09qH+rLu/G7U/C1jl7gvMbFa4/430hBYfDdgSkWTQ7EEiItklkwYlTgImhNtLgLVkYEId6xfdC7MuSndYItJLaPYgEZHsk64+1A48bWY1ZjYjLDvJ3XeH238ETor1QTObYWbVZla9Z8+eVMTaLPKLrm5fPc7hX3RV6+tSGoeI9F5a7l1EJPukK6E+z91HA38JfMXMLog+6MFqMzFXnHH3Sncvc/eyoqKiFIR6mH7RiUiyafYgEZHsk5aE2t3rwvd3gMeAscDbZnYyQPj+Tjpi64h+0YlIsmn2IBGR7JPyhNrMjjKz/pFt4DJgM/A4MDU8bSqwItWxdUa/6EQk2TR7kIhI9klHC/VJwPNm9jvgN8CT7v5LYAFwqZm9DlwS7mcU/aITkWSrGFXM/MkjKC4swIDiwgLmTx6hAYkiIhnMgu7K2amsrMyrq6tTek9NZyUiIiKSe8ysxt3LYh3LpGnzskLFqGIl0CIiIiLSTEuPi4iIiIgkQAm1iIiIiEgClFCLiIiIiCRACbWIiIiISAKUUIuIiIiIJCCrp80zsz3Am52cdgLwbgrCyXaqp/ipruKnuoqf6ip+qqv4qJ7ip7qKXy7X1anuXhTrQFYn1PEws+r25gyUw1RP8VNdxU91FT/VVfxUV/FRPcVPdRU/1VVs6vIhIiIiIpIAJdQiIiIiIgnIhYS6Mt0BZAnVU/xUV/FTXcVPdRU/1VV8VE/xU13FT3UVQ6/vQy0iIiIikky50EItIiIiIpI0SqhFRERERBLQKxJqMzvOzJ4xs9fD9wHtnDc1POd1M5saVX6tmW0ys41m9kszOyF10adOD9TTJ8ys0sxeM7NXzeyvUhd9aiVaV1HHHzezzcmPOH0SqSsz62dmT4ZfT6+Y2YLURp98Zna5mW0zs+1mNivG8SPN7NHw+K/NrCTq2OywfJuZlac08DTobl2Z2aVmVhP+HK8xs4tSHnyKJfJ1FR4fbGb7zezrKQs6TRL8HhxpZi+GP582mVnflAafYgl8D+ab2ZKwjraa2eyUB59u7p71L+C7wKxwexbwbzHOOQ74Q/g+INweABwBvAOcEHWtuel+pkyrp/DYPOC2cLtPpM564yvRugqPTwZ+AmxO9/Nkal0B/YDPhud8AngO+Mt0P1MP1k0e8Hvgk+Hz/Q4Y1uqcvwcWhdvXAI+G28PC848EhoTXyUv3M2VoXY0CBobbw4G6dD9PptZV1PFlwM+Ar6f7eTK1rgjyg43AWeH+8foebLeu/hZ4JNzuB9QCJel+plS+ekULNTAJWBJuLwEqYpxTDjzj7u+5+/vAM8DlgIWvo8zMgGOAXUmPOD0SqSeAG4H5AO7e5O69eaWkhOrKzI4GvgbclvxQ067bdeXuB9x9DYC7/xl4GRiU/JBTZiyw3d3/ED7fIwT1FS26/pYBF4c/iyYR/IL62N3fALaH1+utul1X7r7e3SM/t18BCszsyJREnR6JfF1hZhXAGwR11dslUleXARvd/XcA7r7X3RtTFHc6JFJXTpBHHQEUAH8G/pSasDNDb0moT3L33eH2H4GTYpxTDLwVtb8TKHb3BuDLwCaCRHoYcH8SY02nbteTmRWG+98xs5fN7GdmFuvzvUW36yrc/g7wfeBA0iLMHInWFQDh19hEYFUSYkyXTp87+hx3PwR8QNASFs9ne5NE6iraXwEvu/vHSYozE3S7rsL/7H+D4C+OuSCRr6tPA25mK8Pfe/+SgnjTKZG6WgZ8BOwGdgDfc/f3kh1wJjki3QHEy8yeBf5PjEO3Ru+4u5tZ3HMBmlk+QUI9iuDP0HcDs8nSlsVk1RPB18og4Ffu/jUz+xrwPeCL3Q42zZL4NVUKfMrd/6l1v8VslcSvq8j1jwAeBu5y9z90L0rJdWZ2JvBvBC2LEttc4E533x82WEv7jgDOA84maBxZZWY17t6b/tPfU8YCjcBAgu58z5nZs7n08zxrEmp3v6S9Y2b2tpmd7O67zexkgj7RrdUBE6L2BwFrgdLw+r8Pr/VTgn6gWSmJ9bSX4AfK8rD8Z8BNPRFzuiSxrs4FysysluB77EQzW+vuE8hSSayriErgdXf/QeLRZpQ64JSo/UFhWaxzdob/sTiW4Pstns/2JonUFWY2CHgMuD7y87wXS6SuzgGuNrPvAoVAk5kddPf/THrU6ZFIXe0E1kW6N5rZU8Boetdf0aIlUld/C/wy/Kv/O2b2AlBG0FCZE3pLl4/HgcgMC1OBFTHOWQlcZmYDLJiF4LKwrA4YZmZF4XmXAluTHG+6dLue3N2BJzicFF0MbEluuGmVSF3d6+4D3b2EoHXjtWxOpuOQyPcfZnYbwQ/lf0x+qCn3W+A0MxtiZp8gGMTzeKtzouvvamB1+P32OHBNOKp+CHAa8JsUxZ0O3a6rsLvQkwSDY19IVcBp1O26cvfz3b0k/Pn0A+COXpxMQ2LfgyuBERbMRnQEcCG9+/deInW1A7gIwMyOAsYBr6Yk6kyR7lGRPfEi6L+zCngdeBY4LiwvA+6LOu9GgoE924FpUeU3EyTRGwmSxuPT/UwZWk+nAuvCeloFDE73M2VqXUUdL6H3z/LR7boiaAHx8PtvQ/ianu5n6uH6uQJ4jWD0/K1h2beBz4fbfQn+4rOdIGH+ZNRnbw0/t41eNPtJT9cV8K8E/Tc3RL1OTPfzZGJdtbrGXHr5LB+J1hVwHcHgzc3Ad9P9LJlaV8DRYfkrBP/pmJnuZ0n1S0uPi4iIiIgkoLd0+RARERERSQsl1CIiIiIiCVBCLSIiIiKSACXUIiIiIiIJUEItIiIiIpIAJdQiIq2YWaOZbTCz34VLDn+mi59/wMyu7sL5a81sh0UtXWdmVWa2vyv37cL9bjCzLs093NVn6uRay8zsk+F2rZltCut7k5lNijrPzeyhqP0jzGyPmf1P6+cws1vM7MaeiE9EpKuUUIuItFXv7qXufhYwG5ifgnvuA8YDhAuVnJyCe6ZcuDx4nrdckviz7l5KsFDEXVHlHwHDzawg3L+U9leL/BHwf3s4XBGRuCihFhHp2DHA+wAWWGhmm8PW1L+JKv9PM9tmZs8CJ4blF5lZVeRCZnapmT3Wzn0eIViZDGAysDzqc0eb2aqwtby5FdfMjjKzJ8OW9M1R8Swwsy1mttHMvtfRw4Utz3eZ2a/M7A+RVuj2nik8NsbM/tfMasxspZmdbGbHhucODc952My+FOOWU4i9miZE1XWUp4DPhdvXAg/H+qC7HwBqzWxsR88rIpIMSqhFRNoqCLsgvArcB3wnLJ8MlAJnAZcAC83sZOAqYCgwDLgeiHQRWQOcbmZF4f40gpbUWFYBF5hZHkFi/WjUsYPAVe4+Gvgs8P2we8jlwC53P8vdhwO/NLPjw3jOdPeRwG1xPO/JwHnAlcCCsCzmM5lZPnA3cLW7jwmf53Z3/wC4BXjAzK4BBrj7D2PcazxQ06psjZltBv6XYNXDaI8QLMHeFxgJ/LqD56gGzu/8cUVEepYSahGRtiJdPk4nSFofDBPY84CH3b3R3d8mSADPBi6IKt8FrAbwYCnaHwPXhd04zgV+0c49G4HnCZLpAnevjTpmwB1mtpFgefdi4CRgE3Cpmf2bmZ0fJrUfECTg95vZZOBAHM9b5e5N7r4lvC7tPRNBkj0ceMbMNhAkwIPC530mjOkeYHo79zoZ2NOq7LPhfwhGAP9pZkdHDrj7RqCEoHX6qU6e4x1gYCfniIj0uCPSHYCISCZz9xfN7ASgqNOTY1sMPEGQ5P7M3Q91cO4jwGPA3FblU8L7j3H3BjOrBfq6+2tmNhq4ArjNzFa5+7fDbg8XE/RJvgW4qJMYP47atnbPOnz8FXc/t80Bsz7AGQRJ/ABgZ4zP1wN9Y13Y3X9vZm8TtIr/JurQ48D3gAnA8R3E1je8vohISqmFWkSkA2Z2OpAH7AWeA/7GzPLCbhwXECR+66LKTybolgFA2Lq7i6Ald3Ent3uOYABk637CxwLvhMn0Z4FTw9gGAgfc/SFgITA6bN091t2fAv6JoHtKd7T3TNuAIjM7N4whPxxoSHi/rcDfAovD7iGtbQX+ItYNzexEYAjwZqtDPwLmufumTmL+NLC5k3NERHqcWqhFRNoqCLszQNAiO9XdG8MBhecCvwMc+Bd3/2NYfhGwBdgBvNjqekuBInff2tFNwy4isQYRLgWeMLNNBP2EXw3LRxD0424CGoAvA/2BFWGfYwO+Fv9jtxDzmdz9z+HAxbvM7FiC3yM/MLNDBN08xrr7h2a2juA/EXNaXfdJgpbmZ6PK1phZI5APzAq70zRz9520nP2jPeNp27ovIpJ0Fvz8FhGRZAnnSl7v7venO5Z0C6fAWwOMd/fGHrzuKOBr7v7FnrqmiEi8lFCLiCSRmdUQzKd8qbt/3Nn5ucDMyoGt7r6jB695KfB6q8GcIiIpoYRaRERERCQBGpQoIiIiIpIAJdQiIiIiIglQQi0iIiIikgAl1CIiIiIiCVBCLSIiIiKSgP8PrliPUhyK2EkAAAAASUVORK5CYII=",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# visualize the results\n",
"plt.xlabel('Body Mass Index (BMI)')\n",
"plt.ylabel('Diabetes Risk')\n",
"plt.scatter(diabetes_X_train.loc[:, ['bmi']], diabetes_y_train)\n",
"plt.scatter(diabetes_X_test.loc[:, ['bmi']], diabetes_y_test, color='red', marker='o')\n",
"# plt.scatter(diabetes_X_train.loc[:, ['bmi']], diabetes_y_train_pred, color='black', linewidth=1)\n",
"plt.plot(diabetes_X_test.loc[:, ['bmi']], diabetes_y_test_pred, 'x', color='red', mew=3, markersize=8)\n",
"plt.legend(['Model', 'Prediction', 'Initial patients', 'New patients'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Again, the red dots are the true values of $y$ and the red crosses are the predictions. Note that although the x-axis is still BMI, we used many additional features to make our predictions. As a result, the predictions are more accurate (the crosses are closer to the dots).\n",
"\n",
"We can also confirm our intuition that the fit is good by evaluating the value of our objective."
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training set mean squared error: 1118.22\n",
"Test set mean squared error: 667.81\n",
"Test set mean squared error on random inputs: 15887.97\n"
]
}
],
"source": [
"from sklearn.metrics import mean_squared_error\n",
"\n",
"print('Training set mean squared error: %.2f'\n",
" % mean_squared_error(diabetes_y_train, diabetes_y_train_pred))\n",
"print('Test set mean squared error: %.2f'\n",
" % mean_squared_error(diabetes_y_test, diabetes_y_test_pred))\n",
"print('Test set mean squared error on random inputs: %.2f'\n",
" % mean_squared_error(diabetes_y_test, np.random.randn(*diabetes_y_test_pred.shape)))"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 2.3.5. Summary: Components of a Supervised Machine Learning Problem\n",
"\n",
"In conclusion, we defined in this lecture the task of supervised learning as well as its key elements. Formally, to apply supervised learning, we define a dataset and a learning algorithm.\n",
"\n",
"$$ \\underbrace{\\text{Dataset}}_\\text{Features, Attributes, Targets} + \\underbrace{\\text{Learning Algorithm}}_\\text{Model Class + Objective + Optimizer } \\to \\text{Predictive Model} $$\n",
"\n",
"A dataset consists of training examples, which are pairs of inputs and targets. Each input is a vector of features or attributes. A learning algorithm can be fully defined by a model class, objective and optimizer.\n",
"\n",
"The output of a supervised learning is a predictive model that maps inputs to targets. For instance, it can predict targets on new inputs."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"accelerator": "GPU",
"celltoolbar": "Slideshow",
"colab": {
"collapsed_sections": [],
"name": "neural-ode.ipynb",
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.7"
},
"rise": {
"controlsTutorial": false,
"height": 900,
"help": false,
"margin": 0,
"maxScale": 2,
"minScale": 0.2,
"progress": true,
"scroll": true,
"theme": "simple",
"width": 1200
},
"title": "Lecture 2: Supervised Learning",
"vscode": {
"interpreter": {
"hash": "ccfaef849455cb82dfcf38b07a416fc7e0fa5490bd7d0cd7d98b2a6b1749e9b9"
}
}
},
"nbformat": 4,
"nbformat_minor": 1
}