{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "### Notebook for Exercise Sheet 5:\n", "# Least-square Regression and Regularization\n", "\n", "This notebook is about regularized empirical risk minimization. The focus is on getting some impressions how the regularization parameter affects the computed approximation. Note that points are only given for correctly running code and meaningfull plots. At the end of the notebook you find some tests which you can use to check your code." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# some basic setup\n", "%matplotlib inline\n", "import numpy as np \n", "import matplotlib.pyplot as plt\n", "\n", "# add your required imports here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The toy data\n", "Below you see the toy function which you will approximate in the following, $10$ samples (red dots), and the same $10$ samples heavily corrupted by strong noise." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD7CAYAAACPDORaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4VOXZ+PHvA6hJUVFxTWSJcUERRESginUQkIiVxRUF\nZKvSqGi3t9j+TBMarOV1X2oURdGCjTsVRSEuo0ULiq8sIiiEhCVBLIpVMEEI9++PJxOSkGS2M3PO\nzNyf68pFZubMOTeT5J5n7mczIoJSSqnk1MrtAJRSSsWOJnmllEpimuSVUiqJaZJXSqkkpkleKaWS\nmCZ5pZRKYm3ieTFjjI7XVEqpCIiIieR5cW/Ji4h+iZCfn+96DF750tdCXwt9LVr+ioaWa5RSKolp\nkldKqSSmSd4lPp/P7RA8Q1+LffS12EdfC2eYaOs9YV3MGInn9ZRSKhkYY5BE6XhVSikVP5rklVIq\niWmSV0qpJKZJXimlkljQJG+MmWmM2WqMWRHkuLONMbuNMZc6F55SSqlohNKSfxIY3NIBxphWwF+B\nBU4EpZRSyhlBk7yILAK2BzlsMvAC8JUTQSmllHJG1DV5Y0wGMFxEioCIxnEqpZSKDSdWobwPmFLv\ndouJvqCgoO57n8+ns9qUUqoRv9+P3+935FwhzXg1xnQC5olI9yYeWx/4FjgS2AlcLyKvNHGsznhV\nSqkwRTPjNdSWvKGZFrqInFAvkCexbwb7JXillFLxFzTJG2OeAXxAe2PMRiAfOBAQEZnR6HBtpiul\nlIfoAmVKKeVxukCZUkqpJmmSV0qpJKZJXimlkpgmeaWUSmJOTIZSSqmEV15eRlFRHtXVFaSlZZKb\nW0jnzlluhxU1HV2jlEp55eVl5OcPYuTIUtLToaoKiouzmTq1xBOJXkfXKKVUFIqK8uoSPEB6Oowc\nWUpRUZ67gTlAk7xSKuVVV1fUJfiA9HSorq50JyAHaZJXSqW8tLRMqqoa3ldVBWlpGe4E5CBN8kqp\nlJebW0hxcXZdog/U5HNzC90NzAHa8aqUUtQfXVNJWlqGp0bXRNPxqkleKaU8TkfXKKWUapImeaWU\nSmKa5JVSKolpkldKqSSmSV4ppZKYJnmllEpimuSVUiqJaZJXSqkkpkleKaWSWNAkb4yZaYzZaoxZ\n0czj1xhjltd+LTLGdHM+TKWUUpEIuqyBMaYfsAN4WkS6N/F4X2C1iPzXGJMDFIhI32bOpcsaKEck\n6y4+SjUl5mvXGGM6AfOaSvKNjjsMWCkiHZp5XJO8ippTu/hs3w5r1kD79nDSSWAi+hNSKva8tHbN\nL4DXHT6nUg1Eu4vPtm0wYQJ07gy/+hUMHAjdu8M778QuZqXc4thG3saY/sB4oF9LxxUUFNR97/P5\n8Pl8ToWgUkQ0u/iUlsLgwXDxxbBxI7RrByLwyitwzTXw5z/DddfFKHClQuT3+/H7/Y6cy5FyjTGm\nO/AikCMipS2cR8s1KmpTpozG55vTINFXVYHfP4rp02c3+7yvv4a+feGWW+Cmm/Z/vLQUzj8fHngA\nLr00BoErFaF4lGtM7VdTF++ITfBjWkrwSjklkl18RGxLfejQphM8QHY2/POfMGkSlJXFIHClXBDK\n6JpnAB/QHtgK5AMHAiIiM4wxjwGXAhuwbwS7RaR3M+fSlrxyRLi7+Dz2mP364ANoE6RIeddd8MYb\nUFKinbHKG3RnKKVa8OWX0K2b7Vg9/fTgx+/ZAz16wLRpMHx47ONTKhhN8kq14IYbIC0N7rkn9Oe8\n+aYt26xZAwccELvYlAqFJnmlmvHFF3DOOfD553Y8fDgGDIBrr4WxY2MTm1Kh0iSvVDPGjbMdqnmh\nDaFv4K234MYbYdUqaN3a8dCUCpmXJkMp5RkVFXb8+403Rvb8Cy6Aww6z51AqUWmSV0nrwQdhzBg4\n4ojInm8MTJ4MDz/sbFxKxZOWa1RS+v57yMqCpUvt8gWR2rULOnSA99+369so5QYt1yjVSHEx/Oxn\n0SV4gIMOgvHj4dFHHQlLqbjTJK+S0mOPObcGzYQJMGcO1NQ4cz6l4kmTvEo6y5fDli1w4YXOnO+U\nUyAzE95+25nzKRVPmuRV0nnsMdv6dnLY46hRtjWvVKLRjleVVH78ETIyou9wbWzLFjjtNKisZL9l\njpWKNe14VapWSYktrziZ4AGOOw569YLXXnP2vErFmiZ5lVT+8Q+4+urYnPuyy2Du3NicW6lY0XKN\nSho//GBLNZ9/Dscc4/z5KyvtKpZbt+qiZSq+tFyjFPDqq9C7d2wSPNg3kJNPhnffjc35lYoFTfIq\naRQXx65UEzB8uJZsVGLRco1KCjt32s7RDRvg8MNjd501a2DgQNi0SXeNUvGj5RqV8hYuhD59Ypvg\nAbp0sRuQrFwZ2+so5RRN8iopzJ0Lw4bF51qDB9s9YJVKBJrkVcLbs8eOX49Xks/JgQUL4nMtpaKl\nSV4lvEWL7OSnDh3ic73+/eHDD2HHjvhcT6loBE3yxpiZxpitxpgVLRzzgDFmrTFmmTGmh7MhKtWy\nuXPtqJd4OfhgOPts8Pvjd02lIhVKS/5JYHBzDxpjLgKyReQkYBLwiEOxKRWUSHzr8QGDB2vJRiWG\noEleRBYB21s4ZBjwdO2xS4B2xpgYTUdRqqGVK6FVKzsTNZ5ycrTzVSUGJ2rymcCmercrau9TKuYW\nLrQJN95j1rt3h+++s+PylfKyNvG+YEFBQd33Pp8Pn88X7xBUEthQVsasvDz+b34FR3XLZENZIZ2y\nsuJ2fWPA57N1+bFj43ZZlSL8fj9+hzp9QprxaozpBMwTke5NPPYI8I6IPFt7ew1wvohsbeJYnfGq\norahrIwHBw1iamkpbYGdQH52NpNLSuKa6IuKYMkSmDUrbpdUKSoeM15N7VdTXgGurQ2kL/BtUwle\nKafMysurS/AAbYGppaXMysuLaxz9++sIG+V9Qcs1xphnAB/Q3hizEcgHDgRERGaIyHxjzBBjzDps\no2p8LANWam9FRV2CD2gL7K2sjGscp5wCu3ZBWRnE8QOEUmEJmuRF5JoQjrnJmXCUCq5VZiY7oUGi\n3wm0ysiIaxyBuvw772iSV96lM15VwhlXWMj/65TNztrbgZr8uMLCuMeiJRvldbrUsEpI995dxqv3\n59HvxEpaZWQwrjC+o2sCvvgCBgyAjRt16WEVO9F0vGqSVwlpzBjo1w8mTXI3DhE4/nh47z3IznY3\nFpW8dD15lVL27oWSErjwQrcjaTheXikv0iSvEs7KlXDIId7p7Dz3XHj/fbejUKppmuRVwlm40But\n+IB+/TTJK+/SJK8SjteSfNeusHUr/Oc/bkei1P40yauEUlUFixfboYte0bo19O2rrXnlTXFfoEy1\nrLy8jKKiPKqrK0hLyyQ3t5DOnT1SfPaAf/0LevSAQw91O5KGAnX5eG5eolQotCXvIeXlZeTnD8Ln\nm8OIEX58vjnk5w+ivLzM7dA8Y+FCGDTI7Sj2p52vyqs0yXtIUVEeI0eWkp5ub6enw8iRpRQVxXfh\nLS/zWj0+oE8fWL4cqqvdjkSphjTJe0h1dUVdgg9IT4fq6vguvOVVW7bA5s3Qq5fbkeyvbVs47TRY\nutTtSJRqSJO8h6SlZVJV1fC+qipIS4vvwlteVVICF1wAbTzak3TuubBokdtRKNWQJnkPyc0tpLg4\nuy7RV1VBcXE2ubnxX3jLi7xaqgnQurzyIl27xmP2ja6pJC0tQ0fX1Nq7F447zg6f9MpM18YqK6Fb\nNztevpU2n5SDdIEylfSWL4crrrCrPnrVhrIyhnXLY2DXCg45KdO1lTFV8okmyXu0uqlUQ14v1QT2\nnX1/ZyltP4SdH0L+4sVx33dWqcb0Q6VKCF4dHx/glX1nlWpMk7zyvB9+8N5SBo15Zd9ZpRrTJK88\nz6tLGdQX2He2Pjf2nVWqsZCSvDEmxxizxhjzhTFmShOPH2qMecUYs8wYs9IYM87xSFXK8no9Huy+\ns/nZ3th3Vqn6go6uMca0Ar4ABgCVwEfASBFZU++YPwCHisgfjDFHAp8Dx4jInkbn0tE1KmzdusHj\nj9ulA7xsQ1kZs/LyWOOvRI7OYPqLOrpGOSPWo2t6A2tFZEPtxYqBYcCaescIcEjt94cAXzdO8EpF\nYssWqKjw5lIGjXXKyiJ/9mz+/neYNw86aX5XHhBKuSYT2FTv9uba++p7CDjNGFMJLAducSY8lepK\nSmDAALtme6Lo2xeWLHE7CqUspzpeBwOfiEgGcCbwN2PMwQ6dW6Uwrw+dbMqJJ8KOHfZTiFJuC6Vc\nUwF0rHf7+Nr76hsP3AEgIqXGmDKgC7DfmnwFBQV13/t8Pnw+X1gBq9Sxd69tySda36Ux0Lu3bc3r\nJiIqEn6/H7/f78i5Qul4bY3tSB0AbAE+BK4WkdX1jvkb8JWITDXGHINN7meIyDeNzqUdrypky5bB\nlVd6eymD5hQUwK5dcMcdbkeikkE0Ha9ByzUiUgPcBCwEVgHFIrLaGDPJGHN97WHTgHOMMSuAEuD3\njRO8UuFKhKGTzenb107gUsptukCZ8qyBA+Hmm2HoULcjCd8330DnzrB9e2J1GitvimlLXik3/PCD\nrWknapfNEUfAscfCZ5+5HYlKdZrklSe99573lzIIRks2ygs0yStPWrAAcnLcjiI6ffroeHnlPk3y\nypPeeAMGD3Y7iuhokldeoB2vynM2boSzzoKtWxN7G70ff4TDD4cvv4RDDgl+vFLN0Y5XlVQWLLBD\nJxM5wQMceCB07w4ff+x2JCqVJfifkUpGyVCqCdCSjXKbJnnlKXv2wNtvJ+4kqMZ694YPP3Q7CpXK\nNMkrT1myxE4iOvZYtyNxhrbkldtCWaBMqbhJ5FJNeXkZRUV5VFdXkJaWSW5uISeckEV1tV0TP7Px\nAt1KxYG25JWnLFiQmEm+vLyM/PxB+HxzGDHCj883h/z8QWzYUKYlG+UqHUKpPGPbNsjOhv/8x45M\nSSRTpozG55tDevq++6qqwO8fxU9+Mpvqal2RUkVOh1CqpPDmm3D++YmX4AGqqysaJHiA9HSorq6s\nW1teKTdoklee8frribuUQVpaJlVVDe+rqoK0tAx694alS6Gmxp3YVGqLe5KfMmU05eVl8b6s8ria\nGpg/Hy6+2O1IIpObW0hxcXZdoq+qguLibHJzC2nfHo4+GtascTdGlZrinuQDHVKa6FV9ixdDRgZ0\n6uR2JJHp3DmLqVNL8PtH8fLL/fH7RzF1agmdO2cBOpRSuSfuHa/vvLOvQ2r69Nlxu7bytltvhTZt\nYNo0tyOJjfvvh9Wr4ZFH3I5EJaKE63gNdEgpFTBvHlxyidtRxI625JVbXEnygQ4ppQDWr4evv4az\nz3Y7ktjp0QM+/9zueKVUPMU9ydfvkFIKbCv+4osTf9XJlqSlQdeu8H//53YkKtXE/c+qcYeUUsle\nqgnQko1yQ0gdr8aYHOA+7JvCTBGZ3sQxPuBe4ADgPyLSv4ljdMarauC//4UOHWDLFmjb1u1oYuvp\np+G11+DZZ92ORCWamHa8GmNaAQ8Bg4GuwNXGmC6NjmkH/A34uYicDlwRSTAq9cyfD+edl/wJHrQl\nr9wRSrmmN7BWRDaIyG6gGBjW6JhrgBdFpAJARLY5G6ZKVs8/D5df7nYU8XHSSfaTy9atbkeiUkko\nST4T2FTv9uba++o7GTjCGPOOMeYjY8wYpwJUyWvHDnjrLRjWuMmQpFq1siOIdEVKFU9Odby2AXoC\nFwE5QJ4x5kSHzp0Q9u61e3lu2OB2JInj1VfhnHPgiCPcjiR++vTRJK/iK5RNQyqAjvVuH197X32b\ngW0iUg1UG2PeA84A1jU+WUFBQd33Pp8Pn88XXsQetHkzDB1qh4du2wZXXw333gutW7sdmbc9/zxc\nkWK9N717w4MPuh2FikRlJUyebP/G//Y3OP302F3L7/fj9/sdOVfQ0TXGmNbA58AAYAvwIXC1iKyu\nd0wX4EFsK/4gYAlwlYh81uhcSTe6Ztcu23H4859DXp6tuQ4bZlts//u/bkfnXTt22J2SyspSqyW/\ndSt06WInfyXzvIBks3s3/PSnMHCg3Z7y9tth5Uo47LD4XD+a0TVBW/IiUmOMuQlYyL4hlKuNMZPs\nwzJDRNYYYxYAK4AaYEbjBJ+spk+H446zCd4Y+0N/6SXo1s0m+3PPdS+2rVvtptjffWdbHT/9qXcS\ny/z5Np5USvAAxxwD7drB2rVwyiluR6NCNWMGHH643fjFGPjoI7jzTpvsvU53horCV1/BaafZH3hW\no7ldzz9vF9v65JP4J9aqKvjjH2HWLOjf3/5yfvgh/PijLRVceGF842nKpZfaTz8TJrgdSfxddZWd\n4XvttW5HokJRU2N3LHv++X1Lb5SW2k/rmzax32YxsZBwC5QliwcesH+wjRM82GGBBx0EL7wQ35i2\nb7eJvaLCthZfeglmzoQVK2w/wcSJtjXi5nvt11/bTxipMnSyMe18TSyvv24/gdVfWyk7G844wz7m\ndZrkI7RrFzz2mO2IaYoxMHWq/TgXr4S6axeMGGE79559Fo48smE8Q4bYddufew7+8If4xNSUf/zD\ntmQPPdS9GNyk2wEmlpkzYdKk/e+/8kr7t+R1muQj9PLL0L277URrTk6OLZG8+258YvrVr2yN+777\nbFJvSmam3Ut13jy4++74xNXYrFkwdqw71/aCnj1h1SqornY7EhXMrl3Nz+W49FLbkvf6z1GTfIRe\nfNEOlWyJMXDzzXbDiFh77TX7C/fkk8H7ANq3hzfesEm+pCT2sdW3ahV8+SUMGBDf63rJT35iO12X\nLXM7EhXMe+/ZQQvt2+//2FFHwamnwgcfxD+ucCRtkl+3ztbLJ060HaROqqqChQtDWzlxzBjbkt+0\nKfixkfr+e/txctYsO3IjFB06wDPP2Pg2boxdbI3NmgWjR+scAq3LJ4b5822ZszkDBtiWvpclZZL/\n+mvw+ezH4nbt7NhWJz9SvfWW3QTiqKOCH3vwwXbCz9NPO3f9xu64w3a2hjuvzOezJZ6xY+2M3Vir\nroannoJf/CL21/I6rcsnBk3yHnXbbbZeNmUK3HOP/Wj8l784d/65c2H48NCPnzDBtmBj0QG7fj08\n+ij89a+RPf9//gf27LGvU6w9+yycdRacmFILXjRNV6T0vnXr7ByTHj2aP+acc+DTT+2nac8Skbh9\n2cvFVmWlyOGHi3z11b771q8XOeIIke3boz//nj0iRx1lzxmqvXtFTj1V5L33or9+YyNHihQWRneO\n9etFjjxSZPlyZ2JqTq9eIq++GttrJIo9e0QOOURk2za3I1HNeeABkQkTgh/305+KvP12bGOpzZ0R\n5d2ka8nPmWNb8fVLKVlZtn7+8MPRn/+DDyAjo+mx8c0xBsaPt52iTlq1yo43v+WW6M6TlWVn740e\nbUcTxMLixbaMlpMTm/MnmtatoVcvrct7WbBSTUDfvt7+VJZ0Sf7pp21nYmOTJ8Pjj0dfe547145F\nD9eYMXZi0o4d0V2/vsJC+M1v4JBDoj/X2LF2gsef/hT9uZpyxx021lTvcK1PO1+964cf4P33bX9e\nMH362EaMVyVVkl+9Gr791i4Y1ljPnjYZRrOwm4gdHx9OPT7g2GPhZz9zbvLEqlX2/3Ljjc6czxi7\nPsfTT8O//uXMOQNWrrTJbOJEZ8+b6LTz1bveecf2H4UyWi3Qkvfqii1JleTffhsGDWp6nLgxMG5c\ndKNcVq60/3bvHtnzJ06EJ56I/Pr1TZ0Kv/2tHb3jlKOOsp24Y8c625H0l7/Ar38dnzU+EkmgJe/V\n5JDKQi3VAHTsaH+G8RyKHI6kSvLvvtvyMMLLL7czPX/8MbLzB1rxzc0mDWbIENtj//nnkT0/4NNP\n7SSNG26I7jxNGTrUDsf89a+dOd/y5fbNNzfXmfMlk4wM+8a3fr3bkaj6RMJL8sZ4uy6fNElexCb5\n889v/pgOHeDkk+1HsUhEWo8POOAAu/JgtK35wkLbio/V5tf33msT87x50Z1HBH73O1vnd6LfIBlp\nycZ71qyxfXennRb6c84+G5YujV1M0UiaJL9mjU16HTu2fNzll9slQ8NVXm5XdjznnIjCqzNhgi0Z\n7d4d2fM/+8zW4mPZMj70UDtpadKk6GYLz5tnZ/pef71zsSUb7Xz1nkArPpxP7L16aZKPOb+/5VZ8\nwOWX2xZ5uEl27lxbyoh2dEiXLnYUy/z5kT1/2jRbSnGyFt+U886zfQhXXBFZeevbb2056ZFH7CcY\n1TSvj8xIReGUagLOOsvu8RyPmePhcjXJb9li12QvL4/+XMFKNQGdOsEJJ4S/MmS4s1xbMnGiXaY4\nXGvW2BUknRpRE8zUqXanq/FjyygYNZr8/v2ZOno0G8rKWnyeiB2yeskl4S+1kGrOPtt26P/wg9uR\nqA1lZdx21Whq/P35cE7w3/P6jjzSrgC7br9drT0g0llUkXxRb8brt9+KnHSSyIgRIsccI1JeHvls\nsL17RY49VqSsLLTjp08Xuf760M//1Vci7dqJVFVFFN5+fvjBzpr9/PPwnjdqlMjttzsTQ6g+XbFe\nrjggW3bY3C07QH6bnS3lLUz5vf9+kTPOENmxI46BJrA+fUT8frejSG3l69fLb7PD+z1v7PLLRebM\niU18JOKM1zvvtHt8vvSSXY43mlmbX3wBBx5oN9gNxRVX2OuGWrKZN88OzUxLizjEBtLTbb07nCWI\nly2zrfibbnImhlC9MD2PJ3eXEujjbQtMLS1lVl5ek8fPnm3X0Xn55dh1DCebfv1g0SK3o0h8lZV2\nCHAk5dhZeXlMLQ3997wpXu18dSXJV1VBUdG+2ZW/+Y2tS34W4dbfoZZqArKybMkm1FE2L7zg/FZ1\nN95od0gKpWNTxI6myc+P/25KeysqaJyr2wJ7Kysb3FdTY2e13nqrfTMKZ9mHVKdJPnpvvWW343v/\nfbjrLjukccuW0J8f6u95S3r1svs9e40rSX7+fLuyW3a2vZ2WBtddZ5cdiITfH37t96qr7KqIwWzf\nbn9xwu2ICebYY+1SB9OmBT/21VftL+x11zkbQyhaZWays9F9O4FPtmTw6qv2jfL+++0EsYUL4d//\nDm/omYJzz7WvW02N25Ekps2bYeRIu5FPYMb2JZfA4MHw3/+Gdo7mfs9bZWSEHEfPnvYTt+d+jqHU\ndIAcYA3wBTClhePOBnYDlzbzuIiIXHWVyKOPNqw5rV4tkpFhV+cLx9699nnr1oX3vA8WrZeuB46S\n2873ScGoUc3W3mbNEhk+PLxzh+qrr0Tat2859u3bRTp0EFm4MDYxBNNUrfLmjtlyy+T1MniwyHnn\niUycKLJggf1ZqMiccorIsmVuR5GYhg0TKShoeN/evSLXXSdyzTWhnaN8/Xq5tl10NXkR28/46adh\nPSUkRFGTDyXBtwLWAZ2AA4BlQJdmjnsLeLWlJF9TYxPbxo37/0d69Ah/yc61a0UyM8NLMOF0sgwZ\nIjJ7dngxhWP6dJH+/UVqavbFVjBqlPzJ55P8a0bJiGHrJTc3dtcPRV1M/fu3+IaoIjdxoshDD7kd\nReJZutT+/VdX7//Yzp12ie9QOkN37xY56sj18uuLo/s9v/pq2zB0WqyTfF/g9Xq3b22qNQ/cAuQC\nT7SU5JcuFenSpen/yO23i9x8c3j/+cces6NOwlEwalRdgpd6ib6g0Yk2b7Zr08dylMju3SLnniuS\nl9f0m88VB2TLqpWaVJPdk0/avQFUeC6/XOTee5t//OOPRY4+WqSiouXzzJ8v0rt39PHcfbfITTdF\nf57GoknyodTkM4H6O5Rurr2vjjEmAxguIkVAi/PESkrsSJWmDBkS/iShcDtdIfROlqeegiuvjO0o\nkTZtbC3xuefgxpz9e/if3F3K838NvYdfJSbtfA3f1q02n7S0umnPnnYk26RJLS8E98QTdmG+aHlx\n5msbh85zHzCl3u1mE/3MmQX07AkFBeDz+fDV6zE94wzYuRPWroWTTgp+URHb6RruGuiBTpb6ubtx\nJ8uuXXa25ty54Z07EsccYzt3bzg9+h5+lZiys+2wv40bgy/Noaw5c+wExWDrIt12mx3e+Pe/27Wj\nGlu3zuYRJ1aIPfNMWLHC/iyjment9/vxR7Muen3BmvrYcs0b9W7vV64B1td+lQHfA18CQ5s4l7Rv\nb7foa87EiSL33RfaR5jSUpHjjgu/w6+pssivsxrW5B95RCQnJ7zzRivUMpJKTpddFrvJNMkonD68\nTz6xExCbKttMmGDLpU459VTnO9GJcU2+Nfs6Xg/Edrye2sLxT9JCTb5Dh5b/My++KDJ4cGj/8See\niLyOWb8zcfAJo+T66/Yl+G++sSN2/v3vyM4dKSdm3anEde+94none6JYv94m7XBG402dKtKvn51x\nHrBokZ0t/803zsU2ZozI4487dz6R6JJ80HKNiNQYY24CFmJH0MwUkdXGmEm1F57R+Cktna9Pn5av\nN3CgrY3t3Bm8Fh7J+PiATllZ5M+eDdgJSWeeaT+uXXIJXHONnRXbt29k545Up6wsJpeUcFdeHnsr\nK2mVkcHkwkI66cyilNCvX2RrGqWiV16xf6vhLBh42212/achQ+zcji+/tLnm8cfh8MOdiy1Ql/fM\nTmiRvjtE8gXInXcGf9fy+UReeaXlY/buFenY0Y6vd8Knn9qPf2lpdoRPuOP1lYrWnj0ihx0m8uWX\nbkfiff37i/zzn+E/b88ekb/8ReSEE+z6SvPmOR/b+++L9Orl7DmJoiVvpKUuZ4cZY+SGgaP4/YyW\nW6d33QWlpXbpg+asXWt3MNq0KfKdmpoi4uz5lArH8OF29ubIkW5H4l3ffGPXqfryS/jJT9yOZn8/\n/GBXpdy+HQ46yJlzGmMQkYgyU9yXNfjfN+fw4KBBLS7jOWQIvPZay0OeSkpsacfphKwJXrnpggvs\nrlyqee+9Zzfv8WKCBxvXiSfabTrDEav2dtyTfCiru516qq21rVrV/HnefLP58fahKC8vY8qU0dxy\nS3+mTBlNeXnoa0crFSua5IP717/spjZeFs5iZXv22J3e0tJsHX/PHmdjcWWBsmBjv41peWJUTY1d\nGGvAgMgyvmH4AAAO5ElEQVSuX15eRn7+IHy+OYwY4cfnm0N+/iBN9Mp1XbvCd9/Z8fKqaYmS5EOd\nFFVYaDuEy8rssukPP+xsLK4k+VBWd7v4YtuD3pQlS+ym3MceG9n1i4ryGDmylPR0ezs9HUaOLKWo\nSGeWKncZY/uaIt1sPtnt2GE/4Z99ttuRtCzUJF9RAQ8+CM88AxkZdgLmtGl2MqZT4p7kdwL52dmM\nKyxs8biBA+27W1Mtmueeg8suizyG6uqKugQfkJ4O1dU6s1S5T0s2zVu82C5T3vjv12vOOMO2yoNt\n63j33TB+PBx3nL3dtSt06+bsTPu4J/m7Ro1icklJ0LHfBx5oE/k//tHw/r174fnn7XrwkUpLy6Sq\nquF9VVWQlhb62tFKxUogycdx4FvCSIRSDdhRNb162U7i5mzbBrNm2U2T6hszxuY4p8Q9yefPnh3y\n5J7Ro+HJJxvugP7223DUUdClS+Qx5OYWUlycXZfoq6qguDib3NyWP10oFQ8nnmj/9eSm0C5LlCQP\ncNFF8PrrzT/+wAN2x7nMzIb3X3yxHT1YXe1MHK7t8RqKfv3scKT6tfm7745+n9POnbOYOrUEv38U\nL7/cH79/FFOnltC5s84sVe4zxrbm33rL7Ui8ZfduO2Ll3HPdjiQ0OTnwxhtNP/b993Ye0O9/v/9j\ngUbs4sXOxBH3yVDhXu+NN+CGG+Djj21n1JQpsHKlc5tqK+VFc+bYj+zxWAU1USxZAtdfD8uXux1J\naETg+ONtq7zxlph33gmffGI7XJvyu99Bu3YQGGmeUJOhwpWTYz/SdO0Kv/wlzJ6tCV4lv8GDbaPG\nyVEWiS6RSjVgP5Fde+3+Sxjv3An33GM3vW/OeefZ/68jcXi9JR+wahUcfbT9KKNUKujbF26/PfL5\nIMlm2DC7eGA0gy7ibe1aW3beuHHfEgfTptl81nhQSX1ffw1ZWXYJhzZtkrwlH9C1qyZ4lVoi2Skt\nWe3da3fOSqSWPNjNj/r0sategu1Mv+8+m+hb0r697ZBduTL6GBImySuVaoYMaXl0RipZvRoOO8xO\nGEo099xjB4zceqtdiuX22+1OYMH07AnLlkV/fU3ySnlUz572Y3sLa/mljESrx9d34om2f0UEHnrI\n7jcbijPPtJ2z0dIkr5RHtWplBx5oaz6xkzzY0TXTp9sx8KHSJK9UCrjoIq3LQ+In+UiceaYdLlp/\nMmgkNMkr5WE5OXZq/Pffux2JezZssENJTzrJ7Uji64gj7LaEpaXRnUeTvFIedthhdgjea6+5HYl7\nAq34VNzQ58wzo+981SSvlMdddhm8+KLbUbgnFUs1AT16aJJXKukNGwYLFwZftjZZpXKSP/30lnfI\nC0VISd4Yk2OMWWOM+cIYM6WJx68xxiyv/VpkjOkWXVhKqYAjj7SbZDS32FUy27bNbqzRvbvbkbij\na9c4JHljTCvgIWAw0BW42hjTeKHf9cDPROQMYBrwWHRhKaXqu/xyeOEFt6OIv0WL7PIObdq4HYk7\nDmhdRuv1o6M6RygvXW9grYhsADDGFAPDgDWBA0Sk/qKYi4FGKyQrpaJx6aV2xuSOHXDwwW5HE73y\n8jKKivKorq4gLS2T3NzCJpf6TuVSzYayMh4ZMoiP95YSzY88lHJNJrCp3u3NtJzEfwHo9A2lHHT0\n0XaUzUsvuR1J9MrLy8jPH4TPN4cRI/z4fHPIzx9Eefn+U3tTOcnPystjamkpbaM8j6Mfgowx/YHx\nQL/mjikoKKj73ufz4fP5nAxBqaQ1dqzd6Pnaa92OJDpFRXmMHFlat09rejqMHFlKUVEe06fPrjsu\nsGl3794uBeoiv9/P24sW4cQawaEk+QqgY73bx9fe14AxpjswA8gRke3Nnax+kldKhe6SS+yeChs3\nQseOwY/3qurqiv024k5Ph+rqygb3Jcqm3bHg8/m4oF8/frdhA22BqVGcK5RyzUfAicaYTsaYA4GR\nwCv1DzDGdAReBMaISJTzs5RSTUlLgyuvhKeecjuS6KSlZdbtrxxQVQVpaQ2XmEzlUg3AuMJC8rOz\n2RnleYImeRGpAW4CFgKrgGIRWW2MmWSMub72sDzgCOBhY8wnxpgPo4xLKdWEX/4SHn3U7neaqHJz\nCykuzq5L9FVVUFycTW5uYYPj3n0XUrma2ykri8klJdw1alRU50mYnaGUUtbPfgaTJ8MVV7gdSeT2\nja6pJC0tY7/RNdXVdn7Ali1wyCEuBuoR0ewMpUleqQTz3HN2XfL33ov9tTZvttdbtsyuh969O4wc\nCR06xPa6fr8dMrp4cdBDU0JKbP+nlLJGjID162Hp0thdY9s2Wxo64wz47DNbNhkwwF63Rw+YMiW2\nm4yneqnGSZrklUowBxwAv/89/PnPsTn/u+/a1Q8POsjuSfr44zBhAowbB0VFdiu+1avtWvfffReb\nGPx+TfJO0XKNUgmoutpuKzd3LvTq5dx5n3wS/vAHmDXLrmXfnJoauPFGu9H0W2/ZkT9Oqa6Go46C\nykqtxwdouUapFJOWZmvWf/qTc+e86y6YOtW25FtK8ACtW8PDD9va/Pjxtl7vlA8+sAtzaYJ3hiZ5\npRLUddfB2rXRbygiYt8wnnjCLgh2yimhPa9VK9vy/+ILO6zTKW+8EfxNRoVOyzVKJbCFC20H6apV\nkc0Mramxz1+xwu4l2759+OdYs8auq7NoEXRpvD5tBLp3hxkz7OqTytIhlEqlsNGj7cqUjzwS3vN2\n7YIxY+Cbb2xtP5rVLYuKbB3/gw9sKSdSmzfbET1ffRXdeZKN1uSVSmEPPwxvvgnPPBP6c77/3u44\ntWePLfdEu3zxpEl21M9jUe4ksWABXHihJngnpehS/Eolj0MPtUsQDxoEhx9uhzYGbCgrY1ZeHnsr\nKmiVmcm4wkJ27c5i+HA7c/ahh5zZkKNVK9uav+ACO47/mGMiO8/cuXDVVdHHo/bRco1SSWLxYhg6\nFPLy4KabYGN5GQ8OGlS3JvlO4Jb22by0p4Q7pmcxaZLzMUyZYrfrmz07+LGNffstdOoEmzbZNy61\nj9bklVIAlJbaZQeMgZMZzaMfzWmw6cRO4E8/H8Xd8yLIwiHYudMOf5w5086QDcfTT9tPJHPnxiS0\nhKY1eaUUANnZsGQJ/PGPsGtjxX67CrUFDt5Z2dRTHdG2LTzwANxwQ/jLHjz/fGIvuuZVmuSVSjKt\nWsHw4XD6wMz91iLfCbTKyGjqaY4ZOtQOpbzzztCfs2WLHYI5dGjs4kpVWq5RKkltKNu/Jp+fnc3k\nkhI6Ze2/abaj194AZ51lP1VkZwc//vbb7Y5XTk6qSiZak1dKNaludE1lJa0yMhhXWBjzBB8wfbpd\naGz+fNtH0Jwff9y3Dk/PnnEJLeFokldKec7u3XbxtJtvhokTmz9uxgx48UU7Rl41LZokr+PklVIx\nccABdoLW+efDOefAqafueyzwCePHjRW89nEmf3q6EIjPJ4xUoy15pVRMPfUUFBTA++9DRoa7fQWJ\nSodQKqU8a+xYu+yBzwfLl8OsvLy6BA92WOfU0lJm5eW5GGXyCqlcY4zJAe7DvinMFJHpTRzzAHAR\n9o15nIgsczJQpVTiuvVW24ofOBD6/Nj0+P29lbEbv5/KgrbkjTGtgIeAwUBX4GpjTJdGx1wEZIvI\nScAkIMz18FKP3+93OwTP0Ndin2R+La691g6T7HB2aOP3k/m1iKdQyjW9gbUiskFEdgPFwLBGxwwD\nngYQkSVAO2NMhEsUpQb9Bd5HX4t9kv21SE+HWx8rJD87uy7RB2ry4woLGxyb7K9FvIRSrskENtW7\nvRmb+Fs6pqL2vq1RRaeUSjqdsrKYXFLCXfXG70+O4/j9VKNDKJVScdcpK4v8SJaqVGELOoTSGNMX\nKBCRnNrbtwJSv/PVGPMI8I6IPFt7ew1wvohsbXQuHT+plFIRiOVkqI+AE40xnYAtwEjg6kbHvALc\nCDxb+6bwbeMEH02QSimlIhM0yYtIjTHmJmAh+4ZQrjbGTLIPywwRmW+MGWKMWYftRxkf27CVUkqF\nIq4zXpVSSsVXTGa8GmNyjDFrjDFfGGOmNHPMA8aYtcaYZcaYHrGIwwuCvRbGmGuMMctrvxYZY7q5\nEWc8hPJ7UXvc2caY3caYS+MZXzyF+DfiM8Z8Yoz51BjzTrxjjJcQ/kYONca8UpsrVhpjxrkQZswZ\nY2YaY7YaY1a0cEz4eVNEHP3CvnGsAzoBBwDLgC6NjrkIeK32+z7AYqfj8MJXiK9FX6Bd7fc5qfxa\n1DvuLeBV4FK343bx96IdsArIrL19pNtxu/ha/AG4I/A6AF8DbdyOPQavRT+gB7CimccjypuxaMnr\n5Kl9gr4WIrJYRP5be3Mxdn5BMgrl9wJgMvAC8FU8g4uzUF6La4AXRaQCQES2xTnGeAnltRDgkNrv\nDwG+FpE9cYwxLkRkEbC9hUMiypuxSPJNTZ5qnLiamzyVbEJ5Ler7BfB6TCNyT9DXwhiTAQwXkSIg\nmUdihfJ7cTJwhDHmHWPMR8aYMXGLLr5CeS0eAk4zxlQCy4Fb4hSb10SUN3UylEcYY/pjRyX1czsW\nF90H1K/JJnOiD6YN0BO4ALt+17+NMf8WkXXuhuWKwcAnInKBMSYbKDHGdBeRHW4HlghikeQrgI71\nbh9fe1/jYzoEOSYZhPJaYIzpDswAckSkpY9riSyU16IXUGyMMdja60XGmN0i8kqcYoyXUF6LzcA2\nEakGqo0x7wFnYOvXySSU12I8cAeAiJQaY8qALsDSuEToHRHlzViUa+omTxljDsROnmr8R/oKcC3U\nzahtcvJUEgj6WhhjOgIvAmNEpNSFGOMl6GshIifUfmVh6/I3JGGCh9D+Rv4J9DPGtDbG/ATb0bY6\nznHGQyivxQZgIEBtDfpkYH1co4wfQ/OfYCPKm4635EUnT9UJ5bUA8oAjgIdrW7C7RaTxAnAJL8TX\nosFT4h5knIT4N7LGGLMAWAHUADNE5DMXw46JEH8vpgGz6g0t/L2IfONSyDFjjHkG8AHtjTEbgXzg\nQKLMmzoZSimlkphu/6eUUklMk7xSSiUxTfJKKZXENMkrpVQS0ySvlFJJTJO8UkolMU3ySimVxDTJ\nK6VUEvv/Tj3WO8qwaxoAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def toy(x):\n", " return np.exp(-100*np.power(x-0.5,2)) +\\\n", " 0.5*np.exp(-10*np.power(x-0.2,2)) +\\\n", " 0.5*np.exp(-100*np.power(x-0.9,2)) +\\\n", " 0.1*np.sin(-100*np.power(x-0.5,2))\n", "\n", "X_full = np.linspace(0,1,1000)\n", "X_sample = np.linspace(0,1,10)\n", "\n", "heavy_noise = [ 0.31006098, -0.06900007, 0.49688356, 1.20370015, 0.2254434, -0.15198685,\n", " -0.93082213, 0.16441064, 0.87895284, -0.52823194]\n", "\n", "toy_observed = toy(X_sample)\n", "toy_noisy = toy_observed + heavy_noise\n", "\n", "plt.ylim([0,1.5])\n", "plt.plot(X_full,toy(X_full))\n", "plt.plot(X_sample,toy_observed, 'or')\n", "plt.plot(X_sample,toy_noisy, 'oy')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Regularized least-squares regression\n", "\n", "### Task 1: Eigenvalue decomposition\n", "Implement subroutines for the Gauss kernel\n", "$$\n", " k = k_\\gamma(x,y) = \\exp(-\\gamma \\|x-y\\|_2^2)\n", "$$\n", "and the Dirichlet kernel\n", "$$\n", " k = D_m(x,y) = \\frac{\\sin((m+1/2)(x-y))}{\\sin((x-y)/2)},\n", "$$\n", "which compute the matrix $(k(x,y))_{x \\in X, y \\in Y}$ for two collections of points $X$ and $Y$. You can reuse code which you have written for the previous notebook. Moreover, you can use routines from scikit-learn for kernel evaluations." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def gauss_kernel(X, Y, gamma):\n", " \"\"\"\n", " Evaluates the Gauss kernel k(x,y) = exp(-gamma \\|x-y\\|_2^2) with width gamma\n", " in all tuples (x,y), x in X, y in Y.\n", " \n", " Parameters:\n", " X - array-like with N elements\n", " Y - array-like with M elements\n", " gamma - float\n", " \n", " Returns:\n", " keval - matrix or ndarray of shape (N,M)\n", " \"\"\"\n", " # your code goes here\n", "\n", "\n", "def dirichlet_kernel(X, Y, m):\n", " \"\"\"\n", " Evaluates the Gauss kernel Dirichlet kernel D_m(x,y) = sin((m+0.5)*(x-y))/sin(0.5*(x-y)\n", " in all tuples (x,y), x in X, y in Y.\n", " \n", " Parameters:\n", " X - array-like with N elements\n", " Y - array-like with M elements\n", " gamma - float\n", " \n", " Returns:\n", " keval - matrix or ndarray of shape (N,M)\n", " \"\"\"\n", " # your code goes here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next provide a routine which computes the eigenvalue decomposition of a given kernel matrix. You can use any numpy or scipy routine to achieve this." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def eigenvalue_decomposition(kernelmatrix):\n", " \"\"\"\n", " Computes the eigenvalue decomposition for the given symmetric kernelmatrix.\n", " \n", " Parameters:\n", " kernelmatrix - array-like of shape (N,N)\n", " \n", " Returns:\n", " eigval - array-like with N elements; the eigenvalues\n", " Q - matrix of shape (N,N); the normalized (unit “length”) eigenvectors,\n", " such that the column Q[:,i] is the eigenvector corresponding\n", " to the eigenvalue eigval[i].\n", " \"\"\"\n", " # your code goes here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Task 2: Regularized least-squares\n", "\n", "Using the subroutines from Task 1, implement a subrotuine which numerically solves the regularized empirical risk minimization problem\n", "$$\n", " \\hat f_\\lambda = \\text{argmin}_{f \\in H} \\frac{1}{N} \\sum_{i=1}^N (f(x_i) - y_i)^2 + \\lambda \\|f\\|_H\n", "$$\n", "where $H$ is the reproducing kernel space associated to either the Gauss kernel $k_\\gamma$ or the Dirichlet kernel $D_m$. Note that you are not allowed to use any scikit-learn routine which does the regression for you." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def compute_predictor(kernel, eigval, Q, sample_points, sample_values, regparam):\n", " \"\"\"\n", " Computes the function approximation obtained via regularized empirical risk minimization.\n", " \n", " Parameters:\n", " kernel - function object which takes two array-like arguments X and Y\n", " eigval - array-like with N elements; the eigenvalues\n", " Q - matrix of shape (N,N); the normalized (unit “length”) eigenvectors,\n", " such that the column Q[:,i] is the eigenvector corresponding\n", " to the eigenvalue eigval[i].\n", " sample_points - given sampling points; array-like with N elements\n", " sample_values - values observed at the given sampling points\n", " regparam - float; the regularization parameter\n", " \n", " Returns:\n", " predictor - function object which takes an array-like argument X; predictor(X) returns\n", " an array-like object containing the values of approximant\n", " at the points X\n", " \"\"\"\n", " # your code goes here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Task 3: The regularization parameter\n", "\n", "Consider the Gauss kernel with $\\gamma = 100$. Generate eight plots arranged like\n", "\n", " \n", " \n", " \n", " \n", "
Plot 1Plot 2
Plot 3Plot 4
Plot 5Plot 6
Plot 7Plot 8
\n", "each showing: the toy function evaluated on X_full, the computed approximant $\\hat f_{\\lambda}$ evaluated on X_full and the used data points.\n", "\n", "The plots int the first column are based on uncorrupted data, the plots in the second column on the given noisy data,\n", "\n", "Row 1: $\\lambda = 0.001$\n", "Row 2: $\\lambda = 0.1$\n", "Row 3: $\\lambda = 0.5$\n", "Row 4: $\\lambda = 1$" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# your code goes here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Task 4: Dependency of the RMS on the regularization parameter\n", "\n", "Consider the rooted mean square error (RMS)\n", "$$\n", " \\sqrt{\\frac{1}{|X_{full}|} \\sum_{x \\in X_{full}} (f(x) - \\hat f_\\lambda(x))^2}\n", "$$\n", "where $f$ is the considered toy function and $\\hat f_\\lambda$ the approximation computed with regularization parameter $\\lambda$. \n", "Generate four plots arranged like\n", "\n", " \n", " \n", "
Plot 1Plot 2
Plot 3Plot 4
\n", "where each plot shows two curves: one curve showing how the RMS depends on the regularization parameter $\\lambda$ when uncorrupted observations toy_uncorrupted are used; and one curve showing the dependency for the very noisy observations toy_noisy.\n", "\n", "Plot 1: Gauss kernel with $\\gamma = 100$\n", "\n", "Plot 2: Gauss kernel with $\\gamma = 1$\n", "\n", "Plot 3: Dirichlet kernel with $m = 10$\n", "\n", "Plot 4: Dirichlet kernel with $m = 3$\n", "\n", "As regularization parameters use the values\n", "$$\n", " \\lambda_i = 0.001 + \\frac{1.5-0.0001}{100} i, \\quad i=0,\\dots,100\n", "$$" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# your code goes here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Tests\n", "\n", "This section will be used by the tutor to check your code for correctness. Don't change any code here! You are free to run the tests below yourself to check your code.\n", "\n", "### Tests for Task 1" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def test_kernel(kevaluator):\n", " kernelmatrix = kevaluator(X_sample1, X_sample1)\n", " keval = kevaluator(X_sample1, X_full)\n", " eigval, Q = eigenvalue_decomposition(kernelmatrix)\n", " print('\\t Type checks: '+\\\n", " str(isinstance(Q,np.matrixlib.defmatrix.matrix)))\n", " print('\\t Shape tests: '+\\\n", " str(kernelmatrix.shape == (X_sample1.size, X_sample1.size)) +' '+\\\n", " str(keval.shape == (X_sample1.size, X_full.size)))\n", " print('\\t Checking kernel matrix for symmetry: '+\\\n", " str(np.allclose(kernelmatrix.transpose(),kernelmatrix)))\n", " print('\\t Checking eigenvectors: '+\\\n", " str(np.allclose(np.eye(X_sample1.size), Q.transpose()*Q)))\n", " print('\\t Checking eigenvalue decomposition: '+\\\n", " str(np.allclose(kernelmatrix, Q*np.diag(eigval)*Q.transpose())))\n", " \n", "print('Testing the Gauss kernel:')\n", "test_kernel(lambda X,Y: gauss_kernel(X,Y,1.0))\n", "\n", "print('Test the Dirichlet kernel: ')\n", "test_kernel(lambda X,Y: dirichlet_kernel(X,Y,1.0))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Tests for Task 2" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "kernel = lambda X, Y: gauss_kernel(X,Y,100)\n", "kernelmatrix = kernel(X_sample, X_sample)\n", "eigval, Q = eigenvalue_decomposition(kernelmatrix)\n", "predictor = compute_predictor( kernel, eigval, Q, X_sample, toy_observed, 0.1)\n", "\n", "target_values = [ 0.19781951, 0.24827233, 0.24460145, 0.30744904, 0.56295652, 0.48216615,\n", " 0.09553423, 0.04689467, 0.22911141, 0.13006833]\n", "\n", "print('Checking predictor: '+\\\n", " str(np.allclose(target_values, predictor(X_sample))))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.1" } }, "nbformat": 4, "nbformat_minor": 0 }