{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Sheet 2 Tim Racs and Derrick Hines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import math"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task 2.1.** Implement the function OneStep\n",
    "\n",
    "$f (\\cdot) = \\sum_{l=1}^n \\beta_l y_l \\langle \\cdot, x_l \\rangle + b$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def set_f(beta,y,x_train,b):\n",
    "    def f(vec):\n",
    "        prod=beta*y*x_train.dot(vec)\n",
    "        return prod.sum()+b\n",
    "    return f\n",
    "\n",
    "def OneStep(i,j,beta,y,x,C,b):\n",
    "    f=set_f(beta,y,x,b)\n",
    "    delta=y[i]*(f(x[j])-y[j]-f(x[i])+y[i])\n",
    "    s=y[i]*y[j]\n",
    "    chi=x[i].dot(x[i])+x[j].dot(x[j])-2*x[i].dot(x[j])\n",
    "    gamma=s*beta[i]+beta[j]\n",
    "    if s==1:\n",
    "        L=max(0,gamma-C)\n",
    "        H=min(gamma,C)\n",
    "    else:\n",
    "        L=max(0,-gamma)\n",
    "        H=min(C-gamma,C)\n",
    "    if chi>0:\n",
    "        beta[i]=min(max(beta[i]+delta/chi,L),H)\n",
    "    else:\n",
    "        if delta>0:\n",
    "            beta[i]=L\n",
    "        else:\n",
    "            beta[i]=H\n",
    "    beta[j]=gamma-s*beta[i]\n",
    "    f=set_f(beta,y,x,b)\n",
    "    b=b-(f(x[i])-y[i]+f(x[j])-y[j])/2\n",
    "    \n",
    "    return beta, b\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task 2.2.** draw 20 two-dimensional vectors according to an exponential distribution\n",
    "with $\\lambda = 4$ in each of the coordinate directions, i.e. the j-th coordinate of the\n",
    "i-th vector is drawn i.i.d. according to $(x_i)_j ∼ \\exp(4)$ for all $i = 1, \\dots , 20$\n",
    "and $j = 1, 2$. We assign the label -1 to these $x_i$. Then, we draw 20 two--dimensional\n",
    "vectors according to exp(0.5) in the same way and assign the\n",
    "label 1 to them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "x=np.append(np.random.exponential(1/4,(20,2)),np.random.exponential(2,(20,2)),axis=0)\n",
    "label=np.append(-np.ones(20,int),np.ones(20,int))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(SVM): $\\max_{\\mathbf{\\beta}\\in\\mathbb{R}^n} \\sum_{i=1}^n \\beta_i - \\frac{1}{2} \\sum_{i,j=1}^n \\beta_i  \\beta_j  y_i y_j \\langle \\mathbf{x_i}  , \\mathbf{x_j}\\rangle $ s.t. $0\\leq \\beta_i \\leq C $, $\\sum_i^n \\beta_i y_i =0$\n",
    "\n",
    "**Task 2.3.** Implement a function **SMO** which initializes $\\mathbf{\\beta} = 0$ and $b = 0$ and - in each iteration step - randomly picks $i, j \\in \\{1, \\dots , n\\}$ such that $i \\neq j$ and\n",
    "calls **OneStep** with indices $i, j$ to perform an optimization.\n",
    "\n",
    "**a)** After the last iteration step, we need to compute a final estimate for b.\n",
    "To this end, calculate the median $med$ of $f (\\mathbf{x_k}) - y_k$ for all support\n",
    "vector indices k, i.e. all $k \\in \\{1, \\dots , n\\}$ for which $\\beta_k > 0$. Then, set\n",
    "$b :=  b - med$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def SMO(x,y,C,steps=10000):\n",
    "    n=x.shape[0]\n",
    "    beta=np.zeros(n)\n",
    "    b=0\n",
    "    \n",
    "    for count in range(steps):\n",
    "        i=np.random.randint(n)\n",
    "        j=np.random.randint(n)\n",
    "        while i==j:\n",
    "            j=np.random.randint(n)\n",
    "        \n",
    "        beta, b=OneStep(i,j,beta,y,x,C,b)\n",
    "    \n",
    "    #a\n",
    "    f=set_f(beta,y,x,b)\n",
    "    fval=np.apply_along_axis(f,1,x)\n",
    "    med=np.median((fval-y)[beta>0])\n",
    "    b=b-med\n",
    "    return beta, b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*(b)* Run the *SMO* function with 10,000 iteration steps to compute a support\n",
    "vector classifier f for the n = 40 data points from *task 2.2*. Compute the results for C = 0.01, C = 1 and C = 100. For each C, plot the scattered data and the hyperplane corresponding to f = 0. Compare your results to the separating hyperplane computed by a linear least squares algorithm.\n",
    "\n",
    "*(c)* Count the number of support vectors.\n",
    "*(d)* Furthermore, also count the number of margin defining vector\n",
    "\n",
    "Mark the corresponding $\\mathbf{x_k}$ in your scattered data plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 0.01 \t #support vectors= 36  #margin vectors= 2\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XlcVdX+//HXElFA00wB5+HemyUgojJKIGiaikOGOFTOZl2zrN8thywlS63sdvuW3rpWppWzlvPVtFQccABFUhy7OaXiVKYgobB+f2w8yqDAOQcO5/B5Ph48jmexz96fc6g3i7XXXltprRFCCGH/Kti6ACGEENYhgS6EEA5CAl0IIRyEBLoQQjgICXQhhHAQEuhCCOEgJNCFEMJBSKALIYSDkEAXQggHUbGwDZRSs4CuwHmttc8d7S8AI4GbwGqt9ejC9lWr+v26ce26FpQrhBDlT+KRgxe11u6FbVdooAOzgenAV7calFKRQA/AV2v9p1LKoyhFNa5dl4SZXxW+oRBCCBMVEXCiKNsVOuSitY4DLudp/jvwjtb6z5xtzhe7QiGEEFZl7hh6UyBMKbVTKbVZKRVgzaKEEEIUX1GGXO72uhpAMBAALFJK/UUXsHSjUmo4MBygoWdtc+sUQghRCHMD/TTwbU6A71JKZQO1gAt5N9RazwRmAvg/5CVr9QpRCm4oOO3mRIaTsnUpohhcsjT107NwNjMpzQ30ZUA7YJNSqilQCbho5r6EEFZ22s2J+zw9aFy9OkpJqNsDrTWXrlzhdOp5mqRlmbWPQsfQlVLzgXjgIaXUaaXUUGAW8Bel1H5gATCwoOEWIYRtZDgpakqY2xWljJ+ZJX9VFdpD11r3u8u3njb7qEKIEidhbn8s/ZnJlaJCCOEgJNCtZf1/oU83iAw0Htf/19YVCWFT51JT6TtoIH/1bY5XQGu6RD/BkaNHOX7iBD5BpTPTefykWBo0e4iqdTxL5Xi2JoFuDev/C+9PgdRzoLXx+P4UCXVhN+YucqKxtysVqrvR2NuVuYucLNqf1pqeT/YjIiyMn5N/ImV3IlMmxpJ6oXSvQezWqQu7Nm4u1WPakgS6NXz+b/gzI3fbnxlGuxBl3NxFTgx/oTInTlVAa8WJUxUY/kJli0J9Y9xmnJ0r8tzQYaY2P19fwtqE5tru+IkThD3WgVZhobQKC2X7zh0AnD13jvBOHfELDcEnKIAt27eRlZXFoOeexScogObBgfxr+vRC6wgODKRO7fJz/Yu50xbFnc6nFq9diDJk/JuVSL+e+2Rc+nXF+Dcr8VTv62btc39KCq39Wha6nYe7O+uXr8TFxYWjx47Rb+hgEjZvYd7iRTzW/lHGvzqarKws0tPTSUpO5tezZ9i/czcAv//+u1m1OTIJdGvw8DSGWQpqF6KMO3m64JkVd2u3phs3bjDylX+Q9FMyTk5OHDl2DICAVq0YMmIEN27c4PGu3fDz9eUvjRvzv1+O88Ir/yDqsU50bN++xOuzNzLkYg3DRkBll9xtlV2MdiHKuIb1C76E5G7tReHdrBmJSXsL3e5fM6bj6eHBvu07SNi8hczMTADCQx8hbu066tWtS//hw/hq3jxq1KjBvu3xRISFMeOzmQwb+XyufWVlZeEXGoJfaAgT3n7L7NrtmQS6NXToDK+8Bp61QSnj8ZXXjHYhyrjJEzNxc80d3m6umskTM83eZ7u2Efz5Zyafzf7S1LY7MZHNW7fk2u7KH39Qp3ZtKlSowNcL5pOVZVwheeLkSTzc3Xlm0GCG9h/Inn1JXLx0kezsbKJ7PM5br7/Bnn1Jufbl5ORE0rZ4krbFM+n1N8yu3Z7JkIu1dOgsAS7s0lO9s4A/Gf9mJU6eVjSsb4S50W4epRTfzZvPS2NH884HH+DiUpnGDRvx4Tvv5tpuxLBniO7/FIu/+47I8HCqVKkCwKYtW5j20Yc4OztTtUoVvvrPZ/x65iyDRzxHdnY2AFMnvlloHaPfeJ15ixeRnp5O/YebMmzAQGJfG2/2+yrrVGlese//kJeWG1wIUfIOVqtIs789aOsyhBkOHjtKsz9u5mpTEQGJWmv/wl4rQy5CCOEgJNCFEMJBSKALIYSDkEAXQggHIYEuhBAOQgJdCCEcRFHuWDRLKXU+5+5Eeb/3ilJKK6VqlUx5QojScuCgwifIlQMHrXPJ/+Rp7+Ed6I9vSBB+oSHs3L3bKvs1x4czZpCenp6vPXbKZMbFTszVlpScTDP/VsU+RlJyMmvWrTO7RmsoSg99NtApb6NSqgHQAThp5ZqEEKUsLQ269HIh5ZAiqpcLaWmW7S9+505WrV3Lni3bSI7fyYYVK2lQv751ii2mrKwsPvxkBunX8wd6v5gYFn67NFfbgqVLeDKmd7GPk/RTMmu+L16g37x5s/CNiqHQQNdaxwGXC/jWv4DRgNxLVAg7N2REZc5fUGitSL2gGPp8ZYv2dzb1HLVq1qRyZWM/tWrWom6dOgA09vHi4iXjnvIJe/YQ0cXoL8ZOmUz/Z4bRrmsXHvRrYVo2YNOWOMI7daTnk33xCmjNcy+9aLpadP7iRTQPDsQnKIAxE25f7l+1jicT3n6LoMgIJk97jzNnzxIZ1YXIqNxXcz/0YFPur149118Pi777lr7RvQD4/ocfCGnfjlZhocQMeJpr164BxjIGbR5tT4s2wQRGtOXKlStMmPw2C79dil9oCAuXLuHy5cs83q8vviFBBLeLJHn/ftP7HP7iSDr26M6A4c9Y9DnnZdYYulKqO/Cr1nqfVasRQpS6WV9XZPU6JzIyjKGWjAzFyrVOzPra/JVBOrZrz6lfT9O0pR8jXn4p3xoud5N8YD+rFy8l/ocfmPTuO5w5exaAXYmJ/HPyVH7asYuff/mFb1cs58zZs4yZOIEfV60maVs8u/cksmzVSgDS0tLw8fJi58ZNTBg7jrp16rBx9Ro2rs5/05l+vWJYsHQJADt27aJmjQd48G9/4+Kli7w97V02rFjJni3b8G/Zig+mf0xmZiZ9Bg/k/959j33bd7BhxUqqVKnCpPGv0+eJaJK2xdMnuhcTp0ympa8vyfE7mTJxIgOevR3eiUlJLF+wkHmzvsxXjyWKHehKKTdgPDChiNsPV0olKKUSLlz5rbiHE0KUsHGxzqSl51kPPV0xLtbZ7H1WrVqVxLitzPzoI9xr1aLPoIHMnvtNoa/r0SUKV1dXatWsRWRYOLsSEwAIbN2avzRpgpOTE/16xbA13gjwiEfCcK/lTsWKFXmqdx/itm0DjIW6ons8XqRa+0b3YsnyZWRnZ7Ng6RL6xcQAsGPXblIOHSK046P4hYYwZ95cTpw6xeGjR6jjWZuA1q0BqFatGhUr5v/lt3VHPP379gOMxcouXb7MlStXAOjeuQuurq5Fqq84zPkV/FegCbAv5w7V9YE9SqlArXW+RcG11jOBmWCs5WJBrUKIEjA19gYvvlopV6i7uWneefOGRft1cnIiIiyciLBwmnt7M2fePAY99TQVnSqahkwy8tzpK+9d7289L6j9XutQubi44ORUtDsuNahfn8YNG7J56xaWrlhO/IYfAdBoOkS2Y/6Xs3Ntn7x/f756ClJQfbded2sRMmsrdg9da/2T1tpDa91Ya90YOA20KijMhRBl35D+N4l6LAsXFyOAXFw03TplMfhp80/YHT56hKM5N6sAYwZIowYNAGjcqCGJe42lb5cuX57rdcvXrCYjI4NLly6xaesWAloZveBdiYn8cvw42dnZLFy6lEdCQgjyD2Dztq1cvHSRrKws5i9ZTNtHHimwnvuqVuXq1Wt3rbdfrxheHjeWvzZpQv169QAIDghg284dHPv5ZwDS09M5cvQoDzdtyplzZ9mdmAjA1atXuXnzpnGMa7ePEd4mlLmLFgLGeYBaNWtSrVq1on+IZijKtMX5QDzwkFLqtFJqaIlWJIQodbP+/Sce7hqlNJ7umi9m/GnR/q5dS2Pgc8/iFdAa35AgUg4fIva11wCYOHYco8aMJuyxDvl60YGtWxMVE01w+3a8MXqM6URqSGAgY2Mn4BMUQJPGjejZrTt1atdm6sQ3iYzqQos2wbRq4UePqK4F1jN80GA6R/fMd1L0lpiePTlw8KDpZCiAey13Zn/yKf2GDDZObLaP5NDRI1SqVImFX87hhVdfoUWbYDr06E5GRgaRYeGkHDpkOikaO+41EvbuxTckiLETJzLn05kWfaZFIcvnCuGAzFk+98BBRZ9BLiycnYF3s9IfHY2dMpmqVavyyoujcrVv2hLH+x99xKrFS0q9JluwZPlcucGFEAIA72aa/TvNuym0KBsk0IUQZcLd7iR068SqKJys5SKEEA5CAl0IIRyEBLoQQjgICXQhhHAQEuhCiBKhqlWl/zPDTM9v3ryJe5NGdI3pdY9XFc+wkc+TcuhgkbY9fuIE9R9uarpK9Ra/0BB2JSQU+9hT3p9W7NeUNAl0Icq59z50ZmNc7ijYGFeB9z40fy0XMC5v338whevXjamQ63/8kXp16xZrH4UtL/v59Bl4PdysSPtq3KgRDerVZ8v2baa2Q0cOc/XqNQL9C53inc+Uf75frO211vl+mVibBLoQ5VxAqyx6D3QxhfrGuAr0HuhCQKssi/fduUNHVq9bC8D8JYvp1yvG9L1dCQm0ebQ9LR9pQ5tH23P46BEAZs/9hpgBT9OtdwwdH+9OdnY2I15+Ce9Af7rG9KJL9BMsWfYdABFdOpGwZw9gLJk7flIsLdoEE9wuktTzqfnq6derl2llRYAFS5bQr5fxF8OFixeIfvpJAtqGE9A2nG074gG4du0ag//+HM2DA/ENCWLp8mWMnTiB69ev4xcawlNDhwDwwfSP8QkKwCcogA9nzACMvwqa+bdixMsv0SoslFOnT1v8md6LBLoQ5VxkeDaL5mTQe6ALE952pvdAFxbNySAy3PLeZN9oI0AzMjJIPrCfoDt6wg83bUrc2nXs3bqdSeNf57U3Y03fi9+1izmf/ocfV63h2xXLOX7yJD/t2MXn02cQv3tXgcdKS0sjOCCQfdt3EB4aymezZ+fbpvcT0SxbtcrU81/47VL65gT6qNGjefn5kezeHMfSb+YybOTzALz13rtUr1aNn3bsIjl+J+3C2/LOm5NwdXUlaVs8c7+YReLevXz5zdfs/HETO37YyGdzvmTvPmN18cNHjzKg35Ps3bqdRg0bWvyZ3otcWCSEIDI8m78PvcFb71XijdGZVglzAF8fH46fPMn8JYvp0vGxXN+78scfDHxuOEd//hmlFDdu3F7dsUNkOx544AEAtsbHE9OzJxUqVKC2pyeRYWEFHqtSpUp07WSs1dLaz4/1Gzfm26a2pyfezZrxw6ZNeHp44OzsjI+XNwAbNm0k5fAh07Z/XL3K1atX2bBpIwtmzTa116hRI99+t8Zvp2fXbqZVFJ/o1p0t27fRvUsUjRo2JDgwsCgfl8Uk0IUQbIyrwCdfOPPG6Ew++cKZyPAsq4V6985deGX8eDatWcOly7dvfvbG228RGRbOd/MWcPzECSLuWDiripub6d9FXW/K2dnZtDytk5PTXcffb93QwtPDI9cQUHZ2NvEbfsy3TrnWutDlcu9V4p3vpaTJkIsQ5dytMfNFczKY9PoN0/BL3hOl5hrSfwATxoyhubdPrvYrf1wxnSS9180vHgkJYeny5WRnZ5N6PpVNW7ZaVE909x6s+X6dMdxyx+qKHdu1Z/rM/5ieJyUnF9j+22/GjXqcnZ1Nf1WEh4aybPUq0tPTSUtL47tVKwlrE2pRneaQQBeinNu9xynXmPmtMfXde4p2g4jC1K9Xj1Ejns/XPnrUy4yLjSW0w6NkZd39BGx0j8epX68uPkEBPDtqFEH+/lSvVt3seu6//36CAwLxdPegSePGpvaPpk0jYe8efEOC8ApozaezPgfg9VdH89vvv+MTFECLNsFs3BIHGEvy+oYE8dTQIbTy82PQU08TGNmWoHYRDBswiJYtWphdo7lk+VwhHJA5y+eWZdeuXaNq1apcunSJwMgItq3fQG1PT1uXVSJk+VwhhEPr2rsXv1+5QmZmJm+MGeOwYW6pQgNdKTUL6Aqc11r75LRNA7oBmcDPwGCt9e8lWagQovzatGatrUuwC0UZQ58NdMrTth7w0Vr7AkeAcVauSwghRDEVGuha6zjgcp6277XWtwZ5dgD1S6A2IYQQxWCNWS5DgP9aYT9CCCEsYFGgK6XGAzeBuffYZrhSKkEplXDhym+WHE4IIcQ9mB3oSqmBGCdLn9L3mPuotZ6ptfbXWvu7V89/yawQwjGdS02l76CB/NW3OV4BrekS/QRHjh7l+IkT+AQFlMgx47ZtpVVYKBVrVDct4FWemBXoSqlOwBigu9Y63bolCSFK3aKF4N0Mqt9nPC5aaNHutNb0fLIfEWFh/Jz8Eym7E5kyMZbUC+etVHDBGtZvwOxP/sOTMb1L9DhlVaGBrpSaD8QDDymlTiulhgLTgfuA9UqpJKXUpyVcpxCipCxaCC+MhFOnjEVJTp0ynlsQ6hvjNuPsXJHnht6+wYWfr2++y+GPnzhB2GMdaBUWSquwULbv3AHA2XPnCO/UEb/QEHyCAtiyfRtZWVkMeu5ZfIICaB4cyL+mT8933MaNGuHr40OFCuXzIvhC56FrrfsV0PxFCdQihLCFN2Mh5yYUJtevG+29+5i1y/0pKbT2a1nodh7u7qxfvhIXFxeOHjtGv6GDSdi8hXmLF/FY+0cZ/+posrKySE9PJyk5mV/PnmH/zt0A/P67XPqSl1wpKkR5d7ebLpTwzRgAbty4wchX/kHST8k4OTlx5NgxAAJatWLIiBHcuHGDx7t2w8/Xl780bsz/fjnOC6/8g6jHOtGxffsSr8/elM+/S4QQt9W/y2Ukd2svAu9mzUhM2lvodv+aMR1PDw/2bd9BwuYtZGZmAhAe+ghxa9dRr25d+g8fxlfz5lGjRg32bY8nIiyMGZ/NNN2AQtwmgS5EeTcxFvKsAY6rq9FupnZtI/jzz0w+m/2lqW13YiKbt27Jtd2VP/6gTu3aVKhQga8XzDetunji5Ek83N15ZtBghvYfyJ59SVy8dJHs7GyiezzOW6+/wZ59SWbX56gk0IUo73r3gY+nQ4MGoJTx+PF0s8fPAZRSfDdvPus3/shffZvjHehP7NQp1K1dJ9d2I4Y9w5x5cwluF8mRY8dMd/zZtGULfqEhtHykDUtXLGPU30fw65mzRER1xi80hEF/f5apE9/Md9zdiYnUf7gpi5d9x7OjRuEdWPybP9szWT5XCAfkaMvnlieWLJ8rPXQhhHAQEuhCCOEgJNCFEMJBSKALIYSDkEAXQggHIYEuhBAOQgJdCGE4mAJBAcajFUye9h7egf74hgThFxrCzt27rbJfc3w4Ywbp6fkXho2dMplxsRNztSUlJ9PMv1Wxj5GUnMyadevMrtEaJNCFEJCWBr2i4dAh6NXLeG6B+J07WbV2LXu2bCM5ficbVqykgQVLCVgiKyuLDz+ZQfr1/IHeLyaGhd8uzdW2YOkSs5bfTfopmTXfFy/Qb968WfhGxSCBLoSAEX+HCxeM5XMvnIfnR1i0u7Op56hVsyaVK1cGoFbNWtStY1wl2tjHi4uXLgKQsGcPEV2Me9DHTplM/2eG0a5rFx70a2FaNmDTljjCO3Wk55N98QpozXMvvUh2djYA8xcvonlwID5BAYyZ8Ibp+FXreDLh7bcIioxg8rT3OHP2LJFRXYiM6pyrzocebMr91avn+uth0Xff0je6FwDf//ADIe3b0SoslJgBT3Pt2jXAuCK1zaPtadEmmMCItly5coUJk99m4bdL8QsNYeHSJVy+fJnH+/XFNySI4HaRJO/fb3qfw18cScce3Rkw/BmLPue8JNCFKO++/hrWrYWMDON5Rgas/a/RbqaO7dpz6tfTNG3px4iXX8q3hsvdJB/Yz+rFS4n/4QcmvfsOZ86eBWBXYiL/nDyVn3bs4udffuHbFcs5c/YsYyZO4MdVq0naFs/uPYksW7USgLS0NHy8vNi5cRMTxo6jbp06bFy9ho2r89/+uF+vGBYsXQLAjl27qFnjAR7829+4eOkib097lw0rVrJnyzb8W7big+kfk5mZSZ/BA/m/d99j3/YdbFixkipVqjBp/Ov0eSKapG3x9InuxcQpk2np60ty/E6mTJzIgGdvh3diUhLLFyxk3qwv89VjCQl0Icq72AmQd3w5Pd1oN1PVqlVJjNvKzI8+wr1WLfoMGsjsud8U+roeXaJwdXWlVs1aRIaFsysxAYDA1q35S5MmODk50a9XDFvjjQCPeCQM91ruVKxYkad69yFu2zYAnJyciO7xeJFq7RvdiyXLl5Gdnc2CpUvoFxMDwI5du0k5dIjQjo/iFxrCnHlzOXHqFIePHqGOZ20CWrcGoFq1alSsmH8l8q074unf17idRLu2EVy6fJkrV64A0L1zF1zzLohmBbIeuhDlXewkePUfuUPdzQ3enGTRbp2cnIgICyciLJzm3t7MmTePQU89TUWniqYhk4w/M3K9RilV4POC2u+1DpWLiwtOTk5FqrNB/fo0btiQzVu3sHTFcuI3/AiARtMhsh3zv5yda/vk/fvz1VOQguq79bpbi5BZW1FuQTdLKXVeKbX/jrYHlFLrlVJHcx7l7s9C2Kv+/eGxTuDiYjx3cYFOneHp/mbv8vDRIxzNuVkFGDNAGjVoAEDjRg1J3Gssfbt0+fJcr1u+ZjUZGRlcunSJTVu3ENDK6AXvSkzkl+PHyc7OZuHSpTwSEkKQfwCbt23l4qWLZGVlMX/JYto+8kiB9dxXtSpXr167a739esXw8rix/LVJE+rXqwdAcEAA23bu4NjPPwOQnp7OkaNHebhpU86cO8vuxEQArl69ys2bN41jXLt9jPA2oczNuY3fpi1x1KpZk2rVqhX9QzRDUYZcZgOd8rSNBX7QWj8I/JDzXAhhr/79Cbi7G8vnunvAjH9btLtr19IY+NyzeAW0xjckiJTDh4h97TUAJo4dx6gxowl7rEO+XnRg69ZExUQT3L4db4weYzqRGhIYyNjYCfgEBdCkcSN6dutOndq1mTrxTSKjutCiTTCtWvjRI6prgfUMHzSYztE9850UvSWmZ08OHDxoOhkK4F7LndmffEq/IYONE5vtIzl09AiVKlVi4ZdzeOHVV2jRJpgOPbqTkZFBZFg4KYcOmU6Kxo57jYS9e/ENCWLsxInM+XSmRZ9pURRp+VylVGNgldbaJ+f5YSBCa31WKVUH2KS1fqiw/cjyuUKUDrOWzz2YAoMGwuw50MyrZAq7h9gpk6latSqvvDgqV/umLXG8/9FHrFq8pNRrsgVLls81dwzdU2t9FiAn1D3utqFSajgwHKChZ20zDyeEKHHNvGCn7S7+EZYr8ZOiWuuZwEwweuglfTwhhH2KfW18ge23TqyKwpk7bTE1Z6iFnMfz1itJ3NX6/0KfbhAZaDyuzz+nVghRfpkb6CuAgTn/Hggsv8e2whrW/xfenwKp54yr+VLPGc8l1IUQOYoybXE+EA88pJQ6rZQaCrwDdFBKHQU65DwXJenzf0OeObv8mWG0CyEERRhD11r3u8u32lu5FnEv51OL1y6EKHfk0n974eFZvHYhbExVq0r/Z4aZnt+8eRP3Jo3oGtPrHq8qnmEjnyfl0MEibXv8xAnqP9zUdJXqLX6hIexKSCj2sae8P63YrylpEuj2YtgIqOySu62yi9EuhCU+/BfEbc7dFrfZaLdAlSpV2H8whevXrwOw/scfqVe3brH2Udjysp9Pn4HXw82KtK/GjRrRoF59tmzfZmo7dOQwV69eI9C/0Cne+Uz55/vF2l5rne+XibVJoNuLDp3hldfAs7ZxNZ9nbeN5h4KvfBOiyFq1goEDbod63Gbjeavi3+Qhr84dOrJ63VoA5i9ZTL9eMabv7UpIoM2j7Wn5SBvaPNqew0ePADB77jfEDHiabr1j6Ph4d7Kzsxnx8kt4B/rTNaYXXaKfYMmy7wCI6NKJhD17AGPJ3PGTYmnRJpjgdpGkFjAc2a9XL9PKigALliyhXy/jL4YLFy8Q/fSTBLQNJ6BtONt2xANw7do1Bv/9OZoHB+IbEsTS5csYO3EC169fxy80hKeGDgHgg+kf4xMUgE9QAB/OmAEYfxU082/FiJdfolVYKKdOn7b4M70XCXR70qEzLFwJG3cZjxLmwhrC28Kcr4wQf/st43HOV0a7hfpGGwGakZFB8oH9BN3RE364aVPi1q5j79btTBr/Oq+9GWv6XvyuXcz59D/8uGoN365YzvGTJ/lpxy4+nz6D+N27CjxWWloawQGB7Nu+g/DQUD6bPTvfNr2fiGbZqlWmnv/Cb5fSNyfQR40ezcvPj2T35jiWfjOXYSOfB+Ct996lerVq/LRjF8nxO2kX3pZ33pyEq6srSdvimfvFLBL37uXLb75m54+b2PHDRj6b8yV79+0D4PDRowzo9yR7t26nUcOGFn+m9yKrLQohjPAeOgzeexdGj7FKmAP4+vhw/ORJ5i9ZTJeOj+X63pU//mDgc8M5+vPPKKW4ceOG6XsdItvxwAMPALA1Pp6Ynj2pUKECtT09iQwLK/BYlSpVomsno5PT2s+P9Rs35tumtqcn3s2a8cOmTXh6eODs7IyPlzcAGzZtJOXwIdO2f1y9ytWrV9mwaSMLZs02tdeokX8twq3x2+nZtZtpFcUnunVny/ZtdO8SRaOGDQkODCzKx2UxCXQhhDHM8sXnRph/8TmEh1st1Lt37sIr48ezac0aLl2+bGp/4+23iAwL57t5Czh+4gQRdyycVcXNzfTvoqw3BeDs7GxantbJyemu4++3bmjh6eGRawgoOzub+A0/5lunXGtd6HK59yrxzvdS0mTIRYjy7taY+Zyv4PU3bg+/5D1RaqYh/QcwYcwYmnv75Gq/8scV00nSe9384pGQEJYuX052djap51PZtGWrRfVEd+/Bmu/XGcMtd6yu2LFde6bP/I/peVJycoHtv/32G2D8Arn1V0V4aCjLVq8iPT2dtLQ0vlu1krA2oRbVaQ4JdCHKuz17co+Z3xpTzznZaKn69eoxasTz+dpHj3qZcbGxhHZ4lKysrLvXX7SNAAASFUlEQVS+PrrH49SvVxefoACeHTWKIH9/qlerbnY9999/P8EBgXi6e9CkcWNT+0fTppGwdw++IUF4BbTm01mfA/D6q6P57fff8QkKoEWbYDZuiQOMJXl9Q4J4augQWvn5MeippwmMbEtQuwiGDRhEyxYtzK7RXEVaPtdaZPlcIUqHWcvnlmHXrl2jatWqXLp0icDICLat30BtT8e8BsMWy+cKIUSp6dq7F79fuUJmZiZvjBnjsGFuKQl0IUSZt2nNWluXYBdkDF0IB1Waw6nCOiz9mUmgC+GAXLI0l65ckVC3I1obPzOXLPN/ZjLkIoQDqp+exenU81y4eMHWpYhicMnS1E+/+4yfwkigC+GAnDU0STM/GIR9kiEXIYRwEBYFulLqZaXUAaXUfqXUfKWUS+GvEkIIURLMDnSlVD3gRcBfa+0DOAF9rVWYEEKI4rF0yKUi4KqUqgi4AWcsL0kIIYQ5zA50rfWvwPvASeAscEVr/X3e7ZRSw5VSCUqphAtXfjO/UiGEEPdkyZBLDaAH0ASoC1RRSj2ddzut9Uyttb/W2t+9ev51hIUQQliHJUMujwK/aK0vaK1vAN8CbaxTlhBCiOKyJNBPAsFKKTdlrP7eHija7beFEEJYnSVj6DuBJcAe4Kecfc20Ul1CCCGKyaIrRbXWE4GJVqpFCCGEBeRKUSGEcBAS6EII4SAk0IUQwkFIoAshhIOQQBdCCAchgS6EEA5CAl0IIRyEBLoQQjgICXQhhHAQEuhCCOEgJNCFEMJBSKALIYSDkEAXQggHIYEuhBAOQgJdCCEchAS6EEI4CIsCXSl1v1JqiVLqkFLqoFIqxFqFCSGEKB6L7lgE/B+wVmvdSylVCXCzQk1CCCHMYHagK6WqAeHAIACtdSaQaZ2yhBBCFJclQy5/AS4AXyql9iqlPldKVbFSXUIIIYrJkkCvCLQCPtFatwTSgLF5N1JKDVdKJSilEi5c+c2CwwkhhLgXSwL9NHBaa70z5/kSjIDPRWs9U2vtr7X2d69ew4LDCSGEuBezA11rfQ44pZR6KKepPZBilaqEEEIUm6WzXF4A5ubMcPkfMNjykoQQQpjDokDXWicB/laqRQghhAXkSlEhhHAQEuhCCOEgJNCFEMJBSKALIYSDkEAXQti/+V/B3oTcbXsTjPZyRAJdCGH/HvaC2HG3Q31vgvH8YS/b1lXKLJ2HLoQQttfSH2KnGiHeIxqWLzWetyxfs6qlhy6EcAwt/Y0w/+oL47GchTlIoAshHMXeBKNnPmCo8Zh3TL0cKNVAT7zPDdXWH9W2/P3mFEKUoFtj5rFTYchzt4dfylmo22wM/W6hrjeXrx+AEMIKDqXkHjO/NaZ+KKVcDb0orXXpHczfX5NgXmBL0AshyisVEZCotS70N5PdzHKRHr2wil9+hjdfg4lToMlfbV2NEFZlN4F+NwUFvYS8KND16zD2JTifCmNfhtkLwdXV1lUJYTV2H+gFkd68KNC7k+C3y6A1/HYJ3nvL6KkL4SAcMtDvRoK+HFuzAnZshcxM43lmJsRvMdq7dLdtbUJYicWBrpRyAhKAX7XWXS0vqfTJsE05MHM6ZGTkbsvIMNol0IWDsEYPfRRwEKhmhX2VGdKbdzDDR8JH03KHuosLPPuC7WoSwsosCnSlVH0gCpgM/D+rVFTGSdDbqS7dYVc8bI8zhlsqVYKQMOjczdaVCWE1lvbQPwRGA/dZoRa7JkFvB8ZMgEG9jVkuNWrC6DdsXZEQVmV2oCulugLntdaJSqmIe2w3HBgOQMOG5h7Obsn4fBni6grvfHh7HrpMWRQOxuwrRZVSU4H+wE3ABWMM/Vut9dN3fY0FV4qWBxL0QoiClPiVolrrccA4gJwe+iv3CnNROBm2EUJYolzNQ7dXEvRCiKKwSqBrrTcBm6yxL1F0Mj4vhLiT9NAdjPTmhSi/JNDLiUKDXlYhFMLuyS3oyjnV1p8q/s04PnE0WSd+4XjsGGNVQiGE3ZEeumDWkCF4nD+Pk9Z4pqayYPYM+i1YkG87GbYRomyTQC/nBs2aRdTq1bjlrHHimpFBt5UrGTRrFrOHDMm1rYzPC1G22c0t6ETJOOfpief58/naUz08qJ2aatG+JeiFsA6HuwWdKBljp07l4xdfpGpamqktzc2NMe+8Y/G+ZVqlEKVLAr2cmz1kCJ3WraPbihW4ZWRw3cWFld26MWfw4BI5ngzbCFFyZJaLDXkdOMBPPj54HThg0zqGzJrFeQ8PspUi1dOToV98Ueo1qLb+BX4JIYpOeug24paWxuouXWhw6hSro6LwPnCA9CpVbFJLepUqRK1Zw8I+feizcKHN6iiIDNsIUXQS6DaSd6rgF0OHFjhVsLSkeHvTfP9+mx2/OGTYRoiCSaDbQHGmCoqik6AX5Z1MW7SBkpwqKIpOgl7YC5m2WIaV5FRBUXQyPi8cjQS6DZT2VEFRdDJsI+yZTFu0kXtNFSwr0xnFbTKtUtgDswNdKdVAKbVRKXVQKXVAKTXKmoU5ultTBVO8vIhavdo0VfDWdMZmKSmsjorC7Y5hGVH22H3Q//IzDOpjPAq7Z0kP/SbwD611MyAYeF4p5WWdshxPQb3uW1MFU7y9TW0FTWcU9scuQv76dRj7Epz4Bca+LMsmOwCzA11rfVZrvSfn31eBg0A9axXmSIra677XdEZh/8pcb/7dSfDbZdAafrsE771lu1qEVVhl2qJSqjEQB/horf+463bldNrigj59cp0AXd6jR4EXEcl0RnGnEj0Ru2YFfDQNcjoPALi4wIuvQpfuJXdcYZaiTlu0+KSoUqoqsBR4qaAwV0oNV0olKKUSuHDB0sPZneL0usdOncq1PJfdl8R0Rjnpah9KtDc/c3ruMAfj+czp1juGKHUW9dCVUs7AKmCd1vqDQrcvhz304va6i9qbN5dbWhoHvLxocOoUpxo2tOkaMsJ6it2bv1sPfdRo6NzNusUJi5V4D10ppYAvgINFCfPyqri97pJe+VBOujqmYo/Pd+kOwY9ApUrG80qVICRMwtzOWTLkEgr0B9oppZJyvrpYqS6HMXvIEFZHRZHu4gJQ6EVEd5vOaA1y0tW2bDHUdc+gHzMBajwASkGNmjD6jVKrS5QMWculFNwa5mh46hQnbTjMUVInXb0OHDAtvXvnFExxW1kd6rrzZ3fgokxbLKtK7aSoKFxJ9rqLoyROusqFUEVTVoe67rwWosxNqxTFJj30csbaJ11L+iSuIxg0a1aBi7GN/Phju14uWda3KT1F7aFLoJcz1hz+cdSgsrbydn2BBL31yZBLKbOXud3WHP55Z9y4XGEOUCU9nXfGjbO0TIdSWtcXlBUybGM7EuhWUJRx5AID/wDgk/NYigpaQ8Yc5S2ozFXcmU6OqFTH58vxgmMy5GIFeceRN0ZE0PDUKdOsjwJnOFAFvIBTQEOMULf9pIdikzH0oikrM53shdnDNtevw6DecD4VPGrD7IXg6mrd4mxAhlxKSUFzuzutW4fXHb31Amc4DAHOAxpIBcrGpIdiK+kLoRxFWZnpZC/M7s2X8wXHpIduiQNwroUnnln5T3iB8ad1cvPmeKek5D5xWMmNkXzM7Mw7Thy6AR9jBL2dkXnowpZMvXkHXnBMZrmUtDTACwadnMXH6kWq6oLnX2dT8J9BqXhQmzwzHDwgb5MQomjuNpuI+2vAsu9LvyArkiGXkpYzZDKbIaxWUaQ7uRS4WQWMUL9TWiU3xlTKc+LQDZBziUKY7W4n6Qd98M9yM+NGAt0cs4DVQM5fdtOyX6FiVhbZQDYq16Zpbm7sDgi4PcOhsgsrXboxJ3Iw3Pod4AJ0A8rPpAchrK6os4kceVqlDLmYwxPjhCbgRhoH8KIhp7hJRa5wH7W4jMI433myQQO8Dh68PcOhQkO8sw6Q3iCnJ3Eau57lUpbIWL6w9myisnKRlAy5lKSpmMJ3FkPw4DwV0IA2hTmAAmpdvEjvhQuNGQ73eRHltNqYsngB8MaYuria/GFuoznq9krWlBFg/dlE9ra+jfTQzdUHBi2axce8SFXuHR6pHh7UnpoKI4E7F7S728yWnBOu9j5HvTTJfHhhayXZm5ceekmLhHcYV2iY/+nsbFw5OZbcYQ6QDhR0lfydc9TPAXWRnvo9yDrvoiwoC7156aGbyxMGnS+8h55RqTKuf2ZAc2B/nm+6AdPJfTJ0FvAi5NvlA8BJpKdegPK2+JVwDMXp0UsPvaRNhdmVhrCaKNJzpqvk/dWYjeLFKh/C/1FwD7sR+We2vEr+MAe4jN1eTVrSZE0ZYY9KpEevtTb7C+gEHAaOAWML3b51a/MPVha/KmvtxjX9Cw11Fkpfw02n46I16CyUXkMnbVyDfI+va3fs75rWPHCPbd205osy8L7L4NeC3r11movx2ae7uOj5ffrYvCb5ki+rfUFCUTaz5CbRTsAMoDPGKbx+Sikvy3692JkPIZ0qRLGGFLxoyyZS8SAbxUka0oslhe8j8o5/D8EYV7+bu425C1lTRggsG3IJBI5prf+ntc4EFgA9rFOWnXjOeEjBm+bsJ5EAU7hHkTM9sTC7McbN81ysVCC5mvSuZPErIaCiBa+thzGx7pbTQFDejZRSw4HhADRsaMHhyqgawG+3n94K92K51eu+14QZuZq0ULfWeReivLKkh64KaMs3ZUZrPVNr7a+19sfd3YLDlVEfY9mnWAmj133HxUomrkBNjE/aE5BRBCHEPVgSRaeBBnc8rw+csawcO/QUMBNwusc2zhjBnFcFoCdGr3sIEEXu9V26A5u5+9WkQghxB0sCfTfwoFKqiVKqEtAXWGGdsuzMUGAfRmjXxehx18n53gNANEYwN8t5fkt9cve6Z2EsoXtnj9wbY/66LE0ihCiE2WPoWuubSqmRwDqM/uksrXX5vZ7RG7h4x/MxBWyTgjEfvQdGaC8jd6+7CrAG6AMsRHrkQohiseSkKFrrNRgRJIrKG2PW/r2+L+f1hBBmkCtFhRDCQUigCyGEg5BAF0IIByGBLoQQDkICXQghHIQEuhBCOAgJdCGEcBCle8cipa5irJ/uSGqR+5IiRyDvyT7Ie7IP1nhPjbTWhS6GZdGFRWY4rItwGyV7opRKkPdU9sl7sg/yniwjQy5CCOEgJNCFEMJBlHagzyzl45UGeU/2Qd6TfZD3ZIFSPSkqhBCi5MiQixBCOIhSDXSlVIxS6oBSKlspZddnspVSnZRSh5VSx5RSY21djzUopWYppc4rpRxmAV+lVAOl1Eal1MGc//ZG2bomSymlXJRSu5RS+3Le05u2rslalFJOSqm9SqlVtq7FGpRSx5VSPymlkpRSCSV9vNLuoe8HngDiSvm4VqWUcgJmAJ0xbhDXTynlZduqrGI20MnWRVjZTeAfWutmQDDwvAP8rP4E2mmtWwB+QCelVLCNa7KWUcBBWxdhZZFaa7/SmLpYqoGutT6otXaEC4sCgWNa6/9prTOBBRj3IbJrWus44LKt67AmrfVZrfWenH9fxQiLeratyjLacC3nqXPOl92fDFNK1ce4s+7ntq7FXskYunnqAafueH4aOw+J8kAp1RhoCey0bSWWyxmaSALOA+u11nb/noAPgdFAtq0LsSINfK+USlRKDS/pg1n9SlGl1AagdgHfGq+1Xm7t49mIKqDN7ntIjkwpVRVYCryktf7D1vVYSmudBfgppe4HvlNK+Wit7fbch1KqK3Bea52olIqwdT1WFKq1PqOU8gDWK6UO5fwlXCKsHuha60etvc8y6DTQ4I7n9YEzNqpFFEIp5YwR5nO11t/auh5r0lr/rpTahHHuw24DHQgFuiulugAuQDWl1Dda66dtXJdFtNZnch7PK6W+wxiuLbFAlyEX8+wGHlRKNVFKVQL6AitsXJMogFJKAV8AB7XWH9i6HmtQSrnn9MxRSrkCjwKHbFuVZbTW47TW9bXWjTH+f/rR3sNcKVVFKXXfrX8DHSnhX7qlPW2xp1LqNBACrFZKrSvN41uL1vomMBJYh3GSbZHW+oBtq7KcUmo+EA88pJQ6rZQaauuarCAU6A+0y5k6lpTTC7RndYCNSqlkjM7Feq21Q0zzczCewFal1D5gF7Baa722JA8oV4oKIYSDkCEXIYRwEBLoQgjhICTQhRDCQUigCyGEg5BAF0IIByGBLoQQDkICXQghHIQEuhBCOIj/D/aAB8LhzmiRAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 1 \t #support vectors= 16  #margin vectors= 2\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XlYVdX+x/H3ElFQr2YKOKDivTdLJURlFEHQNGczQKWcNeuaZT15HUuw0sbb7VfaYGVaOUvlmKY5gIoiKJKzdnPKWYsURBTW74+NJIMC5xw4cPi+nqfneNbZZ+/vgXs/Z7PW2msrrTVCCCHKv0rWLkAIIYRlSKALIYSNkEAXQggbIYEuhBA2QgJdCCFshAS6EELYCAl0IYSwERLoQghhIyTQhRDCRlQubAOl1BygJ3BBa+1+R/tzwBjgFrBaaz2+sH3VrXWfdqvXwIxyhRCi4kk8cvCS1tqpsO0KDXRgLjAT+Op2g1IqBOgDeGitbyilnItSlFu9BiTM/qrwDYUQQuRQwd4nirJdoV0uWusY4Eqe5n8Bb2qtb2Rvc6HYFQohhLAoU/vQmwGBSqmdSqktSilvSxYlhBCi+IrS5XK399UG/ABvYIlS6u+6gKUblVKjgFEAjV3qmVqnEEKIQpga6KeBb7MDPF4plQXUBS7m3VBrPRuYDeD1YAtZq1eIUnBTwelqdqTbKWuXIorBIVPjmpaJvYlJaWqgfw90BDYrpZoBVYBLJu5LCGFhp6vZ8TcXZ9xq1UIpCfXyQGvN5ZQUTp+/QNPUTJP2UWgfulJqIRAHPKiUOq2UGgHMAf6ulNoHLAKGFNTdIoSwjnQ7RR0J83JFKeN3Zs5fVYWeoWutI+7y0kCTjyqEKHES5uWPub8zuVJUCCFshAS6paz/Afr3ghAf43H9D9auSAirOnf+PAOGDuEfHg/Twrst3UMf58jRoxw/cQJ339KZ6Tzl1SgaNX+QGvVdSuV41iaBbgnrf4B3Z8D5c6C18fjuDAl1UW7MX2KHW0tHKtWqhltLR+YvsTNrf1pr+j4RQXBgIL8k/8yBXYnMiIzi/MXSvQaxV9fuxG/aUqrHtCYJdEv4/CO4kZ677Ua60S5EGTd/iR2jnqvKiVOV0Fpx4lQlRj1X1axQ3xSzBXv7yjwzYmROm6eHB4HtAnJtd/zECQIf7UybwADaBAawfecOAM6eO0dQ1y54Bvjj7utN7PZtZGZmMvSZp3H39eZhPx/+O3NmoXX4+fhQv17Fuf7F1GmL4k4XzhevXYgyZMq0KqRdzz0Yl3ZdMWVaFZ7sd92kfe47cIC2nq0L3c7ZyYn1y1fi4ODA0WPHiBgxjIQtsSxYuoRHOz3ClH+PJzMzk7S0NJKSk/nt7Bn27dwFwB9//GFSbbZMAt0SnF2MbpaC2oUo406eLnhmxd3aLenmzZuMGfcSST8nY2dnx5FjxwDwbtOG4aNHc/PmTR7r2QtPDw/+7ubG/349znPjXqLHo13p0qlTiddX3kiXiyWMHA1VHXK3VXUw2oUo4xq7FnwJyd3ai6Jl8+YkJu0pdLv/zpqJi7Mze7fvIGFLLBkZGQAEBbQnZu06GjZowKBRI/lqwQJq167N3u1xBAcGMuuz2Ywc82yufWVmZuIZ4I9ngD9TX3/N5NrLMwl0S+jcDcZNBpd6oJTxOG6y0S5EGTc9MoNqjrnDu5qjZnpkhsn77NghmBs3Mvhs7pc5bbsSE9myNTbXdil//kn9evWoVKkSXy9aSGamcYXkiZMncXZy4qmhwxgxaAi79yZx6fIlsrKyCO3zGK+9/Aq79ybl2pednR1J2+JI2hbHqy+/YnLt5Zl0uVhK524S4KJcerJfJnCDKdOqcPK0orGrEeZGu2mUUny3YCEvTBzPm++9h4NDVdwaN+H9N9/Ktd3okU8ROuhJln73HSFBQVSvXh2AzbGxvPPB+9jb21OjenW++vQzfjtzlmGjnyErKwuANyKnFVrH+FdeZsHSJaSlpeH6UDNGDh5C1OQpJn+usk6V5hX7Xg+20HKDCyFK3sGalWn+zwesXYYwwcFjR2n+561cbSrYO1Fr7VXYe6XLRQghbIQEuhBC2AgJdCGEsBES6EIIYSMk0IUQwkZIoAshhI0oyh2L5iilLmTfnSjva+OUUlopVbdkyhNClJb9BxXuvo7sP2iZS/6nv/M2LX288PD3xTPAn527dllkv6Z4f9Ys0tLS8rVHzZjOpKjIXG1Jyck092pT7GMkJSezZt06k2u0hKKcoc8FuuZtVEo1AjoDJy1ckxCilKWmQvcwBw4cUvQIcyA11bz9xe3cyaq1a9kdu43kuJ1sWLGSRq6ulim2mDIzM3n/41mkXc8f6BHh4Sz+NjpX26LoZTwR3q/Yx0n6OZk1PxYv0G/dulX4RsVQaKBrrWOAKwW89F9gPCD3EhWinBs+uioXLiq0Vpy/qBjxbFWz9nf2/Dnq1qlD1arGfurWqUuD+vUBcHNvwaXLxj3lE3bvJri7cb4YNWM6g54aScee3XnAs1XOsgGbY2MI6tqFvk8MoIV3W5554fmcq0UXLl3Cw34+uPt6M2HqX5f716jvwtTXX8M3JJjp77zNmbNnCenRnZAeua/mfvCBZtxXq1auvx6WfPctA0LDAPjxp5/w79SRNoEBhA8eyLVr1wBjGYN2j3SiVTs/fII7kJKSwtTpr7P422g8A/xZHL2MK1eu8FjEADz8ffHrGELyvn05n3PU82Po0qc3g0c9ZdbPOS+T+tCVUr2B37TWey1ajRCi1M35ujKr19mRnm50taSnK1autWPO16avDNKlYydO/XaaZq09Gf3iC/nWcLmb5P37WL00mriffuLVt97kzNmzAMQnJvKf6W/w8454fvn1V75dsZwzZ88yIXIqG1etJmlbHLt2J/L9qpUApKam4t6iBTs3bWbqxEk0qF+fTavXsGl1/pvORISFsyh6GQA74uOpU/t+HvjnP7l0+RKvv/MWG1asZHfsNrxat+G9mR+SkZFB/2FD+L+33mbv9h1sWLGS6tWr8+qUl+n/eChJ2+LoHxpG5IzptPbwIDluJzMiIxn89F/hnZiUxPJFi1kw58t89Zij2IGulKoGTAGmFnH7UUqpBKVUwsWU34t7OCFECZsUZU9qWp710NMUk6LsTd5njRo1SIzZyuwPPsCpbl36Dx3C3PnfFPq+Pt174OjoSN06dQkJDCI+MQEAn7Zt+XvTptjZ2RERFs7WOCPAg9sH4lTXicqVK/Nkv/7EbNsGGAt1hfZ5rEi1DggNY9ny78nKymJR9DIiwsMB2BG/iwOHDhHQ5RE8A/yZt2A+J06d4vDRI9R3qYd327YA1KxZk8qV83/5bd0Rx6ABEYCxWNnlK1dISUkBoHe37jg6OhapvuIw5Sv4H0BTYG/2Hapdgd1KKR+tdb5FwbXWs4HZYKzlYkatQogS8EbUTZ7/d5VcoV6tmubNaTfN2q+dnR3BgUEEBwbxcMuWzFuwgKFPDqSyXeWcLpP0PHf6ynvX+9vPC2q/1zpUDg4O2NkV7Y5LjVxdcWvcmC1bY4lesZy4DRsB0Gg6h3Rk4Zdzc22fvG9fvnoKUlB9t993exEySyv2GbrW+mettbPW2k1r7QacBtoUFOZCiLJv+KBb9Hg0EwcHI4AcHDS9umYybKDpA3aHjx7haPbNKsCYAdKkUSMA3Jo0JnGPsfRt9PLlud63fM1q0tPTuXz5Mpu3xuLdxjgLjk9M5Nfjx8nKymJxdDTt/f3x9fJmy7atXLp8iczMTBYuW0qH9u0LrOdvNWpw9eq1u9YbERbOi5Mm8o+mTXFt2BAAP29vtu3cwbFffgEgLS2NI0eP8lCzZpw5d5ZdiYkAXL16lVu3bhnHuPbXMYLaBTB/yWLAGAeoW6cONWvWLPoP0QRFmba4EIgDHlRKnVZKjSjRioQQpW7ORzdwdtIopXFx0nwx64ZZ+7t2LZUhzzxNC++2ePj7cuDwIaImTwYgcuIkxk4YT+CjnfOdRfu0bUuP8FD8OnXklfETcgZS/X18mBg1FXdfb5q6NaFvr97Ur1ePNyKnEdKjO63a+dGmlSd9evQssJ5RQ4fRLbRvvkHR28L79mX/wYM5g6EATnWdmPvxJ0QMH2YMbHYK4dDRI1SpUoXFX87juX+Po1U7Pzr36U16ejohgUEcOHQoZ1A0atJkEvbswcPfl4mRkcz7ZLZZP9OikOVzhbBBpiyfu/+gov9QBxbPTadl89LvHY2aMZ0aNWow7vmxudo3x8bw7gcfsGrpslKvyRrMWT5XbnAhhACgZXPNvp2m3RRalA0S6EKIMuFudxK6PbAqCidruQghhI2QQBdCCBshgS6EEDZCAl0IIWyEBLoQokSomjUY9NTInOe3bt3CqWkTeoaH3eNdxTNyzLMcOHSwSNseP3EC14ea5VyleptngD/xCQnFPvaMd98p9ntKmgS6EBXc2+/bsykmdxRsiqnE2++bvpYLGJe37zt4gOvXjamQ6zdupGGDBsXaR2HLy34+cxYtHmpepH25NWlCo4auxG7fltN26Mhhrl69ho9XoVO885nxn3eLtb3WOt+XiaVJoAtRwXm3yaTfEIecUN8UU4l+QxzwbpNp9r67de7C6nVrAVi4bCkRYeE5r8UnJNDukU60bt+Odo904vDRIwDMnf8N4YMH0qtfOF0e601WVhajX3yBlj5e9AwPo3vo4yz7/jsAgrt3JWH3bsBYMnfKq1G0aueHX8cQzl84n6+eiLCwnJUVARYtW0ZEmPEXw8VLFwkd+ATeHYLw7hDEth1xAFy7do1h/3qGh/188PD3JXr590yMnMr169fxDPDnyRHDAXhv5oe4+3rj7uvN+7NmAcZfBc292jD6xRdoExjAqdOnzf6Z3osEuhAVXEhQFkvmpdNviANTX7en3xAHlsxLJyTI/LPJAaFGgKanp5O8fx++d5wJP9SsGTFr17Fn63ZenfIyk6dF5bwWFx/PvE8+ZeOqNXy7YjnHT57k5x3xfD5zFnG74gs8VmpqKn7ePuzdvoOggAA+mzs33zb9Hg/l+1Wrcs78F38bzYDsQB87fjwvPjuGXVtiiP5mPiPHPAvAa2+/Ra2aNfl5RzzJcTvpGNSBN6e9iqOjI0nb4pj/xRwS9+zhy2++ZufGzez4aROfzfuSPXuN1cUPHz3K4Ign2LN1O00aNzb7Z3ovcmGREIKQoCz+NeImr71dhVfGZ1gkzAE83N05fvIkC5ctpXuXR3O9lvLnnwx5ZhRHf/kFpRQ3b/61umPnkI7cf//9AGyNiyO8b18qVapEPRcXQgIDCzxWlSpV6NnVWKulracn6zdtyrdNPRcXWjZvzk+bN+Pi7Iy9vT3uLVoCsGHzJg4cPpSz7Z9Xr3L16lU2bN7Eojlzc9pr166db79b47bTt2evnFUUH+/Vm9jt2+jdvQdNGjfGz8enKD8us0mgCyHYFFOJj7+w55XxGXz8hT0hQZkWC/Xe3bozbsoUNq9Zw+Urf9387JXXXyMkMIjvFizi+IkTBN+xcFb1atVy/l3U9abs7e1zlqe1s7O7a//77RtauDg75+oCysrKIm7DxnzrlGutC10u914l3vlZSpp0uQhRwd3uM18yL51XX76Z0/2Sd6DUVMMHDWbqhAk83NI9V3vKnyk5g6T3uvlFe39/opcvJysri/MXzrM5dqtZ9YT27sOaH9cZ3S13rK7YpWMnZs7+NOd5UnJyge2//27cqMfe3j7nr4qggAC+X72KtLQ0UlNT+W7VSgLbBZhVpykk0IWo4HbttsvVZ367T33X7qLdIKIwrg0bMnb0s/nax499kUlRUQR0foTMzLsPwIb2eQzXhg1w9/Xm6bFj8fXyolbNWibXc9999+Hn7YOLkzNN3dxy2j945x0S9uzGw9+XFt5t+WTO5wC8/O/x/P7HH7j7etOqnR+bYmMAY0leD39fnhwxnDaengx9ciA+IR3w7RjMyMFDad2qlck1mkqWzxXCBpmyfG5Zdu3aNWrUqMHly5fxCQlm2/oN1HNxsXZZJUKWzxVC2LSe/cL4IyWFjIwMXpkwwWbD3FyFBrpSag7QE7igtXbPbnsH6AVkAL8Aw7TWf5RkoUKIimvzmrXWLqFcKEof+lyga5629YC71toDOAJMsnBdQgghiqnQQNdaxwBX8rT9qLW+3cmzA3AtgdqEEEIUgyVmuQwHfrDAfoQQQpjBrEBXSk0BbgHz77HNKKVUglIq4WLK7+YcTgghxD2YHOhKqSEYg6VP6nvMfdRaz9Zae2mtvZxq5b9kVghhm86dP8+AoUP4h8fDtPBuS/fQxzly9CjHT5zA3de7RI4Zs20rbQIDqFy7Vs4CXhWJSYGulOoKTAB6a63TLFuSEKLULVkMLZtDrb8Zj0sWm7U7rTV9n4ggODCQX5J/5sCuRGZERnH+4gULFVywxq6NmPvxpzwR3q9Ej1NWFRroSqmFQBzwoFLqtFJqBDAT+BuwXimVpJT6pITrFEKUlCWL4bkxcOqUsSjJqVPGczNCfVPMFuztK/PMiL9ucOHp4ZHvcvjjJ04Q+Ghn2gQG0CYwgO07dwBw9tw5grp2wTPAH3dfb2K3byMzM5OhzzyNu683D/v58N+ZM/Md161JEzzc3alUqWJeBF/oPHStdUQBzV+UQC1CCGuYFgXZN6HIcf260d6vv0m73HfgAG09Wxe6nbOTE+uXr8TBwYGjx44RMWIYCVtiWbB0CY92eoQp/x5PZmYmaWlpJCUn89vZM+zbuQuAP/6QS1/ykitFhajo7nbThRK+GQPAzZs3GTPuJZJ+TsbOzo4jx44B4N2mDcNHj+bmzZs81rMXnh4e/N3Njf/9epznxr1Ej0e70qVTpxKvr7ypmH+XCCH+4nqXy0ju1l4ELZs3JzFpT6Hb/XfWTFycndm7fQcJW2LJyMgAICigPTFr19GwQQMGjRrJVwsWULt2bfZujyM4MJBZn83OuQGF+IsEuhAVXWQU5FkDHEdHo91EHTsEc+NGBp/N/TKnbVdiIlu2xubaLuXPP6lfrx6VKlXi60ULc1ZdPHHyJM5OTjw1dBgjBg1h994kLl2+RFZWFqF9HuO1l19h994kk+uzVRLoQlR0/frDhzOhUSNQynj8cKbJ/ecASim+W7CQ9Zs28g+Ph2np40XUGzNoUK9+ru1Gj3yKeQvm49cxhCPHjuXc8WdzbCyeAf60bt+O6BXfM/Zfo/ntzFmCe3TDM8Cfof96mjcip+U77q7ERFwfasbS77/j6bFjaelT/Js/l2eyfK4QNsjWls+tSMxZPlfO0IUQwkZIoAshhI2QQBdCCBshgS6EEDZCAl0IIWyEBLoQQtgICXQhhOHgAfD1Nh4tYPo7b9PSxwsPf188A/zZuWuXRfZrivdnzSItLf/CsFEzpjMpKjJXW1JyMs292hT7GEnJyaxZt87kGi1BAl0IAampEBYKhw5BWJjx3AxxO3eyau1adsduIzluJxtWrKSRGUsJmCMzM5P3P55F2vX8gR4RHs7ib6NztS2KXmbS8rtJPyez5sfiBfqtW7cK36gYJNCFEDD6X3DxorF87sUL8Oxos3Z39vw56tapQ9WqVQGoW6cuDeobV4m6ubfg0uVLACTs3k1wd+Me9FEzpjPoqZF07NmdBzxb5SwbsDk2hqCuXej7xABaeLflmReeJysrC4CFS5fwsJ8P7r7eTJj6Ss7xa9R3Yerrr+EbEsz0d97mzNmzhPToTkiPbrnqfPCBZtxXq1auvx6WfPctA0LDAPjxp5/w79SRNoEBhA8eyLVr1wDjitR2j3SiVTs/fII7kJKSwtTpr7P422g8A/xZHL2MK1eu8FjEADz8ffHrGELyvn05n3PU82Po0qc3g0c9ZdbPOS8JdCEquq+/hnVrIT3deJ6eDmt/MNpN1KVjJ079dppmrT0Z/eIL+dZwuZvk/ftYvTSauJ9+4tW33uTM2bMAxCcm8p/pb/Dzjnh++fVXvl2xnDNnzzIhciobV60maVscu3Yn8v2qlQCkpqbi3qIFOzdtZurESTSoX59Nq9ewaXX+2x9HhIWzKHoZADvi46lT+34e+Oc/uXT5Eq+/8xYbVqxkd+w2vFq34b2ZH5KRkUH/YUP4v7feZu/2HWxYsZLq1avz6pSX6f94KEnb4ugfGkbkjOm09vAgOW4nMyIjGfz0X+GdmJTE8kWLWTDny3z1mEMCXYiKLmoq5O1fTksz2k1Uo0YNEmO2MvuDD3CqW5f+Q4cwd/43hb6vT/ceODo6UrdOXUICg4hPTADAp21b/t60KXZ2dkSEhbM1zgjw4PaBONV1onLlyjzZrz8x27YBYGdnR2ifx4pU64DQMJYt/56srCwWRS8jIjwcgB3xuzhw6BABXR7BM8CfeQvmc+LUKQ4fPUJ9l3p4t20LQM2aNalcOf9K5Ft3xDFogHE7iY4dgrl85QopKSkA9O7WHce8C6JZgKyHLkRFF/Uq/Pul3KFerRpMe9Ws3drZ2REcGERwYBAPt2zJvAULGPrkQCrbVc7pMkm/kZ7rPUqpAp8X1H6vdagcHByws7MrUp2NXF1xa9yYLVtjiV6xnLgNGwHQaDqHdGThl3NzbZ+8b1++egpSUH2333d7ETJLK8ot6OYopS4opfbd0Xa/Umq9Uupo9qPc/VmI8mrQIHi0Kzg4GM8dHKBrNxg4yORdHj56hKPZN6sAYwZIk0aNAHBr0pjEPcbSt9HLl+d63/I1q0lPT+fy5cts3hqLdxvjLDg+MZFfjx8nKyuLxdHRtPf3x9fLmy3btnLp8iUyMzNZuGwpHdq3L7Cev9WowdWr1+5ab0RYOC9Omsg/mjbFtWFDAPy8vdm2cwfHfvkFgLS0NI4cPcpDzZpx5txZdiUmAnD16lVu3bplHOPaX8cIahfA/Ozb+G2OjaFunTrUrFmz6D9EExSly2Uu0DVP20TgJ631A8BP2c+FEOXVRx+Dk5OxfK6TM8z6yKzdXbuWypBnnqaFd1s8/H05cPgQUZMnAxA5cRJjJ4wn8NHO+c6ifdq2pUd4KH6dOvLK+Ak5A6n+Pj5MjJqKu683Td2a0LdXb+rXq8cbkdMI6dGdVu38aNPKkz49ehZYz6ihw+gW2jffoOht4X37sv/gwZzBUACnuk7M/fgTIoYPMwY2O4Vw6OgRqlSpwuIv5/Hcv8fRqp0fnfv0Jj09nZDAIA4cOpQzKBo1aTIJe/bg4e/LxMhI5n0y26yfaVEUaflcpZQbsEpr7Z79/DAQrLU+q5SqD2zWWj9Y2H5k+VwhSodJy+cePABDh8DcedC8RckUdg9RM6ZTo0YNxj0/Nlf75tgY3v3gA1YtXVbqNVmDOcvnmtqH7qK1PguQHerOd9tQKTUKGAXQ2KWeiYcTQpS45i1gp/Uu/hHmK/FBUa31bGA2GGfoJX08IUT5FDV5SoHttwdWReFMnbZ4PrurhezHC5YrSdzV+h+gfy8I8TEe1+efUyuEqLhMDfQVwJDsfw8Blt9jW2EJ63+Ad2fA+XPG1XznzxnPJdSFENmKMm1xIRAHPKiUOq2UGgG8CXRWSh0FOmc/FyXp848gz5xdbqQb7UIIQRH60LXWEXd5qZOFaxH3cuF88dqFEBWOXPpfXji7FK9dCCtTNWsw6KmROc9v3bqFU9Mm9AwPu8e7imfkmGc5cOhgkbY9fuIErg81y7lK9TbPAH/iExKKfewZ775T7PeUNAn08mLkaKjqkLutqoPRLoQ53v8vxGzJ3RazxWg3Q/Xq1dl38ADXr18HYP3GjTRs0KBY+yhsednPZ86ixUPNi7QvtyZNaNTQldjt23LaDh05zNWr1/DxKnSKdz4z/vNusbbXWuf7MrE0CfTyonM3GDcZXOoZV/O51DOedy74yjchiqxNGxgy+K9Qj9liPG9T/Js85NWtcxdWr1sLwMJlS4kIC895LT4hgXaPdKJ1+3a0e6QTh48eAWDu/G8IHzyQXv3C6fJYb7Kyshj94gu09PGiZ3gY3UMfZ9n33wEQ3L0rCbt3A8aSuVNejaJVOz/8OoZwvoDuyIiwsJyVFQEWLVtGRJjxF8PFSxcJHfgE3h2C8O4QxLYdcQBcu3aNYf96hof9fPDw9yV6+fdMjJzK9evX8Qzw58kRwwF4b+aHuPt64+7rzfuzZgHGXwXNvdow+sUXaBMYwKnTp83+md6LBHp50rkbLF4Jm+KNRwlzYQlBHWDeV0aIv/6a8TjvK6PdTANCjQBNT08nef8+fO84E36oWTNi1q5jz9btvDrlZSZPi8p5LS4+nnmffMrGVWv4dsVyjp88yc874vl85izidsUXeKzU1FT8vH3Yu30HQQEBfDZ3br5t+j0eyverVuWc+S/+NpoB2YE+dvx4Xnx2DLu2xBD9zXxGjnkWgNfefotaNWvy8454kuN20jGoA29OexVHR0eStsUx/4s5JO7Zw5fffM3OjZvZ8dMmPpv3JXv27gXg8NGjDI54gj1bt9OkcWOzf6b3IqstCiGM8B4xEt5+C8ZPsEiYA3i4u3P85EkWLltK9y6P5not5c8/GfLMKI7+8gtKKW7evJnzWueQjtx///0AbI2LI7xvXypVqkQ9FxdCAgMLPFaVKlXo2dU4yWnr6cn6TZvybVPPxYWWzZvz0+bNuDg7Y29vj3uLlgBs2LyJA4cP5Wz759WrXL16lQ2bN7Foztyc9tq1869FuDVuO3179spZRfHxXr2J3b6N3t170KRxY/x8fIry4zKbBLoQwuhm+eJzI8y/+ByCgiwW6r27dWfclClsXrOGy1eu5LS/8vprhAQG8d2CRRw/cYLgOxbOql6tWs6/i7LeFIC9vX3O8rR2dnZ37X+/fUMLF2fnXF1AWVlZxG3YmG+dcq11ocvl3qvEOz9LSZMuFyEqutt95vO+gpdf+av7Je9AqYmGDxrM1AkTeLile672lD9TcgZJ73Xzi/b+/kQvX05WVhbnL5xnc+xWs+oJ7d2HNT+uM7pb7lhdsUvHTsyc/WnO86Tk5ALbf//9d8D4Arn9V0VQQADfr15FWloaqampfLdqJYHtAsyq0xQS6EJUdLt35+4zv92nnj3YaC7Xhg0ZO/rZfO3jx77IpKgoAjo/QmZm5l3fH9rnMVwbNsDd15unx44sAw3lAAAQnUlEQVTF18uLWjVrmVzPfffdh5+3Dy5OzjR1c8tp/+Cdd0jYsxsPf19aeLflkzmfA/Dyv8fz+x9/4O7rTat2fmyKjQGMJXk9/H15csRw2nh6MvTJgfiEdMC3YzAjBw+ldatWJtdoqiItn2spsnyuEKXDpOVzy7Br165Ro0YNLl++jE9IMNvWb6Cei21eg2GN5XOFEKLU9OwXxh8pKWRkZPDKhAk2G+bmkkAXQpR5m9estXYJ5YL0oQtho0qzO1VYhrm/Mwl0IWyQQ6bmckqKhHo5orXxO3PINP13Jl0uQtgg17RMTp+/wMVLF61diigGh0yNa9rdZ/wURgJdCBtkr6FpqunBIMon6XIRQggbYVagK6VeVErtV0rtU0otVEo5FP4uIYQQJcHkQFdKNQSeB7y01u6AHTDAUoUJIYQoHnO7XCoDjkqpykA14Iz5JQkhhDCFyYGutf4NeBc4CZwFUrTWP+bdTik1SimVoJRKuJjyu+mVCiGEuCdzulxqA32ApkADoLpSamDe7bTWs7XWXlprL6da+dcRFkIIYRnmdLk8Avyqtb6otb4JfAu0s0xZQgghisucQD8J+Cmlqilj9fdOQNFuvy2EEMLizOlD3wksA3YDP2fva7aF6hJCCFFMZl0pqrWOBCItVIsQQggzyJWiQghhIyTQhRDCRkigCyGEjZBAF0IIGyGBLoQQNkICXQghbIQEuhBC2AgJdCGEsBES6EIIYSMk0IUQwkZIoAshhI2QQBdCCBshgS6EEDZCAl0IIWyEBLoQQtgICXQhhLARZgW6Uuo+pdQypdQhpdRBpZS/pQoTQghRPGbdsQj4P2Ct1jpMKVUFqGaBmoQQQpjA5EBXStUEgoChAFrrDCDDMmUJIYQoLnO6XP4OXAS+VErtUUp9rpSqbqG6hBBCFJM5gV4ZaAN8rLVuDaQCE/NupJQapZRKUEolXEz53YzDCSGEuBdzAv00cFprvTP7+TKMgM9Faz1ba+2ltfZyqlXbjMMJIYS4F5MDXWt9DjillHowu6kTcMAiVQkhhCg2c2e5PAfMz57h8j9gmPklCSGEMIVZga61TgK8LFSLEEIIM8iVokIIYSMk0IUQwkZIoAshhI2QQBdCCBshgS6EsA3rf4D+vSDEx3hc/4O1Kyp15k5bFEII61v/A7w7A26kG8/PnzOeA3TuZr26SpmcoQshyr/PP/orzG+7kW60VyAS6EKI8u/C+eK12ygJdCFE+efsUrx2GyWBLoQo/0aOhqoOuduqOhjtFYgMigohyr/bA5+ff2R0szi7GGFegQZEQQJdCGErOnercAGel3S5CCGEjZBAFxWHXHgibJx0uYiKQS48ERWAnKGLikEuPBEVgNmBrpSyU0rtUUqtskRBQpQIufBEVACWOEMfCxy0wH6EKDly4YmoAMwKdKWUK9AD+Nwy5QhRQuTCE1EBmDso+j4wHvibBWoRouTIhSeiAjA50JVSPYELWutEpVTwPbYbBYwCaOxSz9TDCWE+ufBE2DhzulwCgN5KqePAIqCjUuqbvBtprWdrrb201l5OtWqbcTghhBD3YnKga60naa1dtdZuwABgo9Z6oMUqE0IIUSwyD10IIWyERa4U1VpvBjZbYl9CCCFMI2foQghhIyTQhRDCRkigC1mFUAgbIastVnSyCqEQNkPO0Cs6WYVQCJshgV7RySqEQtgMCfSKTlYhFMJmSKBXdLIKoRA2QwZFKzpZhVAImyGBbi3rfyg7ISqrEAphEyTQrUGmCgohSoD0oVuDTBUUQpQACXRrkKmCQogSIIFuDTJVUAhRAiTQrUGmCgohSoAMilqDTBUUQpQAc24S3Qj4CqgHZAGztdb/Z6nCbN7dpgqWpemMQohyxZwul1vAS1rr5oAf8KxSqoVlyqqgbk9nPH8OtP5rOqMsZytKgiybbHPMuUn0Wa317ux/XwUOAg0tVZjNWPgV7EnI3bYnwWjPS6YzitIiJw82ySKDokopN6A1sNMS+7MpD7WAqEl/hfqeBOP5QwX8MSPTGUVpkZMHm2R2oCulagDRwAta6z8LeH2UUipBKZWQmJ6K6uBl7iHLl9ZeEPWGEeJzPjEeo94w2vOS6YyitMjJg00yK9CVUvYYYT5fa/1tQdtorWdrrb201l44ORnv6+CV7z+b1toL+oTCV18YjwWFOZTOdEbpNxUgJw82ypxZLgr4AjiotX7P3EIKCnW9JaGALcuhPQmwPBoGjzAeW3sVHOolPZ1R1pARt40cnft/CyDXQtgApbU27Y1KtQdigZ8xpi0CTNZar7nre7y8NAnmh3S5Cvrbfea3u1nyPi9N/XsZIZ6XSz1YvLJ0axHWJ1Nkyw0V7J2otS40MEwOdFNYKtALUmZDfuFXxgDoneG9JwEOHYCIwaVbS4iPMaMhL6VgU3zp1lLRSHgKMxQ10G3mStG79cNbPegLCu27dbmUNGeXgs/Qzek3laAqnHR1iVJiM4F+NzbdN19clu43laAqmntNEZSfk7Agmw/0gpTZs/mSZulBVwmqopEpgqKUVMhAvxuzzuZ//QWmTYbIGdD0HxauzIIsebs5CaqiKYmuLiEKIMvnFqJIc+avX4eJL8CJX2Hii8bzvH79BYb2Nx6z7f/VAfehLdj/q0P+7csDmctcNLJcsiglEugmyBvwi7+cSVrKH8YMkt8vQ+SE3OFdQOCnXq9E94kPcOCEIz0mPkDq9XL4q5CgKprO3WDcZGN6qFLG47jJ0i1VEir4hXM2M23RWobOmcOHzz9PjdTUnLYspQA42bgxbp98DW+9CttjICMDqlSBgA7014tYsf0+0jMq4VAliz4Bv7Mo8ldrfQzTySwXUVbkHaQH4wTDBr48K9w8dKvYD+daueCSeaHAl687OJD88MO0PHAgV+CnVqnGS1kf8OmtETlt1Rwy+fD5kwzvfrnEyxbCJtnwhXNFDfRy+Hd+GZEKdIeJmW9wTVUvcBPH9HS8d+3KFeYA1TPSmHZrcq62tHQ7Js12LalqhbB9Mkgvs1xMNhy4AHMZTle1jl6VVlAtMz3fZpUw1kW485sztUo1JvAmZNzRWA0uvGefb9DV5qdSCmEpMptIztBNMgdYDWTn9ztZ46icmUkWkIXKtWlqtWrs8vYmzcEYPLxe1YGVDr2YFzIMbo8nOgC9gGH5D1XQLBubX51SCFPIIL0EukkmYXS5ANVIZRlhVOYWt7DnMrW5PSqhgUt16tBx0yYuODuTpRTnb7kw4s8v4ADgBCjABWPdymKQkC9AAVNDRQUis4lkUNQkc4DngVRYRH96sYJqpJNBZey5lescPdXRkTEzZxLv68vidv3pn76YAxktjbPyYOAUsBhomecY+4H+d3mtmCpEt8316zC0X/Zsm3owdzE4Olq7KiEsQma5lLT+MHTJHD7keWqQes9Nzzs7U++N8zlfAjmqAR9i9MffKRVogRH2jTHCveBxV5PZXMhHTco3NZTIGdauSgiLqHCrLZa6EHhzyaRCw/yGvT0Tgt6ElyDfpmkY3Td5Az17wBUNnAMaANsx+0z9Tja1ns2aFbBjqxHmYDzGxRrt3XtbtzYhSpGcoZvKBYZeKPwMPb1qVRx/SIfewC1yBlIBqAp8TO7B0Du6c3K5HziJxc/Ui6LMh/xjXeCP3/O331cbvv+x9OsRwsJkHnpJewPmquGspgdp2dNV8n41ZqF4vtr7sBQjiPPOavQk/8yWOwZcc7kCjCigvRSU+QHYUWPAIc/sBgcHePo569QjhJWYe5Porkqpw0qpY0qpiZYqqlwYDmgYzhwu4EwWijSqcT073LNQrONRPvv9GeMsvKBrG3YCd86omo9xFn83KzHO4MuAMjWdsntv8Gtv9J2D8egfCN16WaceIazEnHuK2gFHgM7AaWAXEKG1PnDX99hSlwsYA5anoAX7WUx/hvIlywijMac4SWNasp+0ovSRfJP9OAqjX/1enCn4y6EMK5UuG5nlImxYaQyK+gDHtNb/A1BKLQL6YMywrhjeAAbCAVryMPsA6MEaFtOf/iwuWpgDTMl+LCzMqwFvmlirFZXKAKyjI7z5/l9r0kuYiwrInDP0MKCr1npk9vNBgK/Wekye7UZhnHtC48ZtOXHCrILLHDuMa/vNcXvi+r1+FQ4YX5eLzDxWGVfmB2CFsILSOENXBbTliySt9WxgNmR3udiapzH6yM3ROPuxoO+6218YJlxNWh7JPWCFMJ05gX4aaHTHc1fgjHnllEMfZT9+yt3P1BVGd0lBs1eqANOz/523D70aMBX4GuOKUStMWSwLbGrOvBAlyJxZLruAB5RSTZVSVYABwArLlFXOfARkYvx98g1Q547X6mAE8rW7vDYHeDL7v9lAE4wvgCbZzycA+7DoRUW2oszMshGijDDrwiKlVHfgfYyOgTla6+n33N7WZrmIckPO5kV5ViqX/mut1wBrzNmHEKVB+uZFRSBruYgKS/rmha2RQBciDzmbF+WVBLoQRSAhL8oDCXQhTCRdNqKskUAXwsLkbF5YiwS6EKVAzuZFaZBAF8KK5GxeWFLp3rFIqavA4VI7YOmoC1yydhEWJp+pfJDPVD5Y4jM10Vo7FbZRaZ+hHy7K1U7liVIqQT5T2SefqXyQz2QeuQWdEELYCAl0IYSwEaUd6LNL+XilQT5T+SCfqXyQz2SGUh0UFUIIUXKky0UIIWxEqQa6UipcKbVfKZWllCrXI9lKqa5KqcNKqWNKqYnWrscSlFJzlFIXlFL7rF2LpSilGimlNimlDmb/b2+stWsyl1LKQSkVr5Tam/2Zplm7JktRStkppfYopVZZuxZLUEodV0r9rJRKUkqV+AUGpX2Gvg94HIgp5eNalFLKDpgFdANaABFKqRbWrcoi5gJdrV2Ehd0CXtJaNwf8gGdt4Hd1A+iotW4FeAJdlVJ+Vq7JUsYCB61dhIWFaK09S2PqYqkGutb6oNbaFi4s8gGOaa3/p7XOABYBfaxck9m01jHAFWvXYUla67Na693Z/76KERYNrVuVebThWvZT++z/yv1gmFLKFegBfG7tWsor6UM3TUPg1B3PT1POQ6IiUEq5Aa2BndatxHzZXRNJwAVgvda63H8mjNtZjufut1svjzTwo1IqUSk1qqQPZvErRZVSG4B6Bbw0RWu93NLHsxJVQFu5P0OyZUqpGkA08ILW+k9r12MurXUm4KmUug/4TinlrrUut2MfSqmewAWtdaJSKtja9VhQgNb6jFLKGVivlDqU/ZdwibB4oGutH7H0Psug00CjO567AmesVIsohFLKHiPM52utv7V2PZaktf5DKbUZY+yj3AY6EAD0zr7xvANQUyn1jdZ6oJXrMovW+kz24wWl1HcY3bUlFujS5WKaXcADSqmmSqkqwABghZVrEgVQSingC+Cg1vo9a9djCUopp+wzc5RSjsAjwCHrVmUerfUkrbWr1toN4/9PG8t7mCulqiul/nb730AXSvhLt7SnLfZVSp0G/IHVSql1pXl8S9Fa3wLGAOswBtmWaK33W7cq8ymlFgJxwINKqdNKqRHWrskCAoBBQMfsqWNJ2WeB5Vl9YJNSKhnj5GK91tompvnZGBdgq1JqLxAPrNZary3JA8qVokIIYSOky0UIIWyEBLoQQtgICXQhhLAREuhCCGEjJNCFEMJGSKALIYSNkEAXQggbIYEuhBA24v8Br4lAimTqE34AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 100 \t #support vectors= 13  #margin vectors= 4\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XlcVPX+x/HXV0RxuZopizt2b5ZKiMoqgaBp7maASu5LVrZYj2uupbRo66/bLW2xMq3cpdTUNE0QVBRBkdytm1vuWqggovD9/XEAZVFgZgBn+Dwfjx7D+c6Zcz6D8Z4z3/M936O01gghhLB+lcq7ACGEEJYhgS6EEDZCAl0IIWyEBLoQQtgICXQhhLAREuhCCGEjJNCFEMJGSKALIYSNkEAXQggbUbmoFZRSc4CewFmttdst7c8DzwE3gNVa6/FFbate7Xu0q0sDM8oVQoiKJ/HQ/vNaa8ei1isy0IG5wEzgm5wGpVQw0Adw11pfU0o5FacoV5cGJMz+pugVhRBC5FJBXkeLs16RXS5a6xjgYr7mZ4C3tdbXstc5W+IKhRBCWJSpfejNgQCl1Hal1CallJclixJCCFFyxelyud3r6gC+gBewRCl1ny5k6kal1GhgNEATZxdT6xRCCFEEUwP9BPB9doDHK6WygHrAufwraq1nA7MBPB9oKXP1ClEGris4Ud2OdDtV3qWIEnDI1DRKy8TexKQ0NdCXAx2BaKVUc6AKcN7EbQkhLOxEdTv+4eyEa+3aKCWhbg201lxISeHEmbM0S800aRtF9qErpRYCccADSqkTSqmRwBzgPqXUHmARMLSw7hYhRPlIt1PUlTC3KkoZ/2bmfKsq8ghdax1+m6cGmbxXIUSpkzC3Pub+m8mVokIIYSMk0C1l/U/QvxcEexuP638q74qEKFenz5xhwLCh/NP9IVp6taN7yOMcOnyYI0eP4uZTNiOdp7weQeMWD1CzvnOZ7K+8SaBbwvqf4P0ZcOY0aG08vj9DQl1YjflL7HBtVY1Ktavj2qoa85fYmbU9rTV9nwgnKCCA35N/Zd+ORGZMi+DMubK9BrFX1+7ER20q032WJwl0S/jyE7iWnrftWrrRLsRdbv4SO0Y/X5WjxyuhteLo8UqMfr6qWaEeFbMJe/vKPD1yVG6bh7s7Ae3986x35OhRAh7tTNsAf9oG+LN1+zYATp0+TWDXLnj4++Hm40Xs1i1kZmYy7OmncPPx4iFfb/4zc2aRdfh6e1PfpeJc/2LqsEVxq7NnStYuxF1kymtVSLua92Rc2lXFlNeqMLDfVZO2uWffPtp5tClyPSdHR9av+BEHBwcO//Yb4SOHk7AplgVLl/Bop0eY8vJ4MjMzSUtLIyk5mT9PnWTP9h0A/P333ybVZssk0C3BydnoZimsXYi73LEThY+suF27JV2/fp3nxv2bpF+TsbOz49BvvwHg1bYtI8aM4fr16zzWsxce7u7c5+rK//44wvPj/k2PR7vSpVOnUq/P2kiXiyWMGgNVHfK2VXUw2oW4yzVpVPglJLdrL45WLVqQmLSryPX+M2smzk5O7N66jYRNsWRkZAAQ6P8wMWvX0bBBAwaPHsU3CxZQp04ddm+NIygggFlfzGbUc8/m2VZmZiYe/n54+Psx9c03TK7dmkmgW0LnbjBuMji7gFLG47jJRrsQd7np0zKoXi1veFevppk+LcPkbXbsEMS1axl8Mffr3LYdiYls2hybZ72US5eo7+JCpUqV+HbRQjIzjSskjx47hpOjI08OG87IwUPZuTuJ8xfOk5WVRUifx3jjlVfZuTspz7bs7OxI2hJH0pY4Xn/lVZNrt2bS5WIpnbtJgAurNLBfJnCNKa9V4dgJRZNGRpgb7aZRSvHDgoW8OHE8b3/wAQ4OVXFt0pQP334nz3pjRj1JyOCBLP3hB4IDA6lRowYA0bGxvPfRh9jb21OzRg2++fwL/jx5iuFjniYrKwuAt6a9VmQd4199hQVLl5CWlkajB5szashQIiZPMfl93e1UWV6x7/lASy03uBCi9O2vVZkW/7q/vMsQJtj/22FaXLqRp00FeSVqrT2Leq10uQghhI2QQBdCCBshgS6EEDZCAl0IIWyEBLoQQtgICXQhhLARxblj0Ryl1NnsuxPlf26cUkorpeqVTnlCiLKyd7/Czacae/db5pL/6e+9SytvT9z9fPDw92P7jh0W2a4pPpw1i7S0tALtETOmMyliWp62pORkWni2LfE+kpKTWbNunck1WkJxjtDnAl3zNyqlGgOdgWMWrkkIUcZSU6F7qAP7Dih6hDqQmmre9uK2b2fV2rXsjN1Cctx2Nqz8kcaNGlmm2BLKzMzkw09nkXa1YKCHh4Wx+PvIPG2LIpfxRFi/Eu8n6ddk1vxcskC/ceNG0SuVQJGBrrWOAS4W8tR/gPGA3EtUCCs3YkxVzp5TaK04c04x8tmqZm3v1JnT1Ktbl6pVje3Uq1uPBvXrA+Dq1pLzF4x7yifs3ElQd+N4MWLGdAY/OYqOPbtzv0fr3GkDomNjCOzahb5PDKClVzuefvGF3KtFFy5dwkO+3rj5eDFh6s3L/WvWd2bqm2/gExzE9Pfe5eSpUwT36E5wj7xXcz9wf3PuqV07z7eHJT98z4CQUAB+/uUX/Dp1pG2AP2FDBnHlyhXAmMag/SOdaN3eF++gDqSkpDB1+pss/j4SD38/Fkcu4+LFizwWPgB3Px98OwaTvGdP7vsc/cJzdOnTmyGjnzTr95yfSX3oSqnewJ9a690WrUYIUebmfFuZ1evsSE83ulrS0xU/rrVjzremzwzSpWMnjv95guZtPBjz0osF5nC5neS9e1i9NJK4X37h9Xfe5uSpUwDEJybyf9Pf4tdt8fz+xx98v3IFJ0+dYsK0qWxctZqkLXHs2JnI8lU/ApCamopby5Zsj4pm6sRJNKhfn6jVa4haXfCmM+GhYSyKXAbAtvh46ta5l/v/9S/OXzjPm++9w4aVP7IzdguebdrywcyPycjIoP/wofz3nXfZvXUbG1b+SI0aNXh9yiv0fzyEpC1x9A8JZdqM6bRxdyc5bjszpk1jyFM3wzsxKYkVixazYM7XBeoxR4kDXSlVHZgCTC3m+qOVUglKqYRzKX+VdHdCiFI2KcKe1LR886GnKSZF2Ju8zZo1a5IYs5nZH32EY7169B82lLnzvyvydX2696BatWrUq1uP4IBA4hMTAPBu1477mjXDzs6O8NAwNscZAR70cACO9RypXLkyA/v1J2bLFsCYqCukz2PFqnVASCjLViwnKyuLRZHLCA8LA2Bb/A72HTiAf5dH8PD3Y96C+Rw9fpyDhw9R39kFr3btAKhVqxaVKxf88Nu8LY7BA8IBY7KyCxcvkpKSAkDvbt2pVq1aseorCVM+gv8JNAN2Z9+huhGwUynlrbUuMCm41no2MBuMuVzMqFUIUQreirjOCy9XyRPq1atr3n7tulnbtbOzIyggkKCAQB5q1Yp5CxYwbOAgKttVzu0ySc93p6/8d73PWS6s/U7zUDk4OGBnV7w7LjVu1AjXJk3YtDmWyJUriNuwEQCNpnNwRxZ+PTfP+sl79hSopzCF1ZfzupxJyCytxEfoWutftdZOWmtXrbUrcAJoW1iYCyHufiMG36DHo5k4OBgB5OCg6dU1k+GDTD9hd/DwIQ5n36wCjBEgTRs3BsC1aRMSdxlT30auWJHndSvWrCY9PZ0LFy4QvTkWr7bGUXB8YiJ/HDlCVlYWiyMjedjPDx9PLzZt2cz5C+fJzMxk4bKldHj44ULr+UfNmly+fOW29YaHhvHSpIn8s1kzGjVsCICvlxdbtm/jt99/ByAtLY1Dhw/zYPPmnDx9ih2JiQBcvnyZGzduGPu4cnMfge39mb9kMWCcB6hXty61atUq/i/RBMUZtrgQiAMeUEqdUEqNLNWKhBBlbs4n13By1CilcXbUfDXrmlnbu3IllaFPP0VLr3a4+/mw7+ABIiZPBmDaxEmMnTCegEc7FziK9m7Xjh5hIfh26sir4yfknkj18/ZmYsRU3Hy8aObalL69elPfxYW3pr1GcI/utG7vS9vWHvTp0bPQekYPG063kL4FTormCOvbl7379+eeDAVwrOfI3E8/I3zEcOPEZqdgDhw+RJUqVVj89Tyef3kcrdv70rlPb9LT0wkOCGTfgQO5J0UjJk0mYdcu3P18mDhtGvM+m23W77Q4ZPpcIWyQKdPn7t2v6D/MgcVz02nVoux7RyNmTKdmzZqMe2Fsnvbo2Bje/+gjVi1dVuY1lQdzps+VG1wIIQBo1UKzZ7tpN4UWdwcJdCHEXeF2dxLKObEqiiZzuQghhI2QQBdCCBshgS6EEDZCAl0IIWyEBLoQolSoWjUZ/OSo3OUbN27g2KwpPcNC7/Cqkhn13LPsO7C/WOseOXqURg82z71KNYeHvx/xCQkl3veM998r8WtKmwS6EBXcux/aExWTNwqiYirx7oemz+UCxuXte/bv4+pVYyjk+o0badigQYm2UdT0sl/OnEXLB1sUa1uuTZvSuGEjYrduyW07cOggly9fwduzyCHeBcz4v/dLtL7WusCHiaVJoAtRwXm1zaTfUIfcUI+KqUS/oQ54tc00e9vdOndh9bq1ACxctpTw0LDc5+ITEmj/SCfaPNye9o904uDhQwDMnf8dYUMG0atfGF0e601WVhZjXnqRVt6e9AwLpXvI4yxb/gMAQd27krBzJ2BMmTvl9Qhat/fFt2MwZ86eKVBPeGho7syKAIuWLSM81PjGcO78OUIGPYFXh0C8OgSyZVscAFeuXGH4M0/zkK837n4+RK5YzsRpU7l69Soe/n4MHDkCgA9mfoybjxduPl58OGsWYHwraOHZljEvvUjbAH+Onzhh9u/0TiTQhajgggOzWDIvnX5DHZj6pj39hjqwZF46wYHmH00OCDECND09neS9e/C55Uj4webNiVm7jl2bt/L6lFeY/FpE7nNx8fHM++xzNq5aw/crV3Dk2DF+3RbPlzNnEbcjvtB9paam4uvlze6t2wj09+eLuXMLrNPv8RCWr1qVe+S/+PtIBmQH+tjx43np2efYsSmGyO/mM+q5ZwF44913qF2rFr9uiyc5bjsdAzvw9muvU61aNZK2xDH/qzkk7trF1999y/aN0Wz7JYov5n3Nrt3G7OIHDx9mSPgT7Nq8laZNmpj9O70TubBICEFwYBbPjLzOG+9W4dXxGRYJcwB3NzeOHDvGwmVL6d7l0TzPpVy6xNCnR3P4999RSnH9+s3ZHTsHd+Tee+8FYHNcHGF9+1KpUiVcnJ0JDggodF9VqlShZ1djrpZ2Hh6sj4oqsI6LszOtWrTgl+honJ2csLe3x61lKwA2REex7+CB3HUvXb7M5cuX2RAdxaI5c3Pb69SpU2C7m+O20rdnr9xZFB/v1ZvYrVvo3b0HTZs0wdfbuzi/LrNJoAshiIqpxKdf2fPq+Aw+/cqe4MBMi4V6727dGTdlCtFr1nDh4s2bn7365hsEBwTyw4JFHDl6lKBbJs6qUb167s/FnW/K3t4+d3paOzu72/a/59zQwtnJKU8XUFZWFnEbNhaYp1xrXeR0uXcq8db3Utqky0WICi6nz3zJvHRef+V6bvdL/hOlphoxeAhTJ0zgoVZuedpTLqXkniS9080vHvbzI3LFCrKysjhz9gzRsZvNqiekdx/W/LzO6G65ZXbFLh07MXP257nLScnJhbb/9Zdxox57e/vcbxWB/v4sX72KtLQ0UlNT+WHVjwS09zerTlNIoAtRwe3YaZenzzynT33HzuLdIKIojRo2ZOyYZwu0jx/7EpMiIvDv/AiZmbc/ARvS5zEaNWyAm48XT40di4+nJ7Vr1Ta5nnvuuQdfL2+cHZ1o5uqa2/7Re++RsGsn7n4+tPRqx2dzvgTglZfH89fff+Pm40Xr9r5ExcYAxpS87n4+DBw5grYeHgwbOAjv4A74dAxi1JBhtGnd2uQaTSXT5wphg0yZPvduduXKFWrWrMmFCxfwDg5iy/oNuDg7l3dZpUKmzxVC2LSe/UL5OyWFjIwMXp0wwWbD3FxFBrpSag7QEzirtXbLbnsP6AVkAL8Dw7XWf5dmoUKIiit6zdryLsEqFKcPfS7QNV/besBNa+0OHAImWbguIYQQJVRkoGutY4CL+dp+1lrndPJsAxqVQm1CCCFKwBKjXEYAP1lgO0IIIcxgVqArpaYAN4D5d1hntFIqQSmVcC7lL3N2J4QQ4g5MDnSl1FCMk6UD9R3GPmqtZ2utPbXWno61C14yK4SwTafPnGHAsKH80/0hWnq1o3vI4xw6fJgjR4/i5uNVKvuM2bKZtgH+VK5TO3cCr4rEpEBXSnUFJgC9tdZpli1JCFHmliyGVi2g9j+MxyWLzdqc1pq+T4QTFBDA78m/sm9HIjOmRXDm3FkLFVy4Jo0aM/fTz3kirF+p7uduVWSgK6UWAnHAA0qpE0qpkcBM4B/AeqVUklLqs1KuUwhRWpYshuefg+PHjUlJjh83ls0I9aiYTdjbV+bpkTdvcOHh7l7gcvgjR48S8Ghn2gb40zbAn63btwFw6vRpArt2wcPfDzcfL2K3biEzM5NhTz+Fm48XD/l685+ZMwvs17VpU9zd3KhUqWJeBF/kOHStdXghzV+VQi1CiPLwWgRk34Qi19WrRnu//iZtcs++fbTzaFPkek6Ojqxf8SMODg4c/u03wkcOJ2FTLAuWLuHRTo8w5eXxZGZmkpaWRlJyMn+eOsme7TsA+PtvufQlP7lSVIiK7nY3XSjlmzEAXL9+nefG/ZukX5Oxs7Pj0G+/AeDVti0jxozh+vXrPNazFx7u7tzn6sr//jjC8+P+TY9Hu9KlU6dSr8/aVMzvJUKImxrd5jKS27UXQ6sWLUhM2lXkev+ZNRNnJyd2b91GwqZYMjIyAAj0f5iYteto2KABg0eP4psFC6hTpw67t8YRFBDArC9m596AQtwkgS5ERTctAvLNAU61aka7iTp2COLatQy+mPt1btuOxEQ2bY7Ns17KpUvUd3GhUqVKfLtoYe6si0ePHcPJ0ZEnhw1n5OCh7NydxPkL58nKyiKkz2O88cqr7NydZHJ9tkoCXYiKrl9/+HgmNG4MShmPH880uf8cQCnFDwsWsj5qI/90f4hW3p5EvDWDBi7186w3ZtSTzFswH9+OwRz67bfcO/5Ex8bi4e9Hm4fbE7lyOWOfGcOfJ08R1KMbHv5+DHvmKd6a9lqB/e5ITKTRg81ZuvwHnho7llbeJb/5szWT6XOFsEG2Nn1uRWLO9LlyhC6EEDZCAl0IIWyEBLoQQtgICXQhhLAREuhCCGEjJNCFEMJGSKALIQz794GPl/FoAdPfe5dW3p64+/ng4e/H9h07LLJdU3w4axZpaQUnho2YMZ1JEdPytCUlJ9PCs22J95GUnMyadetMrtESJNCFEJCaCqEhcOAAhIYay2aI276dVWvXsjN2C8lx29mw8kcamzGVgDkyMzP58NNZpF0tGOjhYWEs/j4yT9uiyGUmTb+b9Gsya34uWaDfuHGj6JVKQAJdCAFjnoFz54zpc8+dhWfHmLW5U2dOU69uXapWrQpAvbr1aFDfuErU1a0l5y+cByBh506Cuhv3oI+YMZ3BT46iY8/u3O/ROnfagOjYGAK7dqHvEwNo6dWOp198gaysLAAWLl3CQ77euPl4MWHqq7n7r1nfmalvvoFPcBDT33uXk6dOEdyjO8E9uuWp84H7m3NP7dp5vj0s+eF7BoSEAvDzL7/g16kjbQP8CRsyiCtXrgDGFantH+lE6/a+eAd1ICUlhanT32Tx95F4+PuxOHIZFy9e5LHwAbj7+eDbMZjkPXty3+foF56jS5/eDBn9pFm/5/wk0IWo6L79FtathfR0Yzk9Hdb+ZLSbqEvHThz/8wTN23gw5qUXC8zhcjvJe/ewemkkcb/8wuvvvM3JU6cAiE9M5P+mv8Wv2+L5/Y8/+H7lCk6eOsWEaVPZuGo1SVvi2LEzkeWrfgQgNTUVt5Yt2R4VzdSJk2hQvz5Rq9cQtbrg7Y/DQ8NYFLkMgG3x8dStcy/3/+tfnL9wnjffe4cNK39kZ+wWPNu05YOZH5ORkUH/4UP57zvvsnvrNjas/JEaNWrw+pRX6P94CElb4ugfEsq0GdNp4+5Octx2ZkybxpCnboZ3YlISKxYtZsGcrwvUYw4JdCEquoipkL9/OS3NaDdRzZo1SYzZzOyPPsKxXj36DxvK3PnfFfm6Pt17UK1aNerVrUdwQCDxiQkAeLdrx33NmmFnZ0d4aBib44wAD3o4AMd6jlSuXJmB/foTs2ULAHZ2doT0eaxYtQ4ICWXZiuVkZWWxKHIZ4WFhAGyL38G+Awfw7/IIHv5+zFswn6PHj3Pw8CHqO7vg1a4dALVq1aJy5YIzkW/eFsfgAcbtJDp2COLCxYukpKQA0Ltbd6rlnxDNAmQ+dCEquojX4eV/5w316tXhtdfN2qydnR1BAYEEBQTyUKtWzFuwgGEDB1HZrnJul0n6tfQ8r1FKFbpcWPud5qFycHDAzs6uWHU2btQI1yZN2LQ5lsiVK4jbsBEAjaZzcEcWfj03z/rJe/YUqKcwhdWX87qcScgsrTi3oJujlDqrlNpzS9u9Sqn1SqnD2Y9y92chrNXgwfBoV3BwMJYdHKBrNxg02ORNHjx8iMPZN6sAYwRI08aNAXBt2oTEXcbUt5ErVuR53Yo1q0lPT+fChQtEb47Fq61xFByfmMgfR46QlZXF4shIHvbzw8fTi01bNnP+wnkyMzNZuGwpHR5+uNB6/lGzJpcvX7ltveGhYbw0aSL/bNaMRg0bAuDr5cWW7dv47fffAUhLS+PQ4cM82Lw5J0+fYkdiIgCXL1/mxo0bxj6u3NxHYHt/5mffxi86NoZ6detSq1at4v8STVCcLpe5QNd8bROBX7TW9wO/ZC8LIazVJ5+Co6Mxfa6jE8z6xKzNXbmSytCnn6KlVzvc/XzYd/AAEZMnAzBt4iTGThhPwKOdCxxFe7drR4+wEHw7deTV8RNyT6T6eXszMWIqbj5eNHNtSt9evanv4sJb014juEd3Wrf3pW1rD/r06FloPaOHDadbSN8CJ0VzhPXty979+3NPhgI41nNk7qefET5iuHFis1MwBw4fokqVKiz+eh7PvzyO1u196dynN+np6QQHBLLvwIHck6IRkyaTsGsX7n4+TJw2jXmfzTbrd1ocxZo+VynlCqzSWrtlLx8EgrTWp5RS9YForfUDRW1Hps8VomyYNH3u/n0wbCjMnQctWpZOYXcQMWM6NWvWZNwLY/O0R8fG8P5HH7Fq6bIyr6k8mDN9rql96M5a61MA2aHudLsVlVKjgdEATZxdTNydEKLUtWgJ28vv4h9hvlI/Kaq1ng3MBuMIvbT3J4SwThGTpxTannNiVRTN1GGLZ7K7Wsh+PGu5ksRtrf8J+veCYG/jcX3BMbVCiIrL1EBfCQzN/nkosOIO6wpLWP8TvD8Dzpw2ruY7c9pYllAXQmQrzrDFhUAc8IBS6oRSaiTwNtBZKXUY6Jy9LErTl59AvjG7XEs32oUQgmL0oWutw2/zVCcL1yLu5OyZkrULISocufTfWjg5l6xdiHKmatVk8JOjcpdv3LiBY7Om9AwLvcOrSmbUc8+y78D+Yq175OhRGj3YPPcq1Rwe/n7EJySUeN8z3n+vxK8pbRLo1mLUGKjqkLetqoPRLoQ5PvwPxGzK2xazyWg3Q40aNdizfx9Xr14FYP3GjTRs0KBE2yhqetkvZ86i5YMtirUt16ZNadywEbFbt+S2HTh0kMuXr+DtWeQQ7wJm/N/7JVpfa13gw8TSJNCtReduMG4yOLsYV/M5uxjLnQu/8k2IYmvbFoYOuRnqMZuM5bYlv8lDft06d2H1urUALFy2lPDQsNzn4hMSaP9IJ9o83J72j3Ti4OFDAMyd/x1hQwbRq18YXR7rTVZWFmNeepFW3p70DAule8jjLFv+AwBB3buSsHMnYEyZO+X1CFq398W3YzBnCumODA8NzZ1ZEWDRsmWEhxrfGM6dP0fIoCfw6hCIV4dAtmyLA+DKlSsMf+ZpHvL1xt3Ph8gVy5k4bSpXr17Fw9+PgSNHAPDBzI9x8/HCzceLD2fNAoxvBS082zLmpRdpG+DP8RMnzP6d3okEujXp3A0W/whR8cajhLmwhMAOMO8bI8TffMN4nPeN0W6mASFGgKanp5O8dw8+txwJP9i8OTFr17Fr81Zen/IKk1+LyH0uLj6eeZ99zsZVa/h+5QqOHDvGr9vi+XLmLOJ2xBe6r9TUVHy9vNm9dRuB/v58MXdugXX6PR7C8lWrco/8F38fyYDsQB87fjwvPfscOzbFEPndfEY99ywAb7z7DrVr1eLXbfEkx22nY2AH3n7tdapVq0bSljjmfzWHxF27+Pq7b9m+MZptv0Txxbyv2bV7NwAHDx9mSPgT7Nq8laZNmpj9O70TmW1RCGGE98hR8O47MH6CRcIcwN3NjSPHjrFw2VK6d3k0z3Mply4x9OnRHP79d5RSXL9+Pfe5zsEduffeewHYHBdHWN++VKpUCRdnZ4IDAgrdV5UqVejZ1TjIaefhwfqoqALruDg706pFC36JjsbZyQl7e3vcWrYCYEN0FPsOHshd99Lly1y+fJkN0VEsmjM3t71OnYJzEW6O20rfnr1yZ1F8vFdvYrduoXf3HjRt0gRfb+/i/LrMJoEuhDC6Wb760gjzr76EwECLhXrvbt0ZN2UK0WvWcOHixdz2V998g+CAQH5YsIgjR48SdMvEWTWqV8/9uTjzTQHY29vnTk9rZ2d32/73nBtaODs55ekCysrKIm7DxgLzlGuti5wu904l3vpeSpt0uQhR0eX0mc/7Bl559Wb3S/4TpSYaMXgIUydM4KFWbnnaUy6l5J4kvdPNLx728yNyxQqysrI4c/YM0bGbzaonpHcf1vy8zuhuuWV2xS4dOzFz9ue5y0nJyYW2//XXX4DxAZLzrSLQ35/lq1eRlpZGamoqP6z6kYD2/mbVaQoJdCEqup078/aZ5/SpZ59sNFejhg0ZO+bZAu3jx77EpIgI/Ds/QmZm5m1fH9LnMRo1bICbjxdPjR2Lj6cX5CiVAAAQSklEQVQntWvVNrmee+65B18vb5wdnWjm6prb/tF775Gwayfufj609GrHZ3O+BOCVl8fz199/4+bjRev2vkTFxgDGlLzufj4MHDmCth4eDBs4CO/gDvh0DGLUkGG0ad3a5BpNVazpcy1Fps8VomyYNH3uXezKlSvUrFmTCxcu4B0cxJb1G3Bxts1rMMpj+lwhhCgzPfuF8ndKChkZGbw6YYLNhrm5JNCFEHe96DVry7sEqyB96ELYqLLsThWWYe6/mQS6EDbIIVNzISVFQt2KaG38mzlkmv5vJl0uQtigRmmZnDhzlnPnz5V3KaIEHDI1jdJuP+KnKBLoQtggew3NUk0PBmGdpMtFCCFshFmBrpR6SSm1Vym1Rym1UCnlUPSrhBBClAaTA10p1RB4AfDUWrsBdsAASxUmhBCiZMztcqkMVFNKVQaqAyfNL0kIIYQpTA50rfWfwPvAMeAUkKK1/jn/ekqp0UqpBKVUwrmUv0yvVAghxB2Z0+VSB+gDNAMaADWUUoPyr6e1nq219tRaezrWLjiPsBBCCMswp8vlEeAPrfU5rfV14HugvWXKEkIIUVLmBPoxwFcpVV0Zs793Aop3+20hhBAWZ04f+nZgGbAT+DV7W7MtVJcQQogSMutKUa31NGCahWoRQghhBrlSVAghbIQEuhBC2AgJdCGEsBES6EIIYSMk0IUQwkZIoAshhI2QQBdCCBshgS6EEDZCAl0IIWyEBLoQQtgICXQhhLAREuhCCGEjJNCFEMJGSKALIYSNkEAXQggbIYEuhBA2wqxAV0rdo5RappQ6oJTar5Tys1RhQgghSsasOxYB/wXWaq1DlVJVgOoWqEkIIYQJTA50pVQtIBAYBqC1zgAyLFOWEEKIkjKny+U+4BzwtVJql1LqS6VUDQvVJYQQooTMCfTKQFvgU611GyAVmJh/JaXUaKVUglIq4VzKX2bsTgghxJ2YE+gngBNa6+3Zy8swAj4PrfVsrbWn1trTsXYdM3YnhBDiTkwOdK31aeC4UuqB7KZOwD6LVCWEEKLEzB3l8jwwP3uEy/+A4eaXJIQQwhRmBbrWOgnwtFAtQgghzCBXigohhI2QQBdCCBshgS6EEDZCAl0IIWyEBLoQwjas/wn694Jgb+Nx/U/lXVGZM3fYohBClL/1P8H7M+BaurF85rSxDNC5W/nVVcbkCF0IYf2+/ORmmOe4lm60VyAS6EII63f2TMnabZQEuhDC+jk5l6zdRkmgCyGs36gxUNUhb1tVB6O9ApGTokII65dz4vPLT4xuFidnI8wr0AlRkEAXQtiKzt0qXIDnJ10uQghhIyTQRcUhF54IGyddLqJikAtPRAUgR+iiYpALT0QFYHagK6XslFK7lFKrLFGQEKVCLjwRFYAljtDHAvstsB0hSo9ceCIqALMCXSnVCOgBfGmZcoQoJXLhiagAzD0p+iEwHviHBWoRovTIhSeiAjA50JVSPYGzWutEpVTQHdYbDYwGaOLsYuruhDCfXHgibJw5XS7+QG+l1BFgEdBRKfVd/pW01rO11p5aa0/H2nXM2J0QQog7MTnQtdaTtNaNtNauwABgo9Z6kMUqE0IIUSIyDl0IIWyERa4U1VpHA9GW2JYQQgjTyBG6EELYCAl0IYSwERLoQmYhFMJGyGyLFZ3MQiiEzZAj9IpOZiEUwmZIoFd0MguhEDZDAr2ik1kIhbAZEugVncxCKITNkJOiFZ3MQiiEzZBALy/rf7p7QlRmIRTCJkiglwcZKiiEKAXSh14eZKigEKIUSKCXBxkqKIQoBRLo5UGGCgohSoEEenmQoYJCiFIgJ0XLgwwVFEKUAnNuEt0Y+AZwAbKA2Vrr/1qqMJt3u6GCd9NwRiGEVTGny+UG8G+tdQvAF3hWKdXSMmVVUDnDGc+cBq1vDmeU6WxFaZBpk22OOTeJPqW13pn982VgP9DQUoXZlOL+4chwRlFW5ODBJlnkpKhSyhVoA2y3xPZsSkn+cGQ4oygrcvBgk8wOdKVUTSASeFFrfamQ50crpRKUUgnnUv4yd3fWpyR/ODKcUZQVOXiwSWYFulLKHiPM52utvy9sHa31bK21p9ba07F2HXN2Z51K8odTFsMZpd9UgBw82CiTA10ppYCvgP1a6w8sV5KNKckfTuduMG4yOLuAUsbjuMmWG+Ui/aYih1wLYZPMGYfuDwwGflVKJWW3TdZarzG/LBsyakzeibjgzn84pTnz4Z26f2RoZMUi10LYJJMDXWu9GVAWrMU23U1/ONJvWn7uxusLZNpkmyNXipaFu+UPx8nZ6GYprN1Ud2NQ3W1kumRRRmQul4rE0v2m0idfPDJEUJQRCfSKxNInXSWoike6ukQZkS6XisaS3T8SVMVTGl1dQhRCjtDNtfAb2JWQt21XgtF+qz9+h2H9jcdse/9wwG1YS/b+ka8bxFrIWObikSGCooxIoJvrwZYQMelmqO9KgFfHw/JlN8P76lWY+CIc/QMmvgRXr5J6tRLdJ97PvqPV6DHxflKvWuE/hQRV8ZT29QXipgp+4ZzSWpfZzjwfaKkTZn9T9IrWZleCEep9Qowgt6sEf/8NTi4wdzG88zpsjYGMDKhSBfw70F8vYuXWe0jPqIRDlSz6+P/Foml/lPc7KTkZ5SLuFvlHE4FxgGEDH54qyCtRa+1Z5HplGejK01OTkFDoc3pT4e13s71/OND/tftYPO1/tIr6EL75Cpo2g1N/3gzv+/4FR/4H6Tf/J7teuRrP8zGf3xiZ21bdIZOPXzjGiO4XyuOtCGH9+vcq/FyFswss/rHs67Egqwt0U5Tnh0Dq1Uq0HNaK42er0P+en1mQOQDV6iGIiy3W68/ghAt5Tx463XOdM8t3l0a5Qti+YG9j+Gx+SkFUfNnXY0HFDXSrHuWiOhT5/vKw5AfAiHdcOfuXPR10NB/9NYg3PL5m8t6RxfqFXq9cjanMMG4Rkq26QyZvP3XCYvUJUeHIaCLrDvSSKukHABT+ITBnTV1Wb6tNekYlvNjBeN7l5d1TeKfy87zIe9Qg7ebKVR3A9T744zfIyCDLviob7LtztFUoDruzcvvQe/n9zfBu0t0ihMlKOm+SDbLqLpdy4wycNX6sTip7aUljjnOcJjhwFWfO3pzkxtEZvlkKw/qhz57hT9WYB7P2cq+TPQAnzlWhiVMGe+fupUa1rPJ4N0LYDhs9SV8hulzKzVvA08B1mMMInDiLHZoG/Ik9N/LMWJZ65RLPnTxM/C8bWNy+P/3TF5OaUZPUS0AQcBxWjzssYW6Ohd8Yw0fb3PL/+64EOLAPwoeUX12i7N0t8yaVEwl0U4wAvoVh0XPowWqqY3zFq3Jrp3i2Glev8vakSbi8dYaHMvdARvYT6UAM8DG4DXO7+YJ3AS8g+JaNRAE7gPE3m6xxVFCpybkWIOItI9RzhpFGvFXelQlRpqTLxVSpcPofzjjrs3dc7Zq9PU99/jnzXhwOBW7QBzhBnsEuUUA/YAlGqOdfNoNNfwjcei3Aisib4S6EDZAul9JWAyY+/hYfR75ATVJvu5quVIl5VYbD1UKerAK8na8tGCO8+wEPAyuBLGA4MB0YaHrJ5TkqqNS18TTC/JuvYMhICXNRIUmgm2Hu5hF0ZR29WEl10tHkveNHFooXqn8ITwHXb7OR4YW0BWOE+fJb2o4Co7N/NiPUS8JSo4LKxK4E48h8yEjjsY2nhLqocMzqclFKdQX+C9gBX2qt8x9v5l3flrpcooCON0e5NOE4V6lGJbKoRjpZKNbxKN0pYi6JZ4Cc2WbnA1Mwwvt2mgJHzC//bmL2h8Ctfeb5+9Al1IUNKPUrRZVSdsAhoDNwAuO0XbjWet9tX2NLgf4u8AFwBlqyl8X0Zxhfs4xQmnCcYzShFXtJo0bR2/ou+3E03DqEvVAKowumAivwASCjXISNK4tA9wMitNaPZi9PAtBa33ZogU0FOhhH1IPyNuWEe38Ws49WxdtO0+zHOx2Z37rukWJXKLJZ1fkAIfIpi5OiDYHjtyyfAHwKFKLUaHJ6f5s0MWN3d6GBwBDyHDHvoxUPsadk2zlWzPWqY5wYFSVm0yeEhchmTqCrQtoKHO5rrWcDsyH7CN3WPAV8auY2cj7nCjtCt8P4wGiC2aNcRPFZ1QlhIbKZE+gngMa3LDcCTppXjhXKOaH5Obfv21YYR9eFjW6sws2j7vx96NUxPgolxK2CfAiI8mZOoO8A7ldKNQP+BAYAT1ikKmvzCXlHqowFcubZqosxDmhgEc/lmILRBSNH5BWCdAUJSzJ32GJ34EOMjoE5Wus79vDa3ElRIayAfAhYvzK5UlRrvQZYY842hBClS7qCKg65UlQIUYB0BVknCXQhhNnkW8DdQQJdCFEu5EPA8iTQhRBWQ7qC7kwCXQhhsyratwAJdCGEuIU1fwuQQBdCCDPcTd8CyvYWdEpdBg6W2Q7LRj3gfHkXYWHynqyDvCfrYIn31FRr7VjUSmV9hH6wOFc7WROlVIK8p7ufvCfrIO/JPJXKYidCCCFKnwS6EELYiLIO9NllvL+yIO/JOsh7sg7ynsxQpidFhRBClB7pchFCCBtRpoGulApTSu1VSmUppaz6TLZSqqtS6qBS6jel1MTyrscSlFJzlFJnlVIlvCnq3Usp1VgpFaWU2p/9/97Y8q7JXEopB6VUvFJqd/Z7eq28a7IUpZSdUmqXUmpVeddiCUqpI0qpX5VSSUqpUr8CqayP0PcAjwMxZbxfi1JK2QGzgG5ASyBcKdWyfKuyiLlA1/IuwsJuAP/WWrcAfIFnbeDf6hrQUWvdGvAAuiqlfMu5JksZC+wv7yIsLFhr7VEWQxfLNNC11vu11rZwYZE38JvW+n9a6wxgEdCnnGsym9Y6BrhY3nVYktb6lNZ6Z/bPlzHComH5VmUebbiSvWif/Z/VnwxTSjUCegBflnct1kr60E3TEDh+y/IJrDwkKgKllCvQBthevpWYL7trIgk4C6zXWlv9e8K4neV4bn+7dWukgZ+VUolKqdGlvTOLXymqlNoAuBTy1BSt9QpL76+cqELarP4IyZYppWoCkcCLWutL5V2PubTWmYCHUuoe4AellJvW2mrPfSilegJntdaJSqmg8q7Hgvy11ieVUk7AeqXUgexvwqXC4oGutX7E0tu8C50AGt+y3Ag4WU61iCIopewxwny+1vr78q7HkrTWfyulojHOfVhtoAP+QO/sG887ALWUUt9prQeVc11m0VqfzH48q5T6AaO7ttQCXbpcTLMDuF8p1UwpVQUYAKws55pEIZRSCvgK2K+1/qC867EEpZRj9pE5SqlqwCPAgfKtyjxa60la60Zaa1eMv6eN1h7mSqkaSql/5PwMdKGUP3TLethiX6XUCcAPWK2UWleW+7cUrfUN4DlgHcZJtiVa673lW5X5lFILgTjgAaXUCaXUyPKuyQL8gcFAx+yhY0nZR4HWrD4QpZRKxji4WK+1tolhfjbGGdislNoNxAOrtdZrS3OHcqWoEELYCOlyEUIIGyGBLoQQNkICXQghbIQEuhBC2AgJdCGEsBES6EIIYSMk0IUQwkZIoAshhI34fxvINbuGCqeOAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "xmin,ymin = np.amin(x,axis = 0)\n",
    "xmax,ymax = np.amax(x,axis = 0)\n",
    "xx,yy = np.meshgrid(np.arange(xmin - 1, xmax + 1, 0.05),\n",
    "                    np.arange(ymin - 1, ymax + 1, 0.05))\n",
    "beta=np.empty((3,x.shape[0]))\n",
    "b=np.empty(3)\n",
    "for i,C in enumerate([0.01,1,100]):\n",
    "    beta[i],b[i]=SMO(x,label,C)\n",
    "    print(\"C=\",C,\"\\t #support vectors=\", np.count_nonzero(beta[i]),\n",
    "             \" #margin vectors=\", np.count_nonzero(beta[i,C>beta[i]]))\n",
    "    \n",
    "    #hyperlpane/contour:\n",
    "    f=set_f(beta[i],label,x,b[i])\n",
    "    zz=np.apply_along_axis(f,1,np.array([xx.flatten(),yy.flatten()]).T).reshape(xx.shape)\n",
    "    plt.contourf(xx,yy,zz,0, colors = ['cyan', 'pink'])\n",
    "    for lab,col in zip([-1,1],['b','r']):\n",
    "        for indecies, style, name in zip([beta[i]==0, beta[i]==C,\n",
    "                                          np.logical_and(beta[i]>0,beta[i]<C)],\n",
    "                                         ['o','d','x'],\n",
    "                                        [\"Class \"+str(lab),\"Support Vector\",\"Margin Vector\"]):\n",
    "            plt.plot(x[np.logical_and(label==lab, indecies ),0],\n",
    "                     x[np.logical_and(label==lab, indecies ),1],\n",
    "                     col+style, label=name)\n",
    "    plt.legend(loc = \"best\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task 2.4.** Draw 2, 00 test data points according to the distributions from\n",
    "task 2.2 (1,000 points for class -1 and 1,000 points for class 1). Evaluate the\n",
    "accuracy (percentage of correctly classified data points) for the LLS and SVM\n",
    "models calculated in task task 2.3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#copy thefunctions from exercise 1\n",
    "# Returns confusion matrix for a given vector of prediction labels yp and the real label yr. \n",
    "\n",
    "def conf_matrix(yp, yr, lab1, lab2): #predicted label and real label, label1 and label2\n",
    "    n = yr.shape[0]\n",
    "    t_00 = np.count_nonzero(np.logical_and(yp == lab1, yr == lab1)) \n",
    "    t_11 = np.count_nonzero(np.logical_and(yp == lab2, yr == lab2))\n",
    "    f_01 = np.count_nonzero(np.logical_and(yp == lab1, yr == lab2))\n",
    "    f_10 = n - t_11 - t_00 - f_01\n",
    "    C = np.array([[t_00, f_01], [f_10, t_11]])\n",
    "    return(C)\n",
    "\n",
    "acc = lambda C,n : np.trace(C)/ n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1000  698]\n",
      " [   0  302]]\n",
      "0.651\n",
      "[[978 146]\n",
      " [ 22 854]]\n",
      "0.916\n",
      "[[927 126]\n",
      " [ 73 874]]\n",
      "0.9005\n"
     ]
    }
   ],
   "source": [
    "n=1000\n",
    "x_test=np.append(np.random.exponential(1/4,(n,2)),\n",
    "                 np.random.exponential(2,(n,2)),axis=0)\n",
    "label_test_real=np.append(-np.ones(n,int),np.ones(n,int))\n",
    "for i in range(3):\n",
    "    f=set_f(beta[i],label,x,b[i])\n",
    "    label_test_pred=np.where(np.apply_along_axis(f,1,x_test) <0,-1,1)\n",
    "    conf=conf_matrix(label_test_pred,label_test_real,-1,1)\n",
    "    print(conf)\n",
    "    print(acc(conf,2*n))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(2.3) $KKT_i := (C-\\beta_i) \\max(0, 1-y_i f(\\mathbf{x_i}))+ \n",
    "                    \\beta_i \\max(0, y_i f(\\mathbf{x_i})-1)\n",
    "                    = C \\max(0, 1-y_i f(\\mathbf{x_i})) +\n",
    "                          \\beta_i ( y_i f(\\mathbf{x_i})-1)$   ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def set_KKT(C,beta,y,f,x):\n",
    "    KKT = lambda i : C*max(0,1-y[i]*f(x[i]))-beta[i]*(1-y[i]*f(x[i]))\n",
    "    return KKT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task 2.5.** Repeat task 2.3 and task 2.4 but instead of drawing the indices $i, j$ for each SMO-step randomly, write an outer loop which iterates over all $i \\in \\{ 1, \\dots , n \\}$ and check if $KKT_i > 0$.\n",
    "If this is the case, randomly pick a $j \\neq i$ for which $0 < \\beta_j < C$. \n",
    "If no such j exists, randomly pick a $j \\in \\{ 1, \\dots , n \\} \\setminus \\{ i \\}$. \n",
    "Subsequently, run the OneStep function for the pair $( i, j )$.\n",
    "\n",
    "If $KKT_i = 0$ for each i or if the maximum number of OneStep calls\n",
    "(10,000) is reached, the algorithm terminates.\n",
    "\n",
    "Compare the results achieved with this heuristic with the results achieved by randomly picking i and j. How do their runtimes compare?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def SMO_KKT(x,y,C,steps=10000):\n",
    "    n=x.shape[0]\n",
    "    beta=np.zeros(n)\n",
    "    b=0\n",
    "    \n",
    "    #for count in range(steps):\n",
    "    count=0\n",
    "    for i in range(n):\n",
    "        f=set_f(beta,y,x,b)\n",
    "        KKT=set_KKT(C,beta,y,f,x)\n",
    "        if KKT(i)>0:\n",
    "            possible_j=np.where(np.logical_and(np.logical_and(0<beta,beta<C),\n",
    "                                               range(n)!=i))[0]\n",
    "            if len(possible_j)>0:\n",
    "                j=possible_j[np.random.randint(len(possible_j))]\n",
    "            else:\n",
    "                j=np.random.randint(n)\n",
    "                while i==j:\n",
    "                    j=np.random.randint(n)\n",
    "        \n",
    "            beta, b=OneStep(i,j,beta,y,x,C,b)\n",
    "            count+=1\n",
    "            if count>=steps:\n",
    "                break\n",
    "    \n",
    "    #a\n",
    "    f=set_f(beta,y,x,b)\n",
    "    fval=np.apply_along_axis(f,1,x)\n",
    "    if np.count_nonzero(beta) > 0: \n",
    "        med=np.median((fval-y)[beta>0])\n",
    "        b=b-med\n",
    "    return beta, b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 0.01 \t #support vectors= 0  #margin vectors= 0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XtYlHX6+PH3DWqIWGrgEWW0Wk1R8UyWBu5KpWa5fhVHMs9kbq35+/Ytk1TWQr2qrXbXyqXW1ETEMm3LdlNTNF1TQfGEp1QwD5lamoKkwuf3x4wsR4GZAWS4X9f1XDifeeZ57kG4eeZzuB8xxqCUUqrq86jsAJRSSrmGJnSllHITmtCVUspNaEJXSik3oQldKaXchCZ0pZRyE5rQlVLKTWhCV0opN6EJXSml3ESNknYQkfnAAOBHY0xgnvZngWeA68AqY8wLJR7L19dgsTgerVJKVUfJyeeMMX4l7VZiQgcWAHOBRTcaRCQUeAzoYIz5VUQaliooiwWSkkq1q1JKKTuR9NLsVmKXizFmI/BTgeangTnGmF/t+/xY5gCVUkq5lKN96L8BeonIVhHZICLdXBmUUkqpsitNl0txr6sPBAPdgGUi0soUUbpRRCKBSABatHDwdEoppUriaEI/AXxqT+DbRCQH8AXOFtzRGBMLxAJI165aq1epClD/2jWiT5zg7qwsncpWReQA33l5Ee3vz881azp0DEcT+kqgD5AoIr8BagHnHDyWUsrFok+coHvdutSwWECkssNRpWEMDc6fJ/rECSa1bOnQIUr84y0i8cAWoLWInBCRscB8oJWI7AWWAiOL6m5RSlWOu7OyqHHnnZrMqxIRatx5J3dnZTl8iBKv0I0x1mKeesLhsyqlypUHaDKvikSc6iLT7jWllHITmtBdxBoXxzGLhWwPD45ZLFjj4io7JKUq1bkffmDqsGE8ftddDG3blkn9+pF+6BCn0tIIDwws+QAu8G5UFP2bN6e3j0+FnK+yaUJ3AWtcHO9HRmJJT8fDGCzp6bwfGalJXVUZ/4qDRy3Q3cP29V9O/ugaY3hh0CC6hISw8sgRlqWmMnHWLH46c8YV4ZZar0cfZeG2bRV6zsqkCd0FZkVFUSczM19bncxMZkVFVVJESpXev+JgViT8kA7G2L7OinQuqSetX0+NmjUZPGFCblvroCA69eqVb79TaWmM79WLJzp35onOndn1n/8AcO70aSJ792Z4UBDhgYHs/OYbsrOziR41ivDAQIa1b8+St94qMY72wcH4Nmni+BupYhydtqjyaHH8eJnalbqVvBsFWfmvR8jKtLU/EuHYMY/s3UubLl1K3K9Bw4bMXbOG27y8OH74MC9brSxKSuLfS5YQ/NBDjImKIjs7m6zMTA6lpHD25EkS9u4F4NKFC44F58Y0obvA8RYtsKQXrp1zXFfGqirgTDHXHcW1u9L1a9d47ZlnOJSSgoenJ8cPHQKgbbduvDJmDNevXePBxx+ndVAQzVq14uTRo7z+7LPc378/wWFh5R9gFaNdLi4wNSaGDG/vfG0Z3t5MjYmppIiUKr1GxVx3FNdeGq3ateNAcnKJ+y156y0aNGrEkl27WJSUxPWrVwHo3Ls3sRs34tesGTNGjGDVokXcXr8+S3btoktICB+/8w6vjhuX71jZ2dkMDwpieFAQ86ZPdzz4KkwTugvER0QwPjaWtIAAckRICwhgfGws8REOfl5VqgJNjAGv/NcjeHnb2h3VrU8frv76Kyvefz+3bd/27SRv2JBvv8sXL+LbpAkeHh58+dFHZGdnA3A6PZ36DRsyaPx4Bo4dy4EdO7hw7hw5OTn0GTyYCa+8woEdO/Idy9PTkyUpKSxJSWHCzJmOB1+FaZeLi8RHRGgCV1XSjX7yd6Ns3SyNWtiSuaP95wAiwusrVvDmc8+xcM4canl50dRi4f+9/Xa+/f5n4kReHDyYrz/+mC6hodSuUweA5MREPnr9dWrUrIm3jw/Rixbx48mTzBw9mpycHAD+MHt2iXH89YUX+GrJErIyM+nv789j48YRGR3t+Bu7xUlFrtiXrl2N3uBCqfL3r/378b333soOQzng3P79PFLw/04k2RjTtaTXapeLUkq5CU3oSinlJjShK6WUm9CErpRSbkITulJKuQlN6Eop5SZKc8ei+SLyo/3uRAWfe15EjIj4lk94SqmKcmQfhAfavrrC/JgYhrZrh7VDB4YHBbF361bXHNgBS95+m6wCBfQAYqOjmfvSS/naDqakMMSBKZ8HU1LY/OWXDsfoCqW5Ql8APFywUUSaA30BrUClVBV3JQOe6wfHUmFyf9tjZ+zesoVNX3zB4h07iN+9m3fXrqVR8+auCbaMsrOzWVpMQg+zWlmTkJCvbfXSpTw0fHiZz3PIgYR+/fr1Mp/nZkpM6MaYjcBPRTz1FvACoPcSVaqKmzkGfvrRVj73/Bl4Zaxzxzt3+jR3+PpS67bbAKjn64tf06YADLRYuHDOdk/51KQkngoJAWxXy9NHjODpPn34/T335JYNSE5MJLJ3b/5v0CCGtm3L7AkTcleLfhUfz7D27QkPDORvL76Ye/7ePj7Mmz6dUT16MD8mhrOnTjEhNJQJoaH54rS0bk3devXyfXpYu2wZYcOGAfDt6tWMue8+nujcmSlDhpB5+TJgK2MwpmdPhnfsyMju3bl88SJ/nz6dNQkJDA8KYnVCAhd/+onnH38ca4cOjA4O5vDu3bnvMyYykmfCwoh+8knnvtEFONSHLiIDgZPGmF0ujUYpVeH+OR82rYKr9nsTX82Cbz63tTsqOCyMM99/z+Df/IY5EycWquFSnO927+atVauYv2ULH8ycydlTpwDYt20bk/78Z+L37OHEkSOs//RTzp46xd9efJH31q0jLiWF1O3bSVy5EoArGRncFRjIgq1bGT99On5NmzJv/XrmrV9f6JwPWa2sXroUgD3ffssdd95Ji3vu4cK5c8x/9VXeWbuWxTt2cG/XrsS9+SbXrl5lang4//uXv7Bk1y7eWbsWrzp1eGrmTPqGh7MkJYWw8HBiZ8ygdadOxO/ezcRZs5iRJ3kfSE7mjc8+49UlSxz/JhehzAldRLyBKKBU5cxEJFJEkkQkibNny3o6pVQ5m/sSZBXoYsnKtLU7ytvHh4+Sk5kaG0t9Pz+mhofz+YIFJb6u92OP4VW7NvV8fekaGso++92G2nXvjn+rVnh6evKQ1UrKpk2kbt9Ol5AQ6vv5UaNGDR6OiGDnxo2ArVBXn8GDSxVr32HD+PqTT8jJybF1t1itgC25H01NZez99zM8KIhVCxfyQ3o66QcP4tukCe26dQPA5/bbqVGjcFmslE2beGTECMBWrOzi+fNcvnjR9j4HDsSrdu1SxVcWjhTnugtoCewS213F/YEdItLdGPNDwZ2NMbFALNhruSilbinPzIbX/5g/qXt5w7NznDuup6cnXUJC6BISwt3t2/PFwoU8OmoUnjVq5HaZXM3Kyvcae04p/LiI9pvVoarl5YWnp2ep4mzcvDlNLRZ2bNjAuuXLmb9lC2C7jV6Pvn2JiY/Pt//h3bsLxVmUIuOzv87LXoTM1cp8hW6M2WOMaWiMsRhjLMAJoHNRyVwpdesbOAYe6A+1vGyPa3lBr0fh0dGOHzPt4EGOHz6c+/hQSgpNAgIAaGKxsN9eK33d8uX5Xrfhs8/4NSuLC+fPk5yYSFv7VXDqtm2cPHaMnJwc1iQkEPTAAwT26MGODRu4cO4c2dnZfBUfT+cHHywyHu+6dcm4dKnYeMOsVt6cPBn/u+6ikb8/YLt93a7Nm/n+u+8AyMrMJP3QISxt2nD21Cn2bd8OQMalS1y/fp06deuSmeccnXv35t/2+wonJyZSz9cXn9tvL/030QGlmbYYD2wBWovICRFxcrhEKXWrmT4fGjS0XUDe2Qim/cO54125fJnokSMZ2rYt1g4dOJqamlu2dvyMGbw5aRLje/XCo8BVdLvu3Xmuf3/GBAczdtq03IHU9vfdx9wpUxgWGEjTli0JGTQI3yZN+MPs2UwIDWV4x4606dyZBx97rMh4BkVGMumRRwoNit7wuyFDOLpvX+5gKEB9Pz9mLFhAlNWaO7CZduAANWvVYlZCAm88+yzDO3bkmb59uZqVRZfQUI6mpuYOio6PjmZ/UhLWDh2YO2UK0QsXOvdNLQUtn6uUG3KkfO6RfTA1HGYlwF3tyimwm4iNjqa2jw8jnn8+X3tyYiKL33iDt774ouKDqgTOlM/VG1wopQBbEk8otHxQVSWa0JVSt4Ti7iR0Y2BVlUxruSillJvQhK6UUm5CE7pSSrkJTehKKeUmNKErpcpFNxGm25e+g62yYF8/PyYPGOCyc7w6bhxHU1NLte+ptDT6+/vnrlK9YXhQUG6JgbL4cNasMr+mvGlCV6qaW/QaJBWoWZW03tbujNp16nBk716yrlwBYOuaNfg1a1amY5RUXvblDz6gVdu2pTpWU4uFRs2bs/Obb3Lb0g4cIPPSJdp1716muKDsCd0YU+iPiatpQleqmmvbDV4a+t+knrTe9rhtN+eP3fORR9i8ahUAq+Pjcwtfga2C4piePYno1IkxPXuSdvAgAJ8vWMCUIUOY/OijPBsWRk5ODnMmTmRou3ZMHjCASf368fUnnwDwVEgIqfbFir19fHg3KorhHTsyOjiY82fOFIonzGpljb2yIpCvGNfPZ8/ywuDBPNmtG09268auzZsByLx8mT+NHs2w9u2xdujAuuXL+duUKfx65QrDg4J4OSICgLg33yQ8MJDwwECWvP02YPtUMOTee5kzcSJPdO7Mme+/d/6behOa0JWq5rqGwuxltiQ+b7rt6+xltnZnhQ0bxuqlS/k1K4vDu3cT2KNH7nMBbdoQu3EjcTt38tTMmbw7dWruc3u2bCF64ULeW7eO9Z9+yum0NJbu2UPUBx+wx148q6ArGRkEBgezZNcuOvXuzUp7PfW8fjd0KIkrV+Ze+a9JSKCvfbn/nydNYvjkySzavp3Xli/n1XHjAPjHK6/gc8cdLN2zh/jdu+napw/PzpnDbbVrsyQlhVfj4tifnMznH37Igq1b+fDbb1n5/vsc3LkTgPSDB+n/5JPE7dyZW8+mvOjCIqUUXUNh8NPwj1dg7DTXJHOAezp04HRaGl/Fx3N/v375nrt88SJ/GjmS44cPIyJcv3Yt97nufftyR4MGgK0M7W+HDMHDwwPfxo3pUkw9lpq1atHL3j/fpksXtq1ZU2gf38aNadWuHdu//poGjRpRo2ZN7g4MBGDb2rX5+uMzfvmFjEuX2LZ2LTF5rupvr1+/0HFTNm0iZNAgaturKIb+/vfs/OYbeg8cSJOAANoHB5fq++UsTehKKZLWw/L3bMl8+Xu2hO6qpN5r4ED++vzzzEtM5OL587nt86ZNo0toKK+vWMGptDQm5FkNWjtPednS1puqUbNmbllbT0/PYvvfb9zQokGjRoTl6QLKyclh/pYtheqUG2NKLJd7sxjLq1RuUbTLRalq7kaf+exlMGHmf7tfCg6UOmrgmDGMnT6du9u3z9eecfEiDe2DpF/c5OYXQQ88wPrly8nJyeH8mTPsSEx0Kp4+gwfzny+/ZG1CQr7qisFhYXw8d27u44MpKQD0CAtjWZ72X37+GbD9AbnxqaJz795sWLmSrMxMrmRkkLhiBZ169XIqTkdoQleqmkvdnr/P/Eafeup21xy/kb8/1kmTCrWPeOEF3nnpJcbefz/Z2dnFvr7P4ME09PdnWGAgs596inY9euBzxx0Ox1O3Xj0Cg4Np0KgRzVq2zG1//q9/JdVe7nZo27Z8Om8eAGNffplLP/9MeGAgwzt2JMl+G7tBkZFYO3Tg5YgI2nTuzIBRoxjZvTujevTgsXHjaN2pk8MxOkrL5yrlhhwpn3sry7x8GW8fHy6cP8+o7t35YPNmfBs3ruywyoWWz1VKubXJAwZw6cIFrl+9ythp09w2mTurxIQuIvOBAcCPxphAe9vrwKPAVeAIMNoYc6E8A1VKVV9/d7LfvLooTR/6AuDhAm1rgEBjTAfgEODE/cGVUkq5QokJ3RizEfipQNtqY8yNOUHfAv7lEJtSSqkycMUslzHAv1xwHKWUUk5wKqGLSBRwHYi7yT6RIpIkIkmcPevM6ZRSSt2EwwldREZiGyyNMDeZ+2iMiTXGdDXGdMXPz9HTKaWqmHM//MDUYcN4/K67GNq2LZP69SP90CFOpaURbl9u72o7Nm7kic6dCa5RI7eAV3XiUEIXkYeBF4GBxphM14aklKpoDeLiaG+x0MXDg/YWCw3iiv3QXSrGGF4YNIguISGsPHKEZampTJw1i5+KqIDoSo1btGDGggU8NHx4uZ7nVlWaaYvxQAjgKyIngBnYZrXcBqyx1zj41hgzoRzjVEqVkwZxcQRERuKZabs2uy09nYDISAB+speGLauk9eupUbMmgyf8Ny20DgoCbCVlbziVlsaMESO4kpEBwP/NnUvHnj05d/o0U8PDufzLL2Rfv86U996jQ8+evDJ2LPuTkhARBo4Zw/DJk/Odt6nFAoB4VM9F8CUmdGOMtYjmf5RDLEqpStAsKio3md/gmZlJs6gohxP6kb17adOlS4n7NWjYkLlr1nCblxfHDx/mZauVRUlJ/HvJEoIfeogxUVFkZ2eTlZnJoZQUzp48ScLevQBcuqBLXwrSlaJKVXO1jh8vU7srXb92jdeeeYZDKSl4eHpy/NAhANp268YrY8Zw/do1Hnz8cVoHBdGsVStOHj3K688+y/39+xMcFlbu8VU11fNziVIq19UWLcrUXhqt2rXjQHJyifsteestGjRqxJJdu1iUlMT1q1cBW/XC2I0b8WvWjBkjRrBq0SJur1+fJbt20SUkhI/feSf3BhTqvzShK1XNnYyJIdvbO19btrc3J2NiHD5mtz59uPrrr6zIc9egfdu3k7xhQ779Ll+8iG+TJnh4ePDlRx/lVl08nZ5O/YYNGTR+PAPHjuXAjh1cOHeOnJwc+gwezIRXXuHAjh0Ox+eutMtFqWruRj95s6goah0/ztUWLTgZE+Nw/zmAiPD6ihW8+dxzLJwzh1peXjS1WPh/9ntt3vA/Eyfy4uDBfP3xx3QJDc29sUVyYiIfvf46NWrWxNvHh+hFi/jx5Elmjh6de6PlP8yeXei8+7Zv54VBg/jl55/Z9Pnn/H3GDJbt2+fw+6hqtHyuUm7I3crnVifOlM/VLhellHITmtCVUspNaEJXSik3oQldKaXchCZ0pZRyE5rQlVLKTWhCV0oB4LVvH20DA/Fy0bzt+TExDG3XDmuHDgwPCmLv1q0uOa4jlrz9NlmZhQvDxkZHM/el/HfQPJiSwhAHpnweTElh85dfOhyjK2hCV0rhkZHBPf36UTs1lXv698fDXv3QUbu3bGHTF1+weMcO4nfv5t21a2nUvLmLoi2b7OxslhaT0MOsVtYkJORrW710qUPldw85kNCvX79e8k5loAldKYVlzBhq/vgjYgw1z5whYOxYp4537vRp7vD1pdZttwFQz9cXv6ZNARhosXDh3DkAUpOSeCokBLBdLU8fMYKn+/Th9/fck1s2IDkxkcjevfm/QYMY2rYtsydMyF0t+lV8PMPatyc8MJC/vfhi7vl7+/gwb/p0RvXowfyYGM6eOsWE0FAmhIbmf9+tW1O3Xr18nx7WLltG2LBhAHy7ejVj7ruPJzp3ZsqQIWRevgzYVqSO6dmT4R07MrJ7dy5fvMjfp09nTUICw4OCWJ2QwMWffuL5xx/H2qEDo4ODObx7d+77jImM5JmwMKKffNKp73NBmtCVqubunD+fO1atwiMrCwCPrCzqff45d86f7/Axg8PCOPP99wz+zW+YM3FioRouxflu927eWrWK+Vu28MHMmZw9dQqAfdu2MenPfyZ+zx5OHDnC+k8/5eypU/ztxRd5b9064lJSSN2+ncSVKwG4kpHBXYGBLNi6lfHTp+PXtCnz1q9n3vr1hc75kNXK6qVLAdjz7bfcceedtLjnHi6cO8f8V1/lnbVrWbxjB/d27Urcm29y7epVpoaH879/+QtLdu3inbVr8apTh6dmzqRveDhLUlIICw8ndsYMWnfqRPzu3UycNYsZeZL3geRk3vjsM15dssTh73FRNKErVc35v/QSngW6WDwzM/Ev0LdcFt4+PnyUnMzU2Fjq+/kxNTyczxcsKPF1vR97DK/atann60vX0FD2bdsGQLvu3fFv1QpPT08eslpJ2bSJ1O3b6RISQn0/P2rUqMHDERHs3LjRFr+nJ30GDy5VrH2HDePrTz4hJyfH1t1itd0CYs+333I0NZWx99/P8KAgVi1cyA/p6aQfPIhvkya069YNAJ/bb6dGjcJlsVI2beKRESMAW7Gyi+fPc/niRdv7HDgQr9q1SxVfWWhxLqWquROzZ9Pij3/Ml9Szvb05MWeOU8f19PSkS0gIXUJCuLt9e75YuJBHR43Cs0aN3C6Tq/ZPBTfY74BW+HER7TerQ1XLywtPT89Sxdm4eXOaWizs2LCBdcuXM3/LFsB2G70effsSEx+fb//Du3cXirMoRcZnf52XvQiZq5V4hS4i80XkRxHZm6etgYisEZHD9q/1yyU6pVS5Oz9mDBf79yfHywuAHC8vLjz6KOdHj3b4mGkHD3L88OHcx4dSUmgSEABAE4uF/fZa6euWL8/3ug2ffcavWVlcOH+e5MRE2tqvglO3bePksWPk5OSwJiGBoAceILBHD3Zs2MCFc+fIzs7mq/h4Oj/4YJHxeNetS8alS8XGG2a18ubkyfjfdReN/P0BaB8czK7Nm/n+u+8AyMrMJP3QISxt2nD21Cn2bd8OQMalS1y/fp06deuSmeccnXv35t/2e7MmJyZSz9cXn9tvL/030QGl6XJZADxcoG0K8LUx5h7ga/tjpVQVlTZ/PtcaNsSIcK1RI9L/4dxdJq9cvkz0yJEMbdsWa4cOHE1NJTI6GoDxM2bw5qRJjO/VC48CV9Htunfnuf79GRMczNhp03IHUtvfdx9zp0xhWGAgTVu2JGTQIHybNOEPs2czITSU4R070qZzZx587LEi4xkUGcmkRx4pNCh6w++GDOHovn25g6EA9f38mLFgAVFWa+7AZtqBA9SsVYtZCQm88eyzDO/YkWf69uVqVhZdQkM5mpqaOyg6Pjqa/UlJWDt0YO6UKUQvXOjU97Q0SlU+V0QswBfGmED744NAiDHmtIg0ARKNMa1LPI6Wz1WqQjhSPtdr3z5ahYdzNCGBrHbtyimy4sVGR1Pbx4cRzz+frz05MZHFb7zBW198UeExVQZnyuc62ofeyBhzGsCe1BsWt6OIRAK2W4g7cUsrpVT5ymrXjtS9e0veUd2yyn1Q1BgTC8SC/QpdKaWKcKNLpqAbA6uqZI5OWzxj72rB/vVH14WkimONi+OYxUK2hwfHLBas9gEXpZQCxxP6P4GR9n+PBD5zTTiqONa4ON6PjMSSno6HMVjS03k/MlKTulIqV2mmLcYDW4DWInJCRMYCc4C+InIY6Gt/rMrRrKgo6hSoRVEnM5NZUVGVFJFS6lZTYh+6McZazFO/dXEs6iZaHD9epnalVPWjS/+riOPFzBAqrl2pytZNhOn2pe9gqyzY18+PyQMGuOwcr44bx9HU1FLteyotjf7+/rmrVG8YHhSUW2KgLD6cNavMrylvmtCriKkxMWR4e+dry/D2ZmpMTCVFpNxF49deo26BolV116+n8WuvOXXc2nXqcGTvXrKuXAFg65o1+DVrVqZjlFRe9uUPPqBV27alOlZTi4VGzZuz85tvctvSDhwg89Il2nXvXqa4oOwJ3RhT6I+Jq2lCryLiIyIYHxtLWkAAOSKkBQQwPjaW+IiIyg5NVXEZ3brRaujQ3KRed/16Wg0dSoZ92b0zej7yCJtXrQJgdXx8buErsFVQHNOzJxGdOjGmZ0/SDh4E4PMFC5gyZAiTH32UZ8PCyMnJYc7EiQxt147JAwYwqV8/vv7kEwCeCgkh1b5YsbePD+9GRTG8Y0dGBwdz/syZQvGEWa2ssVdWBPIV4/r57FleGDyYJ7t148lu3di1eTMAmZcv86fRoxnWvj3WDh1Yt3w5f5syhV+vXGF4UBAv238H4958k/DAQMIDA1ny9tuA7VPBkHvvZc7EiTzRuTNnvv/e6e/pzWhCr0LiIyJomZaGZ04OLdPSNJkrl7gUGsrRZctoNXQoTadPp9XQoRxdtoxLxSyTL4uwYcNYvXQpv2ZlcXj3bgJ79Mh9LqBNG2I3biRu506emjmTd6dOzX1uz5YtRC9cyHvr1rH+0085nZbG0j17iPrgA/bYi2cVdCUjg8DgYJbs2kWn3r1Zaa+nntfvhg4lceXK3Cv/NQkJ9LUv9//zpEkMnzyZRdu389ry5bw6bhwA/3jlFXzuuIOle/YQv3s3Xfv04dk5c7itdm2WpKTwalwc+5OT+fzDD1mwdSsffvstK99/n4M7dwKQfvAg/Z98kridO3Pr2ZQXrbaolOJSaChnn36apq+8wqlp01ySzAHu6dCB02lpfBUfz/39+uV77vLFi/xp5EiOHz6MiHD92rXc57r37csdDRoAtjK0vx0yBA8PD3wbN6ZLMbHVrFWLXvb++TZdurBtzZpC+/g2bkyrdu3Y/vXXNGjUiBo1a3J3YCAA29auzdcfn/HLL2RcusS2tWuJyXNVf3v9wrUIUzZtImTQIGrbqyiG/v737PzmG3oPHEiTgADaBweX6vvlLE3oSinqrl+P33vvcWraNPzee49LoaEuS+q9Bg7kr88/z7zERC6eP5/bPm/aNLqEhvL6ihWcSktjQp7VoLXzlJctTb0pgBo1a+aWtfX09Cy2//3GDS0aNGpEWJ4uoJycHOZv2VKoTrkxpsRyuTeLsbxK5RZFu1yUquZu9JkfXbaMUzNn5na/FBwoddTAMWMYO306d7dvn6894+JFGtoHSb+4yc0vgh54gPXLl5OTk8P5M2fYkZjoVDx9Bg/mP19+ydqEhHzVFYPDwvh47tzcxwdTUgDoERbGsjztv/z8M2D7A3LjU0Xn3r3ZsHIlWZmZXMnIIHHFCjr16uVUnI7QhK5UNVdn+/Z8feY3+tTr2Ot9O6ssQaU9AAAOe0lEQVSRvz/WSZMKtY944QXeeeklxt5/P9nZ2cW+vs/gwTT092dYYCCzn3qKdj164HPHHQ7HU7dePQKDg2nQqBHNWrbMbX/+r38l1V7udmjbtnw6bx4AY19+mUs//0x4YCDDO3Ykyf6HblBkJNYOHXg5IoI2nTszYNQoRnbvzqgePXhs3Dhad+rkcIyOKlX5XJedTMvnKlUhHCmfeyvLvHwZbx8fLpw/z6ju3flg82Z8Gzeu7LDKRWWUz1VKqQozecAALl24wPWrVxk7bZrbJnNnaUJXSt3y/u5kv3l1oX3oSrmhHIAK7E5VLmIMzqwl1YSulBv6zsuL6+fPa1KvSozh+vnzfGe/WbcjtMtFKTcU7e9P9IkT3H32rF61VRE52P4QR/v7O3wMTehKuaGfa9ZkUp4peap60D/eSinlJpxK6CIyWUT2icheEYkXEcc7f5RSSjnF4YQuIs2APwJdjTGBgCcw7OavUkopVV6c7XKpAdQWkRqAN3DK+ZCUUko5wuGEbow5CbwBHAdOAxeNMasL7icikSKSJCJJnD3reKRKKaVuypkul/rAY0BLoClQR0SeKLifMSbWGNPVGNMVPz/HI1VKKXVTznS5/A44Zow5a4y5BnwK9HRNWEoppcrKmYR+HAgWEW+xVX//LbDfNWEppZQqK2f60LcCnwA7gD32Y8W6KC6llFJl5NRKUWPMDGCGi2JRSinlBF0pqpRSbkITulJKuQlN6Eop5SY0oSullJvQhK6UUm5CE7pSSrkJTehKKeUmNKErpZSb0ISulFJuQhO6Ukq5CU3oSinlJjShK6WUm9CErpRSbkITulJKuQlN6Eop5SY0oSullJtwKqGLSD0R+UREDojIfhG5z1WBKaWUKhun7lgE/AX4tzHmf0SkFuDtgpiUUko5wOGELiK3A72BUQDGmKvAVdeEpZRSqqyc6XJpBZwFPhSRnSLygYjUcVFcSimlysiZhF4D6Ay8Z4zpBGQAUwruJCKRIpIkIkmcPevE6ZRSSt2MMwn9BHDCGLPV/vgTbAk+H2NMrDGmqzGmK35+TpxOKaXUzTic0I0xPwDfi0hre9NvgVSXRKWUUqrMnJ3l8iwQZ5/hchQY7XxISimlHOFUQjfGpABdXRSLUkopJ+hKUaWUchOa0JVSyk1oQldKKTehCV0ppdyEJnSllFuwxsVxzGIh28ODYxYL1ri4yg6pwjk7bVEppSqdNS6O9yMjqZOZCYAlPZ33IyMBiI+IqMzQKpQYYyruZF27GpKSKux8Sqnq4ZjFgiU9vVB7WkAALdPSKj4gVxNJNsaUOEVcu1yUUlVei+PHy9TurjShK6WqvOMtWpSp3V1pQldKVXlTY2LI8M5/f50Mb2+mxsRUUkSVQxO6UqrKi4+IYHxsLGkBAeSIkBYQwPjY2Go1IAo6KKqUUrc+HRRVSqnqRRO6qjZ04Ylyd7qwSFULuvBEVQfah66qBbdfeKLcW0X1oYuIp4jsFJEvnD2WUuVFF56o6sAVfeiTgP0uOI5S5UYXnqjqwKmELiL+QH/gA9eEo1T50IUnqjpw9gr9beAFIMcFsShVbnThiaoOHB4UFZEBQD9jzEQRCQGeN8YMKGK/SMA2naBFiy4UMTCllFLqJipgUPR+YKCIpAFLgT4isrjgTsaYWGNMV2NMV/z8nDidUkqpm3E4oRtjXjLG+BtjLMAwYJ0x5gmXRaaUUqpMdKWoUkq5CZesFDXGJAKJrjiWUkopx+gVulJKuQlN6Eop5SY0oSutQqiUm9Bqi9WcViFUyn1otcVqTqsQKlUF6B2LVGloFUKl3Icm9GpOqxAq5T40oVdzWoVQKfehCb2a0yqESrkPTeiV5FaaKhgfEUHLtDQ8c3JomZamyVypKkqnLVYCnSqolCoPOm2xEuhUQaVUmei0xVuXThVUSpUHTeiVQKcKKqXKgyb0SqBTBZVS5UETeiXQqYJKqfLgcEIXkeYisl5E9ovIPhGZ5MrA3F1xUwVvpemMSqmqxZkr9OvA/xpj7gWCgT+ISFvXhFU93ZjOaElPx8OY3OmMmtRVedCLBzdkjHHJBnwG9L3pPl26uOZkVWyzLl5sjgUEmGwRcywgwFgXLy5yv2MBAcZAoe1YQEClvwfd3GuzLl5sLnt7m7w/Z5e9vYv92dStkjdIKs1uLpmHLiIWYCMQaIz5pdj9quE89IKLiMA2AFpUn3m2hwceRfx/5IjgmZNT7rGq6kPXQlQxpZyH7nRCFxEfYAMQY4z5tIjnIwHbMsgWLbpQxA+ROyvLL47+kqmKohcPVUxFLCwSkZrAciCuqGQOYIyJNcZ0NcZ0xc/PmdNVSWVZRFQR0xm131SBroVwW4536SDAIuDtUr+mGvahl7VfvLT97Y5s2m+qm/4sVNGtlH3oThyfBwAD7AZS7Fu/m76mGib0W+kXRwdddcu7lefFg24u3so7oTsUUzVM6Jhb5xcnW8QUldCzRSr9e+Tu263yM6BbFd00oetWcCuPK3RNVKX7Ht0qn9J0q6KbJnTdCm6uTiyaqEq3aVeXbk5vmtB1K2pz5RW1JqrSbdrVpZvTW0UuLCqt6riwyJ3pXObS0fUFyml6g4uKU13ndutc5tLRcsmqwlTopwY37HIpTT9ykd0ci40hwPYZiQBje3wLvB9Xv3fdbvIzcAvE5W6b236ftQ+9Yrbi+pFz7H3Jf3v66cJJr5a3sdZcnP8l3qbKJnW3/AXSrcptbn2BoX3oFaO4fuQbckSKfD6NAFqSlr8xAAo2KaVKx63HKrQPvZzFARY4bm7eX1xcsm9BETVe9B7RSjlMb76uCd0xcdjqR6bDVGLIwLukVxRynCL+EOhYolIO00F6TeiOiQLs5c3jiWA8saQRQHEdLzki+R5n1PJmas0CMxy8AZ30oJTDdDaRJnTH5PkEZyWOWUTRguNcok6hpG6ANX362G4IjZDmGcD4q7HE3x4Bd2KrWRkAxAJ6j2ilHKY3X0dnuTi0BRgDxlhZbC6Tf1S92JWTi40ptGsVndlyK24620Y3t9502mI5bouNoaYxx25k9hK2bBFT7K4BxRw/wFTpOeoVubn1dDXddDNGpy2WO1/IPu+BR7E95/+VFhBAy/S0op8UIO8q+RsDrpl52rzRLpmbcOvpakqBTlssdz8VM1OlAANM7RdjS9xFKXiIPAOuuTKBkdiSvSpEp6spZaMJ3VEtSjdl8ZzHncQviqDYC/mCA/DF5aBsbFfumtQL0elqStk4e5Poh0XkoIh8JyJTXBVUlRCTf8piDpBT4DI8A28m5fwFMm5ynM15/h3Hzf9HMrFdwat8dLqaUnaO99HjCRwBWgG1gF1A25u+xl0GRW9sd5p8A5xWFptjBJhsxBwjwFhZXJoxU9ugZ1GzYIra5BZ437fgprNcdHPrrbwHRUXkPiDaGPOQ/fFL9j8Qs4t9jTsNioLtivoJFxwnwP618Lhe0fumueCcSqmqowIGRZsB3+d5fMLeViAOiRSRJBFJ4uxZJ053C4rANaMQxyldHRddTaqUugln0lFR8zYKXe4bY2KNMV2NMV3x83PidLeop1xwjBYUX8fFE11NqpQqlRpOvPYE0DzPY3/glHPhVEHv2r/+nfzzyfMSbFfXRQ2O1uK/V906/1wp5QRnrtC3A/eISEsRqQUMA/7pmrCqmHexTSs0wGJsNVpuuBP4CLhczHPzsSXsCGzJOwC9IldKOcSplaIi0g94G1vHwHxjzE17eN1uUFQppSpCKQdFnelywRjzJfClM8dQSinlGrpSVCml3IQmdKWUchOa0JVSyk1oQldKKTehCV0ppdyEJnSllHITmtCVUspNVOwt6EQuAQcr7IQVwxc4V9lBuJi+p6pB31PV4Ir3FGCMKbEYllMLixxwsDSrnaoSEUnS93Tr0/dUNeh7co52uSillJvQhK6UUm6iohN6bAWfryLoe6oa9D1VDfqenFChg6JKKaXKj3a5KKWUm6jQhC4iQ0Rkn4jkiEiVHskWkYdF5KCIfCciUyo7HlcQkfki8qOI7K3sWFxFRJqLyHoR2W//2ZtU2TE5S0S8RGSbiOyyv6c/VXZMriIiniKyU0S+qOxYXEFE0kRkj4ikiEi53wyioq/Q9wK/BzZW8HldSkQ8gXeAR4C2gFVE2lZuVC6xAHi4soNwsevA/xpj7gWCgT+4wf/Vr0AfY0xHIAh4WESCKzkmV5kE7K/sIFws1BgTVBFTFys0oRtj9htj3GFhUXfgO2PMUWPMVWAp8Fglx+Q0Y8xG4KfKjsOVjDGnjTE77P++hC1ZNKvcqJxjbC7bH9a0b1V+MExE/IH+wAeVHUtVpX3ojmkGfJ/n8QmqeJKoDkTEAnQCtlZuJM6zd02kAD8Ca4wxVf49Ybud5QsUf7v1qsgAq0UkWUQiy/tkLl8pKiJrgcZFPBVljPnM1eerJFJEW5W/QnJnIuIDLAeeM8b8UtnxOMsYkw0EiUg9YIWIBBpjquzYh4gMAH40xiSLSEhlx+NC9xtjTolIQ2CNiBywfxIuFy5P6MaY37n6mLegE0DzPI/9gVOVFIsqgYjUxJbM44wxn1Z2PK5kjLkgIonYxj6qbEIH7gcG2m887wXcLiKLjTFPVHJcTjHGnLJ//VFEVmDrri23hK5dLo7ZDtwjIi1FpBYwDPhnJcekiiAiAvwD2G+MebOy43EFEfGzX5kjIrWB3wEHKjcq5xhjXjLG+BtjLNh+n9ZV9WQuInVEpO6NfwNhlPMf3YqetjhIRE4A9wGrROSrijy/qxhjrgPPAF9hG2RbZozZV7lROU9E4oEtQGsROSEiYys7Jhe4HxgB9LFPHUuxXwVWZU2A9SKyG9vFxRpjjFtM83MzjYBNIrIL2AasMsb8uzxPqCtFlVLKTWiXi1JKuQlN6Eop5SY0oSullJvQhK6UUm5CE7pSSrkJTehKKeUmNKErpZSb0ISulFJu4v8D4Srk4e98f0EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 1 \t #support vectors= 8  #margin vectors= 5\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XlclWX+//HXR0RxSTMF3MWZyVJJUVklEDTN3UxxyX3JGquxfmMuWYqWtk7TNNpiZtrkrpVrmqa4ogiKZO59c8tdCxVEFK7fHwdQBAXOOYAcPs/Ho8fxXOc+9/05h5k3N9d9X9clxhiUUkoVfSUKuwCllFL2oYGulFIOQgNdKaUchAa6Uko5CA10pZRyEBroSinlIDTQlVLKQWigK6WUg9BAV0opB1Eypw1EZCbQEThnjPG8rf0l4EXgJrDSGDMqp31Vqfig8aha3YZylVKq+Ik5tP+CMcY1p+1yDHRgFjAV+Dq9QURCgS5AI2PMdRFxy01RHlWrEz3965w3VEoplUFCfI7lZrscu1yMMZuAS3c0/x14xxhzPW2bc3muUCmllF1Z24deDwgSkR0islFEfOxZlFJKqbzLTZfL3d5XCfAHfICFIvIXk83UjSIyDBgGUNu9qrV1KqWUyoG1gX4S+DYtwKNEJBWoApy/c0NjzHRgOoD3Iw10rl6lCsANgZNlnUhyksIuReWBS4qhZmIKzlYmpbWB/j3QEogQkXpAKeCClftSStnZybJOPODuhkfFiohoqBcFxhguxsdz8uw56iakWLWPHPvQRWQeEAk8IiInRWQIMBP4i4jsBeYDA7LrblFKFY4kJ6GyhnmRImL5mdnyV1WOZ+jGmN53eamv1UdVSuU7DfOix9afmY4UVUopB6GBbi9rf4CenSDU1/K49ofCrkipQnXm7Fl6DRzAXxs9RgOfZrTv9jSHDh/m6LFjePoVzJ3O4yaFU6v+I5Sv5l4gxytsGuj2sPYH+GAKnD0DxlgeP5iioa6KjDkLnfBoWIYSFcvi0bAMcxY62bQ/Ywxdn+lNSFAQv8b9zL6dMUyZEM7Z8wU7BrFT2/ZEbdhYoMcsTBro9jDjE7ielLntepKlXan73JyFTgx7qTTHTpTAGOHYiRIMe6m0TaG+YdNGnJ1L8vyQoRltXo0aEdQ8MNN2R48dI+jJ1jQNCqRpUCDbdmwH4PSZMwS3bYNXYACefj5s3raVlJQUBj7/HJ5+Pjzm78u/p07NsQ5/X1+qVS0+41+svW1R3e7c2by1K3UfGTexFInXMl+MS7wmjJtYij49rlm1z7379tHMq0mO27m5urJ26XJcXFw4fOQIvYcMInrjZuYuWsiTrZ5g3KujSElJITExkdi4OH4/fYq9O3YC8Oeff1pVmyPTQLcHN3dLN0t27Urd546fzP7Oiru129ONGzd4ceQ/if05DicnJw4dOQKAT9OmDB4+nBs3bvBUx054NWrEXzw8+L/fjvLSyH/S4cm2tGnVKt/rK2q0y8Uehg6H0i6Z20q7WNqVus/Vrpn9EJK7tedGw/r1iYndneN2/542FXc3N/Zs2070xs0kJycDEBz4OJtWr6FG9er0GzaUr+fOpVKlSuzZFklIUBDTvpjO0BdfyLSvlJQUvAID8AoMYPxbb1pde1GmgW4PrdvByNfAvSqIWB5HvmZpV+o+N3lCMmXLZA7vsmUMkyckW73Pli1CuH49mS9mfZXRtjMmho1bNmfaLv7yZapVrUqJEiX43/x5pKRYRkgeO34cN1dXnh04iCH9BrBrTywXLl4gNTWVbl2e4s3X32DXnthM+3JyciJ2aySxWyOZ9PobVtdelGmXi720bqcBroqkPj1SgOuMm1iK4yeF2jUtYW5pt46I8N3cebw8ZhTvfPghLi6l8ahdh4/eeTfTdsOHPku3fn1Y9N13hAYHU65cOQAiNm/m/Y8/wtnZmfLlyvH151/w+6nTDBr+PKmpqQC8PWFijnWMeuN15i5aSGJiIjUfrcfQ/gMIf22c1Z/rficFOWLf+5EGRhe4UCr/7a9Qkvp/e7iwy1BW2H/kMPUv38zUJiE+McYY75zeq10uSinlIDTQlVLKQWigK6WUg9BAV0opB6GBrpRSDkIDXSmlHERuViyaKSLn0lYnuvO1kSJiRKRK/pSnlCoov+wXPP3K8Mt++wz5n/z+ezT09aZRgB9egQHs2LnTLvu1xkfTppGYmJilPXzKZMaGT8jUFhsXR33vpnk+RmxcHKvWrLG6RnvIzRn6LKDtnY0iUgtoDRy3c01KqQKWkADtu7uw74DQobsLCQm27S9yxw5WrF7Nrs1biYvcwbply6lVs6Z9is2jlJQUPvp0GonXsgZ677AwFny7JFPb/CWLeSasR56PE/tzHKt+zFug37x5M+eN8iDHQDfGbAIuZfPSv4FRgK4lqlQRN3h4ac6dF4wRzp4XhrxQ2qb9nT57hiqVK1O6tGU/VSpXoXq1agB4eDbgwkXLmvLRu3YR0t5yvhg+ZTL9nh1Ky47tedircca0ARGbNxHctg1dn+lFA59mPP/yPzJGi85btJDH/H3x9PNh9Phbw/3LV3Nn/Ftv4hcawuT33+PU6dOEdmhPaIfMo7kfebgeD1asmOmvh4XffUuvbt0B+PGnnwho1ZKmQYGE9e/L1atXAcs0Bs2faEXj5v74hrQgPj6e8ZPfYsG3S/AKDGDBksVcunSJp3r3olGAH/4tQ4nbuzfjcw77x4u06dKZ/sOetel7vpNVfegi0hn43Rizx67VKKUK3Mz/lWTlGieSkixdLUlJwvLVTsz8n/Uzg7Rp2YoTv5+kXhMvhr/ycpY5XO4m7pe9rFy0hMiffmLSu+9w6vRpAKJiYvjX5Lf5eXsUv/72G98uW8qp06cZPWE861esJHZrJDt3xfD9iuUAJCQk4NmgATs2RDB+zFiqV6vGhpWr2LAy66IzvbuHMX/JYgC2R0VRudJDPPy3v3Hh4gXeev9d1i1bzq7NW/Fu0pQPp/6X5ORkeg4awH/efY8927azbtlyypUrx6Rxr9Pz6W7Ebo2kZ7fuTJgymSaNGhEXuYMpEybQ/7lb4R0TG8vS+QuYO/OrLPXYIs+BLiJlgXHA+FxuP0xEokUk+nz8H3k9nFIqn40NdyYh8Y750BOFseHOVu+zfPnyxGzawvSPP8a1ShV6DhzArDnf5Pi+Lu07UKZMGapUrkJoUDBRMdEA+DZrxl/q1sXJyYne3cPYEmkJ8JDHg3Ct4krJkiXp06Mnm7ZuBSwTdXXr8lSuau3VrTuLl35Pamoq85cspndYGADbo3ay78ABAts8gVdgALPnzuHYiRMcPHyIau5V8WnWDIAKFSpQsmTWX35btkfSr1dvwDJZ2cVLl4iPjwegc7v2lClTJlf15YU1v4L/CtQF9qStUF0T2CUivsaYLJOCG2OmA9PBMpeLDbUqpfLB2+E3+MerpTKFetmyhncm3rBpv05OToQEBRMSFMxjDRsye+5cBvbpS0mnkhldJkl3rPR156r36c+za7/XPFQuLi44OeVuxaVaNWviUbs2G7dsZsmypUSuWw+AwdA6tCXzvpqVafu4vXuz1JOd7OpLf1/6JGT2luczdGPMz8YYN2OMhzHGAzgJNM0uzJVS97/B/W7S4ckUXFwsAeTiYujUNoVBfa2/YHfw8CEOpy1WAZY7QOrUqgWAR53axOy2TH27ZOnSTO9bumolSUlJXLx4kYgtm/FpajkLjoqJ4bejR0lNTWXBkiU8HhCAn7cPG7du4cLFC6SkpDBv8SJaPP54tvU8UL48V65cvWu9vbuH8crYMfy1bl1q1qgBgL+PD1t3bOfIr78CkJiYyKHDh3m0Xj1OnTnNzpgYAK5cucLNmzctx7h66xjBzQOZs3ABYLkOUKVyZSpUqJD7L9EKubltcR4QCTwiIidFZEi+VqSUKnAzP7mOm6tBxODuavhy2nWb9nf1agIDnn+OBj7NaBTgx76DBwh/7TUAJowZy4jRowh6snWWs2jfZs3oENYN/1YteWPU6IwLqQG+vowJH4+nnw91PerQtVNnqlWtytsTJhLaoT2Nm/vTtLEXXTp0zLaeYQMH0a5b1ywXRdOFde3KL/v3Z1wMBXCt4sqsTz+j9+BBlgubrUI5cPgQpUqVYsFXs3np1ZE0bu5P6y6dSUpKIjQomH0HDmRcFA0f+xrRu3fTKMCPMRMmMPuz6TZ9p7mh0+cq5YCsmT73l/1Cz4EuLJiVRMP6Bd87Gj5lMuXLl2fkP0Zkao/YvIkPPv6YFYsWF3hNhcGW6XN1gQulFAAN6xv27rBuUWh1f9BAV0rdF+62klD6hVWVM53LRSmlHIQGulJKOQgNdKWUchAa6Eop5SA00JVS+UIqlKffs0Mznt+8eRPXunXoGNb9Hu/Km6EvvsC+A/tzte3RY8eo+Wi9jFGq6bwCA4iKjs7zsad88H6e35PfNNCVKube+8iZDZsyR8GGTSV47yPr53IBy/D2vfv3ce2a5VbItevXU6N69TztI6fpZWdMnUaDR+vnal8edepQq0ZNNm/bmtF24NBBrly5iq93jrd4ZzHlXx/kaXtjTJZfJvamga5UMefTNIUeA1wyQn3DphL0GOCCT9MUm/fdrnUbVq5ZDcC8xYvo3T0s47Wo6GiaP9GKJo83p/kTrTh4+BAAs+Z8Q1j/vnTqEUabpzqTmprK8FdepqGvNx3DutO+29Ms/v47AELatyV61y7AMmXuuEnhNG7uj3/LUM6eO5ulnt7du2fMrAgwf/Fiene3/MVw/sJ5uvV9Bp8Wwfi0CGbr9kgArl69yqC/P89j/r40CvBjydLvGTNhPNeuXcMrMIA+QwYD8OHU/+Lp54Onnw8fTZsGWP4qqO/dlOGvvEzToEBOnDxp83d6LxroShVzocGpLJydRI8BLox/y5keA1xYODuJ0GDbzyZ7dbMEaFJSEnG/7MXvtjPhR+vVY9PqNezeso1J417ntYnhGa9FRkUx+7PPWb9iFd8uW8rR48f5eXsUM6ZOI3JnVLbHSkhIwN/Hlz3bthMcGMgXs2Zl2abH0934fsWKjDP/Bd8uoVdaoI8YNYpXXniRnRs3seSbOQx98QUA3nzvXSpWqMDP26OIi9xBy+AWvDNxEmXKlCF2ayRzvpxJzO7dfPXN/9ixPoLtP23gi9lfsXuPZXbxg4cP07/3M+zeso06tWvb/J3eiw4sUkoRGpzK34fc4M33SvHGqGS7hDlAI09Pjh4/zrzFi2jf5slMr8VfvsyA54dx+NdfERFu3Lg1u2Pr0JY89NBDAGyJjCSsa1dKlChBVXd3QoOCsj1WqVKl6NjWMldLMy8v1m7YkGWbqu7uNKxfn58iInB3c8PZ2RnPBg0BWBexgX0HD2Rse/nKFa5cucK6iA3Mnzkro71SpUpZ9rslchtdO3bKmEXx6U6d2bxtK53bd6BO7dr4+/rm5uuymQa6UooNm0rw6ZfOvDEqmU+/dCY0OMVuod65XXtGjhtHxKpVXLx0a/GzN956k9CgYL6bO5+jx44RctvEWeXKls34d27nm3J2ds6YntbJyemu/e/pC1q4u7ll6gJKTU0lct36LPOUG2NynC73XiXe/lnym3a5KFXMpfeZL5ydxKTXb2R0v9x5odRag/v1Z/zo0TzW0DNTe/zl+IyLpPda/OLxgACWLF1KamoqZ8+dJWLzFpvq6da5C6t+XGPpbrltdsU2LVsxdfrnGc9j4+Kybf/jD8tCPc7Ozhl/VQQHBvL9yhUkJiaSkJDAdyuWE9Q80KY6raGBrlQxt3OXU6Y+8/Q+9Z27crdARE5q1qjBiOEvZGkfNeIVxoaHE9j6CVJS7n4BtluXp6hZozqefj48N2IEft7eVKxQ0ep6HnzwQfx9fHF3daOuh0dG+8fvv0/07l00CvCjgU8zPps5A4DXXx3FH3/+iaefD42b+7Nh8ybAMiVvowA/+gwZTFMvLwb26YtvaAv8WoYwtP9AmjRubHWN1tLpc5VyQNZMn3s/u3r1KuXLl+fixYv4hoawde06qrq7F3ZZ+UKnz1VKObSOPbrzZ3w8ycnJvDF6tMOGua1yDHQRmQl0BM4ZYzzT2t4HOgHJwK/AIGPMn/lZqFKq+IpYtbqwSygSctOHPgtoe0fbWsDTGNMIOASMtXNdSiml8ijHQDfGbAIu3dH2ozEmvZNnO1AzH2pTSimVB/a4y2Uw8IMd9qOUUsoGNgW6iIwDbgJz7rHNMBGJFpHo8/F/2HI4pZRS92B1oIvIACwXS/uYe9z7aIyZbozxNsZ4u1bMOmRWKeWYzpw9S6+BA/hro8do4NOM9t2e5tDhwxw9dgxPP598OeamrVtoGhRIyUoVMybwKk6sCnQRaQuMBjobYxLtW5JSqsAtXAAN60PFByyPCxfYtDtjDF2f6U1IUBC/xv3Mvp0xTJkQztnz5+xUcPZq16zFrE8/55mwHvl6nPtVjoEuIvOASOARETkpIkOAqcADwFoRiRWRz/K5TqVUflm4AF56EU6csExKcuKE5bkNob5h00acnUvy/JBbC1x4NWqUZTj80WPHCHqyNU2DAmkaFMi2HdsBOH3mDMFt2+AVGICnnw+bt20lJSWFgc8/h6efD4/5+/LvqVOzHNejTh0aeXpSokTxHASf433oxpje2TR/mQ+1KKUKw8RwSFuEIsO1a5b2Hj2t2uXeffto5tUkx+3cXF1Zu3Q5Li4uHD5yhN5DBhG9cTNzFy3kyVZPMO7VUaSkpJCYmEhsXBy/nz7F3h07AfjzTx36cicdKapUcXe3RRfyeTEGgBs3bvDiyH8S+3McTk5OHDpyBACfpk0ZPHw4N27c4KmOnfBq1Ii/eHjwf78d5aWR/6TDk21p06pVvtdX1BTPv0uUUrfUvMswkru150LD+vWJid2d43b/njYVdzc39mzbTvTGzSQnJwMQHPg4m1avoUb16vQbNpSv586lUqVK7NkWSUhQENO+mJ6xAIW6RQNdqeJuQjjcMQc4ZcpY2q3UskUI168n88WsrzLadsbEsHHL5kzbxV++TLWqVSlRogT/mz8vY9bFY8eP4+bqyrMDBzGk3wB27YnlwsULpKam0q3LU7z5+hvs2hNrdX2OSgNdqeKuR0/471SoVQtELI//nWp1/zmAiPDd3Hms3bCevzZ6jIa+3oS/PYXqVatl2m740GeZPXcO/i1DOXTkSMaKPxGbN+MVGECTx5uzZNn3jPj7cH4/dZqQDu3wCgxg4N+f4+0JE7Mcd2dMDDUfrcei77/juREjaOib98WfizKdPlcpB+Ro0+cWJ7ZMn6tn6Eop5SA00JVSykFooCullIPQQFdKKQehga6UUg5CA10ppRyEBrpSymL/PvDzsTzaweT336OhrzeNAvzwCgxgx86ddtmvNT6aNo3ExKwTw4ZPmczY8AmZ2mLj4qjv3TTPx4iNi2PVmjVW12gPGuhKKUhIgO7d4MAB6N7d8twGkTt2sGL1anZt3kpc5A7WLVtOLRumErBFSkoKH306jcRrWQO9d1gYC75dkqlt/pLFVk2/G/tzHKt+zFug37x5M+eN8kADXSkFw/8O589bps89fw5eGG7T7k6fPUOVypUpXbo0AFUqV6F6NcsoUQ/PBly4eAGA6F27CGlvWYM+fMpk+j07lJYd2/OwV+OMaQMiNm8iuG0buj7TiwY+zXj+5X+QmpoKwLxFC3nM3xdPPx9Gj38j4/jlq7kz/q038QsNYfL773Hq9GlCO7QntEO7THU+8nA9HqxYMdNfDwu/+5Ze3boD8ONPPxHQqiVNgwIJ69+Xq1evApYRqc2faEXj5v74hrQgPj6e8ZPfYsG3S/AKDGDBksVcunSJp3r3olGAH/4tQ4nbuzfjcw77x4u06dKZ/sOetel7vpMGulLF3f/+B2tWQ1KS5XlSEqz+wdJupTYtW3Hi95PUa+LF8FdezjKHy93E/bKXlYuWEPnTT0x69x1OnT4NQFRMDP+a/DY/b4/i199+49tlSzl1+jSjJ4xn/YqVxG6NZOeuGL5fsRyAhIQEPBs0YMeGCMaPGUv1atXYsHIVG1ZmXf64d/cw5i9ZDMD2qCgqV3qIh//2Ny5cvMBb77/LumXL2bV5K95NmvLh1P+SnJxMz0ED+M+777Fn23bWLVtOuXLlmDTudXo+3Y3YrZH07NadCVMm06RRI+IidzBlwgT6P3crvGNiY1k6fwFzZ36VpR5baKArVdyFj4c7+5cTEy3tVipfvjwxm7Yw/eOPca1ShZ4DBzBrzjc5vq9L+w6UKVOGKpWrEBoUTFRMNAC+zZrxl7p1cXJyonf3MLZEWgI85PEgXKu4UrJkSfr06MmmrVsBcHJyoluXp3JVa69u3Vm89HtSU1OZv2QxvcPCANgetZN9Bw4Q2OYJvAIDmD13DsdOnODg4UNUc6+KT7NmAFSoUIGSJbPORL5leyT9elmWk2jZIoSLly4RHx8PQOd27Slz54RodqDzoStV3IVPglf/mTnUy5aFiZNs2q2TkxMhQcGEBAXzWMOGzJ47l4F9+lLSqWRGl0nS9aRM7xGRbJ9n136veahcXFxwcnLKVZ21atbEo3ZtNm7ZzJJlS4lctx4Ag6F1aEvmfTUr0/Zxe/dmqSc72dWX/r70ScjsLTdL0M0UkXMisve2todEZK2IHE571NWflSqq+vWDJ9uCi4vluYsLtG0HfftZvcuDhw9xOG2xCrDcAVKnVi0APOrUJma3ZerbJUuXZnrf0lUrSUpK4uLFi0Rs2YxPU8tZcFRMDL8dPUpqaioLlizh8YAA/Lx92Lh1CxcuXiAlJYV5ixfR4vHHs63ngfLluXLl6l3r7d09jFfGjuGvdetSs0YNAPx9fNi6YztHfv0VgMTERA4dPsyj9epx6sxpdsbEAHDlyhVu3rxpOcbVW8cIbh7InLRl/CI2b6JK5cpUqFAh91+iFXLT5TILaHtH2xjgJ2PMw8BPac+VUkXVJ5+Cq6tl+lxXN5j2iU27u3o1gQHPP0cDn2Y0CvBj38EDhL/2GgATxoxlxOhRBD3ZOstZtG+zZnQI64Z/q5a8MWp0xoXUAF9fxoSPx9PPh7oedejaqTPVqlbl7QkTCe3QnsbN/Wna2IsuHTpmW8+wgYNo161rloui6cK6duWX/fszLoYCuFZxZdann9F78CDLhc1WoRw4fIhSpUqx4KvZvPTqSBo396d1l84kJSURGhTMvgMHMi6Kho99jejdu2kU4MeYCROY/dl0m77T3MjV9Lki4gGsMMZ4pj0/CIQYY06LSDUgwhjzSE770elzlSoYVk2fu38fDBwAs2ZD/Qb5U9g9hE+ZTPny5Rn5jxGZ2iM2b+KDjz9mxaLFBV5TYbBl+lxr+9DdjTGnAdJC3e1uG4rIMGAYQG33qlYeTimV7+o3gB2FN/hH2S7fL4oaY6YD08Fyhp7fx1NKFU3hr43Ltj39wqrKmbW3LZ5N62oh7fGc/UpSd7X2B+jZCUJ9LY9rs95Tq5QqvqwN9GXAgLR/DwCW3mNbZQ9rf4APpsDZM5bRfGfPWJ5rqCul0uTmtsV5QCTwiIicFJEhwDtAaxE5DLROe67y04xP4I57drmeZGlXSily0YdujOl9l5da2bkWdS/nzuatXSlV7OjQ/6LCzT1v7UoVMqlQnn7PDs14fvPmTVzr1qFjWPd7vCtvhr74AvsO7M/VtkePHaPmo/UyRqmm8woMICo6Os/HnvLB+3l+T37TQC8qhg6H0i6Z20q7WNqVssVH/4ZNGzO3bdpoabdBuXLl2Lt/H9euXQNg7fr11KhePU/7yGl62RlTp9Hg0fq52pdHnTrUqlGTzdu2ZrQdOHSQK1eu4uud4y3eWUz51wd52t4Yk+WXib1poBcVrdvByNfAvaplNJ97Vcvz1tmPfFMq15o2hQH9b4X6po2W503zvsjDndq1bsPKNasBmLd4Eb27h2W8FhUdTfMnWtHk8eY0f6IVBw8fAmDWnG8I69+XTj3CaPNUZ1JTUxn+yss09PWmY1h32nd7msXffwdASPu2RO/aBVimzB03KZzGzf3xbxnK2Wy6I3t3754xsyLA/MWL6d3d8hfD+Qvn6db3GXxaBOPTIpit2yMBuHr1KoP+/jyP+fvSKMCPJUu/Z8yE8Vy7dg2vwAD6DBkMwIdT/4unnw+efj58NG0aYPmroL53U4a/8jJNgwI5cfKkzd/pvWigFyWt28GC5bAhyvKoYa7sIbgFzP7aEuJvvWl5nP21pd1GvbpZAjQpKYm4X/bid9uZ8KP16rFp9Rp2b9nGpHGv89rE8IzXIqOimP3Z56xfsYpvly3l6PHj/Lw9ihlTpxG5MyrbYyUkJODv48uebdsJDgzki1mzsmzT4+lufL9iRcaZ/4Jvl9ArLdBHjBrFKy+8yM6Nm1jyzRyGvvgCAG++9y4VK1Tg5+1RxEXuoGVwC96ZOIkyZcoQuzWSOV/OJGb3br765n/sWB/B9p828MXsr9i9Zw8ABw8fpn/vZ9i9ZRt1ate2+Tu9F51tUSllCe8hQ+G9d2HUaLuEOUAjT0+OHj/OvMWLaN/myUyvxV++zIDnh3H4118REW7cuJHxWuvQljz00EMAbImMJKxrV0qUKEFVd3dCg4KyPVapUqXo2NZyktPMy4u1GzZk2aaquzsN69fnp4gI3N3ccHZ2xrNBQwDWRWxg38EDGdtevnKFK1eusC5iA/Nnzspor1Qp61yEWyK30bVjp4xZFJ/u1JnN27bSuX0H6tSujb+vb26+LptpoCulLN0sX86whPmXMyA42G6h3rlde0aOG0fEqlVcvHQpo/2Nt94kNCiY7+bO5+ixY4TcNnFWubJlM/6dm/mmAJydnTOmp3Vycrpr/3v6ghbubm6ZuoBSU1OJXLc+yzzlxpgcp8u9V4m3f5b8pl0uShV36X3ms7+G19+41f1y54VSKw3u15/xo0fzWEPPTO3xl+MzLpLea/GLxwMCWLJ0KampqZw9d5aIzVtsqqdb5y6s+nGNpbvlttkV27RsxdTpn2c8j42Ly7b9jz/+ACy/QNL/qggODOT7lStITEwkISGB71YsJ6h5oE11WkMDXanibteuzH3m6X3qaRcbbVWzRg1GDH8hS/uoEa8wNjycwNaYgb5FAAAQEUlEQVRPkJKSctf3d+vyFDVrVMfTz4fnRozAz9ubihUqWl3Pgw8+iL+PL+6ubtT18Mho//j994nevYtGAX408GnGZzNnAPD6q6P4488/8fTzoXFzfzZs3gRYpuRtFOBHnyGDaerlxcA+ffENbYFfyxCG9h9Ik8aNra7RWrmaPtdedPpcpQqGVdPn3seuXr1K+fLluXjxIr6hIWxdu46q7o45BqMwps9VSqkC07FHd/6Mjyc5OZk3Ro922DC3lQa6Uuq+F7FqdWGXUCRoH7pSDqogu1OVfdj6M9NAV8oBuaQYLsbHa6gXIcZYfmYuKdb/zLTLRSkHVDMxhZNnz3H+wvnCLkXlgUuKoWbi3e/4yYkGulIOyNlA3QTrg0EVTdrlopRSDsKmQBeRV0TkFxHZKyLzRMQl53cppZTKD1YHuojUAP4BeBtjPAEnoJe9ClNKKZU3tna5lATKiEhJoCxwyvaSlFJKWcPqQDfG/A58ABwHTgPxxpgf79xORIaJSLSIRJ+P/8P6SpVSSt2TLV0ulYAuQF2gOlBORPreuZ0xZroxxtsY4+1aMes8wkoppezDli6XJ4DfjDHnjTE3gG+B5vYpSymlVF7ZEujHAX8RKSuW2d9bAblbflsppZTd2dKHvgNYDOwCfk7b13Q71aWUUiqPbBopaoyZAEywUy1KKaVsoCNFlVLKQWigK6WUg9BAV0opB6GBrpRSDkIDXSmlHIQGulJKOQgNdKWUchAa6Eop5SA00JVSykFooCullIPQQFdKKQehga6UUg5CA10ppRyEBrpSSjkIDXSllHIQGuhKKeUgbAp0EXlQRBaLyAER2S8iAfYqTCmlVN7YtGIR8B9gtTGmu4iUAsraoSallFJWsDrQRaQCEAwMBDDGJAPJ9ilLKaVUXtnS5fIX4DzwlYjsFpEZIlLOTnUppZTKI1sCvSTQFPjUGNMESADG3LmRiAwTkWgRiT4f/4cNh1NKKXUvtgT6SeCkMWZH2vPFWAI+E2PMdGOMtzHG27ViJRsOp5RS6l6sDnRjzBnghIg8ktbUCthnl6qUUkrlma13ubwEzEm7w+X/gEG2l6SUUsoaNgW6MSYW8LZTLUoppWygI0WVUspBaKArpZSD0EBXSikHoYGulFIOQgNdKeUY1v4APTtBqK/lce0PhV1RgbP1tkWllCp8a3+AD6bA9STL87NnLM8BWrcrvLoKmJ6hK6WKvhmf3ArzdNeTLO3FiAa6UqroO3c2b+0OSgNdKVX0ubnnrd1BaaArpYq+ocOhtEvmttIulvZiRC+KKqWKvvQLnzM+sXSzuLlbwrwYXRAFDXSllKNo3a7YBfidtMtFKaUchAa6Kj504IlycNrloooHHXiiigE9Q1fFgw48UcWAzYEuIk4isltEVtijIKXyhQ48UcWAPc7QRwD77bAfpfKPDjxRxYBNgS4iNYEOwAz7lKNUPtGBJ6oYsPWi6EfAKOABO9SiVP7RgSeqGLA60EWkI3DOGBMjIiH32G4YMAygtntVaw+nlO104IlycLZ0uQQCnUXkKDAfaCki39y5kTFmujHG2xjj7Vqxkg2HU0opdS9WB7oxZqwxpqYxxgPoBaw3xvS1W2VKKaXyRO9DV0opB2GXkaLGmAggwh77UkopZR09Q1dKKQehga6UUg5CA13pLIRKOQidbbG401kIlXIYeoZe3OkshEo5DA304k5nIVTKYWigF3c6C6FSDkMDvbjTWQiVchh6UbS401kIlXIYGuiFZe0P90+I6iyESjkEDfTCoLcKKqXygfahFwa9VVAplQ800AuD3iqolMoHGuiFQW8VVErlAw30wqC3Ciql8oFeFC0MequgUiof2LJIdC3ga6AqkApMN8b8x16FOby73Sp4P93OqJQqUmzpcrkJ/NMYUx/wB14QkQb2KauYSr+d8ewZMObW7Yw6na2yt3lfw+7ozG27oy3tqsiyZZHo08aYXWn/vgLsB2rYqzCHktv5xvV2RlVQHm0A4WNvhfruaMvzR/WcrCizSx+6iHgATYAd9tifQ8nLICK9nVEVlCbeEP62JcS7dIOlSyzPm3gXdmXKBjbf5SIi5YElwMvGmMvZvD5MRKJFJPp8/B+2Hq7oyctZt97OqApSE29LmH/9peVRw7zIsynQRcQZS5jPMcZ8m902xpjpxhhvY4y3a8VKthyuaMrLWXdB3M6oy82pdLujLWfm/YdYHu/sU1dFji13uQjwJbDfGPOh/UpyMG7ulm6W7NrvlN+3M+ocMipdep95ejdLE+/Mz1WRZMsZeiDQD2gpIrFp/7W3U12OI69n3a3bwYLlsCHK8mjPoNWLrirdgX2Zwzu9T/3AvsKtS9nE6jN0Y8wWQOxYi2O6nwYR6UXXwnO/jS/o3T9rW/qZuiqydKRoQbhf5hvPS/dPbt1vQXU/0q4uVUB0LpfixN4XXXUgVO5oV5cqIBroxUnrdjDyNXCvCiKWx5GvWX+WqEGVO9rVpQqIdrnYy2+/wsTXYMIUqPvXwq7m7uzZ/aNBlTv50dWlVDb0DN0erl2DMS/Dsd9gzCuW58WBDoTKHZ0uWRUQDXR7eHcS/HHJ0o/8x0V4783Mr2czmGfO2ofw6PkYJUKb4dHzMeasfahwareFBlXu2LurS91dMR84J8aYAjuY9yMNTPR0B5vNbdUy+Ph9SErK+pp7VfAPhNUrM/U13yxZhmfN58xK6ZfRVrZ0CtNHHqNP60sFUbX96F0u6n5x591EYDnBcIBfnhLiE2OMyfGeUg10Wz3VBv7M+xw1R6lDXY5maqvjfp2jC362U2FKFTM9O2V/rcK9qmWQXhGW20DXLhcrpXeZDP7zAxKlbJ7fX5vjWdqOnytlj9KUKp70Ir0GujXmrH2IYR/U4djZ0nzFYJabjlzDJec33uY4tbO01XZLtleJShU/epFeA90a42bUIPG6U8bzwczkLG7ktvMqkbJMKJH5wmnZ0ilMHvq7HatUqpjRi/Qa6NZI7xp5lfcIYQOJlKMDqzhPlexDvXkQuFfFiHCyRG2eZTqrHuhFLbfriBjquF8vmhdElbqf6N1EOrDIGrXdkjl2tjQ78WEhPejBQnzYyduM5S1ex4UkSmBIRXDyagpTLLML9wr/C8u2PUhScglcrqXSqull5k/4rZA/jQOY97Vl6bTbJ5baHW2ZOTC7SaiU47pf5k0qJHqGboXJQ3/H2SmVCELpwUIW0gNP9vIvRvI6bzGZ1xEgiTIwYCgAM1dVZuX2iiQlW77ypOQSLI98kJmrKmfZv0Pco16QdH1MpQC9bdFqVTo35uJlZwAmMp7xvMnX9KMzy3AmGRBuUpKK/34XmnhTsb0XlxOz/kHk9uANzn6/J+N5+gXX2/voi+w96gUpPcR1fUzlgPS2xXx26YolnEPYwN/5lEm8QWeW4cI1ynGNj51eYWefTyB8LOu+OMi161m/6lLOqbzz3MlMbXdecAVIvO7EgLfr6pn6vej6mEppoFurtlsyIWzI6EOfwCQ2E0QpbvA1fXk25XOmzKlG7xLziVhwghsp2XzVBga1u5ip6djZ7O9FT0kVhn1QR0P9bnR9TKVsXiS6rYgcFJEjIjLGXkUVBZOH/o4PUfRgIRGEEsIG/NnOP/mAvTxGGAtZQE/O/FGKyTdHZ7uP5JvC8H/XyniecK0ETvf4iSRed2LcjBr2/ihF3+3rYw5+3vJ4e5+6UsWELYtEOwHTgNbASWCniCwzxhSLRQn7tL7EiP/+v4x+dB92ZoR7uvS7X25vy0z4dKkbgZ4J9Gl9icHvelCihCEl9e4r++lo0mzca31M7XpRxYgtty36AkeMMf8HICLzgS5AsQh0gP+8dIK+k+sCwvuMyvJ6BKH3CPN0wrgZNbh+Q1i5vSI3bt77jyYdTZoNXR9TKcC2LpcawInbnp9Ma8tERIaJSLSIRJ+Pz/skVvezPq0vUcIOy2QfP1eKsdNrkpDkdM/tdDSpUupebAn07KIsyz2QxpjpxhhvY4y3a8VKNhzu/vRc53Nk87HzpLZbMm8PO0k5l5RM7aWcU3nogRs6mlQplSu2dLmcBGrd9rwmcMq2coqeT16x/JHy+TI3UjNyPfPvOpFUypZOTTsDz/xaqZKpTB76O31aX2JNVMVbI0lLpdIl8A8dSaqykBbalaSyZ/XAIhEpCRwCWgG/AzuBZ4wxv9z1Pd7ehmgHv/NgDjACSL8bsTLwH6BPDq8BJAANsHRk1QZ+AcoVSNVKqfuZSK4GFll9hm6MuSkiLwJrACdg5r3CvNjow62AzstrYAnvVUBPYAEa5kqpPLFpci5jzCosEaTspSGwt7CLUEoVRTpSVCmlHIQGulJKOQgNdKWUchAa6Eop5SA00JVSykFooCullIPQQFdKKQdRoEvQicgV4GCBHbBgVAEuFHYRdqafqWjQz1Q02OMz1THGuOa0kU0Di6xwMDfDV4sSEYnWz3T/089UNOhnso12uSillIPQQFdKKQdR0IE+vYCPVxD0MxUN+pmKBv1MNijQi6JKKaXyj3a5KKWUgyjQQBeRMBH5RURSRaRIX8kWkbYiclBEjojImMKuxx5EZKaInBMRh5nAV0RqicgGEdmf9r+9EYVdk61ExEVEokRkT9pnmljYNdmLiDiJyG4RWVHYtdiDiBwVkZ9FJFZE8n11n4I+Q98LPA1sKuDj2pWIOAHTgHZY1hjqLSINCrcqu5gFtC3sIuzsJvBPY0x9wB94wQF+VteBlsaYxoAX0FZE/Au5JnsZAewv7CLsLNQY41UQty4WaKAbY/YbYxxhYJEvcMQY83/GmGRgPtClkGuymTFmE+BQq1AbY04bY3al/fsKlrCoUbhV2cZYXE176pz2X5G/GCYiNYEOwIzCrqWo0j5069TAsvJnupMU8ZAoDkTEA2gC7CjcSmyX1jURC5wD1hpjivxnAj4CRgGphV2IHRngRxGJEZFh+X0wu48UFZF1QNVsXhpnjFlq7+MVEsmmrcifITkyESkPLAFeNsZcLux6bGWMSQG8RORB4DsR8TTGFNlrHyLSEThnjIkRkZDCrseOAo0xp0TEDVgrIgfS/hLOF3YPdGPME/be533oJFDrtuc1gVOFVIvKgYg4YwnzOcaYbwu7HnsyxvwpIhFYrn0U2UAHAoHOItIecAEqiMg3xpi+hVyXTYwxp9Iez4nId1i6a/Mt0LXLxTo7gYdFpK6IlAJ6AcsKuSaVDRER4EtgvzHmw8Kuxx5ExDXtzBwRKQM8ARwo3KpsY4wZa4ypaYzxwPL/p/VFPcxFpJyIPJD+b6AN+fxLt6BvW+wqIieBAGCliKwpyOPbizHmJvAisAbLRbaFxphfCrcq24nIPCASeERETorIkMKuyQ4CgX5Ay7Rbx2LTzgKLsmrABhGJw3JysdYY4xC3+TkYd2CLiOwBooCVxpjV+XlAHSmqlFIOQrtclFLKQWigK6WUg9BAV0opB6GBrpRSDkIDXSmlHIQGulJKOQgNdKWUchAa6Eop5SD+P2Il8soJxeOpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 100 \t #support vectors= 9  #margin vectors= 9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XtYlVXa+PHv4qCIeAzEAypajQaoeEDJysAZ6aBpjq8gkpmopE6N+b6NqaQyGurPmmpmLB0yUwsRJ8umbCY1QdMxBQRPeCpDU8xTSgqiAuv3x97s5CSw94Ytm/tzXV34rP3s57k32s1iPWvdS2mtEUIIUfc52DoAIYQQ1iEJXQgh7IQkdCGEsBOS0IUQwk5IQhdCCDshCV0IIeyEJHQhhLATktCFEMJOSEIXQgg74VTZCUqpFcAQ4LzW2u+29heBF4ACYKPWenpl13Jv1lx7t25rQbj1V1oTV1uHIISwlbS0i1prj8pOqzShAyuBJcDq4galVDAwDOiutb6hlGpVlZi8W7clNW515SeKMtSjfWwdghDCVpQ6WZXTKh1y0VpvB34u1TwZWKS1vmE853y1AxRCCGFV5o6h/wZ4RCm1Wym1TSkVYM2ghBBCVF9Vhlwqel8LIBAIANYppTrrcko3KqWigCiADp6tzY1TCCFEJcxN6KeBT4wJfI9SqghwBy6UPlFrHQfEAfTp4iO1eoWoBS1u3SLm9Gnuy8+XqWx1RBHwnYsLMV5eXHZ2Nusa5ib0DcBAIFkp9RugAXDRzGsJIaws5vRp+jZpgpO3Nyhl63BEVWhNy0uXiDl9mqmdOpl1iUp/eCulEoBdQBel1Gml1HhgBdBZKXUQWAuMLW+4RQhhG/fl5+N0zz2SzOsSpXC65x7uy883+xKV9tC11uEVvPSM2XcVQtQoB5BkXhcpZdEQmQyvCSGEnZCEbg0JqyE9tWRbeqqhXYh66uJPPzFr1CievvdeQn18mPrkk5w8dozsrCzC/Pwqv4AVvBsdzeD27Rng5lYr97M1SejW0NUHYmb+mtTTUw3HXX1sG5cQVfTveHjKG/o6GL7+O96y62mtmT58OL2Dgtjw/fesy8xkyoIF/HzunDXCrbJHnnqKVXv21Oo9bcncWS7idj37QMxCQxIfNgI+W2847inL9cXd79/xsCAK8vMMxz+dNBwDPBFh3jVTk5JwcnZmxKRJprYu/v4AZGdlmdqys7KYO2YM13NzAfjTkiX06N+fi2fPMissjGu//EJhQQEzli6le//+zB8/nsOpqSilGBoZyehp0+4YR7fAQPM+QB0lCd1aevYxJPPV78Oz462azKWOi6hJ70b/msyL5ecZ2s1N6N8fPEjX3r0rPa9lq1Ys2byZhi4unDp+nFfDw1mdmsp/1qwh8LHHiIyOprCwkPy8PI5lZHDhzBkSDx4E4OqVK+YFZ8ckoVtLeqqhZ/7seMPXnn2khy7qhHOnqtduTQW3brH4hRc4lpGBg6Mjp44dA8AnIID5kZEU3LrFo08/TRd/f9p17syZEyd4/cUXeWjwYAJDQmo+wDpGxtCtoXjMPGYhRE76dfil9INSIe5Cnh2q114VnX19OZKWVul5a956i5aenqzZt4/VqakU3LwJQK8BA4jbvh2Pdu2YO2YMG1evpmmLFqzZt4/eQUH88513eG3ChBLXKiwsZLS/P6P9/Vk2Z475wddhktCt4UhmyTHz4jH1I5m2jUuIKpgSCy6lyu27uBrazRUwcCA3b9zg0/feM7UdSkkhbdu2Euddy8nBvU0bHBwc+PLDDyksLATg7MmTtGjViuETJzJ0/HiO7N3LlYsXKSoqYuCIEUyaP58je/eWuJajoyNrMjJYk5HBpHnzzA++DpMhF2sIf7Zsmwy5iDqieJz83WjDMItnB0MyN3f8HEApxeuffsqbL73EqkWLaODiQltvb/737bdLnPc/U6bwyogRfP3Pf9I7OJhGjRsDkJaczIevv46TszOubm7ErF7N+TNnmDduHEVFRQD8YeHCSuP42/TpfLVmDfl5eQz28mLYhAlExcSY/8Hucqo2V+z36eKjZYOL6pOHoqK6/n34MO4PPGDrMIQZLh4+zBOl/+6UStNaV5oIZMhFCCHshCR0IYSwE5LQhRDCTkhCF0IIOyEJXQgh7IQkdCGEsBNV2bFohVLqvHF3otKvvayU0kop95oJTwhRW74/BGF+hq/WsCI2llBfX8K7d2e0vz8Hd++2zoXNsObtt8nPyyvTHhcTw5KZM0u0Hc3IYKQZUz6PZmSw88svzY7RGqrSQ18JPF66USnVHhgE1ELFh/pL5qCL2nA9F156En7IhGmDDceW2L9rFzu++IKP9u4lYf9+3t2yBc/27a0TbDUVFhaytoKEHhIezubExBJtm9au5bHRo6t9n2NmJPSCgoJq3+dOKk3oWuvtwM/lvPQWMB2QvUSFqOPmRcLP50FruHQO5o+37HoXz56lmbs7DRo2BKC5uzsebdsCMNTbmysXDXvKZ6am8nxQEGDoLc8ZM4bJAwfy+/vvN5UNSEtOJmrAAP40fDihPj4snDTJtFr0q4QERnXrRpifH39/5RXT/Qe4ubFszhye69ePFbGxXMjOZlJwMJOCg0vE6d2lC02aNy/x28OWdesIGTUKgG83bSLywQd5plcvZowcSd61a4ChjEFk//6M7tGDsX37ci0nh3/MmcPmxERG+/uzKTGRnJ9/5uWnnya8e3fGBQZyfP9+0+eMjYrihZAQYp4tZ5W5BcwaQ1dKDQXOaK33WTUaIUSt+9cK2LERbhr3Jr6ZD998bmg3V2BICOd+/JERv/kNi6ZMKVPDpSLf7d/PWxs3smLXLpbPm8eF7GwADu3Zw9S//IWEAwc4/f33JH3yCReys/n7K6+wdOtW4jMyyExJIXnDBgCu5+Zyr58fK3fvZuKcOXi0bcuypCSWJSWVuedj4eFsWrsWgAPffkuze+6hw/33c+XiRVa89hrvbNnCR3v38kCfPsS/+Sa3bt5kVlgY//fXv7Jm3z7e2bIFl8aNeX7ePAaFhbEmI4OQsDDi5s6lS8+eJOzfz5QFC5h7W/I+kpbGG599xmtr1pj/TS5HtRO6UsoViAaqVM5MKRWllEpVSqVeyLlc3dsJIWrYkpmQX2qIJT/P0G4uVzc3PkxLY1ZcHC08PJgVFsbnK1dW+r4Bw4bh0qgRzd3d6RMczCHjbkO+ffvi1bkzjo6OPBYeTsaOHWSmpNA7KIgWHh44OTnxeEQE6du3A4ZCXQNHjKhSrINGjeLrjz+mqKjIMNwSHg4YkvuJzEzGP/QQo/392bhqFT+dPMnJo0dxb9MG34AAANyaNsXJqWxZrIwdO3hizBjAUKws59IlruXkGD7n0KG4NGpUpfiqw5ziXPcCnYB9yrCruBewVynVV2v9U+mTtdZxQBwYarlYEKsQoga8sBBe/2PJpO7iCi8usuy6jo6O9A4KondQEPd168YXq1bx1HPP4ejkZBoyuZmfX+I9xpxS9ric9jvVoWrg4oKjo2OV4mzdvj1tvb3Zu20bW9evZ8WuXYBhG71+gwYRm5BQ4vzj+/eXibM85cZnfJ+LsQiZtVW7h661PqC1bqW19tZaewOngV7lJXMhxN1vaCQ8PBgauBiOG7jAI0/BU+PMv2bW0aOcOn7cdHwsI4M2HTsC0Mbbm8PGWulb168v8b5tn33Gjfx8rly6RFpyMj7GXnDmnj2c+eEHioqK2JyYiP/DD+PXrx97t23jysWLFBYW8lVCAr0efbTceFybNCH36tUK4w0JD+fNadPwuvdePL28AMP2dft27uTH774DID8vj5PHjuHdtSsXsrM5lJICQO7VqxQUFNC4SRPybrtHrwED+E+8YXPWtORkmru749a0adW/iWaoyrTFBGAX0EUpdVopZeHjEiHE3WbOCmjZytCBvMcTZr9v2fWuX7tGzNixhPr4EN69OycyM01layfOncubU6cy8ZFHcCjVi/bt25eXBg8mMjCQ8bNnmx6kdnvwQZbMmMEoPz/adupE0PDhuLdpwx8WLmRScDCje/Sga69ePDpsWLnxDI+KYuoTT5R5KFrsdyNHcuLQIdPDUIAWHh7MXbmS6PBw04PNrCNHcG7QgAWJibzx4ouM7tGDFwYN4mZ+Pr2DgzmRmWl6KDoxJobDqamEd+/OkhkziFm1yrJvahVI+dy7nExbFOYwp3zu94dgVhgsSIR7fWsosDuIi4mhkZsbY15+uUR7WnIyH73xBm998UXtB2UDlpTPlQ0uhBCAIYknllk+KOoSSehCiLtCRTsJFT9YFZWTWi5CCGEnJKELIYSdkIR+F5MHokKI6pCELoQQdkISuhCiRgQoxRzj0ncwVBYc5OHBtCFDrHaP1yZM4ERmZpXOzc7KYrCXl2mVarHR/v6mEgPV8cGCBdV+T02ThC5EPbd6MaSWqlmVmmRot0Sjxo35/uBB8q9fB2D35s14tGtXrWtUVl721eXL6ezjU6VrtfX2xrN9e9K/+cbUlnXkCHlXr+Lbt2+14oLqJ3StdZkfJtYmCV2Ies4nAGaG/prUU5MMxz4Bll+7/xNPsHPjRgA2JSSYCl+BoYJiZP/+RPTsSWT//mQdPQrA5ytXMmPkSKY99RQvhoRQVFTEoilTCPX1ZdqQIUx98km+/vhjAJ4PCiIzNRUwlMx9Nzqa0T16MC4wkEvnzpWJJyQ8nM3GyopAiWJcly9cYPqIETwbEMCzAQHs27kTgLxr1/jzuHGM6taN8O7d2bp+PX+fMYMb168z2t+fVyMiAIh/803C/PwI8/NjzdtvA4bfCkY+8ACLpkzhmV69OPfjj5Z/U+9AEroQ9VyfYFi4zpDEl80xfF24ztBuqZBRo9i0di038vM5vn8/fv36mV7r2LUrcdu3E5+ezvPz5vHurFmm1w7s2kXMqlUs3bqVpE8+4WxWFmsPHCB6+XIOGItnlXY9Nxe/wEDW7NtHzwED2GCsp36734WGkrxhg6nnvzkxkUHG5f5/mTqV0dOmsTolhcXr1/PahAkAvD9/Pm7NmrH2wAES9u+nz8CBvLhoEQ0bNWJNRgavxcdzOC2Nzz/4gJW7d/PBt9+y4b33OJqeDsDJo0cZ/OyzxKenm+rZ1BRZWCSEoE8wjJgM78+H8bOtk8wB7u/enbNZWXyVkMBDTz5Z4rVrOTn8eexYTh0/jlKKglu3TK/1HTSIZi1bAoYytL8dORIHBwfcW7emdwX1WJwbNOAR4/h819692bN5c5lz3Fu3prOvLylff01LT0+cnJ25z88PgD1btpQYj8/95Rdyr15lz5YtxN7Wq2/aokWZ62bs2EHQ8OE0MlZRDP7970n/5hsGDB1Km44d6RYYWKXvl6UkoQshSE2C9UsNyXz9UkNCt1ZSf2ToUP728sssS04m59IlU/uy2bPpHRzM659+SnZWFpNuWw3a6LbyslWtN+Xk7Gwqa+vo6Fjh+HvxhhYtPT0JuW0IqKioiBW7dpWpU661rrRc7p1irKlSueWRIRch6rniMfOF62DSvF+HX0o/KDXX0MhIxs+Zw33dupVoz83JoZXxIekXd9j8wv/hh0lav56ioiIunTvH3uRki+IZOGIE//3yS7YkJpaorhgYEsI/lywxHR/NyACgX0gI625r/+WyYaMeJ2dn028VvQYMYNuGDeTn5XE9N5fkTz+l5yOPWBSnOSShC1HPZaaUHDMvHlPPTLHO9T29vAifOrVM+5jp03ln5kzGP/QQhYWFFb5/4IgRtPLyYpSfHwuffx7ffv1wa9bM7HiaNG+OX2AgLT09adepk6n95b/9jUxjudtQHx8+WbYMgPGvvsrVy5cJ8/NjdI8epBq3sRseFUV49+68GhFB1169GPLcc4zt25fn+vVj2IQJdOnZ0+wYzSXlc+9islJUmMuc8rl3s7xr13B1c+PKpUs817cvy3fuxL11a1uHVSOkfK4Qwq5NGzKEq1euUHDzJuNnz7bbZG6pShO6UmoFMAQ4r7X2M7a9DjwF3AS+B8Zpra/UZKBCiPrrHxaOm9cXVRlDXwk8XqptM+Cnte4OHAMs2B9clEeGW4QQ1VVpQtdabwd+LtW2SWtdPCfoW8CrBmITQghRDdaY5RIJ/NsK1xFCCGEBixK6UioaKADi73BOlFIqVSmVeiHnsiW3E0IIcQdmJ3Sl1FgMD0sj9B3mPmqt47TWfbTWfTyalV0yK4SwTxd/+olZo0bx9L33Eurjw9Qnn+TksWNkZ2URZlxub217t2/nmV69CHRyMhXwqk/MSuhKqceBV4ChWus864YkhKhtLePj6ebtTW8HB7p5e9MyvsJfuqtEa8304cPpHRTEhu+/Z11mJlMWLODnciogWlPrDh2Yu3Ilj40eXaP3uVtVZdpiAhAEuCulTgNzMcxqaQhsNtY4+FZrPakG4xRC1JCW8fF0jIrCMc/QN2t48iQdo6IA+NlYGra6UpOScHJ2ZsSkX9NCF39/wFBStlh2VhZzx4zhem4uAH9asoQe/ftz8exZZoWFce2XXygsKGDG0qV079+f+ePHczg1FaUUQyMjGT1tWon7tvX2BkA51M9F8JUmdK11eDnN79dALEIIG2gXHW1K5sUc8/JoFx1tdkL//uBBuvbuXel5LVu1YsnmzTR0ceHU8eO8Gh7O6tRU/rNmDYGPPUZkdDSFhYXk5+VxLCODC2fOkHjwIABXr8jSl9JkpagQ9VyDU6eq1W5NBbdusfiFFziWkYGDoyOnjh0DwCcggPmRkRTcusWjTz9NF39/2nXuzJkTJ3j9xRd5aPBgAkNCajy+uqZ+/l4ihDC52aFDtdqrorOvL0fS0io9b81bb9HS05M1+/axOjWVgps3AUP1wrjt2/Fo1465Y8awcfVqmrZowZp9++gdFMQ/33nHtAGF+JUkdCHquTOxsRS6upZoK3R15UxsrNnXDBg4kJs3bvDpbbsGHUpJIW3bthLnXcvJwb1NGxwcHPjyww9NVRfPnjxJi1atGD5xIkPHj+fI3r1cuXiRoqIiBo4YwaT58zmyd6/Z8dkrGXIRop4rHidvFx1Ng1OnuNmhA2diY80ePwdQSvH6p5/y5ksvsWrRIhq4uNDW25v/Ne61Wex/pkzhlREj+Pqf/6R3cLBpY4u05GQ+fP11nJydcXVzI2b1as6fOcO8ceNMGy3/YeHCMvc9lJLC9OHD+eXyZXZ8/jn/mDuXdYcOmf056hopn3uXklouwhL2Vj63PrGkfK4MudyFJJkLIcwhCV0IIeyEJHQhhLATktCFEMJOSEIXQgg7IQldCCHshCR0IQQALocO4ePnh4uV5m2viI0l1NeX8O7dGe3vz8Hdu61yXXOseftt8vPKFoaNi4lhycySO2gezchgpBlTPo9mZLDzyy/NjtEaJKELIXDIzeX+J5+kUWYm9w8ejIOx+qG59u/axY4vvuCjvXtJ2L+fd7dswbN9eytFWz2FhYWsrSChh4SHszkxsUTbprVrzSq/e8yMhF5QUFD5SdUgCV0IgXdkJM7nz6O0xvncOTqOH2/R9S6ePUszd3caNGwIQHN3dzzatgVgqLc3Vy5eBCAzNZXng4IAQ295zpgxTB44kN/ff7+pbEBacjJRAwbwp+HDCfXxYeGkSabVol8lJDCqWzfC/Pz4+yuvmO4/wM2NZXPm8Fy/fqyIjeVCdjaTgoOZFBxc8nN36UKT5s1L/PawZd06QkaNAuDbTZuIfPBBnunVixkjR5J37RpgWJEa2b8/o3v0YGzfvlzLyeEfc+awOTGR0f7+bEpMJOfnn3n56acJ796dcYGBHN+/3/Q5Y6OieCEkhJhnn7Xo+1yaJHQh6rl7Vqyg2caNOOTnA+CQn0/zzz/nnhUrzL5mYEgI5378kRG/+Q2LpkwpU8OlIt/t389bGzeyYtculs+bx4XsbAAO7dnD1L/8hYQDBzj9/fckffIJF7Kz+fsrr7B061biMzLITEkhecMGAK7n5nKvnx8rd+9m4pw5eLRty7KkJJYlJZW552Ph4WxauxaAA99+S7N77qHD/fdz5eJFVrz2Gu9s2cJHe/fyQJ8+xL/5Jrdu3mRWWBj/99e/smbfPt7ZsgWXxo15ft48BoWFsSYjg5CwMOLmzqVLz54k7N/PlAULmHtb8j6SlsYbn33Ga2vWmP09Lo8kdCHqOa+ZM3EsNcTimJeHV6mx5epwdXPjw7Q0ZsXF0cLDg1lhYXy+cmWl7xswbBgujRrR3N2dPsHBHNqzBwDfvn3x6twZR0dHHgsPJ2PHDjJTUugdFEQLDw+cnJx4PCKC9O3bDfE7OjJwxIgqxTpo1Ci+/vhjioqKDMMt4YYtIA58+y0nMjMZ/9BDjPb3Z+OqVfx08iQnjx7FvU0bfAMCAHBr2hQnp7JlsTJ27OCJMWMAQ7GynEuXuJaTY/icQ4fi0qhRleKrDinOJUQ9d3rhQjr88Y8lknqhqyunFy2y6LqOjo70Dgqid1AQ93XrxherVvHUc8/h6ORkGjK5afytoJhxB7Syx+W036kOVQMXFxwdHasUZ+v27Wnr7c3ebdvYun49K3btAgzb6PUbNIjYhIQS5x/fv79MnOUpNz7j+1yMRcisrdIeulJqhVLqvFLq4G1tLZVSm5VSx41fZfdnIeqoS5GR5AweTJGLCwBFLi5ceeopLo0bZ/Y1s44e5dTx46bjYxkZtOnYEYA23t4cNtZK37p+fYn3bfvsM27k53Pl0iXSkpPxMfaCM/fs4cwPP1BUVMTmxET8H34Yv3792LttG1cuXqSwsJCvEhLo9eij5cbj2qQJuVevVhhvSHg4b06bhte99+Lp5QVAt8BA9u3cyY/ffQdAfl4eJ48dw7trVy5kZ3MoJQWA3KtXKSgooHGTJuTddo9eAwbwH+PerGnJyTR3d8etadOqfxPNUJUhl5XA46XaZgBfa63vB742HgsrkMJcwhayVqzgVqtWaKW45enJyfct22Xy+rVrxIwdS6iPD+Hdu3MiM5OomBgAJs6dy5tTpzLxkUdwKNWL9u3bl5cGDyYyMJDxs2ebHqR2e/BBlsyYwSg/P9p26kTQ8OG4t2nDHxYuZFJwMKN79KBrr148OmxYufEMj4pi6hNPlHkoWux3I0dy4tAh08NQgBYeHsxduZLo8HDTg82sI0dwbtCABYmJvPHii4zu0YMXBg3iZn4+vYODOZGZaXooOjEmhsOpqYR3786SGTOIWbXKou9pVVSpfK5Syhv4QmvtZzw+CgRprc8qpdoAyVrrLpVdR8rnVk4SurAGc8rnuhw6ROewME4kJpLv61tDkVUsLiaGRm5ujHn55RLtacnJfPTGG7z1xRe1HpMtWFI+19wxdE+t9VkAY1JvVdGJSqkoIAqgg2drM28nhKhp+b6+ZB48WPmJ4q5V4w9FtdZxQBwYeug1fT8hRN1UPCRTWvGDVVE5c6ctnjMOtWD8et56IYmKhMfH84O3N4UODvzg7U248YGLEEKA+Qn9X8BY45/HAp9ZJxxRkfD4eN6LisL75EkctMb75Enei4qSpC6EMKnKtMUEYBfQRSl1Wik1HlgEDFJKHQcGGY9FDVoQHU3jUrUoGuflsSA62kYRCSHuNpWOoWutwyt46bdWjkXcQYdTp6rVLoSof2Tpfx1xqkOHarULYWsBSjHHuPQdDJUFB3l4MG3IEKvd47UJEziRmVmlc7Ozshjs5WVapVpstL+/qcRAdXywYEG131PTJKHXEbNiY8l1dS3RluvqyqzYWBtFJOxF68WLaVKqaFWTpCRaL15s0XUbNW7M9wcPkn/9OgC7N2/Go127al2jsvKyry5fTmcfnypdq623N57t25P+zTemtqwjR8i7ehXfvn2rFRdUP6Frrcv8MLE2Seh1REJEBBPj4sjq2JEipcjq2JGJcXEkRETYOjRRx+UGBNA5NNSU1JskJdE5NJRc47J7S/R/4gl2btwIwKaEBFPhKzBUUIzs35+Inj2J7N+frKNHAfh85UpmjBzJtKee4sWQEIqKilg0ZQqhvr5MGzKEqU8+ydcffwzA80FBZKamAoaSue9GRzO6Rw/GBQZy6dy5MvGEhIez2VhZEShRjOvyhQtMHzGCZwMCeDYggH07dwKQd+0afx43jlHduhHevTtb16/n7zNmcOP6dUb7+/Oq8f/B+DffJMzPjzA/P9a8/TZg+K1g5AMPsGjKFJ7p1YtzP/5o8ff0TiSh1yEJERF0ysrCsaiITllZksyFVVwNDubEunV0Dg2l7Zw5dA4N5cS6dVytYJl8dYSMGsWmtWu5kZ/P8f378evXz/Rax65didu+nfj0dJ6fN493Z80yvXZg1y5iVq1i6datJH3yCWezslh74ADRy5dzwFg8q7Trubn4BQayZt8+eg4YwAZjPfXb/S40lOQNG0w9/82JiQwyLvf/y9SpjJ42jdUpKSxev57XJkwA4P3583Fr1oy1Bw6QsH8/fQYO5MVFi2jYqBFrMjJ4LT6ew2lpfP7BB6zcvZsPvv2WDe+9x9H0dABOHj3K4GefJT493VTPpqZItUUhBFeDg7kweTJt588ne/ZsqyRzgPu7d+dsVhZfJSTw0JNPlnjtWk4Ofx47llPHj6OUouDWLdNrfQcNolnLloChDO1vR47EwcEB99at6V1BbM4NGvCIcXy+a+/e7Nm8ucw57q1b09nXl5Svv6alpydOzs7c5+cHwJ4tW0qMx+f+8gu5V6+yZ8sWYm/r1TdtUbYWYcaOHQQNH04jYxXF4N//nvRvvmHA0KG06diRboGBVfp+WUoS+l1E6rgIW2mSlITH0qVkz56Nx9KlXA0OtlpSf2ToUP728sssS04m59IlU/uy2bPpHRzM659+SnZWFpNuWw3a6LbyslWpNwXg5OxsKmvr6OhY4fh78YYWLT09CbltCKioqIgVu3aVqVOuta60XO6dYqypUrnlkSEXIeq54jHzE+vWkT1vnmn4pfSDUnMNjYxk/Jw53NetW4n23JwcWhkfkn5xh80v/B9+mKT16ykqKuLSuXPsTU62KJ6BI0bw3y+/ZEtiYonqioEhIfxzyRLT8dGMDAD6hYSw7rb2Xy5fBgw/QIp/q+g1YADbNmwgPy+P67m5JH/6KT0fecSiOM0hCV2Ieq7VKl3GAAAQc0lEQVRxSkqJMfPiMfXGxnrflvL08iJ86tQy7WOmT+edmTMZ/9BDFBYWVvj+gSNG0MrLi1F+fix8/nl8+/XDrVkzs+Np0rw5foGBtPT0pF2nTqb2l//2NzKN5W5DfXz4ZNkyAMa/+ipXL18mzM+P0T16kGr8QTc8Korw7t15NSKCrr16MeS55xjbty/P9evHsAkT6NKzp9kxmqtK5XOtRcrn3pkMuQhrMad87t0s79o1XN3cuHLpEs/17cvynTtxb22f1VttUT5XCCFqzbQhQ7h65QoFN28yfvZsu03mlpKELoS46/3DwnHz+kLG0IWwQ0UAtTicKqxEayxZSyoJXQg79J2LCwWXLklSr0u0puDSJb4zbtZtDhlyEcIOxXh5EXP6NPdduCC9tjqiCMMP4hgvL7OvIQldCDt02dmZqbdNyRP1g/zwFkIIO2FRQldKTVNKHVJKHVRKJSilzB/8EUIIYRGzE7pSqh3wR6CP1toPcARG3fldQgghaoqlQy5OQCOllBPgCmRbHlL9JKtEhRCWMjuha63PAG8Ap4CzQI7WelPp85RSUUqpVKVU6oWcy+ZHKoQQ4o4sGXJpAQwDOgFtgcZKqWdKn6e1jtNa99Fa9/FoVraOsBBCCOuwZMjld8APWusLWutbwCdAf+uEJYQQorosSeingECllKsyVH//LXDYOmEJIYSoLkvG0HcDHwN7gQPGa8VZKS4hhBDVZNFKUa31XGCulWIRQghhAVkpKoQQdkISuhBC2AlJ6EIIYSckoQshhJ2QhC6EEHZCEroQQtgJSeh3ASnMJYSwBknoQghhJyShCyGEnZCELoQQdkISuhBC2AlJ6EIIYSckoQshhJ2QhC6EEHZCEroQQtgJSehCCGEnLEroSqnmSqmPlVJHlFKHlVIPWiswIYQQ1WPRjkXAX4H/aK3/RynVAHC1QkxCCCHMYHZCV0o1BQYAzwForW8CN60TVv0hdVyEENZiyZBLZ+AC8IFSKl0ptVwp1dhKcQkhhKgmSxK6E9ALWKq17gnkAjNKn6SUilJKpSqlUi/kXLbgdkIIIe7EkoR+Gjittd5tPP4YQ4IvQWsdp7Xuo7Xu49GshQW3E0IIcSdmJ3St9U/Aj0qpLsam3wKZVolKCCFEtVk6y+VFIN44w+UEMM7ykIQQQpjDooSutc4AZJqGEELcBWSlqBBC2AlJ6EIIYSckoQshhJ2QhC6EEHZCEroQwi6Ex8fzg7c3hQ4O/ODtTXh8vK1DqnWWTlsUQgibC4+P572oKBrn5QHgffIk70VFAZAQEWHL0GqV9NBtSApzCWEdC6KjTcm8WOO8PBZER9soItuQhC6EqPM6nDpVrXZ7JQldCFHnnerQoVrt9koSuhCizpsVG0uua8n9dXJdXZkVG2ujiGxDEroQos5LiIhgYlwcWR07UqQUWR07MjEurl49EAWZ5SKEsBMJERH1LoGXJj10IYSwE5LQRb0hC0+EvZMhF1EvyMITUR9ID13UC7LwRNQHFid0pZSjUipdKfWFNQISoibIwhNRH1ijhz4VOGyF6whRY2ThiagPLEroSikvYDCw3DrhCFEzZOGJqA8s7aG/DUwHiqwQS70ihblqlyw8EfWB2bNclFJDgPNa6zSlVNAdzosCogA6eLY293ZCWEwWngh7Z0kP/SFgqFIqC1gLDFRKfVT6JK11nNa6j9a6j0ezFhbcTgghxJ2YndC11jO11l5aa29gFLBVa/2M1SITQghRLTIPXQgh7IRVVopqrZOBZGtcSwghhHmkhy6EEHZCEroQQtgJSehCqhAKYSek2mI9J1UIhbAf0kOv56QKoRD2QxJ6PSdVCIWwH5LQ6zmpQiiE/ZCEbgN3U2EuqUIohP2QhF7PSRVCIeyHJHQbuZumCiZERNApKwvHoiI6ZWVJMheijpJpizYgUwWFEDVBeug2IFMFhRA1QRK6DchUQSFETZCEbgMyVVAIURMkoduATBUUQtQESeg2IFMFhRA1weyErpRqr5RKUkodVkodUkpNtWZg9q6iqYJ303RGIUTdYkkPvQD4P631A0Ag8AellI91wqqfiqczep88iYPWpumMktSFtf1p8WKCkpJKtAUlJfGnxYttFJGwBks2iT6rtd5r/PNV4DDQzlqB2ZXN/4awpyC4L4Q9VWGClumMorakBASwLjTUlNSDkpJYFxpKSkCAjSMTlrDKwiKllDfQE9htjevZlc3/hjcWwI18w/G5nypcRCTTGUVtSQ4OJnTdOtaFhrJ08mQmL11K6Lp1JAcH2zo0YQGLH4oqpdyA9cBLWutfynk9SimVqpRKvZBz2dLb1T3L3/01mRtV1OuW6YyiNiUHB7N08mTmzJ/P0smTJZnbAYsSulLKGUMyj9daf1LeOVrrOK11H611H49mLSy5Xd10/ly5zeX1umtjOqM8dBXFgpKSmLx0KfNmz2by0qVlxtRF3WPJLBcFvA8c1lq/ab2Q7Ewrz3Kby+t11/R0RnnoKooVj5mHrlvH3HnzTMMvktTrNqW1Nu+NSj0MfAMcAIqMzbO01l9W9J4+XXx0atxqs+5XZ5UeQ8fQ67bFvPMfvL3xPnmyTHtWx450ysqq1ViEbf1p8WJSAgJKDLMEJSURkJLC69On2zAyUS6l0rTWlW6kYHZCN0e9TOhgSOrL34Xz58jq0IFZsbE2WURU6OCAQzl/30VK4VhUVM47hLWEx8ezIDqaDqdOccqG/wZEHVXFhC4rRWvDoCcg8XNI2mPTeuM18dBVxuQrJ0NdorZIQq9HrP3QVRJV1cj6AlFbJKHXI9Z+6CqJqmpkfYGoLbJjUT2TEBFhtSEfSVRVc6pDh3IfRsv6AmFt0kO3VMJqSE8t2Zaeami3c7IQqmqkXLKoLZLQLdXVB2Jm/prU01MNx11vq1N2Wy2XH7y9CR8VD6Wn+yYBdawukiSqqpFyybWnvj+kl2mL1lCcxO+9H45mwmtvQE/jDKP33oG1H0Fhgen03IauTHSOI+FfERCMIZmHAuswHNchMh1P3C1Kb74OtlvzYXUyD72WrVgGq9+Hhi6w6C1DQk9Phf+dAuV8j7NadaRTURZMBpZSJ5O5EHcTu144J/PQa87ihNYkpTf5tSE9lZvrP+G/vaaCkyO8+idDgo+ZWW4yB+hw4ZQhmc/H8FWSuRAWkYf0ktDNEtA1l9CYzoaknp7KzVdnEUoiN8ZMNgy3FNwy9NaHjQDP1uVe45RHB0PPfDaGr1JCQwiLyEN6SehmCe55lXUxJwiN6cymD08TSiK+T/sybpE3sdN+pOjGTTTAhysM4+oNXUq8P7ehK7PyYg3DLPMwfA1FkroQFpCH9JLQzRbc8yqTh11g5d4HiCuIZH68FxnnWjGLBTigUWAYbvnvN9A7ADxbU4Qiy7EjE2/EkdA4ArKLL4YhqafY7OMIUefJbCJ5KGq2pPQmfDh9D0tuTcKVvDuf7OZG/NR9PPNWZ0qc6grEAfXn31uNkMqBwu7JQ9Gak5TehGHR9zH31quVJ3OA3Fyil7ejzKl5QHmr5OMBbwx/O97GY1Eh2R9TCANZ+m+GlCONcW1YSPu8H6v2hlaenDzXoPzXSj+Ajwei+DX5nzQeg/TkKyD7YwphID10M0wP/4nzV5w5RRWenjd0YUfgdMOYenlKXyKa8nvyY5Ge+h3I/phCSEI3W4dWN5lFLLmUfKqugVxc0MBPtOKPDZfx+FcvFj8mLav0A/iKpswWYuipS1Ivl+yPKYTlm0Q/rpQ6qpT6Tik1w1pB1QWxE86QwGgmEkcWHSkCCnEgllm4cZ2BbMWRIg780oHcfMeKL7TT+HUxhmmLd+r0VzTmXs/J/phCGFiySbQj8A7wBOADhCulfO78LvsRMehn7mlaQAIRdCKLGfw/fscWZhu73MkEE8o6AkiBigdcDIuK4oEADHPRI6BUp7+k+rPorcoCUlJKjJkXj6kHpMg8UFG/WLJJ9INAjNb6MePxTACt9cKK3mNP0xYB4je35JnYTtwxYVdFRyCLX4t0PQxsqORcIUT9UQvTFtsBt0/zOG1sKxWHilJKpSqlUi/kXLbgdnefiEE/42BhLgd+7XUHY6jrsgF4mrI9dVfKjrkLIYSRJQm9vFRWpruvtY7TWvfRWvfxaNbCgtvdnZ4fep5yPnb1FI+bJ/FrfZcdwEsYeuTK+FUWIQkh7sCSeeingfa3HXvx62L2euPdaYZfUv7xr1YUmfJ6yZ91ShWhXR0gt5wLNMDQ6y5dEz2YOlsjXQhhG5b00FOA+5VSnZRSDYBRwL+sE1bd8u60HylMSkMnp/FR9A/c0/QWhl675p6mt/hwVhZcAz4C7rntjfcAKzD0ulMombylvosQoposquWilHoSeBtwBFZore84wmtvD0WrQz1a6fMMIYQoXxUfilq09F9r/SXwpSXXEEIIYR2yUlQIIeyEJHQhhLATktCFEMJOSEIXQgg7IQldCCHshCR0IYSwE5LQhRDCTtTqJtFKqavA0Vq7Ye1wBy7aOggrk89UN8hnqhus8Zk6aq09KjuptvcUPVqV1U51iVIqVT7T3U8+U90gn8kyMuQihBB2QhK6EELYidpO6HG1fL/aIJ+pbpDPVDfIZ7JArT4UFUIIUXNkyEUIIexErSZ0pdRIpdQhpVSRUqpOP8lWSj2ulDqqlPpOKTXD1vFYg1JqhVLqvFLqoK1jsRalVHulVJJS6rDx395UW8dkKaWUi1Jqj1Jqn/Ez/dnWMVmLUspRKZWulPrC1rFYg1IqSyl1QCmVoZRKren71XYP/SDwe2B7Ld/XqpRSjsA7wBOADxCulPKxbVRWsRJ43NZBWFkB8H9a6weAQOAPdvB3dQMYqLXuAfgDjyulAm0ck7VMBQ7bOggrC9Za+9fG1MVaTeha68Naa3tYWNQX+E5rfUJrfRNYCwyzcUwW01pvB362dRzWpLU+q7Xea/zzVQzJop1to7KMNrhmPHQ2/lfnH4YppbyAwcByW8dSV8kYunnaAT/ednyaOp4k6gOllDfQE9ht20gsZxyayADOA5u11nX+M2HYznI6UGTrQKxIA5uUUmlKqaiavpnVV4oqpbYArct5KVpr/Zm172cjqpy2Ot9DsmdKKTdgPfCS1voXW8djKa11IeCvlGoOfKqU8tNa19lnH0qpIcB5rXWaUirI1vFY0UNa62ylVCtgs1LqiPE34Rph9YSutf6dta95FzoNtL/t2AvItlEsohJKKWcMyTxea/2JreOxJq31FaVUMoZnH3U2oQMPAUONG8+7AE2VUh9prZ+xcVwW0VpnG7+eV0p9imG4tsYSugy5mCcFuF8p1Ukp1QAYBfzLxjGJciilFPA+cFhr/aat47EGpZSHsWeOUqoR8DvgiG2jsozWeqbW2ktr7Y3h/6etdT2ZK6UaK6WaFP8ZCKGGf+jW9rTF4Uqp08CDwEal1Fe1eX9r0VoXAC8AX2F4yLZOa33ItlFZTimVAOwCuiilTiulxts6Jit4CBgDDDROHcsw9gLrsjZAklJqP4bOxWattV1M87MznsAOpdQ+YA+wUWv9n5q8oawUFUIIOyFDLkIIYSckoQshhJ2QhC6EEHZCEroQQtgJSehCCGEnJKELIYSdkIQuhBB2QhK6EELYif8Pszy6PZAZ0xsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "beta=np.empty((3,x.shape[0]))\n",
    "b=np.empty(3)\n",
    "for i,C in enumerate([0.01,1,100]):\n",
    "    beta[i],b[i]=SMO_KKT(x,label,C)\n",
    "    print(\"C=\",C,\"\\t #support vectors=\", np.count_nonzero(beta[i]),\n",
    "             \" #margin vectors=\", np.count_nonzero(beta[i,C>beta[i]]))\n",
    "    \n",
    "    #hyperlpane/contour:\n",
    "    f=set_f(beta[i],label,x,b[i])\n",
    "    zz=np.apply_along_axis(f,1,np.array([xx.flatten(),yy.flatten()]).T).reshape(xx.shape)\n",
    "    plt.contourf(xx,yy,zz,0, colors = ['cyan', 'pink'])\n",
    "    for lab,col in zip([-1,1],['b','r']):\n",
    "        for indecies, style, name in zip([beta[i]==0, beta[i]==C,\n",
    "                                          np.logical_and(beta[i]>0,beta[i]<C)],\n",
    "                                         ['o','d','x'],\n",
    "                                        [\"Class \"+str(lab),\"Support Vector\",\"Margin Vector\"]):\n",
    "            plt.plot(x[np.logical_and(label==lab, indecies ),0],\n",
    "                     x[np.logical_and(label==lab, indecies ),1],\n",
    "                     col+style, label=name)\n",
    "    plt.legend(loc = \"best\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[   0    0]\n",
      " [1000 1000]]\n",
      "0.5\n",
      "[[   0    0]\n",
      " [1000 1000]]\n",
      "0.5\n",
      "[[443 894]\n",
      " [557 106]]\n",
      "0.2745\n"
     ]
    }
   ],
   "source": [
    "for i in range(3):\n",
    "    f=set_f(beta[i],label,x,b[i])\n",
    "    label_test_pred=np.where(np.apply_along_axis(f,1,x_test) <0,-1,1)\n",
    "    conf_KKT=conf_matrix(label_test_pred,label_test_real,-1,1)\n",
    "    print(conf_KKT)\n",
    "    print(acc(conf_KKT,2*n))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare runtimes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.09 s ± 134 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n",
      "3.66 ms ± 72.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit SMO(x,label,1)\n",
    "%timeit SMO_KKT(x,label,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task 2.6.** Generate 50 uniformly distributed i.i.d. points which lie in $\\{ t \\in \\mathbb{R}^2 | \\| t \\|_2 < 1 \\}$ (e.g. by drawing uniformly distributed points in $(− 1, 1 )^2$  until 50 of them are within the unit sphere) and label them by − 1.\n",
    "Now generate 50 data points, which are uniformly distributed in $\\{ t \\in \\mathbb{R}^2 | 1< \\| t \\|_2 < 2 \\}$ \n",
    "and label them by 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_nonlin=np.random.uniform(-1,1,(50,2))\n",
    "bool_ind=np.linalg.norm(x_nonlin,axis=1)>=1 #array of points not in the unit ball \n",
    "while(np.any(bool_ind)):\n",
    "    x_nonlin[bool_ind]=np.random.uniform(-1,1,(np.count_nonzero(bool_ind),2))\n",
    "    bool_ind=np.linalg.norm(x_nonlin,axis=1)>=1\n",
    "x_nonlin_2=np.random.uniform(-2,2,(50,2))\n",
    "bool_ind=np.logical_or(np.linalg.norm(x_nonlin_2,axis=1)<=1 , np.linalg.norm(x_nonlin_2,axis=1)>=2)\n",
    "while(np.any(bool_ind)):\n",
    "    x_nonlin_2[bool_ind]=np.random.uniform(-2,2,(np.count_nonzero(bool_ind),2))\n",
    "    bool_ind=np.logical_or(np.linalg.norm(x_nonlin_2,axis=1)<=1 , \n",
    "                           np.linalg.norm(x_nonlin_2,axis=1)>=2)\n",
    "x_nonlin=np.append(x_nonlin,x_nonlin_2,axis=0)\n",
    "label_nonlin=np.append(-np.ones(50),np.ones(50))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(a)** Fit a linear SVM for $C = 10$ to the data and plot the scattered data as\n",
    "well as the separating hyperplane"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8U1X+//HXoRTKLkspsjMLww6FrpStIMiiIEKBiiwKouKC/kZBRKGi4Iw6M34dGB1GERyRRaqgiDCgZS9LS7GyCToDLiCyKEtLWdr7++OS2DZJm+UmuUk+z8fDR8ltknsS8J2Tc8/5HKVpGkIIIYJHBX83QAghhLEk2IUQIshIsAshRJCRYBdCiCAjwS6EEEFGgl0IIYKMBLsQQgQZCXYhhAgyEuxCCBFkKvrjpPVq3aQ1b9DQH6cWIiRk16jq7yYIb8jOPqNpWmR5d/NLsDdv0JCsBe/449RChAzVM8bfTRBGU+q4M3eToRghgpS2OcvfTRB+IsEuRBCTcA9NEuxCBDkJ99AjwS5ECJBwDy0S7EKECAn30CHBLkQIkXAPDRLsQoQYCffgJ8EuRAiScA9uEuxChCgJ9+AlwS5ECJNwD04S7EKEOAn34CPBLoSQcA8yEuxCCEDCPZhIsAshrCTcg4MEuxCiBAn3wCfBLoSwIeEe2CTYhRB2SbgHLgl2IYRDEu6BSYJdCFEmCffAI8EuhCiXhHtgkWAXQjhFwj1wSLALIZwm4R4YPA52pVQTpVSGUuqQUuqAUmqKEQ0TQpiThLv5GdFjvw78UdO01kAC8JBSqo0BzyuEMCkJd3PzONg1TTupadreG3++CBwCGnn6vEIIc5NwNy9Dx9iVUs2BaGCXkc8rhDAnCXdzMizYlVLVgXTgMU3TLtj5/SSlVJZSKuv0+Z+NOq0Qws8k3M3HkGBXSoWjh/oSTdM+sHcfTdMWaJoWo2laTGSt2kacVghhEhLu5mLErBgFvAUc0jTtr543SQgRiCTczcOIHnsSMAborZTad+O/gQY8rxAiwEi4m0NFT59A07RtgDKgLUKIIKBtzkL1jPF3M0KarDwVQhhOeu7+JcEuhPAKCXf/kWAXwigbPoWRt0NynP5zw6f+bpHfSbj7hwS7EEbY8Cm8MhdO/Qiapv98Za7/wn3pO5BTKlRzsvTjPibh7nsS7EIY4c1/wJWCkseuFOjH/aFVG0ib/mu452Tpt1v5p4yThLtveTwrRggB/HTKtePeFh0DaS/qYT5kGKxO129H+2+2isyW8R3psQthhPpRrh33hegYPdTfeUv/6cdQt5Ceu29IsAthhImToXJEyWOVI/Tj/pKTpffUx07Qf5Yec/cTCXfvk2AXwghnTsPwURDVAJTSfw4fpR/3B8uYetqLcO8Dvw7LSLiHBAl2IYzQqg2sWQVPzYKM3frPNav8drGSwwdLjqlbxtwPH/RPe+yQcPcepWmaz08a84c2WtYC30+7EsKrLL1kk1ysDBRyQdUFSmVrmlbuGyY9dhFYTDQ/20Z5FyvN3HY/kp678STYRWAx2fzsEsq7WGnmtvuZhLuxZB67CCwmnJ8NlLxYGR1zo53T7Y9z+6vtGz7VF0z9dEqfhjlxMvQd4JtzC5+SHrsIPCacn+30xUp/td1sJQ/skF67cSTYReAx4/zs1LG2IR0dox8vzl9tN1vJAwck3I0hwS4Ci8nnZ5fJn203W8mDMki4e06CXQSWAJif7ZA/227GkgdlkHD3jMxjFyIUWMbYiw/HVI6AJ5429QVUmeNeisxjF0JY9R2gh3jxkgcmD3WQnru7ZLqjEKGi7wDTB7k9Uu7XddJjF0KYnvTcXSPBLoKXLOEPKhLuzpNgDwXBHHBlbSAtS/iDjoS7cyTYQ0GwBlx5qymLL+Ff+IbtEv9AFMwf0k6ScC+fBHsoCMaAA+dWU5qx/IAngvVD2kUS7mWTYA8VwRZw4NxqSjOWH/BEsH5Iu0HC3TEJ9lARbAEH5a+mDOTyA2UJxg9pN0m42yfBHgqCNeDK20A6kMsPlCUYP6Q9IOFuS0oKhIKl7+hjsMV7djlZesCVrj4YaEKtxnjpuu+lb4ewkFjE5GRJAUOCXSm1ELgN+EnTtHbl3V+CXQg3eeNDOog++IM+3H1cK2YR0N+g5xIiuHkyZdHZuu+uCKKZNjIsozMk2DVN2wKcM+K5hAh6ZgvSIJtpI+EuF0+F8D0zBmmQzbQJ9XD3WbArpSYppbKUUlmnz//sq9OKQFBWWYBgZbYgDcKZNqEc7j4Ldk3TFmiaFqNpWkxkrdq+Oq0wO3tlAf40Gwbfogf90H4w+2l/t9J4ZgrSYJ0OS+iGuwzFCP+yVxag8DpcOK8H/c8/w9ZNwdWLN1uQBut8/xtCMdyNmu64FOgF1ANOAbM0TXvL0f1luqOwSo7TA7w8UQ1g+cfeb48vBNH0wkASFFMhfTndUdO0VE3TbtY0LVzTtMZlhboQJTi7mbKjujA+9NLSBmTk1ChxLCOnBi8tbeDaE3ljyqIoVyj13GUoRviXvbIA9jj7AeBFsa3yGJH2G2u4Z+TUYETab4htlefR8xr2gSHKFSrhLsEu/Kv4JsuAhqKoQliJuxSGhfNxW/9fQE2OvsiKtP8yIu03zFzYkBFpv2FF2n9Jjr5o9/7OBra3PjCEfaEQ7hLswv/6DtDHz+9/hH2T3+bBygu5XLshKMWV6nVZpYZS/fZ+/m4loIf7g0NO8/w7DXlwyGmHoQ7OB7arHxjCc8Ee7hLswvcczVtPHUv0iLaMmhtP06JjzBzzPY3DTlLn5dmmCbmMnBq8vjqSZ8ee4PXVkTY98uJcCWxXPjCEMYI53CXYhW+Vt50d5g05S497Rdp/mX3vCWtolxfuzrwWZz4wlmyoQ/OR7amQ3IXmI9uzZEMdw15bqArWcJdgF77lxHZ2rvSKfWnP4WoletyWHvmew9UcPsaZ1+LMB8aSDXWY9Eozjp+qjKYpjp+qzKRXmkm4GyAYw13qsQeCYJr37GjeulKQsbtEyCVHX7S5HUicfS0vLW1AbKu8Escycmqw53A1pqb+CEDzke05fqqyzTmaRV3h2PIvvf9iQkBAzHP3cdle4U1mqwboiXK2s3OnV2xWzr6Wqak/2nxoJUdftIY6wLc/VbJ7DkfHheuCqecuPfZAYQnzIcP02iL+rgboLssYe/HhmMoR+pRHL+18tGRDHWa82Yhvf6pE0/pXmTPxB0b3Dawq09Jj9x1T99ylxx5kzFYN0F3F560rpf/0cqgHw9j0nIk/ULVyYYljVSsXMmfiD35qUfAKhp67BHugMFM1QE9Z5q1n7NZ/enGP0hlvNiL/SskFT/lXwpjxZiOvndMZrq42Hd33HAueOE6zqCsopdEs6goLnjgecN88AkWgh7sEeyAwWzVAg3lzGp8zY9P+mEbozmrT0X3PcWz5lxRlZHNs+ZcS6l4WyOEuY+yBIJhmxZRiGSop3quuWrmwzN5o8THzOjWuA3DuYkW74+fljU07Ov+4/mdYu/Mmr47LW8L8wSGneX11ZEDO/AkFphpzd3KMXYJd+JWrFwXtBXFxpT8UyvvgcHR+hYaGcvi8Rpm5sCHPv9OQZ8eeYPa9Jwx9bmEc04S7XDw1iVDc9s0Frk7jszdmXlzp8fPyxqYdnad4qNt7XmeUN8Rj1oVYwlagDctIsHtTecvnJfRpWv+qU8ctIXn8VPnztkuHdVlj047O78zzlqW82TjulCfwm6Xv2F7PycnSj4eQQAp3CXZvKmv5vBM1U0KBM9P4iockpXrS9rgS1vbOr7A/POnK85Y3GyegFmIF0wI5DwVKuMsYuzeVtXy+fpQe5qUF0xZwTipvAZGjcXB73BkLL33+gQm/sHhdPZcu6JZWIbkLmmb7IaSURlFGttNtM41gWSBnEL+NuTs5xl7RF20JWY7Cu36U463eHB3f8Kne0//plP74iZO9Ov/bl0b3PVdmYDoeAtGoW7PsWTHunj+pXZ5Hq1Wb1r9q98PIlV6/qRRfIDd2QkiHOug9d9NcULVDgt2bJk62v3x+4mQ9pB2Ffmmll+Fbhm0gaMK9LHVqXOfshXCb482irnptOX15HzblmTPxB7uzcQJ2pWjpBXLRMRLuJg53GWP3prKWz9vb69MS+qU5Ueo2WC3ZUIcLebazYCpVLDJ1SAbVStEgXyDnCbOOucsYuz85O7xSTqlbs/OkCJej8fW6Na9x5qMvjG6qsCeIF8gZxWc9dxljDwB9Bzg3lFLWWL23GDSmX3qBkGXaH+BUuDsaXz93Uf7p+oy98JahmBLMNiwjQzGBwJVhGyMYOBXT0yJczs5zF8LfzDQsI8EeCHxc6tadMX3LAiLVqwsVe3dB9epS5oIiZxf7eKtcrSeFv/y996i/zy8cM0u4y/fZQOHssI0RXJyKWXq4pbBIP378VOUbNVdsOdvjtgzXGLlRRlnDQ+Wdy9OhJU/5+/yifGYYlpGLp8LWyNtdWjxV3gIiXxXUclZZF2QvX6lQ5sIkf+9k5O/zC+d5JdylCJhwm4tj+uUNq2jglWl/7g5JOGrv2QsVy70e4O+9Rx2d5/ipSjIkYzL+HJaRYA919gqRuTimX96wimUhkZEbRHiy5Z2rF16Lh6lRF3Pd/VByfB7l8pZ/Mlbvff4Kdwn2UFbW7BcXtq+zd4HTwlurLT2ZbePogqylPEFpxcPUiIu5nnwolfVeuzLbKFj2gg0E/gh3CfZQZtCK1uKrLEEjrIIGeHe1pSdDIo5Whf7fI9+VG9pGrCj15EPJcn4cVKB0dkjIrHvBBitfh7shs2KUUv2B/wPCgDc1TfuTEc8rvMzVQmRl8LS2iqs8LbJVVnvLm4Hj6Wv1dJx+dN9zzHizkUev39/XCkKRL2fLeNxjV0qFAfOBAUAbIFUpFXqFmgORo5Wr3lzRahBvzW/3xYbRRozTe/r6ZeGXf/iq527EUEwc8LWmaf/VNO0qsAwYYsDzCm/z9YpWA1mGJOrWvIY+LKFRpXKRv5vlFCM+lDwdEvLWB6Mony/C3YhgbwR8V+z29zeOCbPz9YpWL7h8pQL6rkqKsxfCA+ICoFGVH4t/u5gz8QdmvNnI6RkuQVV9MgB5O9w9XqCklEoBbtU0beKN22OAOE3THil1v0nAJICmUQ26HA+xXYKE8WSxjq70alTw/yIw4RyXx9x9uEDpe6BJsduNgROl76Rp2gJN02I0TYuJrFXbgNOKYOLOnGq5AKiTGS6By1s9dyOCfQ/we6VUC6VUJWAU8JEBzytChLtzqh1d6KtT43pILbwp7wNOFiKZmzfC3eNg1zTtOvAwsB44BKzQNO2Ap88bsOyt5BRlcrfHae8CYKWKRVzICwvahTf2QrqsGS6yECkwGB3uhixQ0jRtraZpLTVN+62maXOMeM6AZGAd82DkqOfo7pCKvQuANaoWcq2w5D/rYBmWcBTSAxN+cTjDRYZpAoeR4S4rT41k1N6kQdjrL6vn6Mmc6tLzzh3trBQM4+6OQnrtzpscznCR6xCBxahwl2A3khErOYO0119Wz9HIOdXBuPDG8k2nrE1LHC2sCsb3w6Eg6RAZEe4S7EYyYiWnUb1+kymr52jknOpgW3hT/JsOxWraF1dWSJvy/fBGAAdZh8jTcJdgN5IRKzkNrN9iJuX1HI1ayh9sC2/sfdMprryQNt374a0ADsIOkSfhLlvjGcmyYvPNf+hBXD9KD3VnVnJu+FR/nKMFYwFQv6Uscyb+YHcRjTd6jr4uSOZNjsfCNZpFObdNoKnej7IC2JMVz0HaIXK3cJgEu9Hc2ZvU0osp/Q/eIkDqt5S2ZEOdEpUSx/U/w9qdNxm2d2mgKP0+uPK6HVWxtGxeEnC8FcD1o+xv5xjgHSJwL9xlKMYM7PViLFyp3+LHi0elpzJO/lsTm1kwi9fVY87EH7xaOdFsPJ1H7miMfGDCL95ddLT0HcgpNRSQk6Uf94S3KooGcEE7Z7g6LCPBbgaOeitKlbt7kZUfLx7ZC683Vte3Owtm3IstQmpxjKfzyO2NkY/rf4bF6+rZfFhM/lsT6g3uiOrVBdWrC/UGd3T/vW7VBtKm/xruOVn67VYeVuT2VgAHQUG78rgS7h4XAXNHzB/aaFkLSn7yX1PwfdUwCsLsX/kPaqdPQ5Gd7c4qhEFkpO+eww0RhRpDRrfkyyPVnH5MKBWoqpDcBU2z/TetlEZRRrZbz+mo+JlevrjkuSpVLGLhtGPuvdeWMB8yDFanQ9qLEG3ARhGW60muXocSqF6xThUBM80Y+/dVw6gRVZ/mtWqhVIiFe/368OPJkhdOlYIGN0PNWs49R6H9/ToB+N3vPWufA5qmcfb8eSY+9BNTprRw+nGWHmsoBLunOz3Z4/iCqu3/N1evV3D/vY6O0UP9nbdg7ARjQh3cuw4lXGKaoZiCMEXdUAx10MO7wc0QHq7fDg93LdQtj3HluAGU0v/OWv3hsoN7OP42GKgrH50tqFV8UZEq9T54OhvI1Q8Ft9/rnCy9pz52gv6z9Ji7MC3T9NiB0Ax1i5q1XAvy0upF2u/11/PeMIx+CkWdmoVUrVxoZ76147/PQFz5WLruuWVsGyjRIy59P32AREMDp6co2ju3ZWZNnRrXqVSxiKvXf+2X6c/v+gImhyzDMJbhl+iYkreFqZmmx24GP546xajx4/hth/a0ie3CwGF3cuToUY4dP067+FiftGHG7DSatP4D1W92cZaAEb1+N1WLKLRe4Curl27h95WPbnL2Qqi9+2ko6xRFd0K9+MXpsxfC0TSoW/Oa9YLqA0N+IjzMdmvAShWL3HuvDx8sGeLRMfrtwwddfy7hcwEb7EtWhNG8bRUq1KpK87ZVWLLC8eo8Z2iaxtC7UunVvTvf5H7JwT3ZzJ2VxqnTPxnUYufc3n8guzM2u/fgmrXgN7+DP7TWf/og1C0sK0cdf+nSzLHy0QPOFtQyuvCWvQ+Ka4UVqF6lyDp19B+Pf8fbTx0rsQds3ZrX3L9wmjrWtmceHaMfF6ZnqqEYZy1ZEcakRyqTf1lPkePfKSY9Uhm4wugRdmaGOCFjy2bCwyvywISJ1mOdOnQA4Njx49Zjx44fZ8ykieTl5wMw75W/0DU+gZM//sjI8WO5cPEi169f5/W/vUrX+AQmPDSZrJy9KKW49+6xPP7ww2W2IyEuzq32m4WrC2o8Wbzja85eCDX6gqmjD4TSRcFMtcJU+FVA9thnPFfJGuoW+ZcVM55z/4Lc/oMH6dIputz71Y+MZMPqj9m7dTvL317Mo1OfBOC991dwa59b2Lc9ky927KRT+w7sy83lh5Mn2L9rD1/u3M09d9/tdvsChStFpwJtEwhnX5vRhbccfSAoMO17JfwrIIP92+/tf993dNxI165d475HHqZ9Qhwp48Zw8PBhAGI7d+btd98lbe4cvjxwgBo1avCb5s357/+O8cgTf2Tdhg3UrFnT6+3zN1eKTpl9E4jSM2AAp16b0YW35kz8wWZmDejj9mZ5r4S5BORQTNPGGse/sw3xpo3dX2zVtnVrVq5eVe79/jZ/HlH16/PFjp0UFRUREVkXgB5J3diybj2frF/HmEkTefLRxxh71118sSOT9Z9tZP6/FrDiww9Y+I/Xrc9VWFhIlx7dABg8YCCzn3nW7fabibNDAmbeBMLRDJgFTxx3qkaLkcMio/ue4+459tcJmOG9EuYTkD32ObOuUrVKqbnBVTTmzHJ/Cl3vnr24cuUq/1r0tvXYnuxsNm/bWuJ+5y9c4OYGDahQoQL/XraUwkL9K/fxb7+lfmQk942/hwljxrH3i32cOXuGoqIihg25g+efeZa9X+wr8VxhYWHs257Jvu2Zxob6ubOQn1fyWH6eftxEzLwJhNm+TTSLMu97JcwnIIN99IhCFvz9Cs2aFOlfdZsUseDv7l84BX0+9ofvLWVDxuf8tkN72sbFkPbiXBo2uLnE/SZPvI/F7y0hoXcyR77+mmrV9KX0m7ZupVNSItHdupL+0SqmPDiZH06cpNegAXRKSmT8g/fz4qznym3H1GefoXGrluTn59O4VUvS5rqxhWxEBJz44ddwz8/Tb0dElP04HzPlJhA3mO3bhJnfK2E+pqkVc6hmRVp7ael7SLKE+U214ZefoWEjqOp8PRenXTjPoQMHaN3/Vrfqfph1VoyjeizNoq74rVyuWd8r4TsBVytGGKxqNT3Uz56BuvW8Fur8eFIvPla8oiQ4He5mnaLny41BnGXW90qYT0AOxQgn5OfpPfW69fSfpcfcjXDmtO2OTx5sR+ZsHRajH2uP6baUE8IF0mMPRpZhGMvwS9WqJW8b5do1+8fd2A3H2TosRj+2LNJDFoFKeuzBqKCgZIhXrabfLnCwS5O7HFWOdGM3HE9moZhtBosQ/ibBHozq1LXtmVetph83Ur1IbIrDuLkbjiezUMw2g0UIf5NgF+6zVJSsEObxdmSezGk383x4IfwhoIP9wCFFu/gqHDhkTCmBOS+/RNu4GDokxtMpKZFde/YY8rzueHX+fPJvFBorLm3uHKanzSpxbF9uLq1jOrt8jn25uaxdv97tNgJ6uEdGQsZu5/dntcOTedoyx1uIkgI22PPyYODwCA4eVgwaHkGeh5M+MnftYs26dezdup3czF1s/OhjmjRubExjXVRYWMirr88n/7JtsKempLD8g/QSx5alr+SulBEun2ffl7ms/Y9rwX79ehlb8HnAk1koMoNFiJICNtjvnVyZn04rNE1x6rRiwkP2Nvd13slTP1Kvbl0qV9afp17dejS8WV912rxdG86cPQNA1t699BrYH9B7z2Pum0jv2wby+04dreUINm3dQo/+/Rh61yjaxHbhgccepahI3wRh6fsraJ8QR7v4WKbN/LWMQPWbo5j5wvPEJ/dizssvceLkSZIHDSR5UMke8B9+35KbatUq8W1ixYcfMGrYcAD+89lnJPbpTefuSaSMvZtLly4BenmErrf0oWPXBOJ69eT8+fPMnPMCyz9Ip1NSIsvTV3Lu3DnuSB1Fh8R4Enonk7t/v/V1Tnr0YfoNGczYSfd59D6XxVLT3VJj3JVg9uSxwklL37HdHi8nSz8uTMWjYFdKpSilDiilipRSPtsva+G/K/LJ+jAKCvQhmIICxcfrwlj4b/dnb/br3YfvfvieltGdmPz4YzY1YhzJPbCfT95PJ/Ozz5j95z9x4uRJAHZnZ/OXOS/y5c7dfPO///HBR6s5cfIk02bN5PM1n7BveyZ79mazas3HAOTl5dGuTRt2ZWxi5lPTaXjzzWR8spaMTz61OWfq8BSWpa8EYOfu3dStXYff/+53nDl7hhde/jMbP9LLCsdEd+av8/7O1atXGXnPOP7vzy/xxY6dbPzoY6pVq8bsGc8w8s5h7Nueychhw5k1dw7RHTqQm7mLubNmMfb+X0M8e98+Vi9bznsL37ZpjwgRrdro2+NZwt2yfV6rNv5tl7DhaY99P3AnsMWAtjhtelo4efml6rHnK6anub9xc/Xq1cneso0Fr71GZL16jBw/jkVL3i33cUMGDqJKlSrUq1uP5O492J2t/6OP69KF37RoQVhYGKnDU9iWqQd5r27diawXScWKFRk9YiRbtm8H9IJgw4bc4VRbRw0bzsrVqygqKmJZ+kpSU1IA2Ll7DwcPHyap3y10Skpk8XtLOP7dd3x19Ag3RzUgtksXAGrWrEnFirYfgtt2ZjJmVCqgF0U7e+4c58+fB/Tqk1WqVHGqfSJIWbbHS5sOC9+QPVBNzKMFSpqmHQLfb0L9Yto1Hn2yUolwr1pV40/POVgw46SwsDB6de9Br+49aN+2LYvfe4/xo++mYlhF61BKwZWSc8FLv3bLbXvHy6rLExERQViYc9v7NWncmOZNm7J521bSP1pN5sbPAdDQ6Jvcm6VvLypx/9z9+536O7LXPsvjLMXORIiLjoEhw+Cdt2DsBAl1kwrIMfZ7x1xn0K2FREToQRQRoXF7/0Luudv9C3tfHT3C0a+/tt7el5tLsyZNAGjerCnZOXrJ3fTVq0s8bvXaTygoKODs2bNs2raV2M56r3h3djb/O3aMoqIilqen0y0xkfiYWDZv38aZs2coLCxk6cr36dmtm9321KhenYsXLzlsb+rwFB6f/hS/bdGCxo30hTgJsbFs37WTr7/5BoD8/HyOHD1Kq5YtOfHjSfZkZwNw8cb2fTWqV+fipV/P0aNrEktWLAf06wT16tYNic1BhAtysmB1uh7qq9Ntx9yFKZQb7EqpjUqp/Xb+G+LKiZRSk5RSWUqprNPnf3a/xTcs/McV6kfqGyRHRWq8Nf+KR8936VIe4x64nzaxXeiQGM/Brw6T9vTTAMx6ajpTpk2l+619bXrVcV26MChlGAl9evPs1GnWC66JcXE8lTaTdvGxtGjejKG3D+bmBg14cdZzJA8aSMeuCXTu2Ikhg26z255J4+9hwLChNhdPLVKGDuXAoUPWi6YAkfUiWfT6G6Tee49+AbRPMoePHqFSpUosf3sxjzz5BB27JtB3yGAKCgpI7t6Dg4cPWy+epk1/mqycHDokxvPUrFksfmOBR++pMDF3LoRaxtTTXoR7H/h1WEbC3XQMKdurlNoEPKFpmlN/w0aV7T1wSDFyfATLFxXQtrXvyw+nzZ1D9erVeeLRKSWOb9q6hVdee40176/0eZv84dDXR2l9wTvTIIWXFA/p6Bjb2/YsfUe/UFr89zlZcPggpI51vQ0bPtULxv10yq2Sz6EoJMr2tm2tsX/XZX83Qwjj+Crsil8IHTJMH1Yp70KovfCOjnFvnH3Dp3qJZ8s1KzdKPgvHPOqxK6WGAn8HIoFfgH2apt1a3uNko43gYu2xexJK0nuzDTvQa++4WabBKQvf+PVC6L0PeOcc9oy8XQ/z0qIa6CuYhV3O9tg9uniqadqHmqY11jStsqZpUc6EughSllA69WPJTTc22M7DN/SxweTNf5QMdfCovn25/Hkh1FFpZzdKPgtbATkrRpiQJ6Hk60AzK1+Gnb8vhDoq7exGyWdhS4JdGMOTUJLem86XYXf4YMkxdcuY++GDxp/LnomT9WGm4tws+SxsSbALzxVctq3LbuFMKEnvTefLsEsda3vRMzrGvdkt7ug7QL92ENXA45IzD8HXAAAYsElEQVTPwpYEezGqZnXG3DfRevv69etEtmjGbSnDy3iUayY+/BAHDx9y6r7Hjh+ncauW1lWvFp2SEtmd5fpX5rmvvOzyY8p14TxcuACl2gg4H0rSe9OFWtj1HaBfKPWw5LOwFZDB/tKr4WRsKdn0jC0VeOlV92vFgL5sfv+hg1y+rE+h3PD55zRq2NCl5yivrO2b8+bTplVrp56rebNmNGnUmK07tluPHT7yFRcvXiIuxvUpZnP/8opL99c0zeZDxYa9Da0BKlRwPpRCLdDKImEnDBCQwR7buZAR4yKs4Z6xpQIjxkUQ27mwnEeWb0Dffnyyfh0AS1e+T+rwFOvvdmdl0fWWPkR360rXW/rw1dEjACxa8i4pY+/m9hEp9LtjMEVFRUx+/DHaxsVwW8pwBg67k5WrPgSg18D+ZO3dC+ilemfMTqNj1wQSeidzys6Ycurw4dZKjgDLVq4kdbj+DeL0mdMMu/suYnv2ILZnD7bvzATg0qVL3PPgA7RPiKNDYjzpq1fx1KyZXL58mU5JiYyecC8Af533d9rFx9IuPpZX588H9G8JrWM6M/nxx+jcPYnvvv++7DfM0YbWmuZaKEmgCWGYgAz25B5FrFhcwIhxEcx8IZwR4yJYsbiA5B7l9C6dMGqYHqQFBQXkHthPfLGecauWLdmybj0523Ywe8YzPP1cmvV3mbt3s/iNf/L5mrV88NFqjn37LV/u3M2b8+aTuWe33XPl5eWREBvHFzt20iMpiX8tWmRznxF3DmPVmjXWbwLLP0hn1I1gnzJ1Ko8/9DB7Nm8h/d0lTHz4IQCef+nP1KpZky937iY3cxe9e/TkT8/NpkqVKuzbnsmStxaSnZPD2+/+m12fb2LnZxn8a/Hb5HzxBQBfHT3K2NS7yNm2g2ZNm5b9hhm4obUQwhgBu/I0uUcRD064xvMvVeLZqVcNCXWADu3acezbb1m68n0G9is5Lf/8hQuMe2ASR7/5BqUU14r1Vvsm96ZOnToAbMvMJGXoUCpUqECDqCiSu3e3e65KlSpxW3+9Z9qlUyc2ZGTY3KdBVBRtW7fms02biKpfn/DwcNq1aQvAxk0ZHPzqsPW+Fy5e5OLFi2zclMGyhYusx2vXrm3zvNsydzD0ttutVRvvvH0wW3dsZ/DAQTRr2pSEuDhn3i59Q+sfSy00CcXx8dJkwZXwo4AN9owtFXj9rXCenXqV198KJ7lHoWHhPnjAQJ6YMYNNa9dy9tyvO/E8+8LzJHfvwYfvLePY8eP0Klagq1rVqtY/O7uaNzw83FoWNywszOH4vGVjjaj69UsMDRUVFZG58XObOumappVbpresJhZ/LZw7CxERULVY2d78PCgogDp19T1Pa9bUx8UlxHSuLpc3ugaLCHkBORRjGVNfsbiA2c9csw7LlL6g6q57x4xl5rRptG/brsTx8xfOWy+mlrUJR7fERNJXr6aoqIhTP51i09ZtHrVn2OAhrP3Pen0Yplg1x369+zBvwT+tt/fl5to9/vPPejXN8PBw67eMHklJrPpkDfn5+eTl5fHhmo/p3jXJ9uQREXDiBz3MQf954gf9uPU+VWR8vDhXF1zJzkTCYAEZ7Hv2hpUYU7eMue/Z69xGFeVp3KgRUyY/ZHN86pTHmZ6WRlLfWygsdHyhdtiQO2jcqCHt4mO5f8oU4mNiqFWzltvtuemmm0iIjSMqsj4tmje3Hn/t5ZfJytlLh8R42sR24Y2FbwLwzJNT+fmXX2gXH0vHrglkbNU3uJo0Xi/nO3rCvXTu1Inxo+8mLrkn8b17MXHseKI7drQ9edVq0LCRHuZnTus/GzYq2YMXJbm64Ep2JhIGM6Rsr6tCoQjYpUuXqF69OmfPniUuuRfbN2ykQVQAX1A8cxrOnoG69fRx9WK8XrY30Mar3S1w5a+CXCJg+KQImHDsthHD6ZSUSPf+/Xh22rTADvX8PPjlZz3Uf/n512EZX/B2gTB3NpwojzsLrmRnImGggL14anab1q7zdxOMYRlTtwy/VK3q2+GYssarjei1W8a37W044S5Lu5z9llF6k4voGBmOER6RYBdlKygoGeKWMfeCAt8Eu7cLhLmz4YQz+g5w/oOnrIJcEuzCDRLsomx16toeq1rNdxdP60fZH682cgFUdIwe6pbxbV+HqZE7EwmBjLELs/NFgTB/jG9v+FS/yJocp/8MtU1FhFdJsAtzK10grGYtqFwJ5s4yJhD9seGE7BglvEyCvZgfT51i1Phx/LZDe9rEdmHgsDs5cvQox44fp118rFfOuWX7Njp3T6Ji7VrWQmGiFEuBsKefgytX9DLBRgWiPzackB2jhJcFbrCvWA5tW0OtGvrPFcs9ejpN0xh6Vyq9unfnm9wvObgnm7mz0jh1+ieDGmxf08ZNWPT6P7krZYRXzxMUvBGI/thwQnaMEl4WmMG+Yjk88jB8953ec/vuO/22B+GesWUz4eEVeWDCrxttdOrQwWaZ/bHjx+l+a186d0+ic/ckduzaCcDJH3+kR/9+dEpKpF18LFt3bKewsJDxD9xPu/hY2ifE8bd582zO27xZMzq0a0eFCoH5V+FTgRyIxcfUPdltyqg2yLh+UAvMWTHPpcGNzTCsLl/Wj48Y6dZT7j94kC6dosu9X/3ISDas/piIiAiOfv01qRPuIWvzVt57fwW39rmFGU9OpbCwkPz8fPbl5vLDyRPs37UHgF9++cWttokbfDFDxhtKFwWzt9rb2xUxXS1MJgJaYHYTHW3+UN6mEAa4du0a9z3yMO0T4kgZN4aDh/WyubGdO/P2u++SNncOXx44QI0aNfhN8+b893/HeOSJP7JuwwZq1qzp9fYFtUDdQs/eEBLou0z5ascoGdcPKYEZ7I0bu3bcCW1btyZ7X0659/vb/HlE1a/PFzt2krV5K1evXgWgR1I3tqxbT6OGDRkzaSLvvPcetWvX5osdmfTq3p35/1pg3QhDlMPRkEGgbqHnaKhI03xXETOQh7GEywIz2GelQaka5FSpoh93U++evbhy5Sr/WvS29die7Gw2b9ta4n7nL1zg5gYNqFChAv9ettRa5fH4t99SPzKS+8bfw4Qx49j7xT7OnD1DUVERw4bcwfPPPMveL/a53b6QUd5UQDNsoefqWLWjoSJfDiGZoQ3CZwIz2EeMhL/PgyZN9J5bkyb6bTfH1wGUUnz43lI2ZHzObzu0p21cDGkvzqVhg5tL3G/yxPtY/N4SEnonc+Trr607EG3aupVOSYlEd+tK+kermPLgZH44cZJegwbQKSmR8Q/ez4uznrM5757sbBq3asn7qz7k/ilTaBsX4qsNzT5k4M4cdDMMIZmhDcJnpGyv8JihZXuT4+xfXFRK76X7m7slec1QetgMbRAecbZsb2DOihHBy+wzX9wdq3alKJi3mKENwicCcyhGBC+zDxnIWLUIABLswlzMPvPF7B88QiBDMcKMzDxk4OomGkL4gUfBrpR6GbgduAp8A9yjaZosrxTBzcwfPELg+VDMBqCdpmkdgCPAdM+bJIQQwhMeBbumaf/RNM0yz20n4P7ST3ccOgjxsfpPA8x5+SXaxsXQITGeTkmJ7Nqzx5Dndcer8+eTn59vczxt7hymp80qcWxfbi6tYzq7fI59ubmsXb/e7TYKIczJyIun9wK+KxeXlwfDh8HhwzB8uH7bA5m7drFm3Tr2bt1ObuYuNn70MU08KFHgicLCQl59fT75l22DPTUlheUfpJc4tix9pVtlf/d9mcva/7gW7NevGzRfXQjhNeUGu1Jqo1Jqv53/hhS7zwzgOrCkjOeZpJTKUkplnT7/s+ctn/wgnD6tL2Y5/RM85NmshJOnfqRe3bpUrlwZgHp169HwZn3VafN2bThz9gwAWXv30mtgf0DvPY+5byK9bxvI7zt1tJYj2LR1Cz3692PoXaNoE9uFBx57lKKiIgCWvr+C9glxtIuPZdrMZ63nr35zFDNfeJ745F7MefklTpw8SfKggSQPKjmW+4fft+SmWrVKfJtY8eEHjBo2HID/fPYZiX1607l7Eilj7+bSpUuAvsK16y196Ng1gbhePTl//jwz57zA8g/S6ZSUyPL0lZw7d447UkfRITGehN7J5O7fb32dkx59mH5DBjN20n0evc9CCO8r9+Kppmm3lPV7pdQ44Dagj1bGMlZN0xYAC0BfeepiO0v6979h/ToouLH0vKAA1n2qHx8zxq2n7Ne7D7P//CdaRnfill69GDlsGD27dS/3cbkH9rPzswzy8vOI7pbEoFv10N+dnc3B3Vk0a9qU/nfewQcfraZrfALTZs0ke8tWat9Um353DGbVmo+547bbycvLo12bNsx+Rg/7he/+m4xP1lKvbj2bc6YOT2FZ+kriY2PZuXs3dWvX4fe/+x1nzp7hhZf/zMaPPqZatWr8+W9/5a/z/s5T/++PjLxnHMvfXkxsly5cuHCBqlWrMnvGM2Tt3cu8v/wVgEee+CPRHTqwaukyPt+8ibH338e+7ZkAZO/bx7b1G6hSukaPEMJ0PBqKUUr1B6YBgzVNsx038Ja0mVB6/Dk/Xz/upurVq5O9ZRsLXnuNyHr1GDl+HIuWvFvu44YMHESVKlWoV7ceyd17sDtb3yszrksXftOiBWFhYaQOT2FbZiZ79mbTq1t3IutFUrFiRUaPGMmW7dsBCAsLY9iQO5xq66hhw1m5ehVFRUUsS19JakoKADt37+Hg4cMk9buFTkmJLH5vCce/+46vjh7h5qgGxHbpAkDNmjWpWNH2M33bzkzGjEoF9KJoZ8+d4/z58wAMHjBQQl2IAOHpPPZ5QGVgg9J3hdmpadoDHreqPGmz4ck/lgz3qlXhudkePW1YWBi9uvegV/cetG/blsXvvcf40XdTMayidSiloFSBKlVqNxzLbXvHy6rLExERQVhYmFPtbNK4Mc2bNmXztq2kf7SazI2fA6Ch0Te5N0vfXlTi/rn799u0xx577bM8zlLsTAhhfp7OivmdpmlNNE3rdOM/74c66MMtt/aHiBsrACMioP8AuNu9YRiAr44e4ejXX1tv78vNpVmTJgA0b9aU7By95G766tUlHrd67ScUFBRw9uxZNm3bSmxnvVe8Ozub/x07RlFREcvT0+mWmEh8TCybt2/jzNkzFBYWsnTl+/Ts1s1ue2pUr87Fi5cctjd1eAqPT3+K37ZoQeNGjQBIiI1l+66dfP3NNwDk5+dz5OhRWrVsyYkfT7InOxuAixcvcv36df0cl349R4+uSSy5sb3gpq1bqFe3rmwOIkQACtySAv94HSIj9WXnkfVhvmdlXS9dymPcA/fTJrYLHRLjOfjVYdKefhqAWU9NZ8q0qXS/ta9NrzquSxcGpQwjoU9vnp06zXrBNTEujqfSZtIuPpYWzZsx9PbB3NygAS/Oeo7kQQPp2DWBzh07MWTQbXbbM2n8PQwYNtTm4qlFytChHDh0yHrRFCCyXiSLXn+D1Hvv0S+A9knm8NEjVKpUieVvL+aRJ5+gY9cE+g4ZTEFBAcnde3Dw8GHrxdO06U+TlZNDh8R4npo1i8VvLPDoPRVC+Edgl+09dBDGj4NFi6F1GwNb6Jy0uXOoXr06Tzw6pcTxTVu38Mprr7Hm/ZU+b5M/GFq2VwjhUGiU7W3dBnb5bxGREEKYUWAHu5+lPT3D7nHLBVghhPCHwB1jF0IIYZcEuxBCBBkJdiGECDIS7EIIEWQk2ItRNasz5r6J1tvXr18nskUzbksZXsajXDPx4Yc4ePiQU/c9dvw4jVu1tK56teiUlMjurCyXzz33lZddfowQIvAEZrC/+jfYsrnksS2b9eMeqFatGvsPHeTy5csAbPj8cxo1bOjSc5RX1vbNefNp06q1U8/VvFkzmjRqzNYd263HDh/5iosXLxEXU+5UVhtz//KKS/fXNM3mQ0UIYX6BGeydO8O4sb+G+5bN+u3Orm82UdqAvv34ZP06AJaufJ/U4SnW3+3OyqLrLX2I7taVrrf04aujRwBYtORdUsbeze0jUuh3x2CKioqY/PhjtI2L4baU4QwcdicrV30IQK+B/cnauxfQS/XOmJ1Gx64JJPRO5tRPp2zakzp8OMvSf13otGzlSlKH698gTp85zbC77yK2Zw9ie/Zg+069EuOlS5e458EHaJ8QR4fEeNJXr+KpWTO5fPkynZISGT3hXgD+Ou/vtIuPpV18LK/Onw/o3xJax3Rm8uOP0bl7Et99/73H76kQwrcCM9h79ITF7+hh/sLz+s/F7+jHPTRqmB6kBQUF5B7YT3yxnnGrli3Zsm49Odt2MHvGMzz9XJr1d5m7d7P4jX/y+Zq1fPDRao59+y1f7tzNm/Pmk7lnt91z5eXlkRAbxxc7dtIjKYl/LVpkc58Rdw5j1Zo11m8Cyz9IZ9SNYJ8ydSqPP/QwezZvIf3dJUx8+CEAnn/pz9SqWZMvd+4mN3MXvXv05E/PzaZKlSrs257JkrcWkp2Tw9vv/ptdn29i52cZ/Gvx2+R88QUAXx09ytjUu8jZtoNmTZt6/J4KIXwrcBco9egJEybCS3+GqdMMCXWADu3acezbb1m68n0G9ru1xO/OX7jAuAcmcfSbb1BKce3aNevv+ib3pk6dOgBsy8wkZehQKlSoQIOoKJK726/rXqlSJW7rr9eC6dKpExsyMmzu0yAqiratW/PZpk1E1a9PeHg47dq0BWDjpgwOfnXYet8LFy9y8eJFNm7KYNnCRdbjtWvXtnnebZk7GHrb7daqjXfePpitO7YzeOAgmjVtSkJcnDNvlxDChAI32Ldshrfe1EP9rTehRw/Dwn3wgIE8MWMGm9au5ey5c9bjz77wPMnde/Dhe8s4dvw4vYoV6KpWtar1z87W3wkPD7eWxQ0LC3M4Pm/ZWCOqfv0SQ0NFRUVkbvzcpk66pmnlluktq4nFX4sQIvAE5lCMZUx98TvwzLO/DsuUvqDqpnvHjGXmtGm0b9uuxPHzF85bL6aWtQlHt8RE0levpqioiFM/nWLT1m0etWfY4CGs/c96fRimWDXHfr37MG/BP6239+Xm2j3+88/6VoTh4eHWbxk9kpJY9cka8vPzycvL48M1H9O9a5JH7RRCmENgBvvevSXH1C1j7jcuSnqqcaNGTJn8kM3xqVMeZ3paGkl9b6GwsNDh44cNuYPGjRrSLj6W+6dMIT4mhlo1a7ndnptuuomE2DiiIuvTonlz6/HXXn6ZrJy9dEiMp01sF95Y+CYAzzw5lZ9/+YV28bF07JpAxtYtgF4KuENiPKMn3EvnTp0YP/pu4pJ7Et+7FxPHjie6Y0e32yiEMI/ALttrYpcuXaJ69eqcPXuWuORebN+wkQZRUf5ulldI2V4hfCM0yvaa2G0jhvPL+fNcvXqVZ6dNC9pQF0KYjwS7l2xau87fTRBChKjAHGMXQgjhkKmC3R/j/cIz8ncmhPmYJtgjCjXOnj8vQRFANE3/O4solL8zIczENGPsjfML+f7UT5w+c9rfTREuiCjUaJzveOqnEML3TBPs4Rq0yJOAEEIIT5lmKEYIIYQxJNiFECLISLALIUSQ8UtJAaXUaeC4z0/sW/WAM/5uhJfJawwOofAaITheZzNN0yLLu5Nfgj0UKKWynKnpEMjkNQaHUHiNEDqvE2QoRgghgo4EuxBCBBkJdu9Z4O8G+IC8xuAQCq8RQud1yhi7EEIEG+mxCyFEkJFg9xKl1MtKqcNKqVyl1IdKqZv83SZvUEqlKKUOKKWKlFJBNeNAKdVfKfWVUuprpdRT/m6P0ZRSC5VSPyml9vu7Ld6ilGqilMpQSh268e90ir/b5AsS7N6zAWinaVoH4Agw3c/t8Zb9wJ3AFn83xEhKqTBgPjAAaAOkKqXa+LdVhlsE9Pd3I7zsOvBHTdNaAwnAQ0H492hDgt1LNE37j6Zplo1AdwKN/dkeb9E07ZCmaV/5ux1eEAd8rWnafzVNuwosA4b4uU2G0jRtC3DO3+3wJk3TTmqatvfGny8Ch4BG/m2V90mw+8a9wKf+boRwSSPgu2K3vycEAiGYKaWaA9HALv+2xPtMU7Y3ECmlNgIN7PxqhqZpq2/cZwb618ElvmybkZx5nUFI2TkmU8gClFKqOpAOPKZp2gV/t8fbJNg9oGnaLWX9Xik1DrgN6KMF8LzS8l5nkPoeaFLsdmPghJ/aIjyglApHD/UlmqZ94O/2+IIMxXiJUqo/MA0YrGlavr/bI1y2B/i9UqqFUqoSMAr4yM9tEi5SSingLeCQpml/9Xd7fEWC3XvmATWADUqpfUqpN/zdIG9QSg1VSn0PJAKfKKXW+7tNRrhx4fthYD36BbcVmqYd8G+rjKWUWgpkAn9QSn2vlJrg7zZ5QRIwBuh94//DfUqpgf5ulLfJylMhhAgy0mMXQoggI8EuhBBBRoJdCCGCjAS7EEIEGQl2IYQIMhLsQggRZCTYhRAiyEiwCyFEkPn/pufDqf08eUIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "C=10\n",
    "beta,b=SMO_KKT(x_nonlin,label_nonlin,C)\n",
    "#hyperlpane/contour:\n",
    "f=set_f(beta,label_nonlin,x_nonlin,b)\n",
    "xmin,ymin = np.amin(x_nonlin,axis = 0)\n",
    "xmax,ymax = np.amax(x_nonlin,axis = 0)\n",
    "xx,yy = np.meshgrid(np.arange(xmin - 1, xmax + 1, 0.05),\n",
    "                    np.arange(ymin - 1, ymax + 1, 0.05))\n",
    "zz=np.apply_along_axis(f,1,np.array([xx.flatten(),yy.flatten()]).T).reshape(xx.shape)\n",
    "plt.contourf(xx,yy,zz,0, colors = ['cyan', 'pink'])\n",
    "for lab,col in zip([-1,1],['b','r']):\n",
    "    for indices, style, name in zip([beta==0, beta==C,\n",
    "                                      np.logical_and(beta>0,beta<C)],\n",
    "                                     ['o','d','x'],\n",
    "                                     [\"Class \"+str(lab),\"Support Vector\",\"Margin Vector\"]):\n",
    "        plt.plot(x_nonlin[np.logical_and(label_nonlin==lab, indices ),0],\n",
    "                 x_nonlin[np.logical_and(label_nonlin==lab, indices ),1],\n",
    "                 col+style, label=name)\n",
    "plt.legend(loc = \"best\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(b)** Transform the data by the feature map $\\phi : \\mathbb{R}^2 \\to \\mathbb{R}^3$ defined by\n",
    "$\\phi ( t ) : = (t_1 , t_2 , t_1^2 + t_2^2)$ .\n",
    "Fit an SVM for C = 10 to the transformed data. Plot the scattered data\n",
    "and the nonlinear separation curve in a 2d plot (i.e. in the same way as\n",
    "in (a))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "phi= lambda pnt : np.array([pnt[0],pnt[1],pnt[0]**2+pnt[1]**2])\n",
    "x_transf=np.apply_along_axis(phi,1,x_nonlin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd8VFX++P/XSQhdkBKCUhLdFSkBA6QSAwkIUhRECBCpKqKiLvpbBRGFiIJr+ax+XVhd1gJqaBIVC8IGDT2UBDDSUZdgQaobSqjJ/f1xMyFlJplyp928n48Hjzg3M/eeGfB9z7zPOe+jNE1DCCGEeQR4uwFCCCGMJYFdCCFMRgK7EEKYjAR2IYQwGQnsQghhMhLYhRDCZCSwCyGEyUhgF0IIk5HALoQQJlPDGxdt2vBaLaz59d64tBBC+K2cA3tPaJoWXNXzvBLYw5pfT/a8D7xxaSGE8FsqMSrPnudJKkYIIUxGArsQQpiMBHYhhDAZCexCCGEyEtiFEMJkJLALIYTJSGAXQgiTkcAuhBAmI4FdCCFMRgK7EEKYjAR2IYQwGQnsQghhMhLYhRDCZCSwCyGEyUhgF0IIk5HALoQQJiOBXQghTEYCuxBCmIwEdiGEMBkJ7EIIYTIS2IUQwmQksAshhMm4HNiVUq2UUplKqb1Kqd1KqUlGNEwIIYRzahhwjivAXzVN266UugbIUUplaJq2x4BzCyGEcJDLPXZN045omra9+L/PAHuBFq6eVwghhHMMzbErpcKAzsAWI88rhBDCfoYFdqVUfSAdeFzTtNNWfj9BKZWtlMo+nv+HUZcVQghRjiGBXSkVhB7U0zRN+8TaczRNm6dpWqSmaZHBDRsZcVkhhBBWGDErRgHvAns1Tfu7600SQgjhCiN67PHAaKCnUmpn8Z/+BpxXCCGEE1ye7qhp2gZAGdAWIYQQBpCVp0IIYTIS2IUQwmQksAthlIyvYfidkBSt/8z42tstEtWUBHYhjJDxNbw2G47+Dpqm/3xttneDu9xoqi0J7EIY4Z1/wsULZY9dvKAf9wZfvNEIj5HALoQRjh117Li7+dqNRniUBHYhjNAsxLHj7uZrNxrhURLYhTDC+IlQq3bZY7Vq68e9wdduNMKjJLALYYTe/aDvAAgo/l8qIEB/3Lufd9rjazca4VES2IUwQsbXsPIrKCrSHxcV6Y+9NVjZux88+QyENAel9J9PPuO9G43wKCN2UBJCVDZY6a1g2rufBPJqSnrswv/44vxsewcrfbHtwnSkxy78i2V+tqV3bJmfDd7tnTYL0dti7biFr7ZdmI702IV/8dX52fYMVnq77fJtodqQHrvwL746P9vS437nn3pbmoXoQb10T9ybbZdvC9WKBHbhX+xJeXhLVYOV3my7Lw7uCreRVIzwL/48P9ubbffVbzrCLaTHLvyLPSkPX+XNtvvyNx1hOAnswv/48/xsb7V9/MSyOXbwn286wmES2IWoDvz5m45wmAR2IaoLf/6mIxwig6dCCGEyEtiFucmiHFENSWCvLswa4Cp7X7I9nKimJLBXB2YNcFW9L28v4XcXs96khWEksFcHZg1wVb0vMy7KMetNWhhKAnt1YMYAB1W/LzNuD2fWm7QwlAT26sCMAQ6qfl/+XH7AFrPepIWhJLBXB2YMcFD1+zLj9nBmvUkLQ8kCperArKsO7XlfZluUI6UBhB0MCexKqfeAO4BjmqaFG3FOYTCzBTgLs74vW9x1k8742nw3/mrMqB77fGAO8IFB5xPCvBZ9AG3bQ+fIq8d2ZMO+PZAypurXG30zk004TMeQHLumaeuAU0acSwjTa9seUqfqwRz0n6lT9ePeIDNtTEdy7EJ4WudISH1JD+aDhsDydP1x6R68J8lMG9Px2KwYpdQEpVS2Uir7eP4fnrqs8AfVcSVl50g9qH/wrv7TW0EdZKaNCXkssGuaNk/TtEhN0yKDGzby1GWFr7O2kvJvM2HgbXqgH9wHZj7j7VYab0e23lMfc7/+05KW8QazToetxmQeu/Aua/ndwitwOl8P9H/8AevXmKsXb8mpp74E9z10NS3jreBuxvn+1ZxR0x0XAYlAU6XUL8AMTdPeNeLcwuTsyeNevqzfAMwSaPbtKZtTt+Tc9+3xXkqmuk0bNTlDArumaSlGnEdUQ7Y2WS7PRwby0jIaM+2dFhw+VpPWzS4xa/yvjOzt4IQwa1MaO0d6N88uTEVmxQjvsraS0hovDOSpHuUCbRrwOlCgP8w7WotRr9/IqHY3wkj7zqmtrZhueWVRc6LaniOp85mSY5k7rmHbvnpMTrHjpidEORLYhXeVXkl59HdQikIVQGBRYclTCgNqEujGgbwKAdyWaZQE9RIFxcdtBfa04t8fBlqDmhVZ8blF0HTwZZam/kRS5zNk7riGYak3sjT1J7vfgxClSWAX3mfJ7y76gNVnY0hbXMQMnqM1hzlGM9ZpiVxmJCMNWgNndyAv77CDx9OACVy9GeQVP4aywT0JTnwaRM9hN8PDwFvwber+Mj14IRyhNE3z+EUjb26vZc+T6gPVViV1ScKGdyTvaK0KLwkNucihJd87fUmng3lpYejBubxQ4JABz58OvAA8B8y8etha+kZUTyoxKkfTtCr/Mct0R+FZVewAdPhYTasvs3W8MqpHZMkfQ8wC6pY7Vrf4uDWO9PAzgbfQg/pbxY+LqR6RqC2RqKKy7yNzxzW8sqh5lc0W1Y8EduFZVdQlad3sktWX2TpenuHBvLSRwDz0Hrcq/jkP2/n11nYezwSGAUvRe+pLix+XCu5E6cdUUfH7K4pkWOqNRLU959RbEeYmgd1fmGXZfRV1SWaN/5W6tQrL/KpurUJmjf+10tO6LZiXNxI9jVJU/LOy2TD29vC3oQfzpOLHScWPt5V6juXYMPSUzbDivPzjNzv8FoT5yeCpPzBTWVVb89aLpzNa5oTbO1fcI8HcWZagX2pWDLOoeDOYbOW1SVwN9KWPPczVPHzx7y2fgeTihYUMnvqD4XdaD4YhzWHJF55vjyvK36RAr0vi4BJ2hwJ6uSmHVoOrP7CkbIpnzpTp5ZciAd687B08lR67PzBTWVUXdwByuIdu75RDX1c6D2/pzZd+XIr04IUEdn9QRfrC7zhZl8SptIszi4o84RX0AdHSQTkTPa9uLTVTWR7eSq8d9M9Lgnv1JIOn/sDkZVXTMhoTNrwjAUldCRvekbSMxmV+79LAqD1TDtPQ55wHFP9Mc+5SDime5VIy88XSI4+y8fzJWM+5W7sJlOKxQWXhUySw+wMTl1VNy2jMhNdCyTtaC01T5B2txYTXQkuCu9WgVDoQNy3+YysoVzXl0JKqyQM0rqZqJuLeYG9lloutnLkRJLhXLzJ4KryqspWmeb9XPF4hZ15eXcrOLbf2/NLPCcP66lCFHuhtndcoNlabuoukZvybrDz1FWaZf+4mtlaU5h2zEtTBes68NEv+3KKqRUW2UjXl+zvlz2uPqlI8law2dRfpuVcPEtjdqYrl8xL0K1lRWj6FYgmS1nrX5ZUP1pUtKrKVqrHnvJWxleKxBHd7Vpu6icN590UfVNzdaUe2flz4JAns7lTZ8vmqgn41YW2laYXVmaWDpD0cCdbWVocqA85b2WwcsG+1qZvZHdzbti+7dZ9la7+27d3XOOESybG7U1K0HrTLU8r2FEZ/XHTkorSMxoz66EbbC4jCsD+oO5MLL7+AqT+wANt5eXsEUDGdA/pNo8iBtnmAXXl3SzAfNETffLv01n7CYyTH7gtszTNvFuL4oiMTp21GvXhj5fVXKkuBNCn+Y09RLlvKp2r+iWPFvqyxtwCYD7Cr5945Ug/qH7yr/5Sg7tMksLtTZfPPKwv65VX3tE1jG8dDgRPFf+wpyuUIR4p9WeNoiV9ftyNb76mPuV//WT7nLnyKBHZ3qmz+uSOLjqoodevPquwtpgGnrRyviW8HSUdL/HpZpX8PljRM6ktw30P6z9I5d+FzJMfuTZXsJFRGZbn6zK3ub6eL0jIaW63WaFcKIAzr+fUm6D11YSir+fZFH+gDpaXTLzuyYd8eSBnjucYJu3PsEtj9gTeqO9p706mCZWVpwcXAkmN1axVS8G6gfb1XPxqENAtZxOS7ZPDUTDxdK8bAnP60d1qUCeqA/tjexT5+NAgphK+QwO4PPF0rxomcvqWQl0rsSo2eXVGJXYvLBdjYq9TexT7uGoR0pfCXN4qGefD6sjrV/0nZXn/hZKlbpzg4FbN8uqWwOEWSd7QWCs1qJsXuHre9uxA5orIa7VVdy9v13b19feEXpMcuKnJkKibW0y0WGgpVPrQ72uN2dephebZWhU6i8jIAlb3W0ToyzvLQ9aXX7t8ksIuKHMzp2yrkZaGhV2tUSjN22p+zKQlbaaCTVB007anv7k62rpOH51NCwmdJYK/urK1odTCnb7OQV7HQkEscWvI9WpEybhFRVUW2KuPowGvpYGrUYK6zN6XKrmPv+wd9B6fyBccyi48Xk167/5LAXp1VNvuldz99KmXmVv1nJfl9q4W8itWtVcis8b8a33ZXUhK2BmSb2Hh+6WBqxGCuKzcla9e3cCQl4+gOTsKvSGCvzgxa0Tqy9ynmPZlHaMhFQCMwQAM0QkMuMu/JPPsXIznClZSIrVWh/4+qg7YRK0pduSlZrm+LvSkhO3dwkl67fzJkVoxSqi/6/xaBwDuapv3NiPMKN3O0EFklRvY+xcjep1xskANaY31FqiOzbWwF46pm4FT2Wnu4mqcfid5GV94/6EH8Ya7u4OSmbfmE57ncY1dKBQJzgX5AeyBFKSWFmv2Bg7NffIq75rcbPQPHGiPy9Ea8fy/s4CQ8w4hUTDTwg6ZpP2madglYDAwy4LzC3Ty9otVIlpRE6bx4HS+1xVFGBGVXU0Je3MFJuJ8Rgb0F8HOpx78UHxO+zkMrWt2apz1f6r9P4tjMEG8xqvJj6W8Xs9DTM/bOsnFgByfJs/sfI3Ls1jYSq7DYUCk1geI1cq1DmhtwWWEIT65oNVplg5C+vgrT1Tx9ac6sRp1s5VgSkmc3CSN67L8ArUo9bgn8Vv5JmqbN0zQtUtO0yOCGjQy4rDAVZ+Z1e3uxkK/w9mpY4XOMCOzbgJuUUjcopWoCI4DPDTivqC6cnddta7CxMd4t0uVpld3g7FiIJMzH5cCuadoV4FFgFbAXWKpp2m5Xz+u3TLw3qds42+O0NghZE33HJWcW//gDa99sKptlIwuRqiVDFihpmrZC07Q2mqb9SdM0X96wzL2q+96kVbGVbnE2pWJtEPIa4HK555klLWHrm01/bM+ysXMhkjAXWXlqJKP2JjVjr7+ydIsr87rLzzu3tUbKDHl3W99sVlD5LJvSC5EeRoJ6NSCB3UhGrOQ0a6+/snSLkYuNzLjjkuWbjrWVpqDftCpbWFVdFiKZsUPkJAnsRjJiJadRvX5fU1m6xah53eC+FaneUvqbji2V3bR8cSGSOwKwWTtETpLAbiQjVnIaWL/FV2hrs6vuSRu1lN/Im4QvsPZNp7SqbloOLESyxdDNrd0VgM3aIXKSBHYjubKS09KL0axuJOcf9Vsq48metCfqvXhKZWMD9ty0JlMxp56E9QVKnuCuAGzCDpErZM9TozmzktPSiyn/D97CX+q3lJOW0Zhp77TQd1hqDYxFH+gzau9Sf5GG83u22qpiGYp+0/I37grAzUL03r+149WQ9Nh9gbVejIUzvX4vDB6lZTQmbHhHApK6Eja8IxNfb8WE10LJO1oLTVN6cFqAHtTM0JO2lyubaoDtbzr9cesirKdeeYXEzHKJ+B3ZsOgD107sroqi/lzQzg0ksPsCW70VparcvaiEFweP0jIalwnieUdr8fbyZhU3uC5A77WbZbGQPVxd7m9tzGAs+k2y9M3i3uJzll4r0BxIca7Z26KiWDpsGNobb+sHdmRD6lRo62JFbncFYA8VtPMXSrOV03WjyJvba9nzyt75Lyv4pW4gFwKt1RQzuePHocjK1nIBgRAc7LlzOKF2ocagkW34/kA9+19UF/8e0HREAFZK4qEH6SInzxmG9fSMQl95e7HUsVrAuzj1WSdmZpI5+G4YNASWp0PqS9DZgEqPGV/r31KPHdV76uMnVtsA7CiVGJWjaVqVfwk+k2P/pW4g14Q0I6xhQ5SqZsG9WTP4/UjZgVOloPl10KChfecovGL7d3++ybX22aBpGifz8xn/yDEmTbrB/hf6SwVGI7i605M1tgZUNcoGdYofO/lZZwZcowf1D96FMfcbE9TBvyuK+gmfScVcCFQ0qY5BHfTg3fw6CArSHwcFORbULa9x5LgBlNL/ztrefN7GMyr5NuivK0HtrUJZelFR+X/Srs4GcvSm4OxnvSNb76mPuV//ucPAaY/CrXymxw5Uz6Bu0aChY4G8vKbB1nv9Td2XhtEvoWjcoJC6tQor5tStluov5o8rQe2te17+eRr6R6Gh58idmQ1UemZNY/SUy6VSv6+L/n/zaSuvdeKzTszM1HPqlvRL58iyj4VP85keuy/4/ehRRowby586daR9VFf6D7mbAwcPcigvj/AYz5TDmzYzlVbtbqb+dQ7OEjCi1++kerULmfdkHqEhF6m0l27hrytB7R0ItfY8S1A/hHNBvfTMmpPFP5twdUD18eJjtcq9thZOfdaZSz4tG8Q7R+qP9+1x/GTC4/w2sKctDSSsQx0CGtYlrEMd0paW7y06RtM0Bt+TQmJCAj/mfs+ebTnMnpHK0ePHDGqxfe7s25+tmWude3GDhnDjn+HmdvpPDwR1i5G9T3FoyffY/tKloZRGaMhFPnriJ//Mr9tbhdLoDUCs3SguA/W5OnW0IbAcfaDUMoMmBBiMw5+1tjYbUsZU7Jl3jtSPC5/nl4E9bWkgEx6rRd7PAfr0up8DmPBYLZeCe+a6tQQF1eCh+8eXHIvo1ImEbvFlnncoL4+E23vTJSGeLgnxbNqyGYAjv/9O9759iIiPIzwmivWbNlJYWMi4hx4kPCaKjrHRvD5nTpXtiI2O5rrm/rt1YOtml6weDw25RFFmDoeWfM/I3qVKMDqzc5K32FtgzOhCZLZuCKUHZS0rTEuvuv0dWOTkNYVf88vAPu35mhScL9s1LDivmPZ8TafPuWvPHrpGdK7yec2Cg8lY/gXb129kyfsL+MvkpwBY+PFSbu91Gzs3ZvHdps1EdOzEztxcfj3yG7u2bOP7zVu5d9Qop9vnL2aN/5W6tcpOu6xbq5BZ438tc0xbm+364h1Ps7csgtHlE2zdEBSGf1aG1oURXuOXgf3wL9a/79s6bqTLly/zwGOP0jE2muSxo9mzbx8AUV268P5HH5E6exbf797NNddcw41hYfz030M89uRfWZmRQYMGDdzePm8b2ftUSb7dknqZ92Re2V56sdC/XvTtvTrLf5sA+wqMGV2IbBa2t4w38LOSoG4efhnYW7e0PkBn67g9OrRrR87OHVU+7/W5cwhp1ozvNm0me+16Ll3SUw/d429l3cpVtLj+ekZPGM8HCxfSqFEjvtuURWJCAnP/PY/xjz5S5lyFhYVExMcRER/H9BdfcLrtvsaSb7eaeinl8DEb37B8YSqkrW8TYF+BMSMLkY3E9pi0L3xWwuf4ZWCfNeMSdeuU/Zdet47GrBnW87v26NkjkYsXL/Hv+e+XHNuWk8PaDevLPC//9Gmua96cgIAAPly8iMJCPe2Qd/gwzYKDeWDcvdw/eizbv9vJiZMnKCoqYsigu3jh2efY/t3OMucKDAxk58Ysdm7MYuazzznddqtO58NPP8D+vfrP0/nGnt8AtvLxPjEV0tVSAEYLtXHcoM9Keuvm4peBfeSwQub94yKhrYr0r/utipj3j4uMHGZlSb2dlFJ8unARGZnf8qdOHekQHUnqS7O5vvl1ZZ43cfwDLFiYRmzPJA788AP16ulL6desX09EfBydb+1G+uefMenhifz62xESB/QjIj6OcQ8/yEsznq+yHZOfe5aWbdtQUFBAy7ZtSJ3tRGL2dL4+p/1y8eafly/rj30suFvLx/vMVEijZ7a4yo1ljyWom4/P1IrZ26AG7dy09L3a+emHq0G9tKAgfRqkkU7ns3f3btr1vd2puh+lS/tqrZXvlPINw/fK5bpS/tcGCer+xe9qxQgDWQvqlR13luWbQVFh2YqSYHdwH9n7VJkcvOrhI6saZ1F29Sh4/9vESAy96UlQNy+/TMWIKniqbsyJ4xV3fHJhN5y0jMaENr/o3Jx2o+fDm22LvXIkqJub9NjNyFN1Y2x9A3BiNxxLTfeSejO26rBYfTH21XBxlME9ZF8gAb16kB67GXmqboytbwBO7IYz7Z0W1jfmsGcWiq/NYPFREtSrD+mxm5Wr1SLtYflmUJqTu+HYmtOuDmtolVWJBN+bweKDJKhXLxLYhfMsN46Tp/RUjwu74bRudom8o+VLE+rHD639HqhkYNUdm1mYhAT06smvUzG79yrCY+qwe68xpQRmvfoKHaIj6RQXQ0R8HFu2bTPkvM54Y+5cCgrK5xcgdfYspqbOKHNsZ24u7SK7OHyNnbm5rFi1yuk2AnpwDw6GzK32789qhT01ZmwGKTfO8fZnEtSrL78N7OfOQf+htdmzTzFgaG3OnXPtfFlbtvDlypVsX7+R3KwtrP78C1q1bGlMYx1UWFjIG2/NpeB8xcCekpzMkk/SyxxbnL6Me5KHOXydnd/nsuI/jgX2K1cq2YLPBfbWmNHWZlcMWCafweIoq5+RqFb8NrDfN7EWx44rNE1x9Lji/kcqfo13xJGjv9O0SRNq1dLP07RJU66/Tl91GhbenhMnTwCQvX07if37AnrvefQD4+l5R39uirilpBzBmvXr6N63D4PvGUH7qK489PhfKCrSdy5e9PFSOsZGEx4TxZTpV8sI1L8uhOkvvkBMUiKzXn2F344cIWlAf5IGlO0B33xTG65t2LDMt4mln37CiCFDAfjPN98Q16snXRLiSR4zirNnzwJ6eYRut/Xilm6xRCf2ID8/n+mzXmTJJ+lExMexJH0Zp06d4q6UEXSKiyG2ZxK5u3aVvM8Jf3mUPoMGMmbCAy59zpWxt8YMWAleRtZm8VMeCegZX8PwOyEpWv+Z8bV7ryec4lJgV0olK6V2K6WKlFIeW1ny3oc1+GpVIBcu6CmYCxcUX6wM5L0PnR8y6NOzFz//+gttOkcw8YnHK9SIsSV39y6++jidrG++YebLf+O3I/pg4tacHP5v1kt8v3krP/73v3zy+XJ+O3KEKTOm8+2XX7FzYxbbtufw2ZdfAHDu3DnC27dnS+Yapj89leuvu47Mr1aQ+VXF/3FShiazOH0ZAJu3bqVJo8bc9Oc/c+LkCV589WVWf66XFY7s3IW/z/kHly5dYvi9Y/l/L7/Cd5s2s/rzL6hXrx4zpz3L8LuHsHNjFsOHDGXG7Fl07tSJ3KwtzJ4xgzEPXg3iOTt3snzxEha+936F9niT9E49+BlkfK0vQDv6e9kFaRLcfY6rPfZdwN3AOgPaYrepqUGcKyhXj71AMTXV+QU49evXJ2fdBua9+SbBTZsyfNxY5qd9VOXrBvUfQJ06dWjapClJCd3ZmqP/DxbdtSs33nADgYGBpAxNZkOWHsgTb00guGkwNWrUYOSw4azbuBHQC4INGXSXXW0dMWQoy5Z/RlFREYvTl5GSnAzA5q3b2LNvH/F9biMiPo4FC9PI+/ln9h88wHUhzYnq2hWABg0aUKNGxZvghs1ZjB6RAuhF0U6eOkV+vl5fZmC//tSpU8eu9nlDdQzwHn/P7/xTX4BWmgsL0oT7uDQrRtO0veD5TahfSr3MX56qWSa4162r8bfnXVsyHxgYSGJCdxITutOxQwcWLFzIuJGjqBFYoySVcqHcP+zy793y2Nrxyury1K5dm8BA+3aAatWyJWGtW7N2w3rSP19O1upvAdDQ6J3Uk0Xvzy/z/Nxdu+z6O7LWPsvrLMXOfJ0l0PlMaQI38NoNzNbCMycWpAn38ssc+32jrzDg9kJq19YDUe3aGnf2LeTeUc4P7O0/eICDP/xQ8nhnbi6hrVoBEBbampwdesnd9OXLy7xu+YqvuHDhAidPnmTNhvVEddF7xVtzcvjvoUMUFRWxJD2dW+PiiImMYu3GDZw4eYLCwkIWLfuYHrfearU919Svz5kzZ222N2VoMk9MfZo/3XADLVu0ACA2KoqNWzbzw48/AlBQUMCBgwdp26YNv/1+hG05OQCcOXOGK1eu6Nc4e/Ua3bvFk7Z0CaCPEzRt0sRvNwex9GbN0ov3ifdja+GZEwvShHtVGdiVUquVUrus/BnkyIWUUhOUUtlKqezj+X843+Ji7/3zIs2C9Q2SQ4I13p170aXznT17jrEPPUj7qK50iothz/59pD7zDAAznp7KpCmTSbi9d4VedXTXrgxIHkJsr548N3lKyYBrXHQ0T6dOJzwmihvCQhl850Cua96cl2Y8T9KA/tzSLZYut0QwaMAdVtszYdy99BsyuMLgqUXy4MHs3ru3ZNAUILhpMPPfepuU++7VB0B7JbHv4AFq1qzJkvcX8NhTT3JLt1h6DxrIhQsXSErozp59+0oGT1OnPkP2jh10iovh6RkzWPD2PJc+U1/hE0HRCW5vt6MDoeMn6gvQSnNyQZpwL0PK9iql1gBPappm179Ao8r27t6rGD6uNkvmX6BDO8+XH06dPYv69evz5F8mlTm+Zv06XnvzTb78eJnH2+QNe384SLvT7pkG6W6+lLLx6I3HMhBaOrVYqzY8+UzlaxEyvtZz6seOurQgzfBzVRPVomxvh3Yau7ac93YzhB+zFUzdGfBtBvBFH0Db9tC51LV3ZMO+PZAyxthGVDYQWllw7d3PmOBb/sbiRMlnYZtLgV0pNRj4BxAMfKWU2qlp2u2GtMwPpD5jvcqUZQC22nGlB+ZjvTevpG3atofUqZD6kh7cd2RffWw0bw+EOntjEXZxdVbMp8CnBrVF+DNXemDSe9N1jtSDeOpUGDQElqdfDfJGaxaif87WjnuCt28sJueXs2KED3JljrPMj76qc6Qe1D94V//pjqAO3h8IlRk2biWBXRjDlR6Y9N6u2pGt99TH3K//3OGmlFDvfvpgUbriAAAYu0lEQVRAaUhzvTJnSPOqB06N5O0bi8n59eCp8BEXzuvBwdoMK3t6YN5OC/iK0jn1zpHFqZmp7kvHGDUQ6uy1wafGVcxEeuylqAb1Gf3A+JLHV65cIfiGUO5IHlrJqxwz/tFH2LNvr13PPZSXR8u2bUpWvVpExMexNdvxntzs1151+DVVOp0Pp09DuTYC9vfApPem27enbBC35Nz37fFuu9yldz+91LOLJZ9FRX4Z2F95I4jMdWWbnrkugFfecG2z5nr16rFr7x7On9enUGZ8+y0trr/eoXNUVdb2nTlzad+2nV3nCgsNpVWLlqzftLHk2L4D+zlz5izRkY734Gb/32sOPV/TtAo3lQqsbWgNEBBg/1d7b6cFfEXKmIo9886Rxk91FKbnl4E9qkshw8bWLgnumesCGDa2NlFdCqt4ZdX69e7DV6tWArBo2cekDE0u+d3W7Gy63daLzrd2o9ttvdh/8AAA89M+InnMKO4clkyfuwZSVFTExCcep0N0JHckD6X/kLtZ9pk+eSixf1+yt28H9FK902amcku3WGJ7JnHUSk45ZejQkkqOAIuXLSNlqP4N4viJ4wwZdQ9RPboT1aM7GzdnAXD27FnuffghOsZG0ykuhvTln/H0jOmcP3+eiPg4Rt5/HwB/n/MPwmOiCI+J4o25cwH9W0K7yC5MfOJxuiTE8/Mvv1T+gdna0FrTHAvM0nsTwjB+GdiTuhexdMEFho2tzfQXgxg2tjZLF1wgqXsVvUs7jBiiB9ILFy6Qu3sXMaV6xm3btGHdylXs2LCJmdOe5ZnnU0t+l7V1Kwve/hfffrmCTz5fzqHDh/l+81bemTOXrG1brV7r3LlzxEZF892mzXSPj+ff8+dXeM6wu4fw2ZdflnwTWPJJOiOKA/ukyZN54pFH2bZ2HekfpTH+0UcAeOGVl2nYoAHfb95KbtYWenbvwd+en0mdOnXYuTGLtHffI2fHDt7/6EO2fLuGzd9k8u8F77Pju+8A2H/wIGNS7mHHhk2Etq5ifzkDN7QWQhjDbwdPk7oX8fD9l3nhlZo8N/mSIUEdoFN4OIcOH2bRso/p36fsWqv806cZ+9AEDv74I0opLpfqrfZO6knjxo0B2JCVRfLgwQQEBNA8JISkhASr16pZsyZ39NV7pl0jIsjIzKzwnOYhIXRo145v1qwhpFkzgoKCCG/fAYDVazLZs39fyXNPnznDmTNnWL0mk8XvzS853qhRowrn3ZC1icF33FlStfHuOweyftNGBvYfQGjr1sRGR9vzcRVvaF1u4LM65sfL87EFV6J68dvAnrkugLfeDeK5yZd4690gkroXGhbcB/brz5PTprFmxQpOnrq6i89zL75AUkJ3Pl24mEN5eSSWKtBVr+7VTTftrb8TFBRUUhY3MDDQZn7esrFGSLNmZVJDRUVFZK3+tkKddE3TqizTW1kTS78XQB8gPXFcT7sEBenB3LKRdYOG0KCBnheXIKZzZsGV3AiEgfwyFWPJqS9dcIGZz14uScuUH1B11n2jxzB9yhQ6dggvczz/dH7JYGplm3DcGhdH+vLlFBUVcfTYUdas3+BSe4YMHMSK/6zS0zClqjn26dmLOfP+VfJ4Z26u1eN//KFX0wwKCir5ltE9Pp7PvvqSgoICzp07x6dffkFCt/iKFz+dD78fuZpLv3xZf3w6/+pzateR/Hhpji64kp2JhMH8MrBv2x5YJqduyblv227fRhVVadmiBZMmPlLh+ORJTzA1NZX43rdRWGh7oHbIoLto2eJ6wmOieHDSJGIiI2lo6eE64dprryU2KpqQ4GbcEBZWcvzNV18le8d2OsXF0D6qK2+/9w4Azz41mT/+9z/CY6K4pVssmev1Da4mjNPL+Y68/z66REQwbuQoopN6ENMzkfFjxtH5llsqXtzarBdN048L6xxdcCUrb4XBDCnb6yijyvb6srNnz1K/fn1OnjxJdFIiGzNW0zzEDwcU91cy5/5mfdqm28v2+luaYvid1hdchTTXv9GUlxRtPTemlP4tSIhi9pbt9cseuz+4Y9hQIuLjSOjbh+emTPHPoA62Z73YOm40T6QpHN1woiqOLriSuinCYH47eOrr1qxY6e0mGKNpsJ5TL92jVEo/7gnuLu/qjsqSji6XHz/R+qYX1X1mkXCaBHZROcvYgK1ZMe7m7gJh7rpxOFKHReqmCINJYBdVa9DQc4G8PHcXCPOVypLeLMglTEdy7MK3ubtAmLfy20bn9YUoRQK78G3lC4Q1aAi1asLsGd4Z6DSCzFsXbiaBvZTfjx5lxLix/KlTR9pHdaX/kLs5cPAgh/LyCI+Jcss1123cQJeEeGo0alhSKEyUYykQ9szzcPGiXibYqIDojcqSMm9duJn/BvalS6BDO2h4jf5z6RKXTqdpGoPvSSExIYEfc79nz7YcZs9I5ejxYwY12LrWLVsx/61/cU/yMLdexxTcFRA9XVnSV/L6wrT8M7AvXQKPPQo//6z33H7+WX/sQnDPXLeWoKAaPHT/1Y02Ijp1qrDM/lBeHgm396ZLQjxdEuLZtGUzAEd+/53uffsQER9HeEwU6zdtpLCwkHEPPUh4TBQdY6N5fc6cCtcNCw2lU3g4AQH++VfhUf4cEEvn1G3V8ZG8vjCIf86KeT4VijfDKHH+vH582HCnTrlrzx66RnSu8nnNgoPJWP4FtWvX5uAPP5By/71kr13Pwo+Xcnuv25j21GQKCwspKChgZ24uvx75jV1btgHwv//9z6m2iWL+uoVe+bny1laZeiqvb+R8feGz/LObaGvzh6o2hTDA5cuXeeCxR+kYG03y2NHs2aeXzY3q0oX3P/qI1Nmz+H73bq655hpuDAvjp/8e4rEn/8rKjAwaNGjg9vaZmr9uoWcthQT6LlOS1xdu4J+BvWVLx47boUO7duTs3FHl816fO4eQZs34btNmsteu59KlSwB0j7+VdStX0eL66xk9YTwfLFxIo0aN+G5TFokJCcz997ySjTBEFWylDPx1Cz1bqSJNk7y+cAv/DOwzUqFcDXLq1NGPO6lnj0QuXrzEv+e/X3JsW04OazesL/O8/NOnua55cwICAvhw8aKSKo95hw/TLDiYB8bdy/2jx7L9u52cOHmCoqIihgy6ixeefY7t3+10un3VRlVTAX1hCz1Hc9W+UAvGF9ogPMY/A/uw4fCPOdCqld5za9VKf+xkfh1AKcWnCxeRkfktf+rUkQ7RkaS+NJvrm19X5nkTxz/AgoVpxPZM4sAPP5TsQLRm/Xoi4uPofGs30j//jEkPT+TX346QOKAfEfFxjHv4QV6a8XyF627LyaFl2zZ8/NmnPDhpEh2iHd+k2lR8PWXgzBx0X0gh+UIbhMdI2V7hMkPL9vp6CVtHS/Ja+ELpYV9og3CJvWV7/XNWjDAvX5/54myu2hdqwfhCG4RH+GcqRpiXr6cMJFct/IAEduFbfH3mi6/feIRAUjHCF/lyykBqpws/4FJgV0q9CtwJXAJ+BO7VNE2WVwpz8+UbjxC4norJAMI1TesEHACmut4kIYQQrnApsGua9h9N0yzz3DYDzi/9dMbePRATpf80wKxXX6FDdCSd4mKIiI9jy7ZthpzXGW/MnUtBQUGF46mzZzE1dUaZYztzc2kX2cXha+zMzWXFqlVOt1EI4ZuMHDy9D/Bcubhz52DoENi3D4YO1R+7IGvLFr5cuZLt6zeSm7WF1Z9/QSsXShS4orCwkDfemkvB+YqBPSU5mSWfpJc5tjh9mVNlf3d+n8uK/zgW2K9cMWi+uhDCbaoM7Eqp1UqpXVb+DCr1nGnAFSCtkvNMUEplK6Wyj+f/4XrLJz4Mx4/ri1mOH4NHXJuVcOTo7zRt0oRatWoB0LRJU66/Tl91GhbenhMnTwCQvX07if37AnrvefQD4+l5R39uirilpBzBmvXr6N63D4PvGUH7qK489PhfKCoqAmDRx0vpGBtNeEwUU6Y/V3L9+teFMP3FF4hJSmTWq6/w25EjJA3oT9KAsrncm29qw7UNG5b5NrH0008YMWQoAP/55hvievWkS0I8yWNGcfbsWUBf4drttl7c0i2W6MQe5OfnM33Wiyz5JJ2I+DiWpC/j1KlT3JUygk5xMcT2TCJ3166S9znhL4/SZ9BAxkx4wKXPWQjhflUOnmqadltlv1dKjQXuAHpplSxj1TRtHjAP9JWnDrazrA8/hFUr4ULx0vMLF2Dl1/rx0aOdOmWfnr2Y+fLfaNM5gtsSExk+ZAg9bk2o8nW5u3ex+ZtMzhWco/Ot8Qy4XQ/6W3Ny2LM1m9DWrel791188vlyusXEMmXGdHLWrafRtY3oc9dAPvvyC+66407OnTtHePv2zHxWD/bvffQhmV+toGmTphWumTI0mcXpy4iJimLz1q00adSYm/78Z06cPMGLr77M6s+/oF69erz8+t/5+5x/8PT/91eG3zuWJe8vIKprV06fPk3dunWZOe1ZsrdvZ87//R2Ax578K507deKzRYv5du0axjz4ADs3ZgGQs3MnG1ZlUKd8jR4hhM9xKRWjlOoLTAEGappWMW/gLqnToXz+uaBAP+6k+vXrk7NuA/PefJPgpk0ZPm4s89M+qvJ1g/oPoE6dOjRt0pSkhO5szckGILprV2684QYCAwNJGZrMhqwstm3PIfHWBIKbBlOjRg1GDhvOuo0bAQgMDGTIoLvsauuIIUNZtvwzioqKWJy+jJTkZAA2b93Gnn37iO9zGxHxcSxYmEbezz+z/+ABrgtpTlTXrgA0aNCAGjUq3tM3bM5i9IgUQC+KdvLUKfLz8wEY2K+/BHUh/ISr89jnALWADKXvCrNZ07SHXG5VVVJnwlN/LRvc69aF52e6dNrAwEASE7qTmNCdjh06sGDhQsaNHEWNwBolqZQL5QpUqXK74VgeWzteWV2e2rVrExgYaFc7W7VsSVjr1qzdsJ70z5eTtfpbADQ0eif1ZNH788s8P3fXrgrtscZa+yyvsxQ7E0L4PldnxfxZ07RWmqZFFP9xf1AHPd1ye1+oXbwCsHZt6NsPRjmXhgHYf/AAB3/4oeTxztxcQlu1AiAstDU5O/SSu+nLl5d53fIVX3HhwgVOnjzJmg3rieqi94q35uTw30OHKCoqYkl6OrfGxRETGcXajRs4cfIEhYWFLFr2MT1uvdVqe66pX58zZ87abG/K0GSemPo0f7rhBlq2aAFAbFQUG7ds5ocffwSgoKCAAwcP0rZNG377/QjbcnIAOHPmDFeuXNGvcfbqNbp3iyeteHvBNevX0bRJE9kcRAg/5L8lBf75FgQH68vOg5vBXNfKup49e46xDz1I+6iudIqLYc/+faQ+8wwAM56eyqQpk0m4vXeFXnV0164MSB5CbK+ePDd5SsmAa1x0NE+nTic8JoobwkIZfOdArmvenJdmPE/SgP7c0i2WLrdEMGjAHVbbM2HcvfQbMrjC4KlF8uDB7N67t2TQFCC4aTDz33qblPvu1QdAeyWx7+ABatasyZL3F/DYU09yS7dYeg8ayIULF0hK6M6efftKBk9Tpz5D9o4ddIqL4ekZM1jw9jyXPlMhhHf4d9nevXtg3FiYvwDatTewhfZJnT2L+vXr8+RfJpU5vmb9Ol57802+/HiZx9vkDYaW7RVC2FQ9yva2aw9bvLeISAghfJF/B3YvS31mmtXjlgFYIYTwBv/NsQshhLBKArsQQpiMBHYhhDAZCexCCGEyEthLUQ3qM/qB8SWPr1y5QvANodyRPLSSVzlm/KOPsGffXrueeygvj5Zt25SserWIiI9ja3a2w9ee/dqrDr9GCOF//DOwv/E6rFtb9ti6tfpxF9SrV49de/dw/vx5ADK+/ZYW11/v0DmqKmv7zpy5tG/bzq5zhYWG0qpFS9Zv2lhybN+B/Zw5c5boyCqnslYw+/9ec+j5mqZVuKkIIXyffwb2Ll1g7JirwX3dWv1xF8c3myivX+8+fLVqJQCLln1MytDkkt9tzc6m22296HxrN7rd1ov9Bw8AMD/tI5LHjOLOYcn0uWsgRUVFTHzicTpER3JH8lD6D7mbZZ99CkBi/75kb98O6KV6p81M5ZZuscT2TOLosaMV2pMydCiL068udFq8bBkpQ/VvEMdPHGfIqHuI6tGdqB7d2bhZr8R49uxZ7n34ITrGRtMpLob05Z/x9IzpnD9/noj4OEbefx8Af5/zD8JjogiPieKNuXMB/VtCu8guTHzicbokxPPzL7+4/JkKITzLPwN79x6w4AM9mL/4gv5zwQf6cReNGKIH0gsXLpC7excxpXrGbdu0Yd3KVezYsImZ057lmedTS36XtXUrC97+F99+uYJPPl/OocOH+X7zVt6ZM5esbVutXuvcuXPERkXz3abNdI+P59/z51d4zrC7h/DZl1+WfBNY8kk6I4oD+6TJk3nikUfZtnYd6R+lMf7RRwB44ZWXadigAd9v3kpu1hZ6du/B356fSZ06ddi5MYu0d98jZ8cO3v/oQ7Z8u4bN32Ty7wXvs+O77wDYf/AgY1LuYceGTYS2bu3yZyqE8Cz/XaDUvQfcPx5eeRkmTzEkqAN0Cg/n0OHDLFr2Mf373F7md/mnTzP2oQkc/PFHlFJcvny55He9k3rSuHFjADZkZZE8eDABAQE0DwkhKcF6XfeaNWtyR1+9FkzXiAgyMjMrPKd5SAgd2rXjmzVrCGnWjKCgIMLbdwBg9ZpM9uzfV/Lc02fOcObMGVavyWTxe/NLjjdq1KjCeTdkbWLwHXeWVG28+86BrN+0kYH9BxDaujWx0dH2fFxCCB/kv4F93Vp49x09qL/7DnTvblhwH9ivP09Om8aaFSs4eepUyfHnXnyBpITufLpwMYfy8kgsVaCrXt26Jf9tb/2doKCgkrK4gYGBNvPzlo01Qpo1K5MaKioqImv1txXqpGuaVmWZ3sqaWPq9CCH8j3+mYiw59QUfwLPPXU3LlB9QddJ9o8cwfcoUOnYIL3M8/3R+yWBqZZtw3BoXR/ry5RQVFXH02FHWrN/gUnuGDBzEiv+s0tMwpao59unZiznz/lXyeGdurtXjf/yhb0UYFBRU8i2je3w8n331JQUFBZw7d45Pv/yChG7xLrVTCOEb/DOwb99eNqduybkXD0q6qmWLFkya+EiF45MnPcHU1FTie99GYWGhzdcPGXQXLVtcT3hMFA9OmkRMZCQNGzR0uj3XXnstsVHRhAQ344awsJLjb776Ktk7ttMpLob2UV15+713AHj2qcn88b//ER4TxS3dYslcvw7QSwF3ioth5P330SUignEjRxGd1IOYnomMHzOOzrfc4nQbhRC+w7/L9vqws2fPUr9+fU6ePEl0UiIbM1bTPCTE281yCynbK4RnVI+yvT7sjmFD+V9+PpcuXeK5KVNMG9SFEL5HArubrFmx0ttNEEJUU/6ZYxdCCGGTTwV2b+T7hWvk70wI3+Mzgb12ocbJ/HwJFH5E0/S/s9qF8ncmhC/xmRx7y4JCfjl6jOMnjnu7KcIBtQs1WhbYnvophPA8nwnsQRrccE4ChBBCuMpnUjFCCCGMIYFdCCFMRgK7EEKYjFdKCiiljgN5Hr+wZzUFTni7EW4m79EcqsN7BHO8z1BN04KrepJXAnt1oJTKtqemgz+T92gO1eE9QvV5nyCpGCGEMB0J7EIIYTIS2N1nnrcb4AHyHs2hOrxHqD7vU3LsQghhNtJjF0IIk5HA7iZKqVeVUvuUUrlKqU+VUtd6u03uoJRKVkrtVkoVKaVMNeNAKdVXKbVfKfWDUuppb7fHaEqp95RSx5RSu7zdFndRSrVSSmUqpfYW/zud5O02eYIEdvfJAMI1TesEHACmerk97rILuBtY5+2GGEkpFQjMBfoB7YEUpVR777bKcPOBvt5uhJtdAf6qaVo7IBZ4xIR/jxVIYHcTTdP+o2maZSPQzUBLb7bHXTRN26tp2n5vt8MNooEfNE37SdO0S8BiYJCX22QoTdPWAae83Q530jTtiKZp24v/+wywF2jh3Va5nwR2z7gP+NrbjRAOaQH8XOrxL1SDgGBmSqkwoDOwxbstcT+fKdvrj5RSq4HmVn41TdO05cXPmYb+dTDNk20zkj3v04SUlWMyhcxPKaXqA+nA45qmnfZ2e9xNArsLNE27rbLfK6XGAncAvTQ/nlda1fs0qV+AVqUetwR+81JbhAuUUkHoQT1N07RPvN0eT5BUjJsopfoCU4CBmqYVeLs9wmHbgJuUUjcopWoCI4DPvdwm4SCllALeBfZqmvZ3b7fHUySwu88c4BogQym1Uyn1trcb5A5KqcFKqV+AOOArpdQqb7fJCMUD348Cq9AH3JZqmrbbu60yllJqEZAF3KyU+kUpdb+32+QG8cBooGfx/4c7lVL9vd0od5OVp0IIYTLSYxdCCJORwC6EECYjgV0IIUxGArsQQpiMBHYhhDAZCexCCGEyEtiFEMJkJLALIYTJ/P/dhiU14MzCgwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "C=10\n",
    "beta,b=SMO_KKT(x_transf,label_nonlin,C)\n",
    "#hyperlpane/contour:\n",
    "f=set_f(beta,label_nonlin,x_transf,b)\n",
    "xmin,ymin = np.amin(x_transf,axis = 0)[0:2]\n",
    "xmax,ymax = np.amax(x_transf,axis = 0)[0:2]\n",
    "xx,yy = np.meshgrid(np.arange(xmin - 1, xmax + 1, 0.05),\n",
    "                    np.arange(ymin - 1, ymax + 1, 0.05))\n",
    "zz=np.apply_along_axis(f,1,np.apply_along_axis(phi,1,np.array([xx.flatten(),yy.flatten()]).T)).reshape(xx.shape)\n",
    "plt.contourf(xx,yy,zz,0, colors = ['cyan', 'pink'])\n",
    "for lab,col in zip([-1,1],['b','r']):\n",
    "    for indices, style, name in zip([beta==0, beta==C,\n",
    "                                      np.logical_and(beta>0,beta<C)],\n",
    "                                     ['o','d','x'],\n",
    "                                     [\"Class \"+str(lab),\"Support Vector\",\"Margin Vector\"]):\n",
    "        plt.plot(x_transf[np.logical_and(label_nonlin==lab, indices ),0],\n",
    "                 x_transf[np.logical_and(label_nonlin==lab, indices ),1],\n",
    "                 col+style, label=name)\n",
    "plt.legend(loc = \"best\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What does the feature map do and why does it work so well? - It adds the radius^2 as a dimension and it works well, because the original classification only depends on the radius."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task 2.7.** Change your SMO code and your function evaluation of f from\n",
    "(2.2) such that it allows to use a kernel function instead of the scalar product\n",
    "of the input data, i.e. substitute all scalar products by the evaluation of the\n",
    "kernel function. Perform a SVM classification (C = 10) with Gaussian kernel\n",
    "(σ = 1) for the data from task 2.6. Plot the scattered data and the nonlinear\n",
    "separation curve in a 2d plot."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$f(t)=\\sum_{i=1}^n \\beta_i y_i K(t,x_i) +b$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def aux1(vec, kernel):\n",
    "    aux11 = lambda row :kernel(vec, row)\n",
    "    return aux11\n",
    "\n",
    "def set_f(beta,y,x_train,b,kernel):\n",
    "    def f(vec):\n",
    "        # prod=beta*y*kernel(vec,x_train) \n",
    "        prod = beta*y*np.apply_along_axis(aux1(vec, kernel), arr = x_train, axis = 1) \n",
    "        return prod.sum()+b\n",
    "    return f"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def OneStep(i,j,beta,y,x,C,b,kernel):\n",
    "    f=set_f(beta,y,x,b,kernel)\n",
    "    delta=y[i]*(f(x[j])-y[j]-f(x[i])+y[i])\n",
    "    s=y[i]*y[j]\n",
    "    chi= kernel(x[i],x[i]) + kernel(x[j], x[j]) - 2 * kernel(x[i], x[j])\n",
    "    #chi= x[i].dot(x[i])+x[j].dot(x[j])-2*x[i].dot(x[j]) # change all dot products to kernel evaluation \n",
    "    gamma=s*beta[i]+beta[j]\n",
    "    if s==1:\n",
    "        L=max(0,gamma-C)\n",
    "        H=min(gamma,C)\n",
    "    else:\n",
    "        L=max(0,-gamma)\n",
    "        H=min(C-gamma,C)\n",
    "    if chi>0:\n",
    "        beta[i]=min(max(beta[i]+delta/chi,L),H)\n",
    "    else:\n",
    "        if delta>0:\n",
    "            beta[i]=L\n",
    "        else:\n",
    "            beta[i]=H\n",
    "    beta[j]=gamma-s*beta[i]\n",
    "    f=set_f(beta,y,x,b,kernel)\n",
    "    b=b-(f(x[i])-y[i]+f(x[j])-y[j])/2\n",
    "    \n",
    "    return beta, b\n",
    "\n",
    "def SMO_KKT(x,y,C,kernel,steps=10000):\n",
    "    n=x.shape[0]\n",
    "    beta=np.zeros(n)\n",
    "    b=0\n",
    "    \n",
    "    count=0\n",
    "    for i in range(n):\n",
    "        f=set_f(beta,y,x,b,kernel)\n",
    "        KKT=set_KKT(C,beta,y,f,x)\n",
    "        if KKT(i)>0:\n",
    "            possible_j=np.where(np.logical_and(np.logical_and(0<beta,beta<C),\n",
    "                                               range(n)!=i))[0]\n",
    "            if len(possible_j)>0:\n",
    "                j=possible_j[np.random.randint(len(possible_j))]\n",
    "            else:\n",
    "                j=np.random.randint(n)\n",
    "                while i==j:\n",
    "                    j=np.random.randint(n)\n",
    "        \n",
    "            beta, b=OneStep(i,j,beta,y,x,C,b,kernel)\n",
    "            count+=1\n",
    "            if count>=steps:\n",
    "                break\n",
    "    \n",
    "\n",
    "    f=set_f(beta,y,x,b,kernel)\n",
    "    fval=np.apply_along_axis(f,1,x)\n",
    "    if np.count_nonzero(beta) > 0: \n",
    "        med=np.median((fval-y)[beta>0])\n",
    "        b=b-med\n",
    "    return beta, b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "gaussian = lambda t, x : math.exp(-np.linalg.norm(t-x)**2/2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd4VGX+///nTQiEIoiQBKnBXZFugFRCQgKC9CJNRIqAiFhYv7IgIhBRcC3r+nNhcVlFUJEiEbEgbNTQa0IQEULRD2Chs1ISQknO74+TCZlkJplypub9uC6vYU4mc+4Z8DX3vM9dlKZpCCGE8B8VPN0AIYQQxpJgF0IIPyPBLoQQfkaCXQgh/IwEuxBC+BkJdiGE8DMS7EII4Wck2IUQws9IsAshhJ+p6ImT1ql5uxZWt54nTi2EED4r4/DBc5qmBZf1OI8Ee1jdeqQv/MATpxZCCJ+lEiOP2/I4KcUIIYSfkWAXQgg/I8EuhBB+RoJdCCH8jAS7EEL4GQl2IYTwMxLsQgjhZyTYhRDCz0iwCyGEn5FgF0IIPyPBLoQQfkaCXQgh/IwEuxBC+BkJdiGE8DMS7EII4Wck2IUQws9IsAshhJ+RYBdCCD8jwS6EEH5Ggl0IIfyMBLsQQvgZCXYhhPAzTge7UqqhUipNKXVQKfWjUmqSEQ0TQgjhmIoGPMdN4FlN0/YopW4DMpRSqZqmHTDguYUQQtjJ6R67pmknNU3bU/Dny8BBoL6zzyuEEMIxhtbYlVJhQFtgp5HPK4QQwnaGBbtSqjqQAvxF07RLFn4+XimVrpRKP3vxf0adVgghRDGGBLtSKhA91JdqmvappcdomrZQ07QITdMigmvWMuK0QgghLDBiVIwC3gMOapr2pvNNEkII4QwjeuxxwAigs1Jqb8F/PQ14XiGEEA5werijpmlbAGVAW4QQQhhAZp4KIYSfkWAXQgg/I8EuhBGWfQCZ6ebHMtP140K4mQS7EEZo1gKSp90K98x0/X6zFp5rU+rXMLQPJEXpt6lfe64twq2MWCtGCNE2ApJf0cO830BYk6Lfbxvhmfakfg1vzIVrufr906f0+wBde3imTcJtpMcuhFHaRuih/sF7+q2nQh3g3X/dCnWTa7n6ceH3JNiFMEpmut5THzlWvy1ec3enM6ftOy78igS7EEYw1dSTX4ExE26VZTwV7iGh9h0XfkWCXQgjZB2A3v3hby/qFyv/9qJ+P8tD2xKMmwiVg8yPVQ7Sjwu/J8EuhBHqBMOq5fpFSk3Tb1ct1497QtceMPl5CK0LSum3k5+XC6flhIyKEcIIpV2s9FSYdu0hQV5OSY9d+B5vHJ9ty8VKmcQk3ESCXfgW0/jsoiWPN+Z6PtxtuVjpjZOYhF+SYBe+xVvHZ9tysbLoJKZF79waReOu8e7e+E1HuITU2IVv8dbx2aZa9rv/0tsSEqqHevEad9FJTCPHujfUZSZquSHBLnxLSKgeSpaOe5otFyuLT2JqG+GecPfGi7vCZaQUI3yLL4/P9uQkJm/9piNcQoJd+BZfHp+ddcC8pm6qubtjEpPMRC1XpBQjfI+vjs8eNrLkMXeVYsZNNK+xg+980xF2k2AXojyw9eKu8AsS7EKUF776TUfYTWrsQgjhZyTYhf+SKfyinJJgLy/8ddZhaa9LpvCLckpq7OWBv846LOt1eds+pEZY9oH+wVT0NWSm60MmLY26EeWS9NjLA29dX8VZtrwub9qH1AjyLUTYQHrs5YG/zjq05XV5agq/q/jjtxBhOOmxlwf+OuuwrNflbfuQGsXfvoUIw0mwlwe+vL5Kacp6XZ6cwu9Kxb+F+PoHlTCclGLKA3+ddVjW6xo2EtWpWG+2+P1SaBu9MDCLfgsxlZXcva678HpK0zTnn0SpRUBv4Iymaa3KenzEPS209IUyllgYp0SAu5jHQt8Vo2JkpI3PUImRGZqmlfmP3age+2JgHiBpLdzC3UFe1vntCnpngtQVC4mZRtqYev1FvxUIn2RIsGuatkkpFWbEcwlhjafDvDR2Bb23BamMtPE7UmMXXs2bw7w0RdtdIuS9MUg9tWWfcAm3BbtSajwwHqBRaF13nVb4gtSvS1wAVS/P8HSrDGMx5L0tSP1tvH8557bhjpqmLdQ0LULTtIjgmrXcdVrh7UzLApw+BZoGp09x9Y05DFu6FIDEtDTeeewx/vraax5uqDFUpwhUpwiS8i97z5BFfx3vX45JKUZ4loVlAapcu8abzz7LyXr1WN2/PyjFgNWrPdRA4yWmpbFyyBCSVn/KhqQktLc8PGSxtPH+0mv3SYb02JVSy4DtwD1KqV+VUmONeF5RDlhZFiD09Gm+7NWrMNQ3JCW5uWGuE7l7N0NWrix8TeovE0ha/SlTrl/yTIOGjSwZ4G0jZKijDzNkHLu9ZBy7MDk2ahBhx49b/fnsGTOYNXu2G1tUiteASKDoZ0wasBuYYuypvHJylPA4W8exy5ICwmNUpwienzOH7KpVzY5rwLVKlciuUoWn336bxLQ0zzSwuEhgCHqYU3A7pOB4aX4EWhXcWvJakecsoPIjUDulDCIcI8EuPMI0UmTZ8OE8unAhxxo3Jh/IQ5FTuQrd162j91dfgaaxus8AEh/zgnBPAlaih/nMgtuVmPfgi8sGegIHgF4F94sr5QPDV4d7Cs+SYBduZRoVUtSy4cNpcuwYz736Ku/2epTelb9iA0lsSEpiwKzPWHFzKJHs9lCLi0kCHgdeKrgtq/Q/BjiD/jXkNGDp6lMZHxiW3jMhSiM1duFWU3Z+x+7ISLOLoYlpaUTu3s3rUwoK1aYe6+PAAsruFbuTPW1bBDyNeS+9KvBP9MAvbib6B8YMoJTLClJ/L7+kxi68juoUwe7ISFYOGVJYNzcN/dsdWaRQbW+v2F1Mob4SPXhNvWxrVaJplCy95BQct/TcC9BDfYGV51wKhIFKiiBsaGuWpt5h7ysQ5YQEu3ALUylhQ1ISQ1auZOWQIbw4cyYrhwwxG/oH2BZynrAb8x66qYRirUr0ClCt2LGqwN+KHbPlA2Mp+rzt44AGx09X5uF/3CXhLiySYPcVqV/D0D6QFKXfpn7t6RbZrHh9eENSEgsef5yZL73EgscfLxnq9vSK3WkKJb89JGF9qOMY9Aumpr1AgoA+wCPFHmfLB8Z09N5+UTnw8Ed32dp6UY5IsPsCC9PueWOuT4S7pYt+iWlpPL5gAbNnzODxBQvMhzPa2yv2douAEEABocB7Fh5jywfGCSvPf0IuroqSJNh9gYVp91zL1Y/7GFNNfcjKlcyaPbuwLFMY7vb2im1RUJumQsHtUieey17VgLVAC+ArSpZmbNWo7OMS7sJEgt0XWJl2b/W4l7AUNMWn05tq7pG7XdQlL1ab5njBfXeGe0tgf8Gto+ag1+eLqlpwvAgJdwES7L4hJNS+417s9SlTSqz7siEp6dZQR6NZqU0z3TWns4s93ySGAwuBxuhlncYF94eXfKiEu5Bg9wXjJkLlIPNjlYP0417KrnBxZamklNq0W85vjSPfJIYDx4D8glsLoW4i4V6+SbD7gq49YPLzEFoXlNJvJz+vH/d19gZc8XVVlgJ10XuxYRZ+r6zatLXzT8S1Ye+GbxIS7uWXzDwVhrMrUMLQw7S4xui90uKKDof8HX2K/rUiP6+KeYnCFNw5Vh5j7fwKPeitPa+zKhR7/qLnzTfoHAVkpqr/kJmn3sKHx5+7hS2lkqKKrqvyBOahDiV7vWXVpq2dp3joOtKbLq3EY8MoF6NIz738kWB3pbLGn0vo2x5wRUPyEaAjcNHK7xYP69Jq0/YEqbUPAUvKKjHZOMrFKE6F+7IPSm6Tl5muHxdeSYLdlUobf+7Dk44MZUvAWQrJz4AqVp7TnrC2dH5lwPOWVUO3Y5SLURwO92YtzPdANe2R2qyFcY0ThpJgd6XSxp/70aSjouwOD1sCzlJIgh7IlS0cs6fXa+n8E3C+N21LicmOUS5GcSjcTXugJk+DRe94dn9WYRMJdlcqbfy5vZOO/LlsU1bAWQvJC+hT9E1vs6O93uLn/xfO96bdWEO3l8Ph3m8gfPCefiuh7tUk2F2ptPHn9kw6Ku9lG2sLGDZCD9tT6CWaYxjX63W2N+3mGrrLZabDmhQYOVa/LV5zF15Fgt2VSht/bs+kIz8t29hkKXDJwvFKeHdIeqCGbg+7eu2mmnryKzBmwq2yjIS715Jx7J6U+rUezmdO6z31cRMtTzpKitJ76sUpBWm7XN9OO1gMjKXodfIT6L3sOdgecGFYHmdeGzjnSAtFUTaNcV/2gX6htGj5JTMdsg7AsJGua5wowdZx7BXd0RhhRdcets0eDQnVyy+WjruCkf8jF58gZBr2B7aFe2n1deEelv7O20ZInd2LSSnGF7h7rRgjh7c5O3Xeiy9C+gOZvOSfJNh9gbvXinFgeNvS1DsIG9parydX5NbaLZbKKGD7ZB9XXYR0ZuEvT67v7g3nF15PSjG+wtayjVGKDm8bObbMUB//RmNyrgXoB/IKfnCckmuumNja4zaVaxyt0VtSWnmorHM5W1pylgvOrzpFyHoyfkZ67MIyO4a3TX+3/q1QL06j5ExORyYRHcO4iTzWykOTKHulSU+v7+7p8wufIMEuSrJzeNuJM5VKfz4N1wz7c7QkYa0MdJ6yQ9PeRcuMZu08x3GqJCO1dv8iwV7eWZrRmnXAvKZuqrlnHbD4FI1Crpd+DtMSvEZOnXdmyzt7L7wWDVOjLuY6+qFU2nns3fKvWBuWplqbCSZ8jQR7eWZtRmud4JI19bYRVoc6zhn3G1Ur51n8mctmWzpTkrB2Qba2lccXDVMjLuY686Fk6fwm9pRkLLRh/BuNJdz9hAR7eWbQjNbhXS+wcPJxGocWLI5uKre7cralMyURa7NC/z/KDm0jZpQ686FkOr81tpaELLQh51oA09+tb+MTCG9mSLArpborpQ4ppY4qpZ4z4jmFG9i7EFkphne9wLEVP+i9v5sYv3ZLcc6WRCxdkLU1tJ29mOtsnX54QdsssfX1WzlXmddLhE9wOtiVUgHAfKAH0AIYppSShZp9gT0LkdnIbcPmXDW+3R1L6RpRp3f29Vs5V5nXS4RPMKLHHgUc1TTtZ03TrgPLgX4GPK9wNXfPaDWSqXddtC5ubeMNb2PEh5KzJSErbZgz7jc7GiG8lRHBXh/4pcj9XwuOCW/n7hmtrnC1yJ/PY//IEE8wauXHot8u5qDXzW0dZWOlDcO7yiI8/sCImaeWNhIrMddQKTWegjlyjULrGnBaYQgXzGjVNqa7Z1x0aRchvWR5XKtMNX0jODob1VIbNhrUJuFRRvTYfwUaFrnfAPi9+IM0TVuoaVqEpmkRwTVrGXBa4VccGdft6clC3kJmo4pijAj23cDdSqkmSqlKwIPA5wY8rygvHB3Xbe1i4x2Ur0WyyvqAk0XDyh2ng13TtJvAk8B64CCwUtO0H519Xp/lz3uT2sGu0TGO9jgtXQCshL7jkiOTf3yBpZAubZSNM5OhhM8yZBy7pmlrNU1rqmnanzRN8+YNy1yrvO9NWhZrPUdHSyqWLgDeBtwo9jh/KUtYC+meWB9lI2WacklmnhrJqL1J/aTXb9ZrL63n6My47uLjzq0N6vCHuru1kF6L9VE2ch2iXJJgN5IRMzn9tddfWs/RyMlG/rjjkumbTmmbllibWOWP74c1ftIhMoIEu5GMmMlpVK/fSxT22kvrORo1rhtcNyPVU4p+07GmtJD2xvfDFQHsrx0iB0mwG8mImZwGrt/iVcrqORo1ld/IDwlvYOmbTlFlhbQd74dbloNwVQD7WYfIWRLsRnJmJqepF6NZ2kcOp9Zv8TRtY7p7e47uWO/FXUqrhdv6oeVN74erAthfO0QOkj1PjebITE5TL6b4P3gTX1m/pZilqXcw/d36+oqBjYBR6Bf6jNq71FcsxfE9WxthuQxj2rzE17gqgENC9d6/pePlkPTYvYGlXoyJI71+D1w8Wpp6B2FDW1MhqT1hQ1sz8R8NGf9GY46froymKT2clqCHmjf0HN3F2XHk1r7p9MTQSUclyjDLPii5FWJmun7cGS5YURTw7QXtXECC3RtY660oBSu+sD3UPXTxaGnqHWYhfvx0Zd5ZE1Jyg+sc9F57eZoc4+w4cks18lHoH5LFPywmAnUKHqcK/uzoe92shfk+t6Z9cJs5uSK3qwLYHxa0M5DSrNV0XSjinhZa+kLzT/4bCn6tGkBugKU1xfzc2bOQb2FruQoBEBzsvudwQFCeRr/hTfnhcDXbf6kqvn1B0x4VsLAkHnrw5jv4nGGUPkqmqErAIkp9r61eNDWFeb+BsCbFfB9cZ6R+rX9LPXNa76mPm1huA9heKjEyQ9O0Mv8SvKbG/mvVAG4LDSGsZk2UKmfhHhICp06aXzhVCureCTVq2vYceTet/+zPdzvXPis0TeP8xYuMe+IMkyY1sf0XfWUFRiNYq5E7M47cnslF13H8vW4boYf6B+/ByLHGhDq4ZEVRYc5rSjG5AYra5THUQQ/vundCYKB+PzDQvlA3/Y49xw2glP531uyeq1YeUcq3QV+d+WjrglpFJxUV/yft7Gggez8USnmvSx3imJmu99RHjtVvi9fchdfymh47UD5D3aRGTfuCvLg6wZZ7/XVcV4bRT6G4o0YeVSvnlaypW1yqX9c45BrHqezSthnO1nXPiz9OQ38rNPQauSOjgYqOrLkDvcRSdBc70/Nb4si3A1MZxlR+aRthfl94Na/psXuDU6dP8+DoUfypTWtaRLan58AHOHzkCMeOH6dVdKRb2jB9djINm99D9TvtHCVgRK/fQdWC8lg4+TiNQ69Rai+9QNXKecwZ95v79kc1iq0XQi09zhTqx3As1IuOrDlfcFubWxdUJwCWvpxVwuq3g1Lf/6wD5iHeNkK/n3XAzsYLT/DZYF+6MoCwllWoULMqYS2rsHRl8d6ifTRNY8BDw0iMj+enfT9wYHcGc2clc/rsGYNabJs+3XuyK83BbWxq1IS7/gz3NNdv3RDqJsO7XuDYih+w/qVLQymNxqHXWDj5eOEWbD4V7rYuqGX0wluWPihuANW5NXT0X8D7mO8BWxurF07LfN+HjSzZM28boR8XXs8ng33pygDGP1WZ479U0IfX/VKB8U9Vdirc0zZtJDCwIhPGjis8Ft6mDfEd4swed+z4ceLv70q7+DjaxcexbecOAE6eOkVC926Ex8XSKjqSzdu2kpeXx+gJj9EqOpLWMVH8Y968MtsRExXFnXV9d+tAa7vcNw69Tn5aBsdW/GC2r+bS1DtoXPeab2wCYeuCWkYvvGXtA6H4RdnhwDn03rxW8GdPLR0gPMong336i5XIuWreNcy5qpj+YiWHn3P/gQO0D29b5uNCgoNJXfMFezZvZcX7S3h6yl8B+PiTldzf5T72bt3O99t2EN66DXv37eO3k7+zf+duftixi0ceftjh9vmKOeN+o2pl82GXptJLcUXHv/vEJhC2Lotg9PIJ1j4QFHa/VxLq5YNPBvuJXy1/37d23Eg3btzg0aeepHVMFINHjeBAVhYAke3a8f5HH5E8dw4//Pgjt912G3eFhfHz/x3jqcnPsi41lRo1ari8fZ42vOuFwnq7pdJLUdPfrW95EpO3bAJRfAQM2LagltELkc3B+pbxdrxXEurlh08Ge6MGli/QWTtui5bNm5OxN7PMx/1j/jxCQ0L4ftsO0jdu5vp1vfSQENeRTevWU79ePUaMH8cHH39MrVq1+H7bdhLj45n/n4WMe/IJs+fKy8sjPC6W8LhYZr78ksNt9zamerul0ktRJ85Y+YblDUMhrS0FALYtqGXkwlvDsX5N2sb3SkK9fPHJYJ8z6zpVq5j/S69aRWPOLMv1XVt07pTItWvX+c/i9wuP7c7IYOOWzWaPu3jpEnfWrUuFChX4cPky8vL0ssPxEycICQ7m0dGPMHbEKPZ8v5dz58+Rn5/PwH79eemFGez5fq/ZcwUEBLB363b2bt3O7BdmONx2iy5dhJ+PwqGD+u2li8Y+vwGs1uNDrnk+iLxtS7nGVo7bULf3+Hsp3M4ng334kDwW/vMajRvm61/3G+az8J/XGD7EwpR6GymlWP3xMlLTvuNPbVrTMiqC5FfmUq/unWaPmzjuUZZ8vJSYzkkcPnqUatX0qfQbNm8mPC6Wth07kPL5Z0x6fCK//X6SxF49CI+LZfTjj/HKrBfLbMeUGS/QoFlTcnJyaNCsKclzHSjMXrqoj2m/UbD5540b+n0vC/ey6vEeDSRv21LOwbq9hHr55DVrxRysUZHmLpr6Xu78fPRWqBcVGKgPgzTKhfMQFMTB33+n+aWCJQ0y0/WxzjYOiyu6tG+jkOvMGfdbidKN6uSBCTFheN9yuXYu/yuh7n98bq0YYSBLoV7acUcFBcHvv8H1G0AF89mKNhre9YLVGryJKaDcGvBzMJ89Cp7fUm44NtXqJdCFT5ZiRBnctW5M1WpQrz788QcsesepKeevLatLWuZtZsfSMm/jtWW3xvRrG9Oth5ata7jYyke32JNQFyDB7p/qBFNiCqir1o2pWg2qVtVXAOw30OF1RCKbZTMk+a7CcE/LvI0hyXcR2Sy7xGNLhJezm1lY401bypWh1A89Ue5IsPsjd64bk5MNOTlOrwCY1PYyK5N/ZkjyXcxcVI8hyXexMvlnktpetvh4syDzthEsbiaBLoqTGru/cna1SFvkZOs19ttvhzETnF4BMKntZR7vd5aXPqjHjJG/Ww31orSN6VQ40R7N0gwebxgP70IS6MIa6bELx+Xm6jX2SgUTjZxcATAt8zYWrAlmxsjfWbAmuETN3Rpr4+Gd2szCi0nZRZTFp4P9x4OKVtFV+PGgMUsJzHn9NVpGRdAmNprwuFh27t5tyPM64q3588nJKV5fgOS5c5iWPMvs2N59+2ge0c7uc+zdt4+169c73EbuqK3X2ItycAVAU019ZfLPzB7ze2FZxpZwtzYe3qMjWAxmCnMJdGELnw327GzoOSiIA1mKXoOCyC55jc0u23fu5Mt169izeSv7tu/km8+/oGGDBsY01k55eXm8tWA+OVdLBvuwwYNZ8WmK2bHlKat4aPAQu8+z94d9rP2vfcF+82YpW/A5YXdWNbOauqnmvjur7L1Ura1PozXw7TCUMBeO8tlgHzOxMmfOKjRNcfqsYuwTzu3Gc/L0KerUrk3lyvrz1Kldh3p36rNOw1q14Nz5cwCk79lDYs/ugN57HvHoODr37snd4fcWLkewYfMmErp3Y8BDD9Iisj0T/vI0+fn6zsXLPllJ65goWkVHMnXmrWUEqt8ZysyXXyI6KZE5r7/G7ydPktSrJ0m9zPeGvOfuptxes6bZt4mVqz/lwYGDAPjvt98S26Uz7eLjGDzyYa5cuQLoyyN0uK8L93aIISqxExcvXmTmnJdZ8WkK4XGxrEhZxYULF+g/7EHaxEYT0zmJffv3F77O8U8/Sbd+fRk5/lGn3mdrpgw7VaKmntT2MlOGnbLp98tan8ZXQtLr25n6NQztA0lR+m3q155ukbDAqYunSqnBQDLQHIjSNM0t/xoXfViRr9YHkJurl2BycxVfrAtg0YcVGTPCsR5lt85dmP3q32jaNpz7EhMZOnAgnTrGl/l7+37cz45v08jOyaZtxzh63a+H/q6MDA7sSqdxo0Z0f6A/n36+hg7RMUydNZOMTZupdXstuvXvy2dffkH/3n3Izs6mVYsWhWvGLProQ9K+Wkud2nVKnHPYoMEsT1lFdGQkO3btonatO7j7z3/m3PlzvPz6q3zz+RdUq1aNV//xJm/O+yfP/b9nGfrIKFa8v4TI9u25dOkSVatWZfb0F0jfs4d5f38TgKcmP0vbNm34bNlyvtu4gZGPPcrerdsByNi7ly3rU6lSpYpD7683KR6aHpnZWsBrA9yS1K/hjblwLVe/f/qUfh9kc2ov4+yomP3AA8C/DWiLzaYlB5KdU2w99hzFtORAh4O9evXqZGzawuZtW0nbtImho0fxtxdnM3p46Wuo9+vZiypVqlClShWS4hPYlZHO7TVrEtW+PXc1aQLoQbxl+3YCAwNJ7BhPcMF48uFDhrJp61b69+5DQEAAA/v1t6mtDw4cRIeuXfj73FdYnrKKYYMHA7Bj124OZGUR1+0+AK5fv05sVDSHjhzmztC6RLZvD2B1+eAtO7aT8qE++Ltzp0TOX7jAxYv6+jJ9e/T0i1C3xFq4Gh34PhXilrz7r1uhbnItVz8uwe5VnAp2TdMOgvs3oX4l+QZP/7WSWbhXrarxtxedmzIfEBBAYnwCifEJtG7ZkiUff8zo4Q9TMaBiYSklt9g/7OKv3XTf0vHS1uUJCgoiIMC2HaAaNmhAWKNGbNyymZTP17D9m+8A0NDomtSZZe8vNnv8vv37bfo7stQ+0++ZFjsrT3w+iI125rR9x4XH+GSNfcyIm/S6P4+gID2IgoI0+nTP45GHHb+wd+jIYY4cPVp4f+++fTRu2BCAsMaNyMjUl9xNWbPG7PfWrP2K3Nxczp8/z4Ytm4lsp/eKd2Vk8H/HjpGfn8+KlBQ6xsYSHRHJxq1bOHf+HHl5eSxb9QmdOna02J7bqlfn8uUrVts7bNBgnpn2HH9q0oQG9esDEBMZydadOzj6008A5OTkcPjIEZo1bcrvp06yOyMDgMuXL3Pz5k39HFdunSOhQxxLV64A9OsEdWrXLhebgwgbhVjZYN3aceExZQa7UuobpdR+C//1s+dESqnxSql0pVT62Yv/c7zFBRb96xohwfoGyaHBGu/Nv+bU8125ks2oCY/RIrI9bWKjOXAoi+Tnnwdg1nPTmDR1CvH3dy3Rq45q355egwcS06UzM6ZMLbzgGhsVxXPJM2kVHUmTsMYM6NOXO+vW5ZVZL5LUqyf3doih3b3h9OvV22J7xo9+hB4DB5S4eGoyeMAAfjx4sPCiKUBwnWAWL3iHYWMe0S+Adkki68hhKlWqxIr3l/DUXydzb4cYuvbrS25uLknxCRzIyiq8eJo87XnSMzNpExvNc7NmseSdhU69p8KLLfug5CzhzHT9uDXjJkLlIPNjlYP048KrGLJsr1JqAzDZ1ounRi1R+O86AAAaQUlEQVTb++NBxdDRQaxYnEvL5u5ffjh57hyqV6/O5KcnmR3fsHkTb7z9Nl9+ssrtbfKEg0eP3Fq2V/iGoitxto0oed+a1K/1mvqZ03pPfdxEx+vrRj5XOVEulu1t2Vxj/86rnm6GEMZxV9iZZgknT9MXb1uTYttSEF17GNMeGWHjUk7V2JVSA5RSvwKxwFdKKSemMfqe5Oenl+itAyTGJ5Sb3roZR8c4O1IW8EemsDt9CjTtVti5aqx42wg91J1cmdMhpY2wEU5zKtg1TVutaVoDTdMqa5oWqmna/UY1TPgYZ0KpWQu952gKd1NZoFkL17bZ27g77DLT9Z66kytzOkRG2LiUT46KEV7ImVAqWhZwcsMOn+bOsCtaUx8z4db7765wlxE2LiXBLozhbCh5sizgLdwZdlkHzD88nVyZ024ywsalJNiF83KvltyxycTWUPJkWcBbuDPsho0s+eHp4MqcDunaAyY/D6F19X87oXX1+3Lh1BAS7EWoGtUZ8ei4wvs3b94kuEljeg8eVMpv2Wfck09wIOugTY89dvw4DZo1LZz1ahIeF8uudPuDb+4br9v9O2W6dBEuXYJibQRsDyVPlwW8RXkLu649YMUXkLZLv/XX1+kBPhnsr70VSNom86anbarAa285t1lztWrV2H/wAFev6kMoU7/7jvr16tn1HGUta/vuvPm0aNbcpucKa9yYhvUbsHnb1sJjWYcPcfnyFaIi7C9VzP37G3Y9XtO0Eh8qJZw7q18sLa5CBdtDydNlAW8iYScM4JPBHtkujyGjggrDPW1TBYaMCiKyXV4Zv1m2Hl278dX6dQAsW/UJwwYNLvzZrvR0OtzXhbYdO9Dhvi4cOnIYgMVLP2LwyIfpM2Qw3fr3JT8/n4nP/IWWURH0HjyIngMfYNVnqwFI7Nmd9D17AH2p3umzk7m3QwwxnZM4baEePWzQIJan3Bo6uXzVKoYN0r9BnD13loEPP0RkpwQiOyWwdYe+EuOVK1d45PEJtI6Jok1sNClrPuO5WTO5evUq4XGxDB87BoA35/2TVtGRtIqO5K358wH9W0LziHZMfOYvtIuP45dffy39DbthZX0eTbM9lDxdFhDCz/hksCcl5LNySS5DRgUx8+VAhowKYuWSXJISyuhd2uDBgXqQ5ubmsu/H/UQX6Rk3a9qUTevWk7llG7Onv8DzLyYX/mz7rl0seefffPflWj79fA3HTpzghx27eHfefLbv3mXxXNnZ2cRERvH9th0kxMXxn8WLSzxmyAMD+ezLLwu/Caz4NIUHC4J90pQpPPPEk+zeuImUj5Yy7sknAHjptVepWaMGP+zYxb7tO+mc0Im/vTibKlWqsHfrdpa+t4iMzEze/+hDdn63gR3fpvGfJe+T+f33ABw6coSRwx4ic8s2GjcqY3+5QCvfkmR0gxAe47MzT5MS8nl87A1eeq0SM6ZcNyTUAdq0asWxEydYtuoTenYzH5Z/8dIlRk0Yz5GffkIpxY0ivdWuSZ254447ANiyfTuDBwygQoUK1A0NJSne8rrulSpVond3vVfbPjyc1LS0Eo+pGxpKy+bN+XbDBkJDQggMDKRVi5YAfLMhjQOHsgofe+nyZS5fvsw3G9JYvmhx4fFatWqVeN4t27cxoHefwlUbH+jTl83bttK3Zy8aN2pETFSULW8X1AmGU8U2w5DRDTJdXniUzwZ72qYKLHgvkBlTrrPgvUCSEvIMC/e+PXoyefp0Nqxdy/kLt3bimfHySyTFJ7D64+UcO36cxCILdFWrWrXwz7auvxMYGFi4LG5AQIDV+rxpY43QkBCz0lB+fj7bv/muxDrpmqaVuUxvaU0s+loA/QLpubN62SUwUA/zGjX1n9WoCTVq6Bf6JMR0jkyXlw8CYSCfLMWYauorl+Qy+4UbhWWZ4hdUHTVmxEhmTp1K65atzI5fvHSx8GLq4qUfWf39jrGxpKxZQ35+PqfPnGbD5i1OtWdg336s/e96vQxTZDXHbp27MG/hrT1O9u7bZ/H4//6nr6YZGBhY+C0jIS6Oz776kpycHLKzs1n95RfEd4grefJLF+HUyVu19Bs39PuXLt56TFAVueBXlL2Ttdy9lIDwez4Z7Lv3BJjV1E019917bNuooiwN6tdn0sQnShyfMukZpiUnE9f1PvLyrF+oHdivPw3q16NVdCSPTZpEdEQENU09XAfcfvvtxERGERocQpOwsMLjb7/+OumZe2gTG02LyPa8s+hdAF746xT+98cftIqO5N4OMaRt3gToSwG3iY1m+NgxtAsPZ/Twh4lK6kR050TGjRxN23vvLXlyS6NeNE0/Liyzd7KWrJsiDGbIsr32MmrZXm925coVqlevzvnz54lKSmRr6jfUDfXBC4qHShlzf48+bNPly/b6WpliaB+9111caF39G01xSVGWa2NK6d+ChChg67K9Ptlj9wW9hwwiPC6W+O7dmDF1qm+GOlgf9WLtuNHcUaZwdFVKa+ydQSrrpgiD+ezFU2+3Ye06TzfBGHWC9Zp60R6lUvpxd3D1BsquWBfc9Hu2fssYN9G8DSAji4RTJNhF6UzXBqyNinE1V6946KoPDns2pLD3g0CIMkiwi7LVqOm+IC8uJNRyvdqoMoW3rAtu1M5EQiA1duHtXL3ioafq20bX9YUoQoJdeLfiKx7WqAmVK8HcWZ650GkEGbcuXEyCvYhTp0/z4OhR/KlNa1pEtqfnwAc4fOQIx44fp1V0pEvOuWnrFtrFx1GxVs3ChcJEMaYVD59/Ea5d05cJNioQPbFUroxbFy7mu8G+cgW0bA41b9NvV65w6uk0TWPAQ8NIjI/np30/cGB3BnNnJXP67BmDGmxZowYNWbzg3zw0eIhLz+MXXBWI7l4q11vq+sJv+Wawr1wBTz0Jv/yi99x++UW/70S4p23aSGBgRSaMvbXRRnibNiWm2R87fpz4+7vSLj6OdvFxbNu5A4CTp06R0L0b4XGxtIqOZPO2reTl5TF6wmO0io6kdUwU/5g3r8R5wxo3pk2rVlSo4Jt/FW7ly4FYtKbu7G5TRrRB6vp+zTdHxbyYDAWbYRS6elU/PmSoQ0+5/8AB2oe3LfNxIcHBpK75gqCgII4cPcqwsY+QvnEzH3+ykvu73Mf0v04hLy+PnJwc9u7bx28nf2f/zt0A/PHHHw61TRRw9QgZVyk+Vt7SLFN31fWNHK8vvJZvdhOtbf5Q1qYQBrhx4waPPvUkrWOiGDxqBAey9GVzI9u14/2PPiJ57hx++PFHbrvtNu4KC+Pn/zvGU5OfZV1qKjVq1HB5+/yar26AbKmEBPouU1LXFy7gm8HeoIF9x23QsnlzMvZmlvm4f8yfR2hICN9v20H6xs1cv34dgIS4jmxat5769eoxYvw4Pvj4Y2rVqsX327aTGB/P/P8sLNwIQ5Ri2Qcl9zrNTNeP++qeoNZKRZomdX3hEr4Z7LOSodga5FSpoh93UOdOiVy7dp3/LH6/8NjujAw2btls9riLly5xZ926VKhQgQ+XLytc5fH4iROEBAfz6OhHGDtiFHu+38u58+fIz89nYL/+vPTCDPZ8v9fh9pUbzVqYb2Rt2ui6WQv9vjfsCWpvrdob1oLxhjYIt/HNYB8yFP45Dxo21HtuDRvq9x2srwMopVj98TJS077jT21a0zIqguRX5lKv7p1mj5s47lGWfLyUmM5JHD56tHAHog2bNxMeF0vbjh1I+fwzJj0+kd9+P0lirx6Ex8Uy+vHHeGXWiyXOuzsjgwbNmvLJZ6t5bNIkWkbZv0m1XzFtZJ08DRa9o98W3eja0xwZg+4NJSRvaINwG1m2VzjNJcv2LnoHPngPRo6FMROMfW5n2Lskr4k3LD3sDW0QTrF12V7fHBUj/FtmOqxJ0UN9TYreW/eWHrujtWpvWAvGG9og3MI3SzHCf5lq6smv6D11U1mm+AVVT5FatfABEuzCu2QdMK+pm2ruWQc82y4TqVULHyClGOFdho0secybSjGydrrwAU4Fu1LqdaAPcB34CXhE0zSZXin8m9SqhZdzthSTCrTSNK0NcBiY5nyThBBCOMOpYNc07b+appnGue0AHJ/66YiDByA6Ur81wJzXX6NlVARtYqMJj4tl5+7dhjyvI96aP5+cnJwSx5PnzmFa8iyzY3v37aN5RDu7z7F33z7Wrl/vcBuFEN7JyIunYwD3LReXnQ2DBkJWFgwapN93wvadO/ly3Tr2bN7Kvu07+ebzL2joxBIFzsjLy+OtBfPJuVoy2IcNHsyKT1PMji1PWeXQsr97f9jH2v/aF+w3bxo8Xl0IYbgyg10p9Y1Sar+F//oVecx04CawtJTnGa+USldKpZ+9+D/nWz7xcTh7Vp/9d/YMPOHcqISTp09Rp3ZtKleuDECd2nWod6c+6zSsVQvOnT8HQPqePST27A7ovecRj46jc++e3B1+b+FyBBs2byKhezcGPPQgLSLbM+EvT5Ofnw/Ask9W0jomilbRkUydOaPw/NXvDGXmyy8RnZTInNdf4/eTJ0nq1ZOkXua13HvubsrtNWuafZtYufpTHhw4CID/fvstsV060y4+jsEjH+bKlSuAPsO1w31duLdDDFGJnbh48SIz57zMik9TCI+LZUXKKi5cuED/YQ/SJjaamM5J7Nu/v/B1jn/6Sbr168vI8Y869T4LIVyvzIunmqbdV9rPlVKjgN5AF62Uaayapi0EFoI+89TOdpr78ENYvw5yC1ary82FdV/rx0eMcOgpu3XuwuxX/0bTtuHcl5jI0IED6dQxvszf2/fjfnZ8m0Z2TjZtO8bR63499HdlZHBgVzqNGzWi+wP9+fTzNXSIjmHqrJlkbNpMrdtr0a1/Xz778gv69+5DdnY2rVq0YPYLetgv+uhD0r5aS53adUqcc9igwSxPWUV0ZCQ7du2idq07uPvPf+bc+XO8/PqrfPP5F1SrVo1X//Emb877J8/9v2cZ+sgoVry/hMj27bl06RJVq1Zl9vQXSN+zh3l/fxOApyY/S9s2bfhs2XK+27iBkY89yt6t2wHI2LuXLetTqVJ8jR4hhNdxqhSjlOoOTAX6appWsm7gKskzoXj9OSdHP+6g6tWrk7FpCwvffpvgOnUYOnoUi5d+VObv9evZiypVqlCndh2S4hPYlaFPpIlq3567mjQhICCAYYMGs2X7dnbvySCxYzzBdYKpWLEiw4cMZdPWrQAEBAQwsF9/m9r64MBBrFrzGfn5+SxPWcWwwYMB2LFrNweysojrdh/hcbEs+Xgpx3/5hUNHDnNnaF0i27cHoEaNGlSsWPIzfcuO7Yx4cBigL4p2/sIFLl68CEDfHj0l1IXwEc6OY58HVAZSlb4rzA5N01y/sEfybPjrs+bhXrUqvDjbqacNCAggMT6BxPgEWrdsyZKPP2b08IepGFCxsJSSW2xNa1VsNxzTfUvHS1uXJygoiICAAJva2bBBA8IaNWLjls2kfL6G7d98B4CGRtekzix7f7HZ4/ft31+iPZZYap/p90yLnQkhvJ+zo2L+rGlaQ03Twgv+c89qTSNGwP3dIahgBmBQEHTvAQ87VoYBOHTkMEeOHi28v3ffPho3bAhAWONGZGTqS+6mrFlj9ntr1n5Fbm4u58+fZ8OWzUS203vFuzIy+L9jx8jPz2dFSgodY2OJjohk49YtnDt/jry8PJat+oROHTtabM9t1atz+fIVq+0dNmgwz0x7jj81aUKD+vUBiImMZOvOHRz96ScAcnJyOHzkCM2aNuX3UyfZnZEBwOXLl7l586Z+jiu3zpHQIY6lBdsLbti8iTq1a8vmIEL4IN9dUuBfCyA4WF+2NzgE5ju3E8yVK9mMmvAYLSLb0yY2mgOHskh+/nkAZj03jUlTpxB/f9cSveqo9u3pNXggMV06M2PK1MILrrFRUTyXPJNW0ZE0CWvMgD59ubNuXV6Z9SJJvXpyb4cY2t0bTr9evS22Z/zoR+gxcECJi6cmgwcM4MeDBwsvmgIE1wlm8YJ3GDbmEf0CaJckso4cplKlSqx4fwlP/XUy93aIoWu/vuTm5pIUn8CBrKzCi6fJ054nPTOTNrHRPDdrFkveWejUeyqE8AzfXrb34AEYPQoWL4HmLQxsoW2S586hevXqTH56ktnxDZs38cbbb/PlJ6vc3iZPcMmyvUKIEsrHsr3NW8BOz00iEkIIb+Tbwe5hyc9Pt3jcdAFWCCE8wXdr7EIIISySYBdCCD8jwS6EEH5Ggl0IIfyMBHsRqkZ1Rjw6rvD+zZs3CW7SmN6DB5XyW/YZ9+QTHMg6aNNjjx0/ToNmTQtnvZqEx8WyK93+PUDnvvG63b8jhPA9vhnsb/0DNm00P7Zpo37cCdWqVWP/wQNcvXoVgNTvvqN+vXp2PUdZy9q+O28+LZo1t+m5who3pmH9BmzetrXwWNbhQ1y+fIWoCPu3ipv79zfserymaSU+VIQQ3s83g71dOxg18la4b9qo329n/2YTxfXo2o2v1q8DYNmqTxg2aHDhz3alp9Phvi607diBDvd14dCRwwAsXvoRg0c+TJ8hg+nWvy/5+flMfOYvtIyKoPfgQfQc+ACrPlsNQGLP7qTv2QPoS/VOn53MvR1iiOmcxOkzp0u0Z9igQSxPuTXRafmqVQwbpH+DOHvuLAMffojITglEdkpg6w59JcYrV67wyOMTaB0TRZvYaFLWfMZzs2Zy9epVwuNiGT52DABvzvsnraIjaRUdyVvz5wP6t4TmEe2Y+MxfaBcfxy+//ur0eyqEcC/fDPaETrDkAz3MX35Jv13ygX7cSQ8O1IM0NzeXfT/uJ7pIz7hZ06ZsWreezC3bmD39BZ5/MbnwZ9t37WLJO//muy/X8unnazh24gQ/7NjFu/Pms333Lovnys7OJiYyiu+37SAhLo7/LF5c4jFDHhjIZ19+WfhNYMWnKTxYEOyTpkzhmSeeZPfGTaR8tJRxTz4BwEuvvUrNGjX4Yccu9m3fSeeETvztxdlUqVKFvVu3s/S9RWRkZvL+Rx+y87sN7Pg2jf8seZ/M778H4NCRI4wc9hCZW7bRuFEjp99TIYR7+e4EpYROMHYcvPYqTJlqSKgDtGnVimMnTrBs1Sf07Ha/2c8uXrrEqAnjOfLTTyiluHHjRuHPuiZ15o477gBgy/btDB4wgAoVKlA3NJSkeMvruleqVIne3fW1YNqHh5OallbiMXVDQ2nZvDnfbthAaEgIgYGBtGrREoBvNqRx4FBW4WMvXb7M5cuX+WZDGssXLS48XqtWrRLPu2X7Ngb07lO4auMDffqyedtW+vbsReNGjYiJirLl7RJCeCHfDfZNG+G9d/VQf+9dSEgwLNz79ujJ5OnT2bB2LecvXCg8PuPll0iKT2D1x8s5dvw4iUUW6KpWtWrhn21dfycwMLBwWdyAgACr9XnTxhqhISFmpaH8/Hy2f/NdiXXSNU0rc5ne0ppY9LUIIXyPb5ZiTDX1JR/ACzNulWWKX1B10JgRI5k5dSqtW7YyO37x0sXCi6mlbcLRMTaWlDVryM/P5/SZ02zYvMWp9gzs24+1/12vl2GKrObYrXMX5i38d+H9vfv2WTz+v//pWxEGBgYWfstIiIvjs6++JCcnh+zsbFZ/+QXxHeKcaqcQwjv4ZrDv2WNeUzfV3AsuSjqrQf36TJr4RInjUyY9w7TkZOK63kdeXp7V3x/Yrz8N6tejVXQkj02aRHREBDVr1HS4PbfffjsxkVGEBofQJCys8Pjbr79OeuYe2sRG0yKyPe8seheAF/46hf/98QetoiO5t0MMaZs3AfpSwG1ioxk+dgztwsMZPfxhopI6Ed05kXEjR9P23nsdbqMQwnv49rK9XuzKlStUr16d8+fPE5WUyNbUb6gbGurpZrmELNsrhHuUj2V7vVjvIYP44+JFrl+/zoypU/021IUQ3keC3UU2rF3n6SYIIcop36yxCyGEsMqrgt0T9X7hHPk7E8L7eE2wB+VpnL94UYLCh2ia/ncWlCd/Z0J4E6+psTfIyePX02c4e+6sp5si7BCUp9Egx/rQTyGE+3lNsAdq0CRbAkIIIZzlNaUYIYQQxpBgF0IIPyPBLoQQfsYjSwoopc4Cx91+YveqA5zzdCNcTF6jfygPrxH843U21jQtuKwHeSTYywOlVLotazr4MnmN/qE8vEYoP68TpBQjhBB+R4JdCCH8jAS76yz0dAPcQF6jfygPrxHKz+uUGrsQQvgb6bELIYSfkWB3EaXU60qpLKXUPqXUaqXU7Z5ukysopQYrpX5USuUrpfxqxIFSqrtS6pBS6qhS6jlPt8doSqlFSqkzSqn9nm6LqyilGiql0pRSBwv+nU7ydJvcQYLddVKBVpqmtQEOA9M83B5X2Q88AGzydEOMpJQKAOYDPYAWwDClVAvPtspwi4Hunm6Ei90EntU0rTkQAzzhh3+PJUiwu4imaf/VNM20EegOoIEn2+MqmqYd1DTtkKfb4QJRwFFN037WNO06sBzo5+E2GUrTtE3ABU+3w5U0TTupadqegj9fBg4C9T3bKteTYHePMcDXnm6EsEt94Jci93+lHASCP1NKhQFtgZ2ebYnrec2yvb5IKfUNUNfCj6Zrmram4DHT0b8OLnVn24xky+v0Q8rCMRlC5qOUUtWBFOAvmqZd8nR7XE2C3Qmapt1X2s+VUqOA3kAXzYfHlZb1Ov3Ur0DDIvcbAL97qC3CCUqpQPRQX6pp2qeebo87SCnGRZRS3YGpQF9N03I83R5ht93A3UqpJkqpSsCDwOcebpOwk1JKAe8BBzVNe9PT7XEXCXbXmQfcBqQqpfYqpd7xdINcQSk1QCn1KxALfKWUWu/pNhmh4ML3k8B69AtuKzVN+9GzrTKWUmoZsB24Ryn1q1JqrKfb5AJxwAigc8H/h3uVUj093ShXk5mnQgjhZ6THLoQQfkaCXQgh/IwEuxBC+BkJdiGE8DMS7EII4Wck2IUQws9IsAshhJ+RYBdCCD/z/wPwhIKD2fcDFAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "C=10\n",
    "beta,b=SMO_KKT(x_nonlin,label_nonlin,C,gaussian)\n",
    "#hyperlpane/contour:\n",
    "f=set_f(beta,label_nonlin,x_nonlin,b,gaussian)\n",
    "xmin,ymin = np.amin(x_nonlin,axis = 0)\n",
    "xmax,ymax = np.amax(x_nonlin,axis = 0)\n",
    "xx,yy = np.meshgrid(np.arange(xmin - 1, xmax + 1, 0.05),\n",
    "                    np.arange(ymin - 1, ymax + 1, 0.05))\n",
    "zz=np.apply_along_axis(f,1,np.array([xx.flatten(),yy.flatten()]).T).reshape(xx.shape)\n",
    "plt.contourf(xx,yy,zz,0, colors = ['cyan', 'pink'])\n",
    "for lab,col in zip([-1,1],['b','r']):\n",
    "    for indices, style, name in zip([beta==0, beta==C,\n",
    "                                      np.logical_and(beta>0,beta<C)],\n",
    "                                     ['o','d','x'],\n",
    "                                     [\"Class \"+str(lab),\"Support Vector\",\"Margin Vector\"]):\n",
    "        plt.plot(x_nonlin[np.logical_and(label_nonlin==lab, indices ),0],\n",
    "                 x_nonlin[np.logical_and(label_nonlin==lab, indices ),1],\n",
    "                 col+style, label=name)\n",
    "plt.legend(loc = \"best\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task 2.8** Make yourself familiar with the SVC function in scikit-learn, which implements a support vector classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Import MNIST data set\n",
    "\n",
    "import os \n",
    "import gzip \n",
    "from urllib.request import urlretrieve\n",
    "\n",
    "def download(filename , source= \"http://yann.lecun.com/exdb/mnist/\"):\n",
    "    print(\"Downloading %s\" % filename)\n",
    "    urlretrieve(source + filename , filename)\n",
    "    \n",
    "    \n",
    "def load_mnist_images(filename):\n",
    "    if not os.path.exists(filename):\n",
    "        download(filename)\n",
    "    with gzip.open(filename, \"rb\") as f:\n",
    "        data = np.frombuffer(f.read(), np.uint8, offset=16)\n",
    "        data = data.reshape(-1, 28, 28)\n",
    "    return data / np.float32(256)\n",
    "    \n",
    "def load_mnist_labels(filename):\n",
    "    if not os.path.exists(filename):\n",
    "        download(filename)\n",
    "    with gzip.open(filename, \"rb\") as f:\n",
    "         data = np.frombuffer(f.read(), np.uint8, offset=8)\n",
    "    return data \n",
    "\n",
    "X_train = load_mnist_images(\"train-images-idx3-ubyte.gz\")\n",
    "y_train = load_mnist_labels(\"train-labels-idx1-ubyte.gz\")\n",
    "X_test = load_mnist_images(\"t10k-images-idx3-ubyte.gz\")\n",
    "y_test = load_mnist_labels(\"t10k-labels-idx1-ubyte.gz\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(a) Choose a random subset of size 500 from the MNIST training data and use this as your new training data set for crossvalidation. Perform a 5-fold crossvalidation SVM to determine the optimal parameters among $C \\in \\{1,10,100\\}$\n",
    "and $ \\gamma \\in \\{0.1 , 0.01, 0.001\\}$ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC \n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma='auto', kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'C': [1, 10, 100], 'gamma': [0.1, 0.01, 0.001]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=None, verbose=0)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ntr = X_train.shape[0]\n",
    "X_train = X_train.reshape(ntr, -1)\n",
    "nts = X_test.shape[0]\n",
    "X_test = X_test.reshape(nts, -1)\n",
    "ind_subtr = np.random.randint(0, ntr, size = 500)\n",
    "X_subtr = X_train[ind_subtr]\n",
    "y_subtr = y_train[ind_subtr]\n",
    "\n",
    "parameters = {\"C\": [1,10,100], \"gamma\": [0.1, 0.01, 0.001]}\n",
    "svc = SVC()\n",
    "clf = GridSearchCV(svc, parameters, cv = 5)\n",
    "clf.fit(X_subtr, y_subtr)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'C': 10, 'gamma': 0.01}"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.best_params_ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(b) Use the determined optimal parameters to learn a support vector classifier on a random 2000 point subset of the MNIST training data and evaluate the confusion matrix and the accuracy on the whole MNIST testd data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=10, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma=0.01, kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ind_tr2 = np.random.randint(0, ntr, size = 2000)\n",
    "X_tr2 = X_train[ind_tr2]\n",
    "y_tr2 = y_train[ind_tr2]\n",
    "\n",
    "clf2 = SVC()\n",
    "clf2.set_params(**clf.best_params_)\n",
    "clf2.fit(X_tr2, y_tr2)\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 968,    0,    3,    0,    0,    3,    4,    1,    1,    0],\n",
       "       [   0, 1121,    3,    3,    0,    1,    4,    0,    3,    0],\n",
       "       [   9,    3,  958,   14,    5,    4,    8,   10,   20,    1],\n",
       "       [   3,    3,   16,  940,    2,   20,    1,    8,   11,    6],\n",
       "       [   2,    6,    6,    0,  931,    1,    7,    0,    2,   27],\n",
       "       [   8,    2,    8,   21,    8,  825,    7,    2,    7,    4],\n",
       "       [  11,    5,   11,    0,    5,   28,  894,    1,    3,    0],\n",
       "       [   1,   13,   17,    0,   14,    2,    0,  949,    2,   30],\n",
       "       [   6,    4,    9,   13,    9,   22,    3,   10,  885,   13],\n",
       "       [  12,    6,    1,    8,   30,    5,    1,   13,    5,  928]],\n",
       "      dtype=int64)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ypred = clf2.predict(X_test)\n",
    "cm = confusion_matrix(y_test, ypred)\n",
    "cm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9399"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy_score(y_test, ypred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9399"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## check using own function\n",
    "acc(cm, nts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Is our approach of picking a different training set in step(b) and learning with the optimal parameters from (a) valid? Are there potential pitfalls? \n",
    "\n",
    "It is valid since the chosen parameters during crossvalidation should generalize, however it might happen that the resulting hyperparameters are overfitted to the data used in (a) and hence the predictions used when learning on the other set might not be good. \n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}