{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "_You are currently looking at **version 1.0** of this notebook. To download notebooks and datafiles, as well as get help on Jupyter notebooks in the Coursera platform, visit the [Jupyter Notebook FAQ](https://www.coursera.org/learn/python-machine-learning/resources/bANLa) course resource._\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Applied Machine Learning, Module 1:  A simple classification task"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import required modules and load data file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib notebook\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "fruits = pd.read_table('fruit_data_with_colors.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "fruits.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# create a mapping from fruit label value to fruit name to make results easier to interpret\n",
    "lookup_fruit_name = dict(zip(fruits.fruit_label.unique(), fruits.fruit_name.unique()))   \n",
    "lookup_fruit_name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The file contains the mass, height, and width of a selection of oranges, lemons and apples. The heights were measured along the core of the fruit. The widths were the widest width perpendicular to the height."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examining the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# plotting a scatter matrix\n",
    "from matplotlib import cm\n",
    "\n",
    "X = fruits[['height', 'width', 'mass', 'color_score']]\n",
    "y = fruits['fruit_label']\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)\n",
    "\n",
    "cmap = cm.get_cmap('gnuplot')\n",
    "scatter = pd.scatter_matrix(X_train, c= y_train, marker = 'o', s=40, hist_kwds={'bins':15}, figsize=(9,9), cmap=cmap)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# plotting a 3D scatter plot\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111, projection = '3d')\n",
    "ax.scatter(X_train['width'], X_train['height'], X_train['color_score'], c = y_train, marker = 'o', s=100)\n",
    "ax.set_xlabel('width')\n",
    "ax.set_ylabel('height')\n",
    "ax.set_zlabel('color_score')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create train-test split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# For this example, we use the mass, width, and height features of each fruit instance\n",
    "X = fruits[['mass', 'width', 'height']]\n",
    "y = fruits['fruit_label']\n",
    "\n",
    "# default is 75% / 25% train-test split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create classifier object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "knn = KNeighborsClassifier(n_neighbors = 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the classifier (fit the estimator) using the training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "knn.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Estimate the accuracy of the classifier on future data, using the test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "knn.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use the trained k-NN classifier model to classify new, previously unseen objects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# first example: a small fruit with mass 20g, width 4.3 cm, height 5.5 cm\n",
    "fruit_prediction = knn.predict([[20, 4.3, 5.5]])\n",
    "lookup_fruit_name[fruit_prediction[0]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# second example: a larger, elongated fruit with mass 100g, width 6.3 cm, height 8.5 cm\n",
    "fruit_prediction = knn.predict([[100, 6.3, 8.5]])\n",
    "lookup_fruit_name[fruit_prediction[0]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot the decision boundaries of the k-NN classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from adspy_shared_utilities import plot_fruit_knn\n",
    "\n",
    "plot_fruit_knn(X_train, y_train, 5, 'uniform')   # we choose 5 nearest neighbors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How sensitive is k-NN classification accuracy to the choice of the 'k' parameter?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "k_range = range(1,20)\n",
    "scores = []\n",
    "\n",
    "for k in k_range:\n",
    "    knn = KNeighborsClassifier(n_neighbors = k)\n",
    "    knn.fit(X_train, y_train)\n",
    "    scores.append(knn.score(X_test, y_test))\n",
    "\n",
    "plt.figure()\n",
    "plt.xlabel('k')\n",
    "plt.ylabel('accuracy')\n",
    "plt.scatter(k_range, scores)\n",
    "plt.xticks([0,5,10,15,20]);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How sensitive is k-NN classification accuracy to the train/test split proportion?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "t = [0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2]\n",
    "\n",
    "knn = KNeighborsClassifier(n_neighbors = 5)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "for s in t:\n",
    "\n",
    "    scores = []\n",
    "    for i in range(1,1000):\n",
    "        X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 1-s)\n",
    "        knn.fit(X_train, y_train)\n",
    "        scores.append(knn.score(X_test, y_test))\n",
    "    plt.plot(s, np.mean(scores), 'bo')\n",
    "\n",
    "plt.xlabel('Training set proportion (%)')\n",
    "plt.ylabel('accuracy');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}