1. Understanding neural network with TensorFlow Playground.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Understanding neural networks with TensorFlow Playground\n",
    "\n",
    "(This codelab is an adaptation from a blog post [Understanding neural networks with TensorFlow Playground](https://cloud.google.com/blog/big-data/2016/07/understanding-neural-networks-with-tensorflow-playground))\n",
    "\n",
    "**Before start this codelab, please find one or two people close to you as your buddy. We like you to go through the labs with your buddy and discuss about it for better understanding.**\n",
    "\n",
    "You may have heard the buzz about neural networks and deep learning, and want to learn more. But when you learn about the technology from a textbook, many people find themselves overwhelmed by mathematical models and formulas. I certainly was.\n",
    "\n",
    "For people like me, there's an awesome tool to help you grasp the idea of neural networks without any hard math: [TensorFlow Playground](http://playground.tensorflow.org/), a web app written in JavaScript that lets you play with a real neural network running in your browser and click buttons and tweak parameters to see how it works.\n",
    "\n",
    "<br/>\n",
    "\n",
    "![](https://cloud.google.com/blog/big-data/2016/07/images/146798944178238/neural-network-18.png)\n",
    "\n",
    "<br/>\n",
    "\n",
    "In this codelab, I'd like to show how you can play with TensorFlow Playground so that you can understand the core ideas behind neural networks. Then you can understand why people have become so excited by the technology as of late."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1-1. Let the computer solve the problem\n",
    "\n",
    "Computer programming requires a programmer. Humans instruct a computer to solve a problem by specifying each and every step through many lines of code. But with machine learning and neural networks, you can let the computer try to solve the problem itself. A neural network is a **function** that learns the expected output for a given input from **training datasets**.\n",
    "\n",
    "<br/>\n",
    "![](https://cloud.google.com/blog/big-data/2016/07/images/146798944178238/neural-networks-22.png)\n",
    "<br/>\n",
    "\n",
    "For example, to build a neural network that recognizes images of a cat, you train the network with a lot of sample cat images. The resulting network works as a function that takes a cat image as input and outputs the \"cat\" label. Or — to take a more practical example — you can train it to input a bunch of user activity logs from gaming servers and output which users have a high probability of conversion.\n",
    "\n",
    "How does this work? Let's look at a simple classification problem. Imagine you have a dataset such as the one below. Each data point has two values: **x1** (the horizontal axis) and **x2** (the vertical axis). There are two groups of data points, the orange group and blue group.\n",
    "\n",
    "<br/>\n",
    "\n",
    "![](https://cloud.google.com/blog/big-data/2016/07/images/146798944178238/neural-networks-17.png)\n",
    "\n",
    "<br/>\n",
    "\n",
    "How do you write code that classifies whether a data point is orange or blue? Perhaps you draw an arbitrary diagonal line between the two groups like below and define a threshold to determine in which group each data point belongs.\n",
    "\n",
    "<br/>\n",
    "\n",
    "![](https://cloud.google.com/blog/big-data/2016/07/images/146798944178238/neuralnetworks-22.png)\n",
    "\n",
    "<br/>\n",
    "\n",
    "The condition of your IF statement would look like this.\n",
    "\n",
    "<br/>\n",
    "\n",
    "![](https://cloud.google.com/blog/big-data/2016/07/images/146798944178238/neural-networks-3.png)\n",
    "\n",
    "<br/>\n",
    "\n",
    "where **b** is the threshold that determines the position of the line. By putting **w1** and **w2** as weights on x1 and x2 respectively, you make your code more reusable.\n",
    "\n",
    "<br/>\n",
    "\n",
    "![](https://cloud.google.com/blog/big-data/2016/07/images/146798944178238/neural-networks-15.png)\n",
    "\n",
    "<br/>\n",
    "\n",
    "Further, if you tweak the values of w1 and w2, you can rotate the angle of the line as you like. You can also tweak the \"b\" value to move the line position. So you can reuse this condition for classifying any datasets that can be classified by a single straight line.\n",
    "\n",
    "But the thing is, the programmer has to find appropriate values for w1, w2 and b — the so-called **parameters** — and instruct the computer how to classify the data points.\n",
    "\n",
    "<br/>\n",
    "\n",
    "![](https://cloud.google.com/blog/big-data/2016/07/images/146798944178238/neural-networks-2.png)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lab: Solving the simple classification on Playground\n",
    "\n",
    "Now, let's look at how the computer behind TensorFlow Playground solves this particular problem. \n",
    "\n",
    "- [Click this link](http://playground.tensorflow.org/#activation=sigmoid&regularization=L2&batchSize=10&dataset=gauss&regDataset=reg-plane&learningRate=0.0001&regularizationRate=0&noise=0&networkShape=&seed=0.55381&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&showTestData_hide=true&learningRate_hide=true&regularizationRate_hide=true&percTrainData_hide=true&numHiddenLayers_hide=true&discretize_hide=true&activation_hide=true&problem_hide=true&noise_hide=true&regularization_hide=true&dataset_hide=true&batchSize_hide=true&playButton_hide=false) to open Playground for this problem.\n",
    "- On the Playground, click the Play button in the upper left corner. The line between blue and orange data points begins to move slowly. \n",
    "- Hit the reset button and click play again a few times to see how the line moves with different initial values. \n",
    "- Discuss with your buddy about the following.\n",
    " - What do the lines between X1, X2 and the output mean? \n",
    " - How weights and biases works to solve this problem?\n",
    " - If you and your buddy have anything unclear, find a tutor and ask about it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1-2. A neuron: classifying a data point into two kinds\n",
    "\n",
    "What you're seeing is the computer trying to find the best combination of weights and threshold to draw the straight line between two groups.\n",
    "\n",
    "TensorFlow Playground is using a single artificial neuron for this classification. What's an artificial neuron? It’s an idea inspired by the behavior of biological neurons in the human brain.\n",
    "\n",
    "<br/>\n",
    "\n",
    "![](https://cloud.google.com/blog/big-data/2016/07/images/146798944178238/neural-networks-7.png)\n",
    "The network between biological neurons (From: [Wikipedia](https://en.wikipedia.org/wiki/Neuron#/media/File:Chemical_synapse_schema_cropped.jpg))\n",
    "\n",
    "<br/>\n",
    "\n",
    "For a detailed description about the mechanism of a biological neural network, visit [the Wikipedia page](https://en.wikipedia.org/wiki/Neuron#Mechanisms_for_propagating_action_potentials): each neuron gets excited (activated) when it receives electrical signals from other connected neurons. Each connection between neurons has different strengths. Some connections are strong enough to activate other neurons whereas some connections suppress activation. Together, the hundreds of billions of neurons and connections in our brain embody human intelligence.\n",
    "\n",
    "The research into biological neurons led to the creation of a new computing paradigm, the [artificial neural network](https://en.wikipedia.org/wiki/Artificial_neural_network). With artificial neural networks, we mimic the behavior of biological neurons with simple mathematics. To solve the above classification problem, you can use the following simple neural network, which features a single neuron (aka Perceptron).\n",
    "\n",
    "<br/>\n",
    "\n",
    "![](https://cloud.google.com/blog/big-data/2016/07/images/146798944178238/neural-networks-13.png)\n",
    "\n",
    "<br/>\n",
    "\n",
    "x1 and x2 are the input values, and w1 and w2 are weights that represent the strength of each connection to the neuron. b is the so-called **bias**, representing the threshold to determine whether or not a neuron is activated by the inputs. This single neuron can be calculated with the following formula.\n",
    "\n",
    "<br/>\n",
    "\n",
    "![](https://cloud.google.com/blog/big-data/2016/07/images/146798944178238/neural-networks-15.png)\n",
    "\n",
    "<br/>\n",
    "\n",
    "Yes, that's exactly the same formula we used for classifying the datasets with a straight line. And actually, that's the only thing an artificial neuron can do: classify a data point into one of two kinds by examining input values with weights and bias. With two inputs, a neuron can **classify the data points in two-dimensional space into two kinds** with a straight line. If you have three inputs, a neuron can classify data points in three-dimensional space into two parts with a flat plane, and so on. This is called \"dividing n-dimensional space with a [hyperplane](https://en.wikipedia.org/wiki/Hyperplane)\".\n",
    "\n",
    "<br/>\n",
    "\n",
    "![](https://cloud.google.com/blog/big-data/2016/07/images/146798944178238/neural-networks-20.png)\n",
    "A neuron classifies any data point into one of two kinds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lab: What a single neuron does?\n",
    "\n",
    "- Discuss with your buddy what a single neuron does. \n",
    "- What does it mean by \"classifying the data points in two dimensional space into to kind\"? How do you explain it with the Playground case?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What We Learned\n",
    "\n",
    "In this section, we have learned the following concepts.\n",
    "\n",
    "- A Neural Network is just like a **function** in math or programming. It learns the expected output for a given input from **training datasets**\n",
    "- **TensorFlow Playground** let you play with a neural network and see how it works\n",
    "- **A Neuron** classifies a data point into two kinds, by adding all inputs multiplied with **weights** and seeing if it exceeds a threshold (**bias**)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}