09 Error handling and testing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "It is bad when a program crashes without an error message. It is even worse when a program used in \n",
    "engineering, science or finance computes an erroneous result. A crash without an informative error message causes\n",
    "frustration, and it costs time because we get no information on what went wrong, and this makes it hard to fix. \n",
    "An erroneous result used in an engineering design could risk life.\n",
    "With good software engineering we can mitigate these risks.\n",
    "\n",
    "\n",
    "## Bugs and testing\n",
    "\n",
    "Programs have bugs and users make mistakes. It is important to minimise the number of bugs,\n",
    "and where possible detect user mistakes. We do this through careful engineering of a program,\n",
    "error checking and extensive (automated) testing. Testing is critical in engineering programs to build confidence that\n",
    "a program computes the 'right' thing. Good software engineering and thorough testing would have prevented both the Patriot missile and Ariane 5 failures that were discussed in Activity 03.\n",
    "\n",
    "Testing is not a one-off exercise. As you develop programs, it is good practice to add tests as you go. It is also good to test very small units of a program so that if a test fails you can quickly narrow down the possible causes.\n",
    "Modern software development uses *continuous integration*, in which a suite of tests is run every time a change is made to a code. This helps catch issues early. It also helps tremendously when more than one person is working on a program. You may add a feature today and test it, but tomorrow your colleague might make a seemingly unrelated change that breaks your feature. Continuous testing helps in this case.\n",
    "\n",
    "\n",
    "## Objectives\n",
    "\n",
    "- Introduction to raising exceptions\n",
    "- Use exception handling\n",
    "- Creation of tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Errors and exceptions\n",
    "\n",
    "There are two types of program errors: *syntax errors* and *exceptions*.\n",
    "\n",
    "\n",
    "## Syntax errors\n",
    "\n",
    "Syntax errors are when the syntax of a program does not conform to the rules of the language. These are (generally) easy to detect because the interpreter/compiler will print an error. You will have seen plenty of syntax error messages by now!\n",
    "\n",
    "\n",
    "## Exceptions\n",
    "\n",
    "Exceptions are when something unexpected or anomalous occurs during the execution of a program. The syntax is correct, but something goes\n",
    "wrong while a program is running. Simple examples, where a well-engineered program will *raise an exception*, include:\n",
    "\n",
    "- Attempting to divide by zero;\n",
    "- Receiving negative data when only positive data is permitted, e.g. taking the \n",
    "  log of a negative number, or a negative integer for the number students in a class;\n",
    "- Unexpected integer overflows; and\n",
    "- Attempting to compute the dot product between two vectors of different lengths.\n",
    "\n",
    "These are all cases that can be tested in a program. We should check \n",
    "for invalid data and attempts to use our programs beyond the limits of their designs,\n",
    "and in these cases raise an exception with an informative message.\n",
    "\n",
    "\n",
    "### USS Yorktown Smart Ship\n",
    "\n",
    "USS Yorktown was used for the US Navy Smart Ship program. A computer system was fitted to operate a control \n",
    "centre from the bridge. In 1997, a crew member entered data into the system that led to an attempted division \n",
    "by zero. This caused the ship's computer systems and the ship's propulsion systems to shut down.\n",
    "This would have been avoided with good software engineering and exception handling.\n",
    "You can find more at https://en.wikipedia.org/wiki/USS_Yorktown_(CG-48)#Smart_ship_testbed.\n",
    "\n",
    "<a href=\"https://commons.wikimedia.org/wiki/File:Starboard_bow_view_of_USS_Yorktown_(CG-48)_underway_in_Caribbean_c1985.jpg#/media/File:Starboard_bow_view_of_USS_Yorktown_(CG-48)_underway_in_Caribbean_c1985.jpg\"><img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/0/0e/Starboard_bow_view_of_USS_Yorktown_%28CG-48%29_underway_in_Caribbean_c1985.jpg/1200px-Starboard_bow_view_of_USS_Yorktown_%28CG-48%29_underway_in_Caribbean_c1985.jpg\" alt=\"Starboard bow view of USS Yorktown (CG-48) underway in Caribbean c1985.jpg\" width=\"400\"></a><br>By US Navy, unknown scene camera operator - This media is available in the holdings of the <a href=\"//commons.wikimedia.org/wiki/National_Archives_and_Records_Administration\" title=\"National Archives and Records Administration\">National Archives and Records Administration</a>, cataloged under the National Archives Identifier (NAID) <a rel=\"nofollow\" class=\"external text\" href=\"https://catalog.archives.gov/id/6393040\">6393040</a>., Public Domain, <a href=\"https://commons.wikimedia.org/w/index.php?curid=60618\">Link</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Raising exceptions\n",
    "\n",
    "*Raising an exception* is what happens when we trigger an exception. It is sometimes called *throwing an exception*. Python peforms some checks for us. For example, if we have a list of length 5:\n",
    "```python\n",
    "colours = [\"green\", \"blue\", \"yellow\", \"red\", \"orange\"]\n",
    "```\n",
    "and we attempt to index beyond the end of the list:\n",
    "```python\n",
    "c = colours[6]\n",
    "```\n",
    "we will see the error message:\n",
    "```python\n",
    "---------------------------------------------------------------------------\n",
    "IndexError                                Traceback (most recent call last)\n",
    "<ipython-input-26-322e6eedcde5> in <module>()\n",
    "      1 colours = [\"green\", \"blue\", \"yellow\", \"red\", \"orange\"]\n",
    "----> 2 c = colours[6]\n",
    "IndexError: list index out of range\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python can check for some errors, but not all. We need to include checks in our programs to raise exceptions when something unexpected happens or invalid input is received. This reduces the likelihood of our engineering programs computing incorrect results, which could have serious consequences, and it makes it easier to debug programs. We will start with raising exceptions, and move later to how we can handle exceptions when one is raised. \n",
    "\n",
    "Below we build on some examples from earlier notebooks to add exception handling."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parameter validity checking\n",
    "\n",
    "In the first notebook we looked at the gravitational potential $V$ of a body of mass $m$ (point mass) at a distance $r$ from a body of mass $M$:\n",
    "\n",
    "$$\n",
    "V = \\frac{G M m}{r}\n",
    "$$\n",
    "\n",
    "where $G$ is the *gravitational constant*. This expression makes sense only for $G, M, m \\ge 0$ and $r > 0$. Implementing computation of the gravitational potential as a function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.59625857142857\n"
     ]
    }
   ],
   "source": [
    "def gravity_potential(G, M, m, r):\n",
    "    return G*M*m/r\n",
    "    \n",
    "V = gravity_potential(6.674e-11, 1.65e12, 6.1e2, 7e3)\n",
    "print(V)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A user could easily make a typographical error and make $G$, $M$, or $m$ negative, or we might encounter a case where $r$ becomes zero. In any of these cases, the computed gravitational potential would not be sensible but the above program would return a value and proceed.\n",
    "\n",
    "Rather than return an obviously wrong result, we can guard against easy-to-detect errors by checking the validity of the arguments and raising exceptions in the case of invalid data, e.g.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gravity_potential(G, M, m, r):\n",
    "    if G < 0:\n",
    "        raise ValueError(\"Gravitational constant must be greater than or equal to zero\")\n",
    "    if M < 0:\n",
    "        raise ValueError(\"Mass M must be greater than or equal to zero\")\n",
    "    if m < 0:\n",
    "        raise ValueError(\"Mass m must be greater than or equal to zero\")\n",
    "    if r <= 0:\n",
    "        raise ValueError(\"Distance r must be greater than zero\")\n",
    "    return G*M*m/r\n",
    "\n",
    "V = gravity_potential(6.674e-11, 1.65e12, 6.1e2, 7e3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, if we attempt\n",
    "\n",
    "```python\n",
    "V = gravity_potential(-6.674e-11, 1.65e12, 6.1e2, 7e3)\n",
    "```\n",
    "\n",
    "we would see:\n",
    "\n",
    "```python\n",
    "---------------------------------------------------------------------------\n",
    "ValueError                                Traceback (most recent call last)\n",
    "<ipython-input-20-ef1cff047ffd> in <module>()\n",
    "     10     return G*M*m/r\n",
    "     11 \n",
    "---> 12 V = gravity_potential(-6.674e-11, 1.65e12, 6.1e2, 7e3)\n",
    "\n",
    "<ipython-input-20-ef1cff047ffd> in gravity_potential(G, M, m, r)\n",
    "      1 def gravity_potential(G, M, m, r):\n",
    "      2     if G < 0:\n",
    "----> 3         raise ValueError(\"Gravitational constant must be greater than or equal to zero\")\n",
    "      4     if M < 0:\n",
    "      5         raise ValueError(\"Mass M must be greater than or equal to zero\")\n",
    "\n",
    "```\n",
    "```\n",
    "ValueError: Gravitational constant must be greater than or equal to zero\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The error is detected and a message is printed that makes clear what the problem is. The message also shows where in the program the exception was raised.\n",
    "\n",
    "Be liberal with the addition of checks in your programs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exception types\n",
    "\n",
    "Above we have used the exception type '`ValueError`'. There are multiple built-in exception types (see https://docs.python.org/3/library/exceptions.html), and we can even write our own. Pick the exception type that\n",
    "best describes the type of error encountered. Above, we have used the exception type `ValueError` because it describes what we were checking for. If none of the specific built-in exceptions accurately describe the error, '`RuntimeError`' is often the best choice."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Averting the Ariane 5 explosion\n",
    "\n",
    "Recall from Activity 03 that an Ariane 5 rocket exploded shortly after takeoff due to an integer overflow - the navigation software attempted to convert a 64-bit float (it was a velocity component) to a 16-bit integer. The largest value a 16-bit integer can store is 32767, and the velocity of the rocket exceeded 32767.\n",
    "\n",
    "The below snippet of code would have raised an informative exception in pre-launch testing, which could then have been easily fixed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "def cast_velocity(v):\n",
    "    \"Cast velocity to 16-bit with checking for overflow\"\n",
    "    \n",
    "    # Largest number a 16-bit int can store (http://docs.scipy.org/doc/numpy/reference/generated/numpy.iinfo.html)\n",
    "    max16_it = np.iinfo(np.int16).max\n",
    "\n",
    "    # Assign float velocity to 16-bit int if safe, otherwise raise exception\n",
    "    if abs(v) <= max16_it:\n",
    "         return np.int16(v)\n",
    "    else:\n",
    "        raise OverflowError(\"Cannot safely cast velocity to 16-bit int.\")\n",
    "\n",
    "        \n",
    "# Cast rocket velocity to 16-bit int. If velocity is greater than 32767 an exception will be raised.\n",
    "v = cast_velocity(32767.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Increase the velocity to over 32767 to see what happens.\n",
    "\n",
    "This bug should have been detected during pre-launch testing. However, \n",
    "unexpected issues could arise during rocket flight and we cannot just shut down the control software. \n",
    "The program needs to attempt to recover the situation.\n",
    "Next we look at how to handle problems that arise during execution of a program. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Catching and handling exceptions\n",
    "\n",
    "When an exception is raised, by default a program will exit. However, we do not always want to exit a program\n",
    "when we encounter an exception. Sometimes we want to 'catch' the exception and do something else. \n",
    "\n",
    "\n",
    "### Exception handling and the USS Yorktown\n",
    "\n",
    "In the case of the USS Yorktown, the ship's software should not stop if input data \n",
    "leads to a divide-by-zero, nor should it proceed erroneously and without warning.\n",
    "The software needs to 'catch' the divide-by-zero exception, and do something else.\n",
    "This might be reducing propulsion and asking for revised input.\n",
    "\n",
    "In Python we catch exceptions with\n",
    "```python\n",
    "try:\n",
    "    # Attempt to do something here that might raise an exception\n",
    "except FooError:\n",
    "    # If a 'FooError' exception is raised above, do something here. For other exception types\n",
    "    # the program will exit (if FooError is left out, any exception will end up here)\n",
    "```\n",
    "This is the basic and most common construct. It can be made more elaborate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Checking interactive user input\n",
    "\n",
    "Say our program asks the user to enter their age, and they enter an invalid value (negative number, something other than a number, etc). A program should raise an exception when an invalid value is input, but rather than exiting it would be better to ask the user to enter their age again. Here is an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_user_age():\n",
    "    \"Function that asks user for their age. If input is invalid, user is prompted to try again\"\n",
    "    try:\n",
    "        # Get age from user input - if conversion to int fails Python raises an exception\n",
    "        age = int(input('How old are you? '))\n",
    "\n",
    "        # Conversion to int has been successful, but we need to check that age is positive. Raise\n",
    "        # exception if age is less than 0\n",
    "        if age < 0:\n",
    "            raise ValueError(\"Age must be a positive integer\")\n",
    "\n",
    "        return age\n",
    "    except:\n",
    "        # Getting age from user input unsuccessful, so print message\n",
    "        print(\"Invalid age entered. Please try again\")\n",
    "\n",
    "        # Prompt user again to input age\n",
    "        return get_user_age()\n",
    "\n",
    "    \n",
    "# Uncomment the below lines to test\n",
    "# age = get_user_age()\n",
    "# print(age)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example: integer type conversion\n",
    "\n",
    "Below is another (contrived) take on the Ariane 5 rocket. Our preference is to cast the velocity \n",
    "component (float) to a 16-bit integer, but if the value is too large for a 16-bit integer we want to cast to a 32-bit integer. If the float is too large for a 32-bit integer we want to exit the program with an error message.\n",
    "\n",
    "We first provide two functions for converting to a integer: the first to a 16-bit integer and the second to a 32-bit integer: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cast_velocity16(v):\n",
    "    \"Convert to a 16-bit int. Raise exception if this will cause an overflow\"\n",
    "    if abs(v) <= np.iinfo(np.int16).max:\n",
    "         return np.int16(v)\n",
    "    else:\n",
    "        raise OverflowError(\"Cannot safely cast velocity to 16-bit int.\")\n",
    "\n",
    "        \n",
    "def cast_velocity32(v):\n",
    "    \"Convert to a 32-bit int. Raise exception if this will cause an overflow\"\n",
    "    if abs(v) <= np.iinfo(np.int32).max:\n",
    "         return np.int32(v)\n",
    "    else:\n",
    "        raise OverflowError(\"Cannot safely cast velocity to 32-bit int.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now perform a conversion, trying first to convert to a 16-bit integer, and if that is unsuccessful we attempt\n",
    "a conversion to a 32-bit integer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "42767\n",
      "<class 'numpy.int32'>\n",
      "3210\n",
      "<class 'numpy.int16'>\n"
     ]
    }
   ],
   "source": [
    "def cast_velocity(v):\n",
    "    try:\n",
    "        # Try to cast v to a 16-bit int\n",
    "        return cast_velocity16(v)\n",
    "    except OverflowError:\n",
    "        # If cast to 16-bit int failed (and exception raised), try casting to a 32-bit int\n",
    "        try:\n",
    "            return cast_velocity32(v)\n",
    "        except OverflowError:\n",
    "            # If cast to 32-bit int failed, raise exception\n",
    "            raise RuntimeError(\"Could not safely cast velocity to an available int type.\")\n",
    "\n",
    "# Velocity to cast (too large for a 16-bit int)\n",
    "v_int = cast_velocity(42767.0)            \n",
    "print(v_int)\n",
    "print(type(v_int))\n",
    "\n",
    "# Velocity to cast (small enough for a 16-bit int)\n",
    "v_int = cast_velocity(3210.0)            \n",
    "print(v_int)\n",
    "print(type(v_int))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Testing\n",
    "\n",
    "Testing is a critical part of software engineering for enhancing program quality, and for building the\n",
    "confidence we\n",
    "and others will have in a program. Testing is not only performed when developing a new program. Programs should\n",
    "come\n",
    "with a suite of tests that can be run regularly. This helps detect errors that might inadvertently creep into a program. For large projects, these tests might be run nightly or even every time a change is made to a program (the latter is known as *continuous integration*).\n",
    "Any program used for engineering analysis and design should have an \n",
    "extensive suite of tests.\n",
    "It would be negligent to use a program in real-life engineering analysis that is not covered by extensive testing.\n",
    "\n",
    "When testing a program, we should test for both valid and invalid input data. For the valid cases the\n",
    "computed result should be checked against a known correct solution. For the invalid data cases, tests should check that an exception is raised. We will consider the former, and address the latter in an optional section.\n",
    "\n",
    "Ideally, tests for a large program should have different levels of granularity. Some tests should test small blocks (individual functions), ideally in isolation from other parts of a program. The cause of unexpected behaviour in a large program can then be pin-pointed quickly. Higher level tests should test a program as a whole, and would typically represent user cases.\n",
    "\n",
    "The programming examples in the preceding notebooks included little or no checking/testing. From now, we want\n",
    "to add more checks to our programs. For an example of correctness testing, let's\n",
    "consider the Fibonacci series from Activity 04. The function for computing Fibonacci numbers is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(n): \n",
    "    \"Compute the nth Fibonacci number using recursion\"\n",
    "    if n == 0:\n",
    "        return 0  # This doesn't call f, so it breaks out of the recursion loop\n",
    "    elif n == 1:\n",
    "        return 1  # This doesn't call f, so it breaks out of the recursion loop\n",
    "    else:\n",
    "        return f(n - 1) + f(n - 2)  # This calls f for n-1 and n-2 (recursion), and returns the sum "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To build our confidence that the function is correct, we can check a number of computed terms in the series against\n",
    "known results. A helpful tool for this is the `assert` keyword:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert f(0) == 0\n",
    "assert f(1) == 1\n",
    "assert f(2) == 1\n",
    "assert f(3) == 2\n",
    "assert f(10) == 55\n",
    "assert f(15) == 610"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If all the assertions are true, there should be no output. Try changing one of the checks to trigger an assertion failure."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test frameworks (optional)\n",
    "\n",
    "Testing is so important to good software engineering that there are many tools to help us to write and run tests. A popular and powerful testing library for Python is `pytest` (http://doc.pytest.org/en/latest/). Before using it, \n",
    "we need to check that it is installed. We try to import it, and if that fails we install it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "try: \n",
    "    import pytest\n",
    "except:\n",
    "    try:\n",
    "        !{sys.executable} -m pip -q install pytest\n",
    "        import pytest\n",
    "    except ImportError:\n",
    "        !{sys.executable} -m pip -q --user install pytest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have seen some simple testing of the Fibonacci series that checks for correctness. For the gravity potential problem at the start of the notebook we could also add some checks for correctness. But, we also would like \n",
    "an automated process to test that an exception *is* raised when user input is invalid."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider the gravity potential problem from the start of the notebook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gravity_potential(G, M, m, r):\n",
    "    if G < 0:\n",
    "        raise ValueError(\"Gravitational constant must be greater than or equal to zero\")\n",
    "    if M < 0:\n",
    "        raise ValueError(\"Mass M must be greater than or equal to zero\")\n",
    "    if m < 0:\n",
    "        raise ValueError(\"Mass m must be greater than or equal to zero\")\n",
    "    if r <= 0:\n",
    "        raise ValueError(\"Distance r must be greater than zero\")\n",
    "    return G*M*m/r"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now would like to add some tests that check that invalid data raises an exception, i.e. we want to exit with an error if the function `gravity_potential` *does not* raise an exception. We can do this with PyTest:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use PyTest to check that ValueError is raised for invalid input data\n",
    "import pytest\n",
    "\n",
    "# Check that G < zero raises a ValueError\n",
    "with pytest.raises(ValueError):\n",
    "    gravity_potential(-6.674e-11, 1.65e12, 6.1e2, 7e3)\n",
    "\n",
    "# Check that M < zero raises a ValueError\n",
    "with pytest.raises(ValueError):\n",
    "    gravity_potential(6.674e-11, -1.65e12, 6.1e2, 7e3)\n",
    "\n",
    "# Check that m < zero raises a ValueError\n",
    "with pytest.raises(ValueError):\n",
    "    gravity_potential(6.674e-11, 1.65e12, -6.1e2, 7e3)\n",
    "\n",
    "# Check that r < zero raises a ValueError\n",
    "with pytest.raises(ValueError):\n",
    "    gravity_potential(6.674e-11, 1.65e12, 6.1e2, -7e3)\n",
    "\n",
    "# Check that r=0 raises a ValueError\n",
    "with pytest.raises(ValueError):\n",
    "    gravity_potential(6.674e-11, 1.65e12, 6.1e2, 0.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We should test that our programs compute correct results, and we should check that exceptions are indeed raised\n",
    "for invalid input data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercises\n",
    "\n",
    "Complete now the [09 Exercises](Exercises/09%20Exercises.ipynb) notebook."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}