Variation2.ListComprehensions.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[32mimport \u001b[39m\u001b[36m$file.$     \n",
       "\u001b[39m\n",
       "\u001b[32mimport \u001b[39m\u001b[36mcommon._\n",
       "\u001b[39m\n",
       "\u001b[32mimport \u001b[39m\u001b[36mdoobie.implicits._\u001b[39m"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import $file.common\n",
    "import common._\n",
    "import doobie.implicits._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Variation 2. List comprehensions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's pretend that we can use in-memory data structures to consult and manipulate the information stored in the world database. In Scala, we would commonly use case classes, `List`s, `Map`s, and other data types from the Scala standard library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defined \u001b[32mclass\u001b[39m \u001b[36mCountry\u001b[39m\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mCity\u001b[39m\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mWorld\u001b[39m"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "case class Country(\n",
    "    code: String, \n",
    "    name: String, \n",
    "    capital: Option[Int])\n",
    "\n",
    "case class City(\n",
    "    id: Int, \n",
    "    name: String, \n",
    "    countryCode: String, \n",
    "    population: Int)\n",
    "\n",
    "case class World(\n",
    "    countries: Map[String, Country],\n",
    "    cities: Map[Int, City]){\n",
    "    \n",
    "    val allCountries: List[Country] = \n",
    "        countries.values.toList\n",
    "    val allCities: List[City] = \n",
    "        cities.values.toList\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `largeCapitals` query can then be written using for-comprehensions (which build upon common higher-order functions `flatMap`, `filter` and `map`) in a very concise and readable way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defined \u001b[32mfunction\u001b[39m \u001b[36mlargeCapitals\u001b[39m"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def largeCapitals(implicit world: World): List[(String, String)] =\n",
    "    for {\n",
    "        Country(_, name, Some(capitalId)) <- world.allCountries\n",
    "        city <- world.cities.get(capitalId)\n",
    "        if city.population > 8000000\n",
    "    } yield (city.name, name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Modularity FTW!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is just fine, but in a more complex setting, we may greatly benefit from decomposing our queries into smaller and reusable building blocks. This would lead to a query made from query fragments as follows: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defined \u001b[32mfunction\u001b[39m \u001b[36mcity\u001b[39m\n",
       "defined \u001b[32mfunction\u001b[39m \u001b[36mlargeCity\u001b[39m\n",
       "defined \u001b[32mfunction\u001b[39m \u001b[36mlargeCapitalsM\u001b[39m"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def city(maybeId: Option[Int])(implicit world: World): Option[City] = \n",
    "    for {\n",
    "        cityId <- maybeId\n",
    "        city <- world.cities.get(cityId)\n",
    "    } yield city\n",
    "\n",
    "def largeCity(maybeId: Option[Int], size: Long)(implicit world: World): Option[City] = \n",
    "    for {\n",
    "        city <- city(maybeId)\n",
    "        if size < city.population\n",
    "    } yield city\n",
    "\n",
    "def largeCapitalsM(implicit world: World): List[(String, String)] = \n",
    "    for {\n",
    "        country <- world.allCountries\n",
    "        city <- largeCity(country.capital, 8000000)\n",
    "    } yield (city.name, country.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What about testing?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In a general-purpose language setting, we also strive for unit testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[32mimport \u001b[39m\u001b[36morg.scalatest._\n",
       "\n",
       "\u001b[39m\n",
       "defined \u001b[32mclass\u001b[39m \u001b[36mLargeCapitalsSpec\u001b[39m"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import org.scalatest._\n",
    "\n",
    "class LargeCapitalsSpec(largeCapitals: World => List[(String, String)])\n",
    "extends FlatSpec with Matchers{\n",
    "    \n",
    "    val smallWorld: World =         \n",
    "        World(Map(\"ES\" -> Country(\"ES\",\"Spain\",Some(0)),\n",
    "                \"USA\" -> Country(\"USA\", \"United States\", Some(1)),\n",
    "                \"UK\" -> Country(\"UK\", \"United Kingdom\", Some(2)),\n",
    "                \"UNK\" -> Country(\"UNK\", \"Unknown\", None)),\n",
    "        Map(0->City(0,\"Madrid\",\"ES\",9000000),\n",
    "            1->City(1,\"Washington\", \"USA\", 10000000),\n",
    "            2->City(2,\"London\", \"UK\", 500000)))    \n",
    "    \n",
    "    \"large capitals\" should \"be right\" in {\n",
    "        largeCapitals(smallWorld).toSet shouldBe \n",
    "            Set((\"Madrid\", \"Spain\"), (\"Washington\", \"United States\"))\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32mcmd4$Helper$LargeCapitalsSpec:\u001b[0m\n",
      "\u001b[32mlarge capitals\u001b[0m\n",
      "\u001b[32m- should be right\u001b[0m\n",
      "\u001b[32mcmd4$Helper$LargeCapitalsSpec:\u001b[0m\n",
      "\u001b[32mlarge capitals\u001b[0m\n",
      "\u001b[32m- should be right\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "run(new LargeCapitalsSpec(largeCapitals(_)))\n",
    "run(new LargeCapitalsSpec(largeCapitalsM(_)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### But this is all too impractical, right?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is most idiomatic code in Scala, but, of course, normally datasets don't fit into main memory. In our sample world database, it run without problems: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[36mrealWorld\u001b[39m: \u001b[32mWorld\u001b[39m = \u001b[33mWorld\u001b[39m(\n",
       "  \u001b[33mHashMap\u001b[39m(\n",
       "    \u001b[32m\"CYM\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"CYM\"\u001b[39m, \u001b[32m\"Cayman Islands\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m553\u001b[39m)),\n",
       "    \u001b[32m\"MNG\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"MNG\"\u001b[39m, \u001b[32m\"Mongolia\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m2696\u001b[39m)),\n",
       "    \u001b[32m\"SYR\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"SYR\"\u001b[39m, \u001b[32m\"Syria\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3250\u001b[39m)),\n",
       "    \u001b[32m\"LTU\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"LTU\"\u001b[39m, \u001b[32m\"Lithuania\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m2447\u001b[39m)),\n",
       "    \u001b[32m\"GMB\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"GMB\"\u001b[39m, \u001b[32m\"Gambia\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m904\u001b[39m)),\n",
       "    \u001b[32m\"UZB\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"UZB\"\u001b[39m, \u001b[32m\"Uzbekistan\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3503\u001b[39m)),\n",
       "    \u001b[32m\"MAC\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"MAC\"\u001b[39m, \u001b[32m\"Macao\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m2454\u001b[39m)),\n",
       "    \u001b[32m\"KHM\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"KHM\"\u001b[39m, \u001b[32m\"Cambodia\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m1800\u001b[39m)),\n",
       "    \u001b[32m\"ROM\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"ROM\"\u001b[39m, \u001b[32m\"Romania\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3018\u001b[39m)),\n",
       "    \u001b[32m\"BMU\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"BMU\"\u001b[39m, \u001b[32m\"Bermuda\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m191\u001b[39m)),\n",
       "    \u001b[32m\"GUF\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"GUF\"\u001b[39m, \u001b[32m\"French Guiana\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3014\u001b[39m)),\n",
       "    \u001b[32m\"SWE\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"SWE\"\u001b[39m, \u001b[32m\"Sweden\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3048\u001b[39m)),\n",
       "    \u001b[32m\"ARM\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"ARM\"\u001b[39m, \u001b[32m\"Armenia\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m126\u001b[39m)),\n",
       "    \u001b[32m\"LAO\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"LAO\"\u001b[39m, \u001b[32m\"Laos\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m2432\u001b[39m)),\n",
       "    \u001b[32m\"SPM\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"SPM\"\u001b[39m, \u001b[32m\"Saint Pierre and Miquelon\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3067\u001b[39m)),\n",
       "    \u001b[32m\"GNQ\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"GNQ\"\u001b[39m, \u001b[32m\"Equatorial Guinea\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m2972\u001b[39m)),\n",
       "    \u001b[32m\"USA\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"USA\"\u001b[39m, \u001b[32m\"United States\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3813\u001b[39m)),\n",
       "    \u001b[32m\"CXR\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"CXR\"\u001b[39m, \u001b[32m\"Christmas Island\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m1791\u001b[39m)),\n",
       "    \u001b[32m\"VAT\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"VAT\"\u001b[39m, \u001b[32m\"Holy See (Vatican City State)\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3538\u001b[39m)),\n",
       "    \u001b[32m\"MDV\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"MDV\"\u001b[39m, \u001b[32m\"Maldives\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m2463\u001b[39m)),\n",
       "    \u001b[32m\"FJI\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"FJI\"\u001b[39m, \u001b[32m\"Fiji Islands\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m764\u001b[39m)),\n",
       "    \u001b[32m\"HTI\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"HTI\"\u001b[39m, \u001b[32m\"Haiti\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m929\u001b[39m)),\n",
       "    \u001b[32m\"IDN\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"IDN\"\u001b[39m, \u001b[32m\"Indonesia\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m939\u001b[39m)),\n",
       "    \u001b[32m\"LVA\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"LVA\"\u001b[39m, \u001b[32m\"Latvia\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m2434\u001b[39m)),\n",
       "    \u001b[32m\"HUN\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"HUN\"\u001b[39m, \u001b[32m\"Hungary\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3483\u001b[39m)),\n",
       "    \u001b[32m\"BLR\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"BLR\"\u001b[39m, \u001b[32m\"Belarus\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3520\u001b[39m)),\n",
       "    \u001b[32m\"MSR\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"MSR\"\u001b[39m, \u001b[32m\"Montserrat\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m2697\u001b[39m)),\n",
       "    \u001b[32m\"PNG\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"PNG\"\u001b[39m, \u001b[32m\"Papua New Guinea\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m2884\u001b[39m)),\n",
       "    \u001b[32m\"GUY\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"GUY\"\u001b[39m, \u001b[32m\"Guyana\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m928\u001b[39m)),\n",
       "    \u001b[32m\"DNK\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"DNK\"\u001b[39m, \u001b[32m\"Denmark\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3315\u001b[39m)),\n",
       "    \u001b[32m\"FIN\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"FIN\"\u001b[39m, \u001b[32m\"Finland\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3236\u001b[39m)),\n",
       "    \u001b[32m\"DMA\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"DMA\"\u001b[39m, \u001b[32m\"Dominica\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m586\u001b[39m)),\n",
       "    \u001b[32m\"COG\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"COG\"\u001b[39m, \u001b[32m\"Congo\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m2296\u001b[39m)),\n",
       "    \u001b[32m\"CIV\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"CIV\"\u001b[39m, \u001b[32m\"C\\u00f4te d\\u0092Ivoire\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m2814\u001b[39m)),\n",
       "    \u001b[32m\"SDN\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"SDN\"\u001b[39m, \u001b[32m\"Sudan\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3225\u001b[39m)),\n",
       "    \u001b[32m\"CHE\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"CHE\"\u001b[39m, \u001b[32m\"Switzerland\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m3248\u001b[39m)),\n",
       "    \u001b[32m\"VIR\"\u001b[39m -> \u001b[33mCountry\u001b[39m(\u001b[32m\"VIR\"\u001b[39m, \u001b[32m\"Virgin Islands, U.S.\"\u001b[39m, \u001b[33mSome\u001b[39m(\u001b[32m4067\u001b[39m)),\n",
       "..."
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val realWorld = World(\n",
    "    Map.from(\n",
    "        sql\"select code, code, name, capital from country\"\n",
    "            .query[(String, Country)].to[List].transact(xa).unsafeRunSync),\n",
    "    Map.from(\n",
    "        sql\"select id, id, name, countryCode, population from city\"\n",
    "            .query[(Int, City)].to[List].transact(xa).unsafeRunSync))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "186074 nanos\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\u001b[36mres9\u001b[39m: \u001b[32mList\u001b[39m[(\u001b[32mString\u001b[39m, \u001b[32mString\u001b[39m)] = \u001b[33mList\u001b[39m(\n",
       "  (\u001b[32m\"Jakarta\"\u001b[39m, \u001b[32m\"Indonesia\"\u001b[39m),\n",
       "  (\u001b[32m\"Ciudad de M\\u00e9xico\"\u001b[39m, \u001b[32m\"Mexico\"\u001b[39m),\n",
       "  (\u001b[32m\"Moscow\"\u001b[39m, \u001b[32m\"Russian Federation\"\u001b[39m),\n",
       "  (\u001b[32m\"Seoul\"\u001b[39m, \u001b[32m\"South Korea\"\u001b[39m)\n",
       ")"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "largeCapitals(realWorld).timed(100).nanos"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ideally, we would like to program our queries as we did here, with higher-order functions, and be able to access the database. How do we do that? Let's see our first contender, the [Repository Pattern](Variation3.Repositories.ipynb)!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Scala",
   "language": "scala",
   "name": "scala"
  },
  "language_info": {
   "codemirror_mode": "text/x-scala",
   "file_extension": ".sc",
   "mimetype": "text/x-scala",
   "name": "scala",
   "nbconvert_exporter": "script",
   "version": "2.13.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}