Lec07b.MooresLaw.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Moore's Law == More Parallelism\n",
    "\n",
    "Moore’s law -- The number of transistors that can be inexpensively placed on an integrated circuit is increasing exponentially, doubling approximately every two years.\n",
    "\n",
    "* The observation has held for half a century\n",
    "\n",
    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/9/9d/Moore%27s_Law_Transistor_Count_1971-2016.png\" width=\"768\" title=\"Moore's Law\" />\n",
    "\n",
    "Moore's law is commonly misinterpreted as meaning that processing power increases exponentially. This was the case for a long time, but not what Moore's law says.\n",
    "\n",
    "Moore's law continues to hold, but it is not always helpful:\n",
    "\n",
    "* More transistors has become more cores (independent processing units on the same chip)\n",
    "* Pipelined multicore (N k-flop cores) are not as useful as a big (Nk flop) processor\n",
    "* In 2004, a 'processor' turned into a parallel computer (Itanium 2)\n",
    "* This has progressed to processors with 16 cores, GPUs, etc.\n",
    "\n",
    "The chip vendors tell us we have faster processors, not parallel computers. So we (the programmers) must write parallel code to make software faster on multiple cores with the same clock speed and number of transistors.  \n",
    "  \n",
    "#### Conclusion: \n",
    "\n",
    "Moore's law does not mean that we get more processing power over time. It's not that simple. It's implementation has meant that we get more processing power at the expense of needing to encode more parallelism in programs.\n",
    "\n",
    "### Dennard Scaling: Why the Big Lie?\n",
    "\n",
    "As transistors get smaller their power density stays constant so that power is in proportion with area.\n",
    "Thus, voltage and current must scale downward.\n",
    "* Performance per watt increases exponentially \n",
    "* Smaller transistors lead to faster clock rates\n",
    "\n",
    "However, Dennard scaling ended in 2006. At the same time as the advent of multicore.\n",
    "* But Moore’s law still alive\n",
    "* Absent Dennard scaling, only choice was to turn to multicore processors\n",
    "\n",
    "<img src=\"http://15418.courses.cs.cmu.edu/spring2016content/lectures/01_whyparallelism/images/slide_033.jpg\" width=\"512\" title=\"Dennard Scaling\" />\n",
    "\n",
    "__Note__: ILP = Instruction level parallelism\n",
    "* Mostly this means pipelines and scalar processing.\n",
    "\n",
    "### Conclusion:\n",
    "\n",
    "Moore's law scaling as is commonly misinterpreted ended in 2006. This has been replaced with multi-core."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Is Moore's Law Dead?\n",
    "\n",
    "Likely, but we don't care. There will always be multicore\n",
    "\n",
    "* Why is Moore's law dead?\n",
    "  * Limits on miniturization of CMOS technology at 5 nm\n",
    "  * Dark silicon -- power density (post Dennard Scaling) means that not all transistors can be on.  Estimates of 50-80% dark currently.\n",
    "  * DARPA PM predicts that we will see a 50x increase in \"electronic\" over the next 30 years.  \n",
    "      * Compare with 1M times over the previous 30 years.\n",
    "      \n",
    "* What else?\n",
    "  * quantum computing\n",
    "  * biological processors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}