Base/frameworks/pytorch/practice/60分钟入门PyTorch-1.PyTorch是什么？.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 60分钟入门深度学习工具-PyTorch(一、PyTorch 是什么)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**作者**：Soumith Chintala\n",
    "\n",
    "\n",
    "原文翻译自：https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html\n",
    "    \n",
    "中文翻译、注释制作：黄海广\n",
    "\n",
    "github：https://github.com/fengdu78\n",
    "\n",
    "代码全部测试通过。\n",
    "\n",
    "配置环境：PyTorch 1.0，python 3.6，\n",
    "\n",
    "主机：显卡：一块1080ti；内存：32g（注：绝大部分代码不需要GPU）\n",
    "![公众号](images/gongzhong.jpg)\n",
    "### 目录\n",
    "* 1.[Pytorch是什么？](60分钟入门PyTorch-1.PyTorch是什么？.ipynb)\n",
    "* 2.[AUTOGRAD](60分钟入门PyTorch-2.AUTOGRAD.ipynb)\n",
    "* 3.[神经网络](60分钟入门PyTorch-3.神经网络.ipynb)\n",
    "* 4.[训练一个分类器](60分钟入门PyTorch-4.训练一个分类器.ipynb)\n",
    "* 5.[数据并行](60分钟入门PyTorch-5.数据并行.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 一、PyTorch 是什么"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "他是一个基于Python的科学计算包，目标用户有两类\n",
    "\n",
    "* 为了使用GPU来替代numpy\n",
    "* 一个深度学习研究平台：提供最大的灵活性和速度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 开始"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 张量（Tensors)\n",
    "张量类似于numpy的ndarrays，不同之处在于张量可以使用GPU来加快计算。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "构建一个未初始化的5*3的矩阵："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0., 0., 0.],\n",
      "        [0., 0., 0.],\n",
      "        [0., 0., 0.],\n",
      "        [0., 0., 0.],\n",
      "        [0., 0., 0.]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.Tensor(5, 3)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "构建一个零矩阵，使用long的类型\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0, 0, 0],\n",
      "        [0, 0, 0],\n",
      "        [0, 0, 0],\n",
      "        [0, 0, 0],\n",
      "        [0, 0, 0]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.zeros(5, 3, dtype=torch.long)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从数据中直接构建一个张量(tensor)："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([5.5000, 3.0000])\n"
     ]
    }
   ],
   "source": [
    "x = torch.tensor([5.5, 3])\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "或者在已有的张量(tensor)中构建一个张量(tensor). 这些方法将重用输入张量(tensor)的属性，例如， dtype，除非用户提供新值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.]], dtype=torch.float64)\n",
      "tensor([[ 0.4742, -0.2118,  1.8551],\n",
      "        [-0.2984, -0.4816, -2.0312],\n",
      "        [-0.3725,  1.5228,  0.5752],\n",
      "        [-1.1978,  0.4541,  1.7682],\n",
      "        [-1.8775, -0.4712, -0.2484]])\n"
     ]
    }
   ],
   "source": [
    "x = x.new_ones(5, 3, dtype=torch.double)      # new_* methods take in sizes\n",
    "print(x)\n",
    "\n",
    "x = torch.randn_like(x, dtype=torch.float)    # 覆盖类型!\n",
    "print(x)                                      # result 的size相同"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.]], dtype=torch.float64)\n",
      "tensor([[-0.5474,  1.0514, -0.9948],\n",
      "        [-1.7351,  0.1658, -1.2403],\n",
      "        [ 1.4564, -0.1534,  0.2367],\n",
      "        [ 1.2123,  0.7427, -0.6357],\n",
      "        [-0.1546,  0.5876, -0.4393]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.ones(5, 3, dtype=torch.double)      # new_* methods take in sizes\n",
    "print(x)\n",
    "\n",
    "x = torch.randn_like(x, dtype=torch.float)    # 覆盖类型!\n",
    "print(x)         "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "获取张量(tensor)的大小"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([5, 3])\n",
      "torch.Size([5, 3])\n"
     ]
    }
   ],
   "source": [
    "print(x.size())\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** 注意 **\n",
    "\n",
    "`torch.Size`实际上是一个元组，所以它支持元组的所有操作。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 操作\n",
    "张量上的操作有多重语法形式，下面我们以加法为例进行讲解。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 语法1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.2687,  1.3077, -0.1701],\n",
      "        [-0.7820,  1.0664, -0.9537],\n",
      "        [ 2.3488,  0.0306,  1.0304],\n",
      "        [ 1.6032,  0.9730,  0.1564],\n",
      "        [ 0.3977,  1.0480,  0.2768]])\n"
     ]
    }
   ],
   "source": [
    "y = torch.rand(5, 3)\n",
    "print(x + y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 语法二"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.2687,  1.3077, -0.1701],\n",
      "        [-0.7820,  1.0664, -0.9537],\n",
      "        [ 2.3488,  0.0306,  1.0304],\n",
      "        [ 1.6032,  0.9730,  0.1564],\n",
      "        [ 0.3977,  1.0480,  0.2768]])\n"
     ]
    }
   ],
   "source": [
    "print(torch.add(x, y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 语法三：\n",
    "给出一个输出张量作为参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.2687,  1.3077, -0.1701],\n",
      "        [-0.7820,  1.0664, -0.9537],\n",
      "        [ 2.3488,  0.0306,  1.0304],\n",
      "        [ 1.6032,  0.9730,  0.1564],\n",
      "        [ 0.3977,  1.0480,  0.2768]])\n"
     ]
    }
   ],
   "source": [
    "result = torch.empty(5, 3)\n",
    "torch.add(x, y, out=result)\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 语法四：\n",
    "原地操作（in-place）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.2687,  1.3077, -0.1701],\n",
      "        [-0.7820,  1.0664, -0.9537],\n",
      "        [ 2.3488,  0.0306,  1.0304],\n",
      "        [ 1.6032,  0.9730,  0.1564],\n",
      "        [ 0.3977,  1.0480,  0.2768]])\n"
     ]
    }
   ],
   "source": [
    "# 把x加到y上\n",
    "y.add_(x)\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** 注意**\n",
    "\n",
    "任何在原地(in-place)改变张量的操作都有一个`_`后缀。例如`x.copy_(y)`, `x.t_()`操作将改变x.\n",
    "\n",
    "你可以使用所有的`numpy`索引操作。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "你可以使用各种类似标准NumPy的花哨的索引功能"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.5474,  1.0514, -0.9948],\n",
      "        [-1.7351,  0.1658, -1.2403],\n",
      "        [ 1.4564, -0.1534,  0.2367],\n",
      "        [ 1.2123,  0.7427, -0.6357],\n",
      "        [-0.1546,  0.5876, -0.4393]])\n",
      "tensor([ 1.0514,  0.1658, -0.1534,  0.7427,  0.5876])\n"
     ]
    }
   ],
   "source": [
    "print(x)\n",
    "print(x[:, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "调整大小：如果要调整张量/重塑张量，可以使用`torch.view`："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 4]) torch.Size([16]) torch.Size([2, 8])\n"
     ]
    }
   ],
   "source": [
    "x = torch.randn(4, 4)\n",
    "y = x.view(16)\n",
    "z = x.view(-1, 8)  # -1的意思是没有指定维度\n",
    "print(x.size(), y.size(), z.size())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1, 16])\n"
     ]
    }
   ],
   "source": [
    "yy = x.view(1,16)\n",
    "print( yy.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果你有一个单元素张量，使用`.item()`将值作为Python数字"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([-0.8757])\n",
      "-0.8757410645484924\n"
     ]
    }
   ],
   "source": [
    "x = torch.randn(1)\n",
    "print(x)\n",
    "print(x.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 稍后阅读\n",
    "\n",
    "[这里](https://pytorch.org/docs/stable/torch.html)描述了一百多种张量操作，包括转置，索引，数学运算，线性代数，随机数等。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## numpy桥\n",
    "把一个torch张量转换为numpy数组或者反过来都是很简单的。\n",
    "\n",
    "Torch张量和numpy数组将共享潜在的内存，改变其中一个也将改变另一个。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 把Torch张量转换为numpy数组"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([1., 1., 1., 1., 1.])\n"
     ]
    }
   ],
   "source": [
    "a = torch.ones(5)\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1. 1. 1. 1. 1.]\n",
      "<class 'numpy.ndarray'>\n"
     ]
    }
   ],
   "source": [
    "b = a.numpy()\n",
    "print(b)\n",
    "print(type(b))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过如下操作，我们看一下numpy数组的值如何在改变。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([2., 2., 2., 2., 2.])\n",
      "[2. 2. 2. 2. 2.]\n"
     ]
    }
   ],
   "source": [
    "a.add_(1)\n",
    "print(a)\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 把numpy数组转换为torch张量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看看改变numpy数组如何自动改变torch张量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2. 2. 2. 2. 2.]\n",
      "tensor([2., 2., 2., 2., 2.], dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "a = np.ones(5)\n",
    "b = torch.from_numpy(a)\n",
    "np.add(a, 1, out=a)\n",
    "print(a)\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "所有在CPU上的张量，除了字符张量，都支持在numpy之间转换。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## CUDA张量\n",
    "\n",
    "可以使用.to方法将张量移动到任何设备上。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([0.8831])\n",
      "cuda\n",
      "tensor([1.], device='cuda:0')\n",
      "tensor([1.8831], device='cuda:0')\n",
      "tensor([1.8831], dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "# let us run this cell only if CUDA is available\n",
    "# We will use ``torch.device`` objects to move tensors in and out of GPU\n",
    "x = torch.randn(1)\n",
    "print(x)\n",
    "if torch.cuda.is_available():\n",
    "    device = torch.device(\"cuda\")          # a CUDA device object\n",
    "    print(device)\n",
    "    y = torch.ones_like(x, device=device)  # directly create a tensor on GPU\n",
    "    print(y)\n",
    "    x = x.to(device)                       # or just use strings ``.to(\"cuda\")``\n",
    "    z = x + y\n",
    "    print(z)\n",
    "    print(z.to(\"cpu\", torch.double))       # ``.to`` can also change dtype together!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本章的官方代码：\n",
    "* Python：[tensor_tutorial.py](download/tensor_tutorial.py)\n",
    "* Jupyter notebook:[tensor_tutorial.ipynb](download/tensor_tutorial.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "pth",
   "language": "python",
   "name": "pth"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}