example_jupyter.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# *CoastSat*: example at Narrabeen-Collaroy, Australia\n",
    "\n",
    "This software is described in details in:\n",
    "* Vos K., Splinter K.D., Harley M.D., Simmons J.A., Turner I.L. (2019). CoastSat: a Google Earth Engine-enabled Python toolkit to extract shorelines from publicly available satellite imagery. Environmental Modelling and Software. 122, 104528. https://doi.org/10.1016/j.envsoft.2019.104528\n",
    "\n",
    "It enables the users to extract time-series of shoreline change over the last 30+ years at their site of interest.\n",
    "There are three main steps:\n",
    "1. Retrieval of the satellite images of the region of interest from Google Earth Engine\n",
    "2. Shoreline extraction at sub-pixel resolution\n",
    "3. Intersection of the shorelines with cross-shore transects\n",
    "\n",
    "## Initial settings\n",
    "\n",
    "Refer to the **Installation** section of the README for instructions on how to install the Python packages necessary to run the software, including Google Earth Engine Python API. If that step has been completed correctly, the following packages should be imported without any problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import pickle\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "import matplotlib.pyplot as plt\n",
    "from coastsat import SDS_download, SDS_preprocess, SDS_shoreline, SDS_tools, SDS_transects"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Retrieval of the images from GEE\n",
    "\n",
    "Define the region of interest (`polygon`), the date range (`dates`) and the satellite missions (`sat_list`) from which you wish to retrieve the satellite images. The images will be cropped on the Google Earth Engine server and only the region of interest will be downloaded as a .tif file. The files will stored in the directory defined in `filepath`.\n",
    "\n",
    "Make sure the area of your ROI is smaller than 100 km2 (if larger split it into smaller ROIs)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# region of interest (longitude, latitude)\n",
    "polygon = [[[151.2957545, -33.7012561],\n",
    "            [151.297557, -33.7388075],\n",
    "            [151.312234, -33.7390216],\n",
    "            [151.311204, -33.701399],\n",
    "            [151.2957545, -33.7012561]]]   \n",
    "# date range\n",
    "dates = ['2017-12-01', '2018-01-01']\n",
    "# satellite missions\n",
    "sat_list = ['S2']\n",
    "# name of the site\n",
    "sitename = 'NARRA'\n",
    "# directory where the data will be stored\n",
    "filepath = os.path.join(os.getcwd(), 'data')\n",
    "# put all the inputs into a dictionnary\n",
    "inputs = {'polygon': polygon, 'dates': dates, 'sat_list': sat_list, 'sitename': sitename, 'filepath':filepath}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Retrieve satellite images from GEE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metadata = SDS_download.retrieve_images(inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**If you have already retrieved the images**, just load the metadata file by only running the section below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metadata = SDS_download.get_metadata(inputs) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Shoreline extraction\n",
    "\n",
    "This section maps the position of the shoreline on the satellite images. The user can define the cloud threhold (`cloud_thresh`) and select the spatial reference system in which to output the coordinates of the mapped shorelines (`output_epsg`). See http://spatialreference.org/ to find the EPSG number corresponding to your local coordinate system. Make sure that your are using cartesian coordinates and not spherical coordinates (lat,lon) like WGS84. \n",
    "\n",
    "To quality control each shoreline detection and manually validate the mapped shorelines, the user has the option to set the parameter `check_detection` to **True**. To save a figure for each mapped shoreline set `save_figure` to **True**. \n",
    "\n",
    "The other parameters are for advanced users only and are described in the README."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "settings = { \n",
    "    # general parameters:\n",
    "    'cloud_thresh': 0.5,        # threshold on maximum cloud cover\n",
    "    'output_epsg': 28356,       # epsg code of spatial reference system desired for the output   \n",
    "    # quality control:\n",
    "    'check_detection': True,    # if True, shows each shoreline detection to the user for validation\n",
    "    'save_figure': True,        # if True, saves a figure showing the mapped shoreline for each image\n",
    "    # add the inputs defined previously\n",
    "    'inputs': inputs,\n",
    "    # [ONLY FOR ADVANCED USERS] shoreline detection parameters:\n",
    "    'min_beach_area': 4500,     # minimum area (in metres^2) for an object to be labelled as a beach\n",
    "    'buffer_size': 150,         # radius (in metres) of the buffer around sandy pixels considered in the shoreline detection\n",
    "    'min_length_sl': 200,       # minimum length (in metres) of shoreline perimeter to be valid\n",
    "    'cloud_mask_issue': False,  # switch this parameter to True if sand pixels are masked (in black) on many images  \n",
    "    'sand_color': 'default',    # 'default', 'dark' (for grey/black sand beaches) or 'bright' (for white sand beaches)\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [OPTIONAL] Save .jpg of the satellite images \n",
    "Saves .jpg files of the preprocessed satellite images (cloud masking + pansharpening/down-sampling) under *./data/sitename/jpeg_files\\preprocessed*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SDS_preprocess.save_jpg(metadata, settings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [OPTIONAL] Digitize a reference shoreline\n",
    "Creates a reference shoreline which helps to identify outliers and false detections. The reference shoreline is manually digitised by the user on one of the images. The parameter `max_dist_ref` defines the maximum distance from the reference shoreline (in metres) at which a valid detected shoreline can be. If you think that the default value of 100 m will not capture the full shoreline variability of your site, increase this value to an appropriate distance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib qt\n",
    "settings['reference_shoreline'] = SDS_preprocess.get_reference_sl(metadata, settings)\n",
    "settings['max_dist_ref'] = 100 # max distance (in meters) allowed from the reference shoreline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Batch shoreline detection\n",
    "Extracts the 2D shorelines from the images in the spatial reference system specified by the user in `'output_epsg'`. The mapped shorelines are saved into `output.pkl` (under *./data/sitename*) and `output.geojson` (to be used in a GIS software).\n",
    "\n",
    "If you see that the sand pixels on the images are not being identified, change the parameter `sand_color` from `default` to `dark` or `bright` depending on the color of your beach. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%matplotlib qt\n",
    "output = SDS_shoreline.extract_shorelines(metadata, settings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Simple plot of the mapped shorelines. The coordinates are stored in the output dictionnary together with the exact dates in UTC time, the georeferencing accuracy and the cloud cover."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure()\n",
    "plt.axis('equal')\n",
    "plt.xlabel('Eastings')\n",
    "plt.ylabel('Northings')\n",
    "plt.grid(linestyle=':', color='0.5')\n",
    "for i in range(len(output['shorelines'])):\n",
    "    sl = output['shorelines'][i]\n",
    "    date = output['dates'][i]\n",
    "    plt.plot(sl[:,0], sl[:,1], '.', label=date.strftime('%d-%m-%Y'))\n",
    "plt.legend()\n",
    "mng = plt.get_current_fig_manager()                                         \n",
    "mng.window.showMaximized()    \n",
    "fig.set_size_inches([15.76,  8.52])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Shoreline analysis\n",
    "\n",
    "In this section we show how to compute time-series of cross-shore distance along user-defined shore-normal transects."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**If you have already mapped the shorelines**, just load the output file (`output.pkl`) by running the section below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filepath = os.path.join(inputs['filepath'], sitename)\n",
    "with open(os.path.join(filepath, sitename + '_output' + '.pkl'), 'rb') as f:\n",
    "    output = pickle.load(f) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 3 options to define the coordinates of the shore-normal transects:\n",
    "\n",
    "**Option 1**: the user can interactively draw the shore-normal transects along the beach by calling:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib qt\n",
    "transects = SDS_transects.draw_transects(output, settings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Option 2**: the user can load the transect coordinates (make sure the spatial reference system is the same as defined previously by the parameter *output_epsg*) from a .geojson file by calling:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "geojson_file = os.path.join(os.getcwd(), 'examples', 'NARRA_transects.geojson')\n",
    "transects = SDS_tools.transects_from_geojson(geojson_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Option 3**: manually provide the coordinates of the transects as shown in the example below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transects = dict([])\n",
    "transects['Transect 1'] = np.array([[342836, 6269215], [343315, 6269071]])\n",
    "transects['Transect 2'] = np.array([[342482, 6268466], [342958, 6268310]])\n",
    "transects['Transect 3'] = np.array([[342185, 6267650], [342685, 6267641]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, intersect the transects with the 2D shorelines to obtain time-series of cross-shore distance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# defines the along-shore distance over which to consider shoreline points to compute the median intersection (robust to outliers)\n",
    "settings['along_dist'] = 25 \n",
    "cross_distance = SDS_transects.compute_intersection(output, transects, settings) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the time-series of shoreline change along each transect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import gridspec\n",
    "import numpy as np\n",
    "fig = plt.figure()\n",
    "gs = gridspec.GridSpec(len(cross_distance),1)\n",
    "gs.update(left=0.05, right=0.95, bottom=0.05, top=0.95, hspace=0.05)\n",
    "for i,key in enumerate(cross_distance.keys()):\n",
    "    if np.all(np.isnan(cross_distance[key])):\n",
    "        continue\n",
    "    ax = fig.add_subplot(gs[i,0])\n",
    "    ax.grid(linestyle=':', color='0.5')\n",
    "    ax.set_ylim([-50,50])\n",
    "    ax.plot(output['dates'], cross_distance[key]- np.nanmedian(cross_distance[key]), '-^', markersize=6)\n",
    "    ax.set_ylabel('distance [m]', fontsize=12)\n",
    "    ax.text(0.5,0.95, key, bbox=dict(boxstyle=\"square\", ec='k',fc='w'), ha='center',\n",
    "            va='top', transform=ax.transAxes, fontsize=14)\n",
    "mng = plt.get_current_fig_manager()                                         \n",
    "mng.window.showMaximized()    \n",
    "fig.set_size_inches([15.76,  8.52])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}