python/how-tos/How_to_Convert_to_ENVI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How To: Convert EMIT .nc to .envi\n",
    "\n",
    "There are currently 2 similar methods to convert the EMIT netCDF4 files to `.envi` format. Note these only support L1B Radiance, L1B Obs, L2A Reflectance, L2A Reflectance Uncertainty, or L2A Mask to .envi. They do not yet support the L2B Mineral or L2B Mineral Uncertainty products.\n",
    "1. The `write_envi` function in EMIT tools. This function is still being developed but will currently:\n",
    "  - Write a GLT output to use for orthocorrection later \n",
    "  - Functions from `emit_tools` can be used beforehand to orthorectify if so desired\n",
    "2. The `reformat.py` script available in the [emit-sds/emit-utils](https://github.com/emit-sds/emit-utils) repository can be used to convert EMIT netCDF files (as delivered to the LP DAAC) to ENVI files. This script also can apply the included GLT to orthorectify the image if desired.\n",
    "\n",
    "This jupyter notebook walks through how to use both methods to provide users with programmatic routes to accomplish their EMIT reformatting workflows.\n",
    "\n",
    "**Requirements:**\n",
    "+ A NASA [Earthdata Login](https://urs.earthdata.nasa.gov/) account is required to download EMIT data   \n",
    "+ Selected the `emit_tutorials` environment as the kernel for this notebook.\n",
    "  + For instructions on setting up the environment, follow the the `setup_instructions.md` included in the `/setup/` folder of the repository.  \n",
    "\n",
    "**Learning Objectives**\n",
    "+ How to use the `write_envi` function from `emit_tools` module to convert an EMIT netCDF4 to a `.envi` file.\n",
    "+ How to use the `reformat.py` function from the `emit-utils` repository to convert an EMIT netCDF4 to a `.envi` file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "Import packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import earthaccess"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Authenticate using Earthdata Login and Download the required Granules\n",
    "\n",
    "Login to your NASA Earthdata account and create a `.netrc` file using the `login` function from the `earthaccess` library. If you do not have an Earthdata Account, you can create one [here](https://urs.earthdata.nasa.gov/home). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "earthaccess.login(persist=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this notebook we will download the files necessary using `earthaccess`. You can also access the data in place or stream it, but this can slow due to the file sizes. Provide a URL for an EMIT L2A Reflectance granule."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = 'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2ARFL.001/EMIT_L2A_RFL_001_20220903T163129_2224611_012/EMIT_L2A_RFL_001_20220903T163129_2224611_012.nc'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get an HTTPS Session using your earthdata login, set a local path to save the file, and download the granule asset - This may take a while, the reflectance file is approximately 1.8 GB. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get Https Session using Earthdata Login Info\n",
    "fs = earthaccess.get_fsspec_https_session()\n",
    "# Retrieve granule asset ID from URL (to maintain existing naming convention)\n",
    "granule_asset_id = url.split('/')[-1]\n",
    "# Define Local Filepath\n",
    "fp = f'../../data/{granule_asset_id}'\n",
    "# Download the Granule Asset if it doesn't exist\n",
    "if not os.path.isfile(fp):\n",
    "    fs.download(url, fp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets create an output folder where we will save the `.envi` files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "outpath = '../../data/envi' \n",
    "if not os.path.exists(outpath):\n",
    "    os.makedirs(outpath)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Method 1: Using `write_envi` from the `emit_tools` module.\n",
    "\n",
    "Import the necessary packages for this method. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append('../modules/')\n",
    "import emit_tools as et"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open the granule using the `emit_xarray` function. We can orthorectify here if so desired."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds = et.emit_xarray(fp, ortho=True)\n",
    "ds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, write the dataset as an `.envi` output. If we chose not to orthorectify, you can include a `glt` file to orthorectify later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "et.write_envi(ds, outpath, overwrite=False, extension='.img', interleave='BIL', glt_file=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Method 2: Using reformat.py from emit-utils\n",
    "\n",
    "### 2.1 Clone and Install emit-utils\n",
    "\n",
    "Clone the [emit-utils](https://github.com/emit-sds/emit-utils) repository."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!git clone https://github.com/emit-sds/emit-utils.git ../emit_utils/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This will copy the `emit-utils` repository to a folder within this repository. \n",
    "\n",
    "After you have copied it, use `pip` package manager to install the directory as a package to ensure you have all of the dependencies and be used in the command line. \n",
    "\n",
    "> **This requires that some dependencies already be installed to work properly on Windows. If you have created the Python environment described in the [setup instructions](../../setup/setup_instructions.md) it should work.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip install --editable ../emit_utils"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After successfully installing `emit-utils`, you can use the scripts contained within as part of your workflows. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Executing the Reformat Script\n",
    "\n",
    "Before calling the `reformat.py` script, make sure you have an output directory for the `.envi` files that will be produced."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "outpath = '../../data/envi' \n",
    "if not os.path.exists(outpath):\n",
    "    os.makedirs(outpath)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, execute the `reformat.py` script contained in the emit-utils repository. When executing this script, provide the path to the `.nc` file, followed by the directory to place the `.envi` files in. If you wish to apply the GLT or orthorectify, include `--orthorectify` as an argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!python ../emit_utils/emit_utils/reformat.py ../../data/EMIT_L2A_RFL_001_20220903T163129_2224611_012.nc ../../data/envi/ --orthorectify"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This will orthorectify the image, create an ENVI header, and save it in `.envi` format inside the `../data/envi` folder."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contact Info:  \n",
    "\n",
    "Email: LPDAAC@usgs.gov  \n",
    "Voice: +1-866-573-3222  \n",
    "Organization: Land Processes Distributed Active Archive Center (LP DAAC)¹  \n",
    "Website: <https://lpdaac.usgs.gov/>  \n",
    "Date last modified: 07-06-2023  \n",
    "\n",
    "¹Work performed under USGS contract G15PD00467 for NASA contract NNG14HH33I.  "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "ee32eb811eece122811842209709e899a392a6a8deb39746eb88e988164e27f9"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}