SDAP-399: Quickstart update - Docker (#185)

* Fix for null count field in return for matchup_spark request

* CDMS-159: Initial update pass

Still need to replace tags with variables and verify

* CDMS-159: Updated Jupyter notebook

The quickstart has the user add a cell, so I added it into the notebook. This will require the nexusjpl/jupyter image being rebuild and pushed to dockerhub

* CDMS-159: Cleaned up

Used env variables for image tags.

Added solr-cloud-init to list of images

Updated points

Misc cleanup

* CDMS-159: Fixed some typos

* CDMS-159: Fixed jupyter update

* CDMS-159: Fixed jupyter update

* CDMS-159: Shutdown instructions

* CDMS-159: Updated dockerhub url

* CDMS-159: Updated changelog

* Updated files for jupyter docker image

Tested build and verified added cell is present in the notebook

* Update quickstart.rst

Small updates for text clarity

* Updated title.

Replaced run time w/ placeholder

* Fixed formatting errors

* Changed requirements & git commit for nexuscli in dockerfile

* Temporarily pointed jupyter image to rileykkjpl/jupyter

* Made comment clearer

* Undid changes from improper merge

Accidentally merged from wrong branch I think. Just noticed it will lead to undoing accepted changes in Matchup.py on merge of this branch.

* Undid changes from improper merge

Accidentally merged from wrong branch I think. Just noticed it will lead to undoing accepted changes in Matchup.py on merge of this branch.

* Added missing mkdir

* Improved clarity of Cassandra start block

* Added note for occasional GI container failure

* Initial guide completion time estimate

* Fix for duplicate messages on RMQ

Cause was the CM flagging partially downloaded granules then the completely downloaded granule. Fixed by downloading the granules before CM start. Also added a note for mitigating this issue in the case the user wishes to download granules while the CM is running.

* Rearranged headers

* Cleaned up some commands

* Fix for cli import?

* Updated docker image to use up to date branch instead of old commit

* Updated docker image to use up to date branch instead of old commit

* Fixed nexuscli import issue

* Newline @ end of changelog to maybe fix PR oddity??

PR shows Security header line being deleted for some reason even though it is present in this branch

* Added a 'known issues' section

- GI crash
- RMQ blocking

* Link fix?

* Replace non-working section link with non-link

* Bumped jupyter image

* Prep for moving rmq monitor download location to main repo

Pending addition of monitor script into repo

* Added note for pending dataset migration to AWS

* Pointed jupyter image to nexusjpl

* Moved changelog entries to more suitable section

* Pointed RMQ script to version in main repo

* Bumped webapp image version

+ Fixed bad variable for jupyter image version (included name)

* It looks like the images are taking up less space?

* Removed excess space

Looks nicer

Co-authored-by: rileykk <[email protected]>

CHANGELOG.md

@@ -28,6 +28,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Matchup returns numSecondary and numPrimary counts rather than insitu/gridded
 - SDAP-402: Changed matchup matchOnce logic to match multiple points if same time/space
 - Bumped ingress timeout in Helm chart to reflect AWS gateway timeout
+- SDAP-399: Updated quickstart guide for standalone docker deployment of SDAP.
+- SDAP-399: Updated quickstart Jupyter notebook
 ### Deprecated
 ### Removed
 - removed dropdown from matchup doms endpoint secondary param

docker/jupyter/Dockerfile

@@ -37,9 +37,9 @@ RUN mkdir -p /home/jovyan/Quickstart && \
     git init && \
     git remote add -f origin ${APACHE_NEXUS} && \
     git config core.sparseCheckout true && \
-    echo "client" >> .git/info/sparse-checkout && \
+    echo "integrations/python-client" >> .git/info/sparse-checkout && \
     git pull origin ${APACHE_NEXUS_BRANCH} && \
-    cd client && \
+    cd integrations/python-client && \
     python setup.py install
 
-COPY ["Time Series Example.ipynb", "/home/jovyan/Quickstart/Time Series Example.ipynb"]
+COPY ["Time Series Example.ipynb", "/home/jovyan/Quickstart/Time Series Example.ipynb"]

docker/jupyter/Time Series Example.ipynb

@@ -2,7 +2,11 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "metadata": {},
+   "metadata": {
+    "pycharm": {
+     "name": "#%% md\n"
+    }
+   },
    "source": [
     "# Start Here\n",
     "\n",
@@ -12,7 +16,34 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "metadata": {},
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    },
+    "pycharm": {
+     "name": "#%%\n"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "os.environ[\"PROJ_LIB\"] = \"/opt/conda/share/proj\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    },
+    "pycharm": {
+     "name": "#%%\n"
+    }
+   },
    "outputs": [],
    "source": [
     "%matplotlib inline\n",
@@ -89,7 +120,11 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {},
+   "metadata": {
+    "pycharm": {
+     "name": "#%% md\n"
+    }
+   },
    "source": [
     "# Run Time Series and Plot\n",
     "\n",
@@ -103,7 +138,11 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "metadata": {},
+   "metadata": {
+    "pycharm": {
+     "name": "#%%\n"
+    }
+   },
    "outputs": [],
    "source": [
     "import time\n",
@@ -139,7 +178,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -153,9 +192,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.5"
+   "version": "3.9.7"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 2
-}
+ "nbformat_minor": 4
+}

docker/jupyter/requirements.txt

@@ -1,4 +1,3 @@
-shapely
-requests
-numpy
-cassandra-driver==3.9.0
+shapely==1.6.4.post2
+requests==2.21.0
+numpy>=1.13.3