AGVis Update 4.0 (#72)

* Legend.js

* Window.js

* Window.js

* Updating Window.js comments

* Updating CanvasLayer.js comments

* Added CommunicationLayer.js header comment.

* Overhauling developer comments

* Updating ContMulti.js comments

* Dev comments

* Dev comments

* Dev comments

* LayerControl.js Comments

* CanvasLayer comments

* ContMulti.js comments

* NDArray comments

* Fixed header comment formatting

* Fixed header comment formatting

* PlaybackControl.js Comments

* Updating PlaybackControl.js comments

* PlayMulti.js comments

* PlayMulti.js comments

* SearcLayer.js Comments

* TimeBox.js Comments

* TimeBox.js Comments

* TopMulti.js comments

* TopMulti.js comments

* Finishing touches

* Finishing touches

* Added windows support for the agvis run command

* Updated usage documentation for Windows

* Setting up tests folder

* Runs server start command from the app.py directory

* basic test setup

* updated requirements

* updating requirements

* Initial restructure of AGVis backend

* AGVis backend restructure

* Finished backend restructure. Added dev mode.

* fixed typo

* fixed typo in go.sh

* Updated agvis repository Docker uses

* Fixed agvis pip install in Docker container

* Added shell=False to subprocess

* Changed subprocess command for testing

* Disabled running from directory. Currently breaks Docker.

* reenabled popen for subprocess

* reenabled popen for subprocess

* Fixed Docker issue. Added ability to build different branches using ./go.sh build <BRANCH_NAME>

* removed ! from go.sh

* Removed all syntax errors relating to an extra ! at the end

* Cleaned up object oriented implementation of flask app

* Testing Web App

* Writing tests

* tests

* conftest

* dime testing

* Added header comments. Finished initial dime_client test for agvis

* Updated test name

* disabled docker test, looking into js testing

* adding tests for web

* client tests

* removed redundant test

* Added header comment and removed unused imports

* Created docker test file

* debug ./go.sh file not being found

* debug ./go.sh file not being found

* debug ./go.sh file not being found

* Added checking out of code

* updated go.sh script to include docker testing

* removed interactive tag

* Disabling basic dime testing. Will keep the file in case it is needed in the future, but testing it here is has both project design and bug problems.

* added PR unit test protection for master

* Custom Flask Configurations for AGVis

* Include pip upgrade in Dockerfile

* Update citation

* Update copyright

* Updated max zoom

* Rename test.yml to pythonapp.yml

Conform with rest of LTB.

* Update pythonapp.yml

* Update dependency versions on pythonapp.yml

* Added linting with flake8

* Remove flake 8 until its recommended fixes are corrected.

* Publish Python to PyPI in pythonapp.yml

* Fixed broken tests. Added agvis selftest functionality

* Fixing last unittest

* Removing problematic test working locally but not on github

* Cleaning up code for codacy

---------

Co-authored-by: Jinning Wang <[email protected]>

.github/workflows/pythonapp.yml

@@ -0,0 +1,61 @@
+name: Python application
+
+on: [push, pull_request]
+
+jobs:
+  build:
+    name: Build distribution
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: "3.8"
+    - name: Install pypa/build
+      run: >-
+        python3 -m
+        pip install
+        build
+        --user
+    - name: Build a binary wheel and a source tarball
+      run: python3 -m build
+    - name: Store the distribution packages
+      uses: actions/upload-artifact@v4
+      with:
+        name: python-package-distributions
+        path: dist/
+
+  test:
+    name: Run test suite
+    needs:
+    - build
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v4
+    - name: Build the Docker image
+      run: ./go.sh build
+    - name: Run tests
+      run: ./go.sh run_tests
+    # Developer note: Fix flake8 errors before uncommenting here.
+    # - name: Lint with flake8 for pull requests
+    #   if: github.event_name == 'pull_request'
+    #   run: |
+    #     pip install flake8  # specify flake8 to avoid unknown error
+    #     # stop the build if there are Python syntax errors or undefined names
+    #     flake8 .
+
+  publish-to-pypi:
+    name: Publish Python distribution to PyPI
+    if: github.ref == 'refs/heads/master' && github.event_name == 'push'
+    needs:
+      - build
+      - test
+    runs-on: ubuntu-latest
+    steps:
+      - name: Publish Python dist to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          user: __token__
+          password: ${{ secrets.PYPI_PASSWORD }}

CITATION.bib

@@ -0,0 +1,10 @@
+@INPROCEEDINGS{10318583,
+  author={Parsly, Nicholas and Wang, Jinning and West, Nick and Zhang, Qiwei and Cui, Hantao and Li, Fangxing},
+  booktitle={2023 North American Power Symposium (NAPS)}, 
+  title={DiME and AGVis: A Distributed Messaging Environment and Geographical Visualizer for Large-Scale Power System Simulation}, 
+  year={2023},
+  volume={},
+  number={},
+  pages={1-5},
+  keywords={Power system simulation;Data visualization;Distributed databases;Nonhomogeneous media;Real-time systems;Power systems;North America;Power grid;Open-source software;Large-scale system;High-concurrency Data;Geovisualization;Digital twin},
+  doi={10.1109/NAPS58826.2023.10318583}}

Dockerfile

@@ -1,5 +1,6 @@
 # Start with a base Python 3.10 image from the Debian Buster distribution
 FROM python:3.10-buster AS base
+ARG BRANCH_NAME
 
 # Switch to the root user and set the working directory to /root
 USER root
@@ -18,14 +19,12 @@ RUN apt update \
         libev-dev \
     && rm -rf /var/lib/apt/lists/*
 
+RUN python3 -m pip install --upgrade pip
+
 # Install the necessary Python packages
 RUN python3 -m pip install \
         kvxopt \
         git+https://github.com/cuihantao/andes.git@develop \
-        --no-cache-dir \
-    && python3 -m pip install \
-        git+https://github.com/zmalkmus/agvisdev.git \
-        # git+https://github.com/CURENT/agvis.git \
         --no-cache-dir
 
 # Create a new user named 'cui' and a work directory
@@ -50,6 +49,11 @@ RUN git clone https://github.com/CURENT/dime.git && \
     rm -rf dime \
     && rm -rf /tmp/dime
 
+RUN git clone --single-branch --branch unittesting https://github.com/CURENT/agvis.git && \
+    cd agvis && \
+    python3 -m pip install -e . && \
+    cd ..
+
 # Switch to the 'cui' user and set the working directory to the new user's work directory
 USER cui
 WORKDIR /home/cui/work
@@ -62,4 +66,4 @@ COPY . .
 
 # Set the entrypoint and command for the container
 ENTRYPOINT []
-CMD []
+CMD []

README.md

@@ -36,13 +36,13 @@ AGVis is currently under active development. Use the following resources to get
 + Report bugs or issues by submitting a [GitHub issue][GitHub issues]
 + Submit contributions using [pull requests][GitHub pull requests]
 + Read release notes highlighted [here][release notes]
-+ Check out and cite our [paper][arxiv paper]
++ Check out and cite our [paper][naps paper]
 
 # Citing AGVis
 
 If you use AGVis for research or consulting, please cite the following publications in your publication:
 
-> Parsly, N., Wang, J., West, N., Zhang, Q., Cui, H., & Li, F. (2022). "DiME and AGVIS A Distributed Messaging Environment and Geographical Visualizer for Large-scale Power System Simulation". arXiv. https://doi.org/https://arxiv.org/abs/2211.11990v1
+> N. Parsly, J. Wang, N. West, Q. Zhang, H. Cui and F. Li, "DiME and AGVis: A Distributed Messaging Environment and Geographical Visualizer for Large-Scale Power System Simulation," 2023 North American Power Symposium (NAPS), Asheville, NC, USA, 2023, pp. 1-5, doi: 10.1109/NAPS58826.2023.10318583.
 
 Please refer as **LTB AGVis** for the first occurence and then refer as **AGVis**.
 
@@ -70,7 +70,7 @@ AGVis is licensed under [GPL v3 License](./LICENSE)
 [readthedocs]: https://agvis.readthedocs.io
 [advanced usage]: https://agvis.readthedocs.io/en/latest/usage/index.html
 [release notes]: https://agvis.readthedocs.io/en/latest/release-notes.html
-[arxiv paper]: https://arxiv.org/abs/2211.11990
+[naps paper]: https://ieeexplore.ieee.org/document/10318583
 [tutorial]: https://agvis.readthedocs.io/en/latest/getting_started/tutorial/index.html
 [LTB Repository]: https://github.com/CURENT
 [Visualization Gallery]: https://ltb.readthedocs.io/projects/agvis/en/latest/getting_started/testcases.html#visualization-gallery

agvis/__init__.py

@@ -2,7 +2,7 @@
 __version__ = _version.get_versions()['version']
 
 from agvis.main import config_logger  # NOQA
-from agvis.app import run_app
+from agvis.web import AgvisWeb
 
 __author__ = 'Nicholas West, Nicholas Parsly, Zack Malkmus, and Jinning Wang'
 

agvis/cli.py

@@ -43,6 +43,7 @@ def create_parser():
     run = sub_parsers.add_parser('run')
     run.add_argument('--host', default='127.0.0.1', help='Host to bind the server (default: 127.0.0.1)')
     run.add_argument('--port', default=8810, type=int, help='Port to bind the server (default: 8810)')
+    run.add_argument('--dev', default=False, type=bool, help='Run AGVis in development mode (default: False)')
     # run.add_argument('--static', default=None, help='Static path to serve (default: None)')
 
     misc = sub_parsers.add_parser('misc')
@@ -54,8 +55,6 @@ def create_parser():
 
     selftest = sub_parsers.add_parser('selftest', aliases=command_aliases['selftest'])
 
-    demo = sub_parsers.add_parser('demo')  # NOQA
-
     return parser
 
 
@@ -107,7 +106,6 @@ def main():
     # Run the command
     if args.command is None:
         parser.parse_args(sys.argv.append('--help'))
-
     else:
         cmd = args.command
         for fullcmd, aliases in command_aliases.items():

agvis/flask_configurations.py

@@ -0,0 +1,21 @@
+# ================================================================================
+# File Name:   flask_configurations.py
+# Author:      Zack Malkmus
+# Date:        1/18/2024 (created)
+# Description: Configuration file for FLASK APP
+# ================================================================================
+
+class DefaultConfig(object):
+    DEBUG = False
+    TESTING = False
+    CSRF_ENABLED = True
+
+class ProductionConfig(DefaultConfig):
+    DEBUG = False
+
+class DevelopmentConfig(DefaultConfig):
+    DEVELOPMENT = True
+    DEBUG = True
+
+class TestingConfig(DefaultConfig):
+    TESTING = True

agvis/main.py

@@ -8,14 +8,17 @@
 import pprint
 import logging
 import tempfile
-
+import subprocess
 from ._version import get_versions
-
 from andes.utils.misc import is_interactive
 import agvis
+from agvis.web import AgvisWeb
 
-logger = logging.getLogger(__name__)
+agvis_web = AgvisWeb()
 
+app = agvis_web.app
+
+logger = logging.getLogger(__name__)
 
 def config_logger(stream_level=logging.INFO, *,
                   stream=True,
@@ -171,8 +174,8 @@ def remove_output(recursive=False):
 
 
 def run(filename='', input_path='', verbose=20,
-        host='localhost', port=8810, socket_path=None,
-        static=None,
+        host='localhost', port=8810, dev=False,
+        socket_path=None, static=None,
         **kwargs):
     """
     Entry point to run AGVis.
@@ -209,7 +212,7 @@ def run(filename='', input_path='', verbose=20,
     cases = _find_cases(filename, input_path) #NOQA
 
     # Run the flask web app
-    agvis.app.run_app("agvis.app:app", host=host, port=port)
+    agvis_web.run(host=host, port=port, dev=dev)
 
     return True
 
@@ -266,7 +269,7 @@ def print_license():
     print(f"""
     AGVis version {agvis.__version__}
 
-    Copyright (c) 2020-2023 Nick West, Nicholas Parsly, Jinning Wang
+    Copyright (c) 2020-2024 Nick West, Nicholas Parsly, Jinning Wang
 
     This program is free software; you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
@@ -298,10 +301,6 @@ def misc(show_license=False, clean=True, recursive=False,
         remove_output(recursive)
         return
 
-    if demo is True:
-        demo(**kwargs)
-        return
-
     if version is True:
         versioninfo()
         return
@@ -313,15 +312,11 @@ def selftest(**kwargs):
     """
     TODO: Run unit tests.
     """
-    logger.warning("Tests have not been implemented")
-
-
-def demo(**kwargs):
-    """
-    TODO: show some demonstrations from CLI.
-    """
-    logger.warning("Demos have not been implemented")
-
+    # logger.warning("Tests have not been implemented")
+    logger.info('Running tests...')
+    path = os.path.dirname(os.path.abspath(__file__))
+    test_path = os.path.join(path, '../tests')
+    subprocess.run(['pytest', '-v', test_path])
 
 def versioninfo():
     """