Create code tests to ensure the code quality.
For now, we can cover the python code part first, such as the independent data reader and the backend framework.

Additionally, some GitHub actions can be added to alleviate manual efforts. Such as actions to automatically publish to PyPi and Conda.

Reference:
ANDES code test
ANDES GiiHub workflow
pytest
CI/CD in software development

Leaflet allows extending its functionality by using third-party plugins, https://leafletjs.com/plugins.html.
Is it worth integrating some of them? Such as basemap selection?

The screenshots added to the documentation turned out to be too large:

1. Built-in cases map at https://github.com/CURENT/agvis/tree/master/docs/source/getting_started/cases
2. Screenshots to demonstrate usage at https://github.com/CURENT/agvis/tree/master/docs/source/usage/diagrams

This enlarges the package size and thus prevents us from publishing it in PyPi. It also slows down the documentation loading speed.

We might need to replace these figures with compressed ones to address this issue.

As Zack suggested, a quick solution can be using the flask to avoid it. Remember to use the branch master

cd /Users/jinningwang/Documents/work/agvis/agvis
flask run -p 8810

When using AGVis on windows, an error occurred that "no module named 'fcntl'", as shown in the below screenshot. It seems that fcntl does not support windows platform. Can we do a quick fix on this?

08/30/2023

We can have a different way to implement the online web application of AGVis, because we have both a server and a domain to deploy the web application.

03/20/2023

Create an online web application of AGVis using Pyodide, some examples:
MATPOWER App by Richard Lincoln
Pyodide Doc

@nparsly Nicholas, I've reorganized and improved the AGVis documentation, see #38. The online documentation is available at https://agvis.readthedocs.io/en/latest/. Given your and Nick's thesis, it might be helpful to give more detail in the Development section.

Animation calculation

In the Animations section, it might be helpful to explain more how the animation is made.

AGVis input data

In the Test Cases section, I made a clarification that the devices BusFreq are required in the test case to measure the Bus voltage, angle, and frequency.

AGVis logging info

It might also be helpful to have logging messages tell the variables' name, size, etc, that are used for visualization.

The docker build function failed on my end. I am using Mac OS, and additional error message is excerpted below.

(ltb) jinningwang@jwmba-3 agvis % agvis

    _   _____   ___     | Version 3.2.1.post46+g8f69adc
   /_\ / __\ \ / (_)___ | Python 3.11.6 on Darwin, 10/20/2023 03:16:01 PM
  / _ \ (_ |\ V /| (_-< | 
 /_/ \_\___| \_/ |_/__/ | Web-based geographical visualizer.

usage: agvis [-h] [-v {1,10,20,30,40}] {run,misc,selftest,st,demo} ...

positional arguments:
  {run,misc,selftest,st,demo}
                        [run] serve the web; [selftest] run self test;

options:
  -h, --help            show this help message and exit
  -v {1,10,20,30,40}, --verbose {1,10,20,30,40}
                        Verbosity level in 10-DEBUG, 20-INFO, 30-WARNING, or 40-ERROR.
(ltb) jinningwang@jwmba-3 agvis % ./go.sh build
./go.sh: line 48: syntax error near unexpected token `newline'
./go.sh: line 48: `    google-chrome --incognito http://localhost:8810/ 2> /dev/null > /dev/null &!'

AGVis accept S_Set sheet as configuration settings.

However, it can be easier and more consistent for an LTB user to use if saving the configuration in a separate file, let's say, agvis.rc, to follow the overall style of LTB.

Reference:
ANDES Config

When using ANDES, DiME, and AGVis, together, I ran into an issue, the visualization using MS Edge browser is a lot behind the ANDES simulation speed. After switching to Google Chrome, the issue disappeared.

All three modules, namely ANDES, DiME, and AGVis, are running on WSL, Ubuntu 22.04.
MS Edge browser is running on win10.
ANDES is running in quasi-real-time mode with an IEEE 39-bus system.

If this is the case, we might need to mark this as a known issue in the documentation.

I would like to suggest adding a legend bar to AGVis. The legend bar would enhance the visualization experience by providing a clear and concise representation of the color-coded values used in the visualization.

Currently, the color-coded values are shown in a scale that is located in the config panel. However, this scale is not very visible and can be easily overlooked by users.

By adding a legend bar to the visualization window corresponding to the visualized variable, users would be able to quickly understand the meaning of the colors and the corresponding values. This would also make it easier to interpret the simulation results and make informed decisions based on the visualization.

Issue
When using AGVis, the zoom-in seems to be limited somewhere, as shown in the Figure below:

Desire Feature
First, improve zoom-in limitation.
Can we made some change to alleviate this limitation so AGVis is capable of distribution system visualization? For example, a microgrid on UTK Campus size as attached in the case file. UTCampus.xlsx

Second, is it possible to add option to switch base-map? Maybe street-map?

Version

(agvis) jinningwang@jwmba-3 agvis % agvis run

    _   _____   ___     | Version 3.2.1.post96+gbd5488c
   /_\ / __\ \ / (_)___ | Python 3.9.18 on Darwin, 02/23/2024 07:18:28 AM
  / _ \ (_ |\ V /| (_-< | 
 /_/ \_\___| \_/ |_/__/ | Web-based geographical visualizer.

Working directory: "/Users/jinningwang/Documents/work/agvis"
AGVis will serve static files from directory "/Users/jinningwang/Documents/work/agvis/agvis/static"
at the URL http://127.0.0.1:8810. Open your web browser and navigate to the URL to access the application.

Starting AGVis... Press Ctrl+C to stop.

[2024-02-23 07:18:28 -0500] [63127] [INFO] Starting gunicorn 21.2.0
[2024-02-23 07:18:28 -0500] [63127] [INFO] Listening at: http://127.0.0.1:8810 (63127)
[2024-02-23 07:18:28 -0500] [63127] [INFO] Using worker: sync
[2024-02-23 07:18:28 -0500] [63128] [INFO] Booting worker with pid: 63128

Visualization of static results

For now, AGVis is designed to show dynamic results from ANDES. In the future, we may also integrate dispatch results, or named static results, from AMS in this visualization. The dispatch results typically include power flow results and nodal price.

In the future, we may also integrate with AMS to visualize the dispatch results. The button to visualize power flow and nodal price may need to be added.

Power flow

We can use arrows on the transmission lines to show the power flow between buses. We can use the boldness or color of arrows to represent the numerical value of the power flow. An example is shown in the picture:

Picture source: Schäfer, Mirko, et al. "Principal Cross-Border Flow Patterns in the European Electricity Markets." 2019 16th International Conference on the European Energy Market (EEM). IEEE, 2019.

Another practice in the industry can be seen from the following examples:

• NYISO Loads and Flows on NYISO REAL-TIME DASHBOARD

However, this might not be compatible with AGVis implementation.

Price map

Another practice in the industry to visualize prices is to fill the price signal areas on the map in a pattern, as shown in the two following examples

• PJM Locational Marginal Pricing Map
• ERCOT Real-Time Locational Prices

However, similarly, this might not be compatible with AGVis implementation. We can stick with the frequency contour map as shown below, but discussions are very welcomed.

Regards,
Jinning

The below picture shows the current AGVis UI.

However, we might need to adapt this UI to feature MultiLayer.

1. The lighting button symbol stands for the power grid. The lighting button will control the power grid topology and all related animation layers with buttons $\theta$, "V", and "f". For easy use, let us call these "ANDES sets", maybe a drop-down list.
2. Similarly, "AMS sets", using the symbol (maybe a simpler one, but use this as reference) stands for static results visualization. There will also be several buttons, $ for Locational Marginal Price (LMP), for available capacity, F for power flow.
3. The WiFi button stands for the communication network (I guess, not sure).

We can come up with some draft overall UI first.

There are three buttons for the dynamic results in the left side bar, voltage angle, voltage magnitude, and frequency.

There can be a user-defined button there that the user can define the symbol and the linked variable. Then it will be much more easier to visualize non-dynamic results.

For example, the user can define a button and give it a math label $ for LMP when visualize user input file.

There are little to no comments in the javascript files. This will provide a lot of confusion for anyone looking at the code. The files should follow a common commenting style, outlined here.

Uncaught TypeError: Cannot read properties of undefined (reading 'updatePlaybackBar') at Window.js:29:16

While this error does not effect functionality, it still is bad practice to let this hang.

Animations are drawn over the portion of the map and nodes that they occupy. Lengthy animations that take place over a large portion of the map can block the user's view, especially if it has values consistently near the maximum or minimum of the range given for the animation. Adding in an opacity adjuster for each set of animation data could help alleviate this issue.

Using Python Code for Interactive AGVis Web Sessions

The functionality of interaction through Python code will improve the applicability. The main use of AGVis can be classified into two aspects: 1) starting, and 2) visualizing.
AGVis web is hosted using Python http.server. The visualization part is using the Leaflet. The data exchange between ANDES and AGVis is through a socket file using DiME.

It is possible to let AGVis interact with the web and the Python code through the socket file, namely, DiME client and DiME server.

Starting

Description

The modules cli and system module (available at cli and system) allows users to launch AGVis from either the command line or IPython as showing below:

Command line:

agvis run --host 127.0.0.1 --port 8810

IPython:

import agvis

web = agvis.webapp()
web.run(open_browser=True)

Update on 04/28/2023

We have developed two modules, cli and web, which enhance the starting process of AGVis web. For more information on how to use the command line interface, please refer to our documentation on Command line usage.

In this implementation, we use http.server and socketserver.TCPServer to host the web application in the background. However, Flask can be a better choice for its flexibility and usability to allow interactive usage.

Reference: https://flask.palletsprojects.com/en/2.3.x/

Visualizing

Description

After starting a web using the following code:

web = agvis.webapp()
web.run(open_browser=True)

The further interaction can be:

web.load(agvis.get_case('ieee39.xlsx'))  # Load topology into the web
web.play(andes_case='ieee39.xlsx', andes_output='ieee39_out.csv')  # play the animation with given input

web.refresh()  # Refresh the web

...

web.stop()

One possible approach for integration is to use DiME, which can start and run a DiME server backend for communication between ANDES and AGVis. However, this approach may require the packaging of DiME. See CURENT/dime#47 for more discussion.

Then, this improvement makes it possible to integrate AGVis as a built-in visualizer for ANDES and AMS in the future.

The scenario often arises where the user loads multiple sets of simulation data into a single topology. In such cases, it is more convenient to use separate input ports for the topology and simulation data. This approach also follows the overall LTB style, as in ANDES and AMS where the topology and simulation data are separated.

When using AGVis to visualize only Topology using the MultiLayer feature, it does not support the "Toggle Bus Labels". Is it possible to also support this in the MultiLayer feature?

I created a basic github workflow to run all agvis tests that at this time is pushed to development. This should be discussed and improved upon to include things such as pushing to pip.

When first using AGVis and loading a simulation, I was confused when it initially ran out of sync with the timer. It then displayed the playback popup in the bottom left corner. I might like to add a visual queue to the page to show that the simulation is loading in AGVis during that first run.

PyData Sphinx Theme can be more informative and can be easier for the LTB management.
Maybe we can transfer the documentation from MKDocs to PyData Sphinx Theme. The existing ANDES, AMS, and LTB documentation can be good example to use the theme.

Related links:
The PyData Sphinx Theme
ANDES docs
AMS docs
LTB docs

is it possible to view the inner-substation topology (oneline diagram) using Agvis?