Giter VIP home page Giter VIP logo

review's Introduction

reView

GitHub Pytests GitHub Super-Linter codecov

reView is a data portal for reviewing outputs from the National Renewable Energy Laboratory's (NREL) Renewable Energy Potential Model (reV). It is currently limited to tables generated by the reV Supply Curve module, which provide technical potential data for a given technology under a set of land-use, technology performance, and cost assumptions. Once a user has generated tables withreV, reView allows that user to view the data as an interactive map linked to an interactive graph, allowing for quick exploration of reV outputs. Other functionality allows the user to filter the dataset based on variable thresholds, calculate differences between two tables, compare multiple tables in one chart, group results by region, and calculate least-cost scenarios at each point. More functionality is coming soon.


Installation

  1. Clone the reView repository.

    Using ssh:

    git clone [email protected]:NREL/reView.git
    

    Using https:

    git clone https://github.com/NREL/reView.git
    
  2. cd into the repository: cd reView.

  3. Install Package and Dependencies

    1. RECOMMENDED OPTION
      Using conda/mamba ():

      1. Create a conda environment:
        conda env create -f environment.yml
        
      2. Activate the environment:
        conda activate review
        
      3. Install reView:
        For users:
        pip install -e .
        
        For developers:
        pip install -e '.[dev]'
        
    2. CHOOSE YOUR OWN ADVENTURE
      Using pip:

      1. Create and activate an environment with your tool of choice
      2. Install GEOS dependency (https://libgeos.org/usage/install/).
      3. Install reView:
        For users:
        pip install -e .
        
        For developers:
        pip install -e '.[dev]'
        

    Note:
    The pip approach has not been tested by the reView package maintainers.

  4. Create a project config pointing to directory containing reV supply curve tables. Save as json dictionary in configs/ (e.g. configs/sample.json). If the user doesn't have reV data, they can get started with supply-curves for NREL's standard scenarios based on the Electricity Annual Technology Baseline through NREL's Geospatial Data Science group. Supply curves for wind power are available here and those for solar photovoltaic power are available here.

    {
        "project_name": <"Your Project Name>",
        "directory": <"Local path to folder containing reV supply curve tables">
    }

Running reView

  1. Run the reView command:
    reView
    
  2. Open your browser and enter the URL output from command above. The default port is 8050.
    http://localhost:8050
    

Running reView with Gunicorn (Unix only)

  1. To run reView using gunicorn (a Python WSGI HTTP Server for UNIX) for better performance, make sure to follow the installation steps as outlined above, but when you get to the last step of #2, include the gunicorn dependency:

    pip install -e '.[gunicorn]'
    

    or, for more development tools:

    pip install -e '.[dev,gunicorn]'
    
  2. Run reView using gunicorn:

    cd reView
    gunicorn index:server
    
  3. Open your browser and enter the URL output from command above. The default port is 9875.

    http://localhost:9875
    

Command-Line Tools

Additional command-line tools are available as part of this package. A list of these tools can be accessed using the command reView-tools. More information on the usage of these tools can be found in the Usage documentation.

Development

This repository uses pylint to lint the code, please follow this guideline if you wish to contribute to this repository. In addition, your code must pass all existing tests. You will need to download the Chrome Driver in order to run the integration tests included with this repository.

Before pushing code, the following commands should be run to ensure GitHub actions will succeed:

pylint reView
pylint tests

flake8 reView
flake8 tests

pytest

review's People

Contributors

mjgleason avatar ppinchuk avatar williamstravis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

review's Issues

Time Series

Why this feature is necessary:
This is a longer-term goal, but at some point it would be great to have reView display time-series. Perhaps not the entire time series since those are so large, but at least diurnal profiles and seasonal patterns.

Mismatched fonts

There's a mix of serif and sans-serif fonts in headings, plot titles, axis labels, menu names, etc. This isn't exactly a bug, but the tool would look much nicer with a uniform font family. Also, there are a lot of different font sizes. Consider choosing a limited number of larger fonts to emphasize the most important headings and information (see screenshot).

Screenshots
Attached.

Charge code
Sorry. I don't have a charge code for this :[

Screen Shot 2023-09-11 at 15 53 16

Add existing solar and wind farms

Why this feature is necessary:
Because it would be useful to see the current build out and how it relates to the various technical potential maps reView displays.

A possible solution is:
For wind setup methods to use the USWTDB's API service to build and periodically update their dataset. For solar we can use Berkely Lab's dataset here, though they do not appear to have an API and the historical datasets don't have a consistent url pattern. Might be worth it to reach out to them directly.

It would be nice to overlay these datasets on top of the tech potential maps, though that could become a little complicated on the UI side since these will be time series and will have their own variable drop downs. It might be worth it to include them in the ReEDS page, though we'll still have mismatching variables and available dates. We could start with a separate page and to get it working and then think about integrating them will our model visualizations.

UnicodeDecodeError

Bug Description
Receiving this error when reading both chart and map table. Possibly due to moved cached folder. Breaks completely in debug mode, but some sort of exception handler in production reloads the file and it works, though the error is still thrown.

Full Traceback

  File "/Users/twillia2/github/reView/reView/utils/classes.py", line 181, in _callback_func
    return func(*args, **kwargs)
  File "/Users/twillia2/github/reView/reView/pages/rev/controller/callbacks.py", line 764, in figure_chart
    dfs = cache_chart_tables(signal_dict, region)
  File "/Users/twillia2/anaconda3/envs/review/lib/python3.9/site-packages/flask_caching/__init__.py", line 834, in decorated_function
    cache_key = decorated_function.make_cache_key(f, *args, **kwargs)
  File "/Users/twillia2/anaconda3/envs/review/lib/python3.9/site-packages/flask_caching/__init__.py", line 591, in make_cache_key
    fname, version_data = self._memoize_version(
  File "/Users/twillia2/anaconda3/envs/review/lib/python3.9/site-packages/flask_caching/__init__.py", line 541, in _memoize_version
    version_data_list = list(self.cache.get_many(*fetch_keys))
  File "/Users/twillia2/anaconda3/envs/review/lib/python3.9/site-packages/cachelib/base.py", line 49, in get_many
    return [self.get(k) for k in keys]
  File "/Users/twillia2/anaconda3/envs/review/lib/python3.9/site-packages/cachelib/base.py", line 49, in <listcomp>
    return [self.get(k) for k in keys]
  File "/Users/twillia2/anaconda3/envs/review/lib/python3.9/site-packages/flask_caching/backends/filesystemcache.py", line 120, in get
    pickle_time = pickle.load(f)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 3: invalid start byte

Fix positioning of "Additional Scenarios" tab

Bug Description
"Additional Scenarios" tab is located too low for browser's zoomed to 100%. As a result, it is only partially visible.
Screen Shot 2023-11-09 at 3 44 39 PM

To Reproduce
Steps to reproduce the problem behavior

  1. Launch reView by running: reView
  2. Open http://localhost:8050/ in your browser.
    3.See error

Expected behavior
"Additional Scenarios" tab should show up higher so that it is fully visible at browser zoom 100%.

Screenshots
See above.

System (please complete the following information):

  • OS: MacOS Monterey
  • Compute hardware: MacBook Pro

Bespoke Layout Performance

Bug Description
This function is not quick enough as caching does not appear to working properly. Check the caching or only pull one point at a time from the data set.

ReEDS Buildout Page Features

Incorporate more functionality into the ReEDS buildout page:

  1. Linked chart/graph selections
  2. Variable Options
  3. Graph Element options
  4. Filters
  5. Titles

Replace --tech with --breaks-style in make-maps CLI

Why this feature is necessary:
Currently, the make-maps command allows the user to select from two technologies (wind and solar), which results in the use of hard-coded legend breaks for the various maps.

@ppinchuk raised the point that this is limiting for other technologies (e.g., geothermal) and, even for wind and solar, the hard-coded legend breaks may not always be suitable for every supply curve.

A possible solution is:
As discussed in PR 38, a possible solution would be to replace the existing --tech option with a --breaks-style option. The user could then choose from a set of options including wind and solar (which would use the existing hard-coded breaks for those technologies) or auto, which would use an automatic classification (e.g., Jenks-Fisher with 5 breaks).

I have considered the following alternatives:
An alternative solution would be to implement the new --breaks-style and enable the user to select any classification from https://pysal.org/mapclassify/api.html#classifiers. The user would have to specify the classifier name (and if applicable, a number of breaks), e.g., --breaks-style fisherjenks:5.

Additional context
NA

Charge code
NA. This is a nice to have at the moment.

Urgency / Timeframe
Not urgent at all.

Data Download Button

Why this feature is necessary:
This is necessary for users who want to work with a reView data set outside of reView. It would download the data set behind the current set of selections, filters, and map functions as a CSV or GeoPackage.

Publishable Map/Graph Image Download

Why this feature is necessary:
reView is able to make many useful maps and graphs quickly and it is possible to download these images. However, these are not quite high enough quality for publishing and they contain branding. This feature will apply a set of higher quality graphing/mapping methods to the currently selected dataset and return a high resolution PNG ready for publishing.

Flask Caching Security Issue

Bug Description
We have a flag for a potential security involving flask-caching. There was some recent activity in this packages repo that might result in this being resolved soon. Flagging this in issues until this resolves it self or we find an alternative.

pallets-eco/flask-caching#209

Default Project

Bug Description
A file containing the default project wasn't cloning to new install. Also, the default should be one of the self contained sample datasets, then the user can change that file themselves if they want reView to start with a specific project everytime.

Caching Problem

Bug Description
A clear and concise description of what the bug is.

Full Traceback
Traceback (most recent call last):
File "/Users/astanley/Programs/miniconda3/envs/review/lib/python3.9/site-packages/flask_caching/init.py", line 834, in decorated_function
cache_key = decorated_function.make_cache_key(f, *args, **kwargs)
File "/Users/astanley/Programs/miniconda3/envs/review/lib/python3.9/site-packages/flask_caching/init.py", line 591, in make_cache_key
fname, version_data = self._memoize_version(
File "/Users/astanley/Programs/miniconda3/envs/review/lib/python3.9/site-packages/flask_caching/init.py", line 541, in _memoize_version
version_data_list = list(self.cache.get_many(*fetch_keys))
File "/Users/astanley/Programs/miniconda3/envs/review/lib/python3.9/site-packages/cachelib/base.py", line 49, in get_many
return [self.get(k) for k in keys]
File "/Users/astanley/Programs/miniconda3/envs/review/lib/python3.9/site-packages/cachelib/base.py", line 49, in
return [self.get(k) for k in keys]
File "/Users/astanley/Programs/miniconda3/envs/review/lib/python3.9/site-packages/flask_caching/backends/filesystemcache.py", line 120, in get
pickle_time = pickle.load(f)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa2 in position 1: invalid start byte
127.0.0.1 - - [09/Jun/2022 10:39:27] "POST /_dash-update-component HTTP/1.1" 200 -
reView (INFO) - [classes.py:178] : Running _capacity_print... (Trigger: reeds_mapcap.children)
127.0.0.1 - - [09/Jun/2022 10:39:27] "POST /_dash-update-component HTTP/1.1" 200 -
reView (INFO) - [classes.py:178] : Running figure_map_reeds... (Trigger: years_reeds.value)
Exception possibly due to cache backend.
Traceback (most recent call last):
File "/Users/astanley/Programs/miniconda3/envs/review/lib/python3.9/site-packages/flask_caching/init.py", line 834, in decorated_function
cache_key = decorated_function.make_cache_key(f, *args, **kwargs)
File "/Users/astanley/Programs/miniconda3/envs/review/lib/python3.9/site-packages/flask_caching/init.py", line 591, in make_cache_key
fname, version_data = self._memoize_version(
File "/Users/astanley/Programs/miniconda3/envs/review/lib/python3.9/site-packages/flask_caching/init.py", line 541, in _memoize_version
version_data_list = list(self.cache.get_many(*fetch_keys))
File "/Users/astanley/Programs/miniconda3/envs/review/lib/python3.9/site-packages/cachelib/base.py", line 49, in get_many
return [self.get(k) for k in keys]
File "/Users/astanley/Programs/miniconda3/envs/review/lib/python3.9/site-packages/cachelib/base.py", line 49, in
return [self.get(k) for k in keys]
File "/Users/astanley/Programs/miniconda3/envs/review/lib/python3.9/site-packages/flask_caching/backends/filesystemcache.py", line 120, in get
pickle_time = pickle.load(f)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa2 in position 1: invalid start byte

Code Sample
A copy-pastable example if possible

# Your code here

To Reproduce
Steps to reproduce the problem behavior

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

System (please complete the following information):

  • OS: [e.g. OSX, Windows, Linux]
  • Compute hardware (e.g. Eagle, AWS)
  • Version [e.g. 0.1]

Additional context
Add any other context about the problem here.

Charge code
If you are at NREL and fixing this bug is urgent, please provide a charge code for our time.

Chart elements do not move dynamically when Options are expanded (Chrome)

Bug Description
When using Chrome, if you expand the "Options" section, the chart elements below do not dynamically move with the expanded menu. Instead, the Options show up behind the charts.
Screen Shot 2023-11-09 at 3 38 56 PM

This can be corrected by clicking the "X Variable" tab in the right chart menu, but then upon minimizing the "Options", the charts again do not move dynamically and white space remains:
Screen Shot 2023-11-09 at 3 42 23 PM

To Reproduce
Steps to reproduce the problem behavior

  1. Launch reView by running: reView
  2. Open http://localhost:8050/ in Chrome
  3. Scroll down past the "Options" section.
  4. See error

Expected behavior
Charts should move down and up dynamically as the options menu is expanded and minimized.

Screenshots
See above.

System (please complete the following information):

  • OS: MacOS Monterey
  • Compute hardware: MacBook Pro

Add support for new PV and bespoke wind capacity attributes

Bug Description
Neither bespoke wind nor new PV supply curves have the standard capacity column any longer. Instead, they have capacity_mw and capacity_mw_ac & capacity_mw_dc, respectively.

This causes the initial "cumulative capacity" chart to show "No data loaded":
Screen Shot 2023-11-09 at 3 33 38 PM

To Reproduce
Steps to reproduce the problem behavior

  1. Launch reView by running: reView
  2. Open http://localhost:8050 in your broswer.
  3. Scroll down past the "Options" section.
  4. See error

Expected behavior
Initial chart should look more like this:
Screen Shot 2023-11-09 at 3 37 09 PM

Screenshots
See above

System (please complete the following information):

  • OS: MacOS Monterey
  • Compute hardware: MacBook Pro

Update CLIs to use "review" base command

Why this feature is necessary:
This syntax will make it easier to find available review CLI commands. For our reVX CLI, we have a lot of commands that are not intuitively associated with reVX (e.g., setbacks) while for the reV model we have a reV base command to give the user a nice place to start. I find the latter much more intuitive and easy to work with.

A possible solution is:
Use a generic review CLI with command options. So, you would type review and then the help file would list the available commands. The option would then run the target command, so, something like this:

review make-maps -i input.csv -t wind -o wind_maps/ -d 300

Sample Dataset

Why this feature is necessary:
At the moment we don't have any way to provide users with our supply curve tables. It would be good to have at least a small sample dataset in package data so a user can see if reView is working

A possible solution is:
Take a supply curve and subset for a medium size state, add to package data, build config on installation or runtime that points to it.

Area based weighting with Bespoke Hybrids

Since bespoke, we are taking area weighted means instead of capacity weighted means. This will be a problem for hybrid datasets since there is no singular capacity field for the combination of wind and solar, just individual wind and solar area fields.

Figure something out in this case.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.