rubyforgood / ghgvc Goto Github PK

Sauce Labs graciously allows free testing for open source projects. Since cross-browser compatibility is a goal of the project, we should try to use this tool

https://saucelabs.com/open-source

Investigate whether this tool would be a good fit

Spec tests fail without this. Need to add the chromedriver & any other dependencies so we can run feature tests

After bootstrap upgrade, icons disappeared

R server

The instruction "Create a new empty folder on your host OS and clone this project & the R server" is a bit confusing; should I also be downloading an "R server"? How do I go about doing this?

Cloning syntax

git clone [email protected]/rubyforgood/ghgvc.git && cd ghgvc should be git clone [email protected]:rubyforgood/ghgvc.git && cd ghgvc

Creating databses

When trying to create dbs, I got the following errors:

#<Mysql2::Error: Can't connect to MySQL server on 'database' (111)>
Couldn't create database for {"adapter"=>"mysql2", "encoding"=>"utf8", "pool"=>5, "host"=>"database", "username"=>"root", "password"=>"root_password", "port"=>3306, "timeout"=>5000, "database"=>"climate_calculator_dev"}, {:charset=>"utf8"}
(If you set the charset manually, make sure you have a matching collation)
Created database 'climate_calculator_dev'
#<Mysql2::Error: Can't connect to MySQL server on 'database' (111)>
Couldn't create database for {"adapter"=>"mysql2", "encoding"=>"utf8", "pool"=>5, "host"=>"database", "username"=>"root", "password"=>"root_password", "port"=>3306, "timeout"=>5000, "database"=>"climate_calculator_test"}, {:charset=>"utf8"}
(If you set the charset manually, make sure you have a matching collation)
Created database 'climate_calculator_test'

After running docker-compose run --rm app bundle exec rails db:create again, it seemed create dbs fine (I got the following return)

Starting ghgvc_database_1 ...
Starting ghgvc_r_1 ... done
Created database 'climate_calculator_dev'
Created database 'climate_calculator_test'

Additional Docker commands would be helpful

I suggest adding

docker-compose run --rm app bin/rake routes
docker-compose run --rm web bin/rails c
docker-compose run --rm app rspec 

Issues with testing

When I ran docker-compose run --rm app rspec , I got 3/5 failing tests; 2 of the errors were due to a problem locating chrome webdriver

Failure/Error: before { visit root_path }

     Selenium::WebDriver::Error::WebDriverError:
        Unable to find chromedriver. Please download the server from http://chromedriver.storage.googleapis.com/index.html and place it somewhere on your PATH. More info at https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver.

& I suspect the third is related.

Biome
     Failure/Error: response = connection.eval("ghgvcr::get_biome(latitude, longitude)")

     Rserve::Connection::EvalError:
       eval failed: Packet[cmd=2130771970,len=<nil>, con='<nil>', status=error:'Unknown variable/method'(127)]

Failure to get biomes

When I click on the map in the browser (both on localhost:3000 and 127.0.0.1:3000), I get the following errors on the server:

Started GET "/get_biome?lng=-94&lat=38" for 172.18.0.1 at 2017-07-24 14:04:37 +0000
app_1       | Cannot render console from 172.18.0.1! Allowed networks: 127.0.0.1, ::1, 127.0.0.0/127.255.255.255
app_1       | Processing by WorkflowsController#get_biome as */*
app_1       |   Parameters: {"lng"=>"-94", "lat"=>"38"}
app_1       | Completed 500 Internal Server Error in 1668ms (ActiveRecord: 0.0ms)
app_1       |
app_1       |
app_1       |
app_1       | Rserve::Connection::EvalError (eval failed: Packet[cmd=2130771970,len=<nil>, con='<nil>', status=error:'Unknown variable/method'(127)]):
app_1       |
app_1       | app/services/biome.rb:11:in `list'
app_1       | app/controllers/workflows_controller.rb:62:in `get_biome'

The root page looks like this below the map:

I'm guessing all these errors are due to not having the R server, but I've never used it and not sure what I need to do for full setup.

It's important that the application works across multiple browsers and platforms

Look at using Sauce & Travis per here: https://docs.travis-ci.com/user/gui-and-headless-browsers/

Related to ebimodeling#12

• doc folder
• for deletion folder
• INSTALL.md
• junk.rb
• ghgvc.sh
• rb files in Rails root (looks like testing files)
• GHGVC-wireframes.pdf
• nginx.conf
• temp folder (has javascripts that don't appear to be referenced)
• script/rails (replaced by bin/rails)
• spec folder (needs to be updated)
• Anything else that looks like it isn't necessary

Remove ActiveSupport.halt_callback_chains_on_return_false

Look at the .css.scss files, I believe the .css should be dropped (deprecation warning when running tests)

Deprecated parameter. Minor, but generates an unnecessary warning in console

Button broke after the JS/CSS update. Triage & resolve to add behavior as before

Integrate Travis CI with the project

One graph shows miles driven, the other is the more technical version. Allow toggling between on UI when presented.

With a JSON string below, assigned to a variable data:

{"sites":{"site_1_data":{"lat":"-8.75","lng":"-65.21","ecosystems":{"Broadleaf_Evergreen_Forest":{"4":[""],"OM_ag":["300.2348334"],"OM_root":["33.61702128"],"OM_wood":["43.64411765"],"OM_litter":["6.904033613"],"OM_peat":["0"],"OM_SOM":["11.3607"],"fc_ag_wood_litter":["0.52"],"fc_root":["0"],"fc_peat":["0"],"fc_SOM":["0"],"Ec_CO2":["35.909"],"Ec_CH4":["0.294"],"Ec_N2O":["0.0059"],"k_ag_wood_litter":["0.167"],"k_root":["0.04"],"k_peat":["0"],"k_SOM":["0.4"],"termite":["3"],"Ed_CO2_ag_wood_litter":["41.66666667"],"Ed_CO2_root":["41.66666667"],"Ed_CO2_peat":["45"],"Ed_CO2_litter":["48.33333333"],"Ed_CH4_ag_wood_litter":["0.0275"],"Ed_CH4_root":["0"],"Ed_CH4_peat":["0"],"Ed_CH4_litter":["0"],"Ed_N2O_ag_wood_litter":["0"],"Ed_N2O_root":["0"],"Ed_N2O_peat":["0"],"Ed_N2O_litter":["0"],"F_CO2":["-14.52279202"],"F_CH4":["-0.22584375"],"F_N2O":["0.081607401"],"rd":["0"],"tR":["-9999"],"FR_CO2":["-9999"],"FR_CH4":["-9999"],"FR_N2O":["-9999"],"dfc_ag_wood_litter":["-9999"],"dfc_root":["0"],"dfc_peat":["0"],"dk_ag_wood_litter":["0.167"],"dk_root":["0.04"],"dk_peat":["0"],"age_transition":["-9999"],"new_F_CO2":["-9999"],"new_F_CH4":["-9999"],"new_F_N2O":["-9999"],"F_anth":["0"],"sw_radiative_forcing":["0.3979"],"latent":["0.5731"],"sensible":["MAP"],"code":["T2"],"vegtype":["Broadleaf Evergreen Forest"],"name":["Broadleaf_Evergreen_Forest"],"NACP_FI_AGB_US":["NA"],"NACP_LiDAR_Boreal_AGB":["0"],"LiDAR_AGB_Boreal_Eurasia":[""],"SOC":["51.7241"],"biophysical_net":["0.5731"],"in_synmap":["true"]}}}},"options":{"T_E":"50","T_A":"100","r":"0.00","co2":"TRUE","n2o":"TRUE","ch4":"TRUE","storage":"TRUE","flux":"TRUE","disturbance":"TRUE","includeANTH":"TRUE","includeBIO":"TRUE"}}

The command below fails on the Ruby app side (this repo):

connection.eval("ghgvcr::calc_ghgv(data)").to_ruby

With the following error messages listed:

Error in eco_params$options : $ operator is invalid for atomic vectors
Rserve::Connection::EvalError (eval failed: Packet[cmd=2130771970,len=<nil>, con='<nil>', status=error:'Unknown variable/method'(127)])

This looks like an error on the ghgvcr repo side of things (the R application), rather than the ruby/rails side. Noting here for triage & investigation further (even though likely broken on the ghgvcr repo).

Unclear if this is related to this bit of code leftover with the XML stuff (thought was ripped out):

https://github.com/jaydorsey/ghgvcR/blob/master/R/ghgvc.R#L49

Steps to Reproduce

Use the jay/compose_improvements branch for testing. The branch uses an image I pushed up that uses a slightly different path for the data folder (/app/data) so we could use compose locally without having to spend a lot of time thinking about how the data gets loaded/mounted (the command downloads the data to a mounted volume which is preserved across up/down calls)

docker-compose up on the branch and visit http://localhost:3000 and try to generate a graph to see the error

Desired behavior

• Add the JSON snippet above as a test sample/suite against the calc_ghgv method?
• Should also work on the Ruby side of things
• ???

@ValentineHerr I can schedule a time to chat about this if you've got time in the upcoming weeks, if you think it's something you can investigate & resolve.

Please add a link at the top of the page, following "Guide", entitled "Methods & Sources". I will provide a .pdf documenting the methods and sources, which will address ebimodeling#105. For now, you can link to a blank pdf, which I can easily replace.

Improved README, but also look into Vagrant or Docker

Google map doesn't work 100% correctly on Firefox. Review & fix

I think this content

should be removed from the home page. It also shows up on the result page, which is correct

Rather than clicking on the map, allow for entering the location by typing in the lat/lng coordinates.

This will allow easier reconciliation of the results on the original versus this fork.

Need to add a scripted wait to the app server, so it waits on MySQL server to spin up before trying to connect.

This error most noticeably occurs when running the database migrations for the first time.

Look at integrating this script: https://github.com/vishnubob/wait-for-it

Related to #66

On chrome (possibly others), the map is allowed to zoom out too far

These can live in the public folder, don't need to be anywhere near the asset pipeline

The netcdf (*.nc) files are very large in size (1 file is 10GB alone). Come up with a better strategy, maybe gitlfs?

Any new strategy should seek to lighten the size of the repo & make it easier to jump into the project.

Also:

Update the setup & installation instructions
Maybe also filter branch the previous commits to get rid of history (the repo is rather large as-is)

It would be nice to have a small switch at the top of the page allowing the user to toggle between technical and layperson displays. Differences would include:
1- graph (produced in R) would have different units and labels. Units would be driving equivalents (miles driven per 100sq. feet cleared), which would be calculated by applying a simple conversion factor prior to final calculated values prior to plotting (for now, [value in CO2-eq]*1.86, but @teixeirak needs to verify).
2- The .csv for download would only show up on the technical display.
3- The users guide (.pdf) to which the "guide" link directs would differ.

I'd appreciate other suggestions from a layperson's perspective!

Currently, no footer exists. Could point back to the repo, possibly the organizations (Smithsonian, UI), etc.

The about/guide links and header are misaligned after the Bootstrap/Rails/jQuery upgrade

Ecosystem edit link no longer appears after bootstrap/jquery upgrade

Running thru the demo, only CSV export works for now but some work has been done on getting the app to work with JSON

The R scripts work great with JSON, but the output/download link is for CSV only.

Need to look at getting this PR merged in, because the generated file will have the wrong extension anyways

Opening this issue to make sure someone double checks it is okay if the function get.biome() defined here, apparently needs 5 arguments but that up to 9 are given in line 40 of get.biome.R

Needs the MYSQL_USER & MYSQL_PASSWORD env set in travis per docs & .travis.yml updated

https://docs.travis-ci.com/user/database-setup/#MySQL

There are a number of controllers, routes, views, css, js, etc that appear to be scaffolded but are not actually used. Remove these.

Add 5 options in a "settings" button and form with the following:

• Anaysis time (T_A): slider ranging from 1-500, default is 50
• Emissions time (T_E): slider ranging from 1-100, default is 50
• Anthropogenic effects (includeANTH): boolean, default is 1
• Biophysical effects (includeBIO): boolean default is 1
• Decay rate (r): slider 0-0.3, default is 0

Select greenhouse gases to include:

• Carbon Dioxide (co2): default checked
• Nitrous Oxide (n20): default checked
• Methane (ch4): default checked

Select sources to include:

• Storage (storage): default checked
• Flux (flux): default checked
• Disturbance (disturbance): default checked

All the above should be variable names in the "options".

Currently, the scripts write & read text files on the file system. This prevents multiple users from using the application at the same time and may lead to inconsistent results or errors

Look at a gem like https://github.com/clbustos/rinruby, https://github.com/alexgutteridge/rsruby, or https://github.com/clbustos/Rserve-Ruby-client possibly?

We don't really need to be on 2.4.1, so relax by letting the user decide. We can specify which ruby version in the .travis.yml file.

Not correct:

Screenshot from live site (correct):

Appears to be an issue w/ the SVG but unclear if it's from conversion on the Rails side, or the R application

Adding rspec, so test folder is not needed

Add an icon next to the label for each of the advanced settings. Include a description of the settings

Icons can be the simple glyphicon-question-sign or similar. No descriptions at present, but putting these in the en.yml might make it easier to switch them out later.

Related to #29

Deprecated, no longer needed

The graphic isn't present but the screen does dim. Possibly a lightbox issue.

We're attempting to remove lightbox from the application, so this can be swapped for a native bootstrap modal

Screenshot of original behavior (dim + present below) for reference