Giter VIP home page Giter VIP logo

mbsstools's Introduction

Erik W. Leppo

Working as a data scientist in the environmental field with Tetra Tech (since 1994).

Working in R since 2006 and Shiny since 2017. Using GitHub for version control since 2016.

๐Ÿ”ฅ GitHub Stats

Anurag's GitHub stats

GitHub Streak

mbsstools's People

Contributors

leppott avatar tivasdnr avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

mbsstools's Issues

Shiny app fails

Describe the bug
Fails with non-standard files.

To Reproduce
Load files in Shiny app.

Expected behavior
Should handle files that worked a week ago.

Screenshots
NA

Additional context
Sample files being tested.

metric.values - Fish example not working

Example code fails.

Metrics, MBSS Index, Fish

myIndex <- "MBSS.2005.Fish"

Thresholds

thresh <- metrics_scoring

get metric names for myIndex

(myMetrics.Fish <- as.character(droplevels(unique(thresh[thresh[,"Index.Name"]==myIndex,"Metric"]))))

Taxa Data

myDF.Fish <- taxa_fish
myMetric.Values.Fish <- metric.values(myDF.Fish, "SampleID", "fish", myMetrics.Fish)
View(myMetric.Values.Fish)

Add Shiny App

Is your feature request related to a problem? Please describe.
Add a Shiny app for metric calculation.

Describe the solution you'd like

  • upload a file
  • choose MMI
  • calculate
  • download results as zip file

Describe alternatives you've considered
NA

Additional context
May need to include

  • instructions on file input
  • MMI basics
  • Review / qc file input structure / content
  • Review results (table and plot - use dygraphs or plotly)

Update metric.scores warning and error messages.

Is your feature request related to a problem? Please describe.
Warning and error messages in metric.scores don't always space and/or paste correctly.

Describe the solution you'd like
See BioMonTools package for updates.

Describe alternatives you've considered
NA

Additional context
Non-critical.

FIBI - message about 0 ind always shows.

Describe the bug
QC message about samples with 0 individuals and scoring a 1 is always showing. Even if count is zero.

To Reproduce
Calc FIBI.

Expected behavior
Should only show when count is greater than 0.

Screenshots
image

Additional context
Add if/then to code is metric.scores().

metric.values() - not all required fields specified

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
See images below from the code and the help file.

Expected behavior
NA

Screenshots

Fish
image

Bugs
image

Additional context
Checked both bugs and fish and missing 3 required fields from fish index; NATIVE_MBSS, SILT, and TOTBIOM.

PHIcalc - fails on example.

data.PHI <- MBSS.PHI
PHI <- PHIcalc(data.PHI)
Error in fun.df[, name.Remote_020][boo.BioRegion == TRUE]/20 :
non-numeric argument to binary operator

FFG Piercer not recognized

Describe the bug
There are two invertebrates in MD with the FFG characteristic "Piercer". If this designation is included in the dataset, it fails to run. Please add this to the valid inputs list.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
NA

Additional context
Add any other context about the problem here.

metric.values has error in R v4.0

Describe the bug
Calculating metrics for fish results in error and no calculation.

To Reproduce
Run example code in metric.values and get error.

# Metrics, MBSS Index, Fish
myIndex <- "MBSS.2005.Fish"
# Thresholds
thresh <- metrics_scoring
# get metric names for myIndex
(myMetrics.Fish <- as.character(droplevels(unique(thresh[thresh[,
"Index.Name"]==myIndex,"MetricName.Other"]))))
# Taxa Data
myDF.Fish <- taxa_fish
myMetric.Values.Fish <- metric.values(myDF.Fish, "fish", myMetrics.Fish)
View(myMetric.Values.Fish)

Expected behavior
Should run without error.

Screenshots
If applicable, add screenshots to help explain your problem.
image

Additional context
Did not test other R versions.

Sessioninfo below.
image

Vignette not installing

Describe the bug
I get an error code if I have "build_vignettes = TRUE" that says "failed re-building". If I dont put that in there, it works fine.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
NA

Additional context
Add any other context about the problem here.

zip file not always created

Describe the bug
Zip file not created on some computers.

Works on ShinyApps.io and most test machines.

To Reproduce
Run the shiny app but the zip file never gets created so cannot download data.

Expected behavior
Zip file should always get created.

Screenshots
NA

Additional context
utils::zip is the command.

Update (modernize) GitHub Actions

Is your feature request related to a problem? Please describe.
Update GitHub Actions to current standards.

Describe the solution you'd like
See baytrends package.

Describe alternatives you've considered
Not good to leave "as is" since not all actions working.

Additional context
Add any other context or screenshots about the feature request here.

Change to scoring thresholds

Is your feature request related to a problem? Please describe.
Per email from Matt, 20190221.

Matt sent revised file.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

FIBI for HIGHLAND not scoring.

Email from DNR staff on 2017-10-26

Hi Erik,

I am attempting to calculate a FIBI using the package you developed and I found an issue in the metric.scores function, specifically line 181 of the scrip https://github.com/leppott/MBSStools/blob/master/R/metric.scores.R t:

myTF <- DF_Metrics[,IndexName]=="MBSS.2005.Fish" & DF_Metrics[,IndexRegion]=="WARM"

Basically, there is no "WARM" stratum, but rather a warmwater highland stratum, which we designate as "HIGHLAND". I think that changing that in the package script would fix the issues I'm having calculating a FIBI. Let me know what you think.

Thanks,

Jonathan

Function to determine strata from lat-long

Is your feature request related to a problem? Please describe.
Most users won't know the strata for a given site but should have GIS coordinates. Keep users in the R environment instead of requiring GIS (of which most may not have access).

Describe the solution you'd like
Build function to determine strata based on input coordinates.

Describe alternatives you've considered
Access to GIS software is a high bar for most. Keep it simple and stay in R.

Additional context
Use the absolute coordinates. Not checking for location on a blue line (stream). May add distance to blue line and/or edge of strata polygon.

Exclude column not working

Describe the bug
The exclude column does not seem to be working. I have tried using the Y/N and TRUE/FALSE designations, but all taxa are being counted in the metrics. This was not a problem when I ran these data in the Fall. Regardless of how this column is formatted, the code does run. Invalid input in this column should trigger a fatal error.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
NA

Additional context
Add any other context about the problem here.

Change levels of Exclude for consistency with database export

Is your feature request related to a problem? Please describe.
Benthic macroinvertebrate sample database populates a factor to identify non-unique taxon (EXCLUDE) with levels "TRUE" and "FALSE". MBSStools metric.values.R sets levels of Exclude as "Y" and null. This has created problems reading data in this column when there are long strings of null values and currently requires additional tidying that is not required.

Describe the solution you'd like
Change factor for EXCLUDE from "Y" to "TRUE" when the row is a non-unique taxon to establish consistency with level used in the benthic sample database. This change should be applied in the following lines of metric.values.R

212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 226, 245, 246, 247, 255, 256, 257, 258, 259, 277, 278, 279, 280, 281, 289, 290, 291, 295, 296, 297 (twice), 313, and 439

Describe alternatives you've considered
We have currently made these changes with additional, personal lines of code or directly in flat files, which is not desired.

Consistency of variables, names and values

Describe the bug
Check for and ensure consistency of variable names and examples between functions. For example, Highland vs. Highlands.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
Should be the same across datasets and functions.

Screenshots
NA

Additional context
Add any other context about the problem here.

Feature - MMI calc

Add feature for calculating MBSS fish and bug MMI (Southerland et al. 2006).

IBI calculation - potential data QC checks

Is your feature request related to a problem? Please describe.
Potential checks:

  1. Strata spellings. Make sure all are of the correct convention (i.e., spelling).
  2. Exclude column. TRUE/FALSE vs. other formats.
  3. Add a master taxa list check. User provides and the master taxa list and reports back non-matches.
    a. Can do for fish and bugs. Just need the lists. If posted online much easier to update/distribute to the users but can put in a current version.
  4. Ensure samples with 0 organisms get an IBI of 1.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

PHIcalc? example data seems corrupted

Describe the bug
Ran example code from PHIcalc? on 10/18/2018 in R 3.4.0 and see many NA in raw values for PHI data table. Referenced a handful of sites in database and see they should have data.

To Reproduce
Steps to reproduce the behavior:

  1. Go to PHIcalc? http://127.0.0.1:11817/library/MBSStools/html/PHIcalc.html
  2. Copy and paste example code into R console and run
  3. Visually examine InStrmHab_020 for StationID 1:10
  4. See NA
  5. Those sites have Instream Habitat scores

Expected behavior
Function ran as expected, but View(PHI) returns table of suspect data

Screenshots
image

Desktop (please complete the following information):

  • OS: Windows 7 Enterprise
  • Browser Chrome
  • Version Unknown

Shiny Download button not working

Describe the bug
If running SHINY on your local machine using the function in R, when you click "download data" you get an extensionless filename suggestion, that gives the error "failed-no file" when you try to save it.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
NA

Additional context
Add any other context about the problem here.

Add integration with master taxa list in Shiny app

Is your feature request related to a problem? Please describe.
Want users to be able to use the official MBSS master taxa list in calculations.

Describe the solution you'd like
Shiny App: Add ability to merge the standard master taxa lists (taxa characteristics) with uploaded sample data.

Describe alternatives you've considered
Need in the Shiny app.

Additional context
NA

MapTaxObs

Error when running MapTaxObs function when loading the county boundary shapefile from GIT.

First ran the function with the directories that already existed in my wd. After the error I deleted the GIS and Maps directories and re-ran the function pulling the GIS data from GIT, but got the same error.

library(MBSStools)

Set Working Directory

wd <- getwd()

Create Example Data if Needed

Create Directories

myDir.create <- file.path(wd,"Data")
ifelse(dir.exists(myDir.create)==FALSE,dir.create(myDir.create),"Directory already exists")
[1] "Directory already exists"
myDir.create <- file.path(wd,"GIS")
ifelse(dir.exists(myDir.create)==FALSE,dir.create(myDir.create),"Directory already exists")
[1] TRUE
myDir.create <- file.path(wd,"Maps")
ifelse(dir.exists(myDir.create)==FALSE,dir.create(myDir.create),"Directory already exists")
[1] TRUE

Create Data

Taxa Data

myFiles <- c("AllFish_95to16.xls", "TaxaMapsCrossWalk20170731.xlsx")
file.copy(file.path(path.package("MBSStools"),"extdata",myFiles),file.path(wd,"Data",myFiles))
[1] FALSE FALSE

GIS

unzip(file.path(path.package("MBSStools"),"extdata","MD_GIS.zip"),exdir=file.path(wd,"GIS"))

Inputs

Obs <- "AllFish_95to16.xls"
XWalk <- "TaxaMapsCrossWalk20170731.xlsx"
wd <- getwd()

Create maps

MapTaxaObs(Obs, XWalk, wd)
C:/Users/mashton/Documents
OGR data source with driver: ESRI Shapefile
Source: "C:\Users\mashton\Documents\GIS", layer: "MD_State_Boundary"
with 2 features
It has 9 fields
Integer64 fields read as strings: OBJECTID
OGR data source with driver: ESRI Shapefile
Source: "C:\Users\mashton\Documents\GIS", layer: "MD_Coast_Hydrology"
with 1117 features
It has 9 fields
Integer64 fields read as strings: OBJECTID
OGR data source with driver: ESRI Shapefile
Source: "C:\Users\mashton\Documents\GIS", layer: "MD_Boundary_County_Detailed"
with 24 features
It has 7 fields
Integer64 fields read as strings: OBJECTID DISTRICT COUNTY_FIP COUNTYNUM
Error in as.double(y) :
cannot coerce type 'S4' to vector of type 'double'

Update documentation

Is your feature request related to a problem? Please describe.
Preparing for release to a wider audience. Need more background/explanation in descriptions.

Describe the solution you'd like
Update help documentation, R vignette, and possibly develop a user manual.

Describe alternatives you've considered
NA

Additional context
NA

Failed install - R v3.6.0 - staged install

Describe the bug*
Staged install implemented in R v.3.6.0 is causing the package to not install.

StagedInstall = FALSE in DESCRIPTION is a quick (temporary) fix.

Due to use system.file(package="ContDataQC").

Staged install installs to temp directory and fails.

To Reproduce
Fails on devtools::install_github("leppott/MBSStools")

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.
image

Additional context
https://developer.r-project.org/Blog/public/2019/02/14/staged-install/index.html

metric.scores - Family BIBI incorrect upper end of scale

Describe the bug
Unlike genus BIBI the Family BIBI uses "greater than" instead of "greater than or equal" for its thresholds. The values entered in to the rules table did not adjust for this and values at the upper threshold are being scored a "5" instead of a "3".

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots

Metric thresholds from report.

image

Additional context
Revised threshold values provided by MBSS staff.

image

Unlike the BioMonTools package that uses an Excel file the extdata folder the MBSStools package has the values in the package as an internal data file. Revise the .tab file and process data script in "data-raw" folder.

Error - FlowSum()

Matt, email 20170623.

Still get an error message (below) even when the excel sheet being read in only has one site's (ANTI-101-S-2015) worth of data. All other command lines work up until I request results.

Error in model.frame.default(formula = data.calc[, name.DisCell] ~ data.calc[, :
invalid type (list) for variable 'data.calc[, name.DisCell]'

If I do it with two sites worth (ANTI-101-S and ANTI-101-X) I get the same error message as before multiple zero values per channel.

Error in try(if (sum(tbl.zero$Freq) != nrow(tbl.zero)) stop(msg.dupChan)) :

More than one zero value for Lateral Distance ('Lat. Loc (m)').

This indicates more than one channel ('SumFlowChannelNum') per sample ('SITEYR'). Or an error in the data.

Please correct then retry. Sample and Channel IDs are presented below:

Add examples for getting data from a database for metric calculation

Is your feature request related to a problem? Please describe.
Provide examples for getting data from a database (or Excel) and then calculating the BIBI and FIBI.

Describe the solution you'd like
Another Vignette but also include in the README.

Describe alternatives you've considered
Work with a flat table as well as get from a database query.

Additional context
Need to include the required fields.

IBI Narrative

Is your feature request related to a problem? Please describe.
Add IBI Narrative

Describe the solution you'd like
Put into example for metric.scores.

Describe alternatives you've considered
It could be in the function but the factor doesn't always translate outside of the function.

Additional context
Added to BioMonTools inside the function but have returned as text.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.