Workflow Updates for the NAWQA Pesticide National Synthesis Project
usgs-r / nawqa-pesticide-nsp Goto Github PK
View Code? Open in Web Editor NEWWorkflow Updates for the NAWQA Pesticide National Synthesis Project
Workflow Updates for the NAWQA Pesticide National Synthesis Project
You could make this object a separate target so it is only created once in the pipeline, rather than for every map.
Originally posted by @ceenell in #8 (comment)
In corporate multiple viz-related updates:
A reproducible example of this issue is available on the $debug-targets-error
branch.
In the $main
branch, I am using tarchetypes::tar_files()
instead of tar_target()
to create targets because i want to track all the data input files that are being used to generate the pesticide maps. Using tar_files
allows me to track all of the input files and to then branch over them to load the data (doc reference here).
The problem with using tar_files()
is that is results in downstream targets that appear to be outdated when they are not. Here is a zoomed in example:
This is a known issue with tar_files
. Here is a snippet from the tarchetypes
doc:
tar_files_input() is like tar_files() but more convenient when the files in question already
exist and are known in advance. Whereas tar_files() always appears outdated (e.g. with tar_outdated())
because it always needs to check which files it needs to branch over, tar_files_input() will ap-
pear up to date if the files have not changed since last tar_make(). In addition, tar_files_input()
automatically groups input files into batches to reduce overhead and increase the efficiency of par-
allel processing.
The problem is, when I make the switch from tar_files
to tar_files_input
, it works for some targets (e.g., p1_pest_hi_dbf
and p1_pest_lo_dbf
) but not others (e.g., p1_pest_bin_csv
, p1_pest_label_csv`). Here is the error that I bump into after making the switch:
Error in purrr::map_chr(., ~grep(paste("(\\.", .x, "\\.)", sep = ""), :
object 'p1_pest_of_interest' not found
Error in `tar_throw_run()`:
! callr subprocess failed: object 'p1_pest_of_interest' not found
Visit https://books.ropensci.org/targets/debugging.html for debugging advice.
Run `rlang::last_error()` to see where the error occurred.
AND, weirdly, when I debug using tar_option_set(debug = "p1_pest_bin_csv")
, tar_make(callr_function = NULL)
and then "step into the current function call" using the debugger, it works without an issue (successfully cycling through the 69 pesticides of interest listed in p1_pest_of_interest
.
$debug-targets-error
In addition to converting tar_files
calls to tar_files_input
in 1_fetch.R
I also had to make the following changes to the repo to get it to run:
library(tidyverse)
to the top-level of _targets.R
(rather than in tar_options_set
to fix this error:Error in poi %>% purrr::map_chr(~grep(paste("(\\.", .x, "\\.)", sep = ""), :
could not find function "%>%"
_targets.R
to 1_fetch.R
to prevent this error:Error in list.files(path_dbfs, full.names = T, pattern = "H_") :
object 'path_dbfs' not found
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.