nceas / arcticdatautils Goto Github PK
View Code? Open in Web Editor NEWUtility functions in R for processing data for the Arctic Data Center
Home Page: https://nceas.github.io/arcticdatautils/
License: Apache License 2.0
Utility functions in R for processing data for the Arctic Data Center
Home Page: https://nceas.github.io/arcticdatautils/
License: Apache License 2.0
At least one package can be removed as a dep now that it's no longer a core part:
Probably more?
Also consider a helper for other common things to reference
doc <- new("eml")
my_atts <- set_attributes(...)
doc <- eml_set_shared_attributes(doc@dataset@otherEntity, my_atts)
something like that
Just got wind of this: https://github.com/jeroenooms/xslt. This will be a more popular project than the one I'm currently using and it looks like the installation is easier.
Functions like set_rights_and_access
can be passed any Subject
and a common mistake is to pass in an HTTPS ORCID URI for a Subject
. This is a problem because DataONE only uses the HTTP form and doesn't consider the HTTPS variant to be equivalent. So, currently, the user has to be careful to not to use the HTTPS form.
The various functions that set rightsHolder
and accessPolicy
information on SysMeta could easily check for this specific case.
Right now, if one or more data objects in your publish_update
command doesn't have its sysmeta fileName
property set, you get:
System Metadata for object with PID 'arctic-data.9451.1' did not have its fileName property set. This will result in 'NA' being set for the EML entityName and objectName (which we don't want). Edit each data object first to give it a fileName property.
I want to improve this error so the student knows what to do.
As changed in DataONEorg/rdataone@a3ed1bb, createObject
and updateObject
now return strings (instead of XML) containing the identifier of the created/updated object.
I need to modify code that uses these two functions to handle this change.
Just warn after the package is created. We had a package come in from the package with an object that had a format ID of 'xml'
The routines in this package that generate resource maps aren't smart (enough). They totally disregard the triples in previous versions of the resource map and generate the next version's resource map from scratch. This isn't usually a problem but it will be once/if we add PROV triples to resource maps because they'll get removed when we update a package.
Off the top of my head, the only function that needs changing is:
update_resource_map
I think what I want to do is have the routine split the resource map into two sets of triples:
cito:identifier
and suchThen, when updating a resource map, the routine would only need to generate a new resource map from scratch and then add the triples from (2).
These could be returned as a named vector with the names being the sysmeta fileName
I see that the solr query in get_package
uses rows=1000. Probably should:
similar to pid_to_eml_other_entity
, this function will use sysmeta to generate a physical section for a dataTable object, in addition to inserting attributes and factors.
arguments
mn = member node instance
pid = pid of object published on that member node
attributes = data frame of attributes with columns attributeName, attributeDefinition, measurementScale, domain, formatString, definition, unit, numberType, missingValueCode, missingValueCodeExplanation
factors = data frame of factors with column names attributeName, code, definition. This argument is not required for the function to run
this function would need to call pid_to_eml_physical
(which is a wrapper around sysmeta_to_eml_physical
) and is part of arcticdatautils
in addition to set_attributes
(which is the function that will take the attributes and factors dataframes as arguments)
it might be nice if we also incorporate an attribute validation check within this function
We put arcticdata.io into the production environment today (mn@env = 'prod
') so I need to adjust the package around that change.
esp w/in publish_update
Metadata documents created or edited by the registry script end up with EML access
elements at the /eml/dataset
level which are no longer respected by Metacat. The R package currently leaves these in tact when updating packages but it would be better if it removed them.
In the publish_update
function, when updating the packageId
and system
, also remove any EML access
elements before calling updateObject()
.
It's common to want to set the userId
on an existing party in an EML record and it'd be nice if there was a helper function to automate this. The helper function should do some sanity-checking on the provided value and try to figure out the directory
automatically like the current eml_party
function does.
@amoeba Edit: An MRE for this is:
library(arcticdatautils)
library(dataone)
mn <- MNode("https://dev.nceas.ucsb.edu/knb/d1/mn/v2")
pkg <- create_dummy_package(mn)
new_pkg <- publish_update(mn, pkg$metadata, pkg$resource_map, pkg$data)
cat(rawToChar(getObject(mn, new_pkg$resource_map)))
Noting the output:
<rdf:Description rdf:about="_:r1511139237r13053r1">
<foaf:name rdf:resource=""DataONE R Client"^^<http://www.w3.org/2001/XMLSchema#string"/>
</rdf:Description>
[original issue below]
library(dataone)
library(datapack)
library(arcticdatautils)
library(EML)
cn <- CNode('STAGING2')
mn <- getMNode(cn,"urn:node:mnTestKNB")
#write metadata and attach a data file in registry on dev.nceas.ucsb.edu
id <- 'knb.109096.1'
#read in registry EML
outpath <- '~/example.xml'
writeBin(getObject(mn, id), outpath)
#make edits and save EML
eml <- read_eml(outpath)
eml@dataset@abstract@[email protected][[2]] <- new('para', .Data = 'and edited using R') #this does not actually change anything, annoyingly
write_eml(eml, outpath)
#get ids from initial submission
ids <- get_package(mn, id)
#update EML with new version using publish_update
id_new <- publish_update(mn, metadata_pid = ids$metadata, resource_map_pid = ids$resource_map, data_pids = ids$data, metadata_path = outpath)
These steps generate a resource map with this, evidently problematic line:
<rdf:Description rdf:about="https://cn-stage-2.test.dataone.org/cn/v1/resolve/knb.109095.1">
<dcterms:identifier rdf:resource=""knb.109095.1"^^<http://www.w3.org/2001/XMLSchema#string"/>
</rdf:Description>
Link to package: https://dev.nceas.ucsb.edu/#view/urn:uuid:0c720fed-bbe9-4076-9e59-7636730b3d5a
Attached is a list of the 854 resource maps with this problem on the ADC. There are likely some on the KNB as well
Need to add a userID argument to eml_party functions to easily incorporate adding ORCIDs to EML in datateam workflow
eml_creator
eml_contact
eml_associated_party
Jeanette requested helpers for metadata provider and associatedparty. So write these.
> get_all_versions(env$mn, "doi:10.18739/A2NQ09")
Your login certificate is missing or unreadable. You must login again via CILogon using 'dataone::downloadCert(CertificateManager())'
Attempting to call as public user without being authenticated.
Your login certificate is missing or unreadable. You must login again via CILogon using 'dataone::downloadCert(CertificateManager())'
Attempting to call as public user without being authenticated.
Your login certificate is missing or unreadable. You must login again via CILogon using 'dataone::downloadCert(CertificateManager())'
Attempting to call as public user without being authenticated.
Your login certificate is missing or unreadable. You must login again via CILogon using 'dataone::downloadCert(CertificateManager())'
Attempting to call as public user without being authenticated.
Error in get_chain_neighbors(mn, pid) :
trying to get slot "obsoletes" from an object of a basic class ("NULL") with no slots
In addition: Warning message:
In .local(x, ...) :
Error getting SystemMetadata: READ not allowed on urn:uuid:49295b90-d47c-447a-bdb2-2809b1240971
>
It just returns an empty list()
right now.
Jesse mentioned this in Slack, here's an example he pasted:
get_related_pids(mn,"urn:uuid:0599306e-2ed9-4f5f-a5aa-f1f05e70cebe")
identifier documents1
"urn:uuid:0599306e-2ed9-4f5f-a5aa-f1f05e70cebe" "urn:uuid:0599306e-2ed9-4f5f-a5aa-f1f05e70cebe"
documents2 documents3
"resource_map_urn:uuid:ff45529d-83d2-437b-bf62-3484ff4a5b4c" "resource_map_urn:uuid:3a731bb6-447a-4e61-95d8-3cb305eab60f"
resourceMap1 resourceMap2
"resource_map_urn:uuid:0599306e-2ed9-4f5f-a5aa-f1f05e70cebe" "resource_map_urn:uuid:cccda1ff-cac3-4dc2-8f35-90723622503f"
> get_package(mn,"urn:uuid:0599306e-2ed9-4f5f-a5aa-f1f05e70cebe")
$metadata
[1] "urn:uuid:0599306e-2ed9-4f5f-a5aa-f1f05e70cebe"
$resource_map
[1] "resource_map_urn:uuid:cccda1ff-cac3-4dc2-8f35-90723622503f"
$data
character(0)
$child_packages
[1] "resource_map_urn:uuid:ff45529d-83d2-437b-bf62-3484ff4a5b4c"
[2] "resource_map_urn:uuid:3a731bb6-447a-4e61-95d8-3cb305eab60f"
See what's going on.
If two tokens are set, weird stuff happens.
It would be nice to have a simple function that queries the Solr index to see if the object with the given PID is indexed. e.g.,
is_indexed(my_pid)
returns boolean
@csjx found that many objects, ostensibly created by this package, failed to sync from the test arctic MN into a test CN env. It looks like this is caused by my use of serialize=TRUE
when calling digest::digest
and not using file=TRUE
.
The following code shows the issue and the proper code for computing checksums on the file itself, which is what's needed when synchronizing:
> library(openssl)
> library(digest)
>
> digest::digest("~/tmp/science_metadata.xml", algo = "sha256")
[1] "b482585513b413e8d2e78f7671a9c58a021551c97a6570fbf97e99eac99df852"
> digest::digest("~/tmp/science_metadata.xml", algo = "sha256", serialize=FALSE, file=TRUE)
[1] "76cc4dce5b7489d73b8a35a4842b76639c3c13d4aaa3fb0f574290798c7923ef"
> system("shasum -a 256 /Users/mecum/tmp/science_metadata.xml")
76cc4dce5b7489d73b8a35a4842b76639c3c13d4aaa3fb0f574290798c7923ef /Users/mecum/tmp/science_metadata.xml
> sha256(file("~/tmp/science_metadata.xml"))
sha256 76:cc:4d:ce:5b:74:89:d7:3b:8a:35:a4:84:2b:76:63:9c:3c:13:d4:aa:a3:fb:0f:57:42:90:79:8c:79:23:ef
md5
or sha1
instead of sha256
as it is sufficient for our (non-cryptographic) purposesUsing update_object
to update a metadata record doesn't set the packageId (eml@packageId) as the pid of the most recent version.
I'd be happy to update this (if it's a worthy change) - filing it as an issue so I don't forget.
Interns may like it, so ask them and find a way to keep it in if they use it.
isAuthorized
Function eml_metadata_provider
needs updating.
Error in getClass(Class, where = topenv(parent.frame())) :
“metadata_provider” is not a defined class
When creating new System Metadata records, this package is sometimes using a checksumAlgorithm
of SHA1 and sometimes SHA-1. This is technically valid but, in reality, probably a bad thing. Also see https://github.nceas.ucsb.edu/KNB/arctic-data/issues/283 as it relates to the issue there.
publish_update, when updating a parent, does add the updated child resource map to the list of parent child pids but it doesn't remove the old parent child pid if it's still referenced in parent_child_pids. This would be a nice safeguard.
Optionally, can updating the parent be done automatically? I think so now.......
They're out of sync with the func sig
the list of formatIDs called by the package needs to be updated. I couldn't find the actual list but did find where it is referenced:
from publish_object
if (!(format_id %in% D1_FORMATS)) {
stop(call. = FALSE,
paste0("The format_id of '", format_id, "' is not a valid format ID. See https://cn.dataone.org/cn/v2/formats for the current list. This package stores a copy and may be out of date with that list so please email the author if needed."))
}```
Looks like the eml_party function doesn't support adding URLs. Should be a quick change!
While working on registering arcticdata.io in production, @csjx noticed most objects have their replicationAllowed
property set to TRUE
in their System Metadata. We need this to be off for now due to NCEI not being a Tier 4 MN. datapack
sets this to TRUE
by default and sets 3 replicas.
Go through everywhere in the code the Data Team uses and manually disable replication on every call.
publish_object
defaults to application/octet-stream if the format_id
arg is left off. Turn on guessing via guess_format_id
. Also use the custom NetCDF-specific format ID guesser for all .nc
files.
> library(arcticdatautils)
> env <- env_load('production')
> get_all_versions(env$mn, 'doi:10.18739/A2T67B')
Your login certificate is missing or unreadable. You must login again via CILogon using 'dataone::downloadCert(CertificateManager())'
Attempting to call as public user without being authenticated.
Your login certificate is missing or unreadable. You must login again via CILogon using 'dataone::downloadCert(CertificateManager())'
Attempting to call as public user without being authenticated.
Your login certificate is missing or unreadable. You must login again via CILogon using 'dataone::downloadCert(CertificateManager())'
Attempting to call as public user without being authenticated.
Error in get_chain_neighbors(node, pid) :
trying to get slot "obsoletes" from an object of a basic class ("NULL") with no slots
In addition: Warning message:
In .local(x, ...) :
Error getting SystemMetadata: READ not allowed on arctic-data.9827.4
The list of Imports and Suggests in in the DESCRIPTION is probably due for some trimming down. At the very least, ncdf4
should be removed and guarded with requireNamespace
in relevant functions.
These functions trust the result but I think getSystemMetadata doesn't error out when it fails.
While running the articdatautils, three tests are skipped, and fifteen failures are detected. I believe this issue has to do with my session token. I set a browser() breakpoint, which can be seen in the dump below, but can be ignored.
It appears that the issue isn't with code integrity, rather it is a problem with my session token. I believe this is the case because before the failures, tests were skipped due to one not being set. The failures are also occurring in code that depends on the token.
Every error also fails at the same point, and follows the same trace.
This can be marked as very low priority, I'm not using arcticdatautils-only playing with the code base.
options(dataone_test_token = "...")
devtools::test()
Loading arcticdatautils
Testing arcticdatautils
access: 1234Called from: eval(code, test_env)
Browse[1]> c
5
dataone: SSS
formats: ..
editing: 6789ab.cdef..
EML: ............................E
environment: ......
helpers: E
inventory: .......................
packaging: ....
sysmeta: .....
util: ...........
Skipped ---------------------------------------------------------------------------------------------------------------------------------------
permissions can be checked (@test_dataone.R#7) - No token set. Skipping test.
permissions can be checked on multiple objects (@test_dataone.R#16) - No token set. Skipping test.
checking permissions on a missing object returns an error (@test_dataone.R#27) - No token set. Skipping test.
Failed ----------------------------------------------------------------------------------------------------------------------------------------
Error: get_package works for a simple package (@test_access.R#10) --------------------------------------------------------------------------
parse error: premature EOF
(right here) ------^
1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_access.R:10
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: get_package works for a package with a child package (@test_access.R#24) ------------------------------------------------------------
parse error: premature EOF
(right here) ------^
1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_access.R:24
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: get_package works for an object in two packages (@test_access.R#48) -----------------------------------------------------------------
parse error: premature EOF
(right here) ------^
1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_access.R:48
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: get_package works the same when given a metadata pid as it does when given a resource map pid (@test_access.R#62) -------------------
parse error: premature EOF
(right here) ------^
1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_access.R:62
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: get_package returns a warning if the package doesn't exist (@test_access.R#74) ------------------------------------------------------
parse error: premature EOF
(right here) ------^
1: create_dummy_package(mnn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_access.R:74
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: we can publish an update (@test_editing.R#11) ---------------------------------------------------------------------------------------
parse error: premature EOF
(right here) ------^
1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:11
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: an identifier can be manually specified when publishing an update (@test_editing.R#30) ----------------------------------------------
parse error: premature EOF
(right here) ------^
1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:30
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: we can create a resource map (@test_editing.R#50) -----------------------------------------------------------------------------------
parse error: premature EOF
(right here) ------^
1: create_dummy_metadata(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:50
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:14
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: we can update a resource map (@test_editing.R#66) -----------------------------------------------------------------------------------
parse error: premature EOF
(right here) ------^
1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:66
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: an object can be published with a SID (@test_editing.R#97) -------------------------------------------------------------------------
parse error: premature EOF
(right here) ------^
1: publish_object(mn, tmp, "text/csv", sid = new_sid) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:97
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/editing.R:55
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: SIDs are maintained when publishing an update to an object with a SID (@test_editing.R#118) ----------------------------------------
parse error: premature EOF
(right here) ------^
1: publish_object(mn, metadata_path, "eml://ecoinformatics.org/eml-2.1.1", sid = new_sid) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:118
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/editing.R:55
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: we can publish an update to an object (@test_editing.R#139) ------------------------------------------------------------------------
parse error: premature EOF
(right here) ------^
1: create_dummy_object(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:139
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:58
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: we can publish an update to an object and specify our own format id (@test_editing.R#160) ------------------------------------------
parse error: premature EOF
(right here) ------^
1: create_dummy_object(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:160
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:58
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: extra statements are maintained between updates (@test_editing.R#179) --------------------------------------------------------------
parse error: premature EOF
(right here) ------^
1: create_dummy_package(mn, 3) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:179
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
Error: rightsholder is properly set back after publishing an update (@test_editing.R#218) -------------------------------------------------
parse error: premature EOF
(right here) ------^
1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:218
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)
DONE ==========================================================================================================================================
sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] dataone_2.1.0 arcticdatautils_0.6.1 datapack_1.3.1 EML_1.0.3 XML_3.98-1.9 testthat_1.0.2
loaded via a namespace (and not attached):
[1] Rcpp_0.12.12 compiler_3.4.1 plyr_1.8.4 bindr_0.1 base64enc_0.1-3 tools_3.4.1 ncdf4_1.16 digest_0.6.12
[9] uuid_0.1-2 jsonlite_1.5 memoise_1.1.0 tibble_1.3.4 pkgconfig_2.0.1 rlang_0.1.2 rstudioapi_0.7 commonmark_1.4
[17] curl_2.8.1 yaml_2.1.14 bindrcpp_0.2 withr_2.0.0 httr_1.3.1 stringr_1.2.0 dplyr_0.7.3 roxygen2_6.0.1
[25] xml2_1.1.1 devtools_1.13.3 redland_1.0.17-9 glue_1.1.1 parsedate_1.1.3 R6_2.2.2 hash_2.2.6 magrittr_1.5
[33] assertthat_0.2.0 stringi_1.1.5 crayon_1.3.4
This is primarily aimed at NetCDF files.
To speed up creating dataTable
and attribute
elements, we could extract as much information as possible that's relevant to creating an EML dataTable
from an individual NetCDF file.
Write one for NetCDF and maybe think of others.
Not just arcticdata pids
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.