nceas / arcticdatautils Goto Github PK

At least one package can be removed as a dep now that it's no longer a core part:

• ncdf4

Probably more?

Also consider a helper for other common things to reference

doc <- new("eml")
my_atts <- set_attributes(...)
doc <- eml_set_shared_attributes(doc@dataset@otherEntity, my_atts)

something like that

Just got wind of this: https://github.com/jeroenooms/xslt. This will be a more popular project than the one I'm currently using and it looks like the installation is easier.

Functions like set_rights_and_access can be passed any Subject and a common mistake is to pass in an HTTPS ORCID URI for a Subject. This is a problem because DataONE only uses the HTTP form and doesn't consider the HTTPS variant to be equivalent. So, currently, the user has to be careful to not to use the HTTPS form.

The various functions that set rightsHolder and accessPolicy information on SysMeta could easily check for this specific case.

Right now, if one or more data objects in your publish_update command doesn't have its sysmeta fileName property set, you get:

System Metadata for object with PID 'arctic-data.9451.1' did not have its fileName property set. This will result in 'NA' being set for the EML entityName and objectName (which we don't want). Edit each data object first to give it a fileName property.

I want to improve this error so the student knows what to do.

As changed in DataONEorg/rdataone@a3ed1bb, createObject and updateObject now return strings (instead of XML) containing the identifier of the created/updated object.

I need to modify code that uses these two functions to handle this change.

Just warn after the package is created. We had a package come in from the package with an object that had a format ID of 'xml'

The routines in this package that generate resource maps aren't smart (enough). They totally disregard the triples in previous versions of the resource map and generate the next version's resource map from scratch. This isn't usually a problem but it will be once/if we add PROV triples to resource maps because they'll get removed when we update a package.

Off the top of my head, the only function that needs changing is:

• update_resource_map

I think what I want to do is have the routine split the resource map into two sets of triples:

1. Basic housekeeping triples such as

• ore:aggregates / cito:documents statements
• other basic statements like cito:identifier and such

2. Extra triples

Then, when updating a resource map, the routine would only need to generate a new resource map from scratch and then add the triples from (2).

These could be returned as a named vector with the names being the sysmeta fileName

I see that the solr query in get_package uses rows=1000. Probably should:

• Increase the count
• Warn if the number of rows returned is as the max
• Allow the user to specify rows

similar to pid_to_eml_other_entity, this function will use sysmeta to generate a physical section for a dataTable object, in addition to inserting attributes and factors.

arguments

• mn = member node instance

• pid = pid of object published on that member node

• attributes = data frame of attributes with columns attributeName, attributeDefinition, measurementScale, domain, formatString, definition, unit, numberType, missingValueCode, missingValueCodeExplanation

• factors = data frame of factors with column names attributeName, code, definition. This argument is not required for the function to run

this function would need to call pid_to_eml_physical (which is a wrapper around sysmeta_to_eml_physical) and is part of arcticdatautils in addition to set_attributes (which is the function that will take the attributes and factors dataframes as arguments)

it might be nice if we also incorporate an attribute validation check within this function

We put arcticdata.io into the production environment today (mn@env = 'prod') so I need to adjust the package around that change.

esp w/in publish_update

Metadata documents created or edited by the registry script end up with EML access elements at the /eml/dataset level which are no longer respected by Metacat. The R package currently leaves these in tact when updating packages but it would be better if it removed them.

In the publish_update function, when updating the packageId and system, also remove any EML access elements before calling updateObject().

It's common to want to set the userId on an existing party in an EML record and it'd be nice if there was a helper function to automate this. The helper function should do some sanity-checking on the provided value and try to figure out the directory automatically like the current eml_party function does.

@amoeba Edit: An MRE for this is:

library(arcticdatautils)
library(dataone)

mn <- MNode("https://dev.nceas.ucsb.edu/knb/d1/mn/v2")
pkg <- create_dummy_package(mn)
new_pkg <- publish_update(mn, pkg$metadata, pkg$resource_map, pkg$data)
cat(rawToChar(getObject(mn, new_pkg$resource_map)))

Noting the output:

<rdf:Description rdf:about="_:r1511139237r13053r1">
    <foaf:name rdf:resource="&quot;DataONE R Client&quot;^^&lt;http://www.w3.org/2001/XMLSchema#string"/>
</rdf:Description>

[original issue below]

library(dataone)
library(datapack)
library(arcticdatautils)
library(EML)

cn <- CNode('STAGING2')
mn <- getMNode(cn,"urn:node:mnTestKNB")

#write metadata and attach a data file in registry on dev.nceas.ucsb.edu
id <- 'knb.109096.1'

#read in registry EML
outpath <- '~/example.xml'
writeBin(getObject(mn, id), outpath)

#make edits and save EML
eml <- read_eml(outpath)
eml@dataset@abstract@[email protected][[2]] <- new('para', .Data = 'and edited using R') #this does not actually change anything, annoyingly
write_eml(eml, outpath)

#get ids from initial submission
ids <- get_package(mn, id)
#update EML with new version using publish_update
id_new <- publish_update(mn, metadata_pid = ids$metadata, resource_map_pid = ids$resource_map, data_pids = ids$data, metadata_path = outpath)

These steps generate a resource map with this, evidently problematic line:

  <rdf:Description rdf:about="https://cn-stage-2.test.dataone.org/cn/v1/resolve/knb.109095.1">
    <dcterms:identifier rdf:resource="&quot;knb.109095.1&quot;^^&lt;http://www.w3.org/2001/XMLSchema#string"/>
  </rdf:Description>

Link to package: https://dev.nceas.ucsb.edu/#view/urn:uuid:0c720fed-bbe9-4076-9e59-7636730b3d5a

Attached is a list of the 854 resource maps with this problem on the ADC. There are likely some on the KNB as well

bad_rms_ADC.txt

Need to add a userID argument to eml_party functions to easily incorporate adding ORCIDs to EML in datateam workflow

eml_creator
eml_contact
eml_associated_party

Jeanette requested helpers for metadata provider and associatedparty. So write these.

It just returns an empty list() right now.

Jesse mentioned this in Slack, here's an example he pasted:

get_related_pids(mn,"urn:uuid:0599306e-2ed9-4f5f-a5aa-f1f05e70cebe")
                                                  identifier                                                   documents1 
             "urn:uuid:0599306e-2ed9-4f5f-a5aa-f1f05e70cebe"              "urn:uuid:0599306e-2ed9-4f5f-a5aa-f1f05e70cebe" 
                                                  documents2                                                   documents3 
"resource_map_urn:uuid:ff45529d-83d2-437b-bf62-3484ff4a5b4c" "resource_map_urn:uuid:3a731bb6-447a-4e61-95d8-3cb305eab60f" 
                                                resourceMap1                                                 resourceMap2 
"resource_map_urn:uuid:0599306e-2ed9-4f5f-a5aa-f1f05e70cebe" "resource_map_urn:uuid:cccda1ff-cac3-4dc2-8f35-90723622503f" 
> get_package(mn,"urn:uuid:0599306e-2ed9-4f5f-a5aa-f1f05e70cebe")
$metadata
[1] "urn:uuid:0599306e-2ed9-4f5f-a5aa-f1f05e70cebe"

$resource_map
[1] "resource_map_urn:uuid:cccda1ff-cac3-4dc2-8f35-90723622503f"

$data
character(0)

$child_packages
[1] "resource_map_urn:uuid:ff45529d-83d2-437b-bf62-3484ff4a5b4c"
[2] "resource_map_urn:uuid:3a731bb6-447a-4e61-95d8-3cb305eab60f"

See what's going on.

If two tokens are set, weird stuff happens.

It would be nice to have a simple function that queries the Solr index to see if the object with the given PID is indexed. e.g.,

is_indexed(my_pid)

returns boolean

@csjx found that many objects, ostensibly created by this package, failed to sync from the test arctic MN into a test CN env. It looks like this is caused by my use of serialize=TRUE when calling digest::digest and not using file=TRUE.

The following code shows the issue and the proper code for computing checksums on the file itself, which is what's needed when synchronizing:

> library(openssl)
> library(digest)
> 
> digest::digest("~/tmp/science_metadata.xml", algo = "sha256")
[1] "b482585513b413e8d2e78f7671a9c58a021551c97a6570fbf97e99eac99df852"
> digest::digest("~/tmp/science_metadata.xml", algo = "sha256", serialize=FALSE, file=TRUE)
[1] "76cc4dce5b7489d73b8a35a4842b76639c3c13d4aaa3fb0f574290798c7923ef"
> system("shasum -a 256 /Users/mecum/tmp/science_metadata.xml")
76cc4dce5b7489d73b8a35a4842b76639c3c13d4aaa3fb0f574290798c7923ef  /Users/mecum/tmp/science_metadata.xml
> sha256(file("~/tmp/science_metadata.xml"))
sha256 76:cc:4d:ce:5b:74:89:d7:3b:8a:35:a4:84:2b:76:63:9c:3c:13:d4:aa:a3:fb:0f:57:42:90:79:8c:79:23:ef 

• Change the places where I calculated checksums to calculate checksums based upon the file on disk
• Consider changing the checksums algorithm to md5 or sha1 instead of sha256 as it is sufficient for our (non-cryptographic) purposes

Using update_object to update a metadata record doesn't set the packageId (eml@packageId) as the pid of the most recent version.
I'd be happy to update this (if it's a worthy change) - filing it as an issue so I don't forget.

Interns may like it, so ask them and find a way to keep it in if they use it.

• Checking object existence
• Checking object formatType/Id
• Checking isAuthorized

Function eml_metadata_provider needs updating.

Error in getClass(Class, where = topenv(parent.frame())) :
“metadata_provider” is not a defined class

When creating new System Metadata records, this package is sometimes using a checksumAlgorithm of SHA1 and sometimes SHA-1. This is technically valid but, in reality, probably a bad thing. Also see https://github.nceas.ucsb.edu/KNB/arctic-data/issues/283 as it relates to the issue there.

1. Find the right way to write algorithm names
2. Change the relevant sections of code to reflect that change

publish_update, when updating a parent, does add the updated child resource map to the list of parent child pids but it doesn't remove the old parent child pid if it's still referenced in parent_child_pids. This would be a nice safeguard.

Optionally, can updating the parent be done automatically? I think so now.......

They're out of sync with the func sig

the list of formatIDs called by the package needs to be updated. I couldn't find the actual list but did find where it is referenced:

from publish_object

  if (!(format_id %in% D1_FORMATS)) {
    stop(call. = FALSE,
         paste0("The format_id of '", format_id, "' is not a valid format ID. See https://cn.dataone.org/cn/v2/formats for the current list. This package stores a copy and may be out of date with that list so please email the author if needed."))
  }```

Looks like the eml_party function doesn't support adding URLs. Should be a quick change!

While working on registering arcticdata.io in production, @csjx noticed most objects have their replicationAllowed property set to TRUE in their System Metadata. We need this to be off for now due to NCEI not being a Tier 4 MN. datapack sets this to TRUE by default and sets 3 replicas.

Go through everywhere in the code the Data Team uses and manually disable replication on every call.

publish_object defaults to application/octet-stream if the format_id arg is left off. Turn on guessing via guess_format_id. Also use the custom NetCDF-specific format ID guesser for all .nc files.

The list of Imports and Suggests in in the DESCRIPTION is probably due for some trimming down. At the very least, ncdf4 should be removed and guarded with requireNamespace in relevant functions.

These functions trust the result but I think getSystemMetadata doesn't error out when it fails.

Issue

While running the articdatautils, three tests are skipped, and fifteen failures are detected. I believe this issue has to do with my session token. I set a browser() breakpoint, which can be seen in the dump below, but can be ignored.

It appears that the issue isn't with code integrity, rather it is a problem with my session token. I believe this is the case because before the failures, tests were skipped due to one not being set. The failures are also occurring in code that depends on the token.

Every error also fails at the same point, and follows the same trace.

This can be marked as very low priority, I'm not using arcticdatautils-only playing with the code base.

Dump

options(dataone_test_token = "...")
devtools::test()
Loading arcticdatautils
Testing arcticdatautils
access: 1234Called from: eval(code, test_env)
Browse[1]> c
5
dataone: SSS
formats: ..
editing: 6789ab.cdef..
EML: ............................E
environment: ......
helpers: E
inventory: .......................
packaging: ....
sysmeta: .....
util: ...........

Skipped ---------------------------------------------------------------------------------------------------------------------------------------

1. permissions can be checked (@test_dataone.R#7) - No token set. Skipping test.

2. permissions can be checked on multiple objects (@test_dataone.R#16) - No token set. Skipping test.

3. checking permissions on a missing object returns an error (@test_dataone.R#27) - No token set. Skipping test.

Failed ----------------------------------------------------------------------------------------------------------------------------------------

1. Error: get_package works for a simple package (@test_access.R#10) --------------------------------------------------------------------------
   parse error: premature EOF

                 (right here) ------^
   

1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_access.R:10
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

2. Error: get_package works for a package with a child package (@test_access.R#24) ------------------------------------------------------------
   parse error: premature EOF

                 (right here) ------^
   

1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_access.R:24
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

3. Error: get_package works for an object in two packages (@test_access.R#48) -----------------------------------------------------------------
   parse error: premature EOF

                 (right here) ------^
   

1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_access.R:48
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

4. Error: get_package works the same when given a metadata pid as it does when given a resource map pid (@test_access.R#62) -------------------
   parse error: premature EOF

                 (right here) ------^
   

1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_access.R:62
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

5. Error: get_package returns a warning if the package doesn't exist (@test_access.R#74) ------------------------------------------------------
   parse error: premature EOF

                 (right here) ------^
   

1: create_dummy_package(mnn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_access.R:74
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

6. Error: we can publish an update (@test_editing.R#11) ---------------------------------------------------------------------------------------
   parse error: premature EOF

                 (right here) ------^
   

1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:11
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

7. Error: an identifier can be manually specified when publishing an update (@test_editing.R#30) ----------------------------------------------
   parse error: premature EOF

                 (right here) ------^
   

1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:30
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

8. Error: we can create a resource map (@test_editing.R#50) -----------------------------------------------------------------------------------
   parse error: premature EOF

                 (right here) ------^
   

1: create_dummy_metadata(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:50
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:14
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

9. Error: we can update a resource map (@test_editing.R#66) -----------------------------------------------------------------------------------
   parse error: premature EOF

                 (right here) ------^
   

1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:66
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

10. Error: an object can be published with a SID (@test_editing.R#97) -------------------------------------------------------------------------
    parse error: premature EOF

                 (right here) ------^
    

1: publish_object(mn, tmp, "text/csv", sid = new_sid) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:97
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/editing.R:55
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

11. Error: SIDs are maintained when publishing an update to an object with a SID (@test_editing.R#118) ----------------------------------------
    parse error: premature EOF

                 (right here) ------^
    

1: publish_object(mn, metadata_path, "eml://ecoinformatics.org/eml-2.1.1", sid = new_sid) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:118
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/editing.R:55
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

12. Error: we can publish an update to an object (@test_editing.R#139) ------------------------------------------------------------------------
    parse error: premature EOF

                 (right here) ------^
    

1: create_dummy_object(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:139
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:58
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

13. Error: we can publish an update to an object and specify our own format id (@test_editing.R#160) ------------------------------------------
    parse error: premature EOF

                 (right here) ------^
    

1: create_dummy_object(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:160
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:58
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

14. Error: extra statements are maintained between updates (@test_editing.R#179) --------------------------------------------------------------
    parse error: premature EOF

                 (right here) ------^
    

1: create_dummy_package(mn, 3) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:179
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

15. Error: rightsholder is properly set back after publishing an update (@test_editing.R#218) -------------------------------------------------
    parse error: premature EOF

                 (right here) ------^
    

1: create_dummy_package(mn) at C:\Users\Data\Documents\GitHub\arcticdatautils/tests/testthat/test_editing.R:218
2: get_token_subject() at C:\Users\Data\Documents\GitHub\arcticdatautils/R/helpers.R:98
3: dataone::getTokenInfo(dataone::AuthenticationManager()) at C:\Users\Data\Documents\GitHub\arcticdatautils/R/util.R:607
4: dataone::getTokenInfo(dataone::AuthenticationManager())
5: getTokenDetails("dataone_test_token")
6: parseAuthToken(authToken)
7: fromJSON(payloadChar)
8: fromJSON_string(txt = txt, simplifyVector = simplifyVector, simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, flatten = flatten,
...)
9: parseJSON(txt, bigint_as_char)
10: parse_string(txt, bigint_as_char)

DONE ==========================================================================================================================================

Session Info

sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] dataone_2.1.0 arcticdatautils_0.6.1 datapack_1.3.1 EML_1.0.3 XML_3.98-1.9 testthat_1.0.2

loaded via a namespace (and not attached):
[1] Rcpp_0.12.12 compiler_3.4.1 plyr_1.8.4 bindr_0.1 base64enc_0.1-3 tools_3.4.1 ncdf4_1.16 digest_0.6.12
[9] uuid_0.1-2 jsonlite_1.5 memoise_1.1.0 tibble_1.3.4 pkgconfig_2.0.1 rlang_0.1.2 rstudioapi_0.7 commonmark_1.4
[17] curl_2.8.1 yaml_2.1.14 bindrcpp_0.2 withr_2.0.0 httr_1.3.1 stringr_1.2.0 dplyr_0.7.3 roxygen2_6.0.1
[25] xml2_1.1.1 devtools_1.13.3 redland_1.0.17-9 glue_1.1.1 parsedate_1.1.3 R6_2.2.2 hash_2.2.6 magrittr_1.5
[33] assertthat_0.2.0 stringi_1.1.5 crayon_1.3.4

This is primarily aimed at NetCDF files.

To speed up creating dataTable and attribute elements, we could extract as much information as possible that's relevant to creating an EML dataTable from an individual NetCDF file.

Write one for NetCDF and maybe think of others.

Not just arcticdata pids