Giter VIP home page Giter VIP logo

Comments (5)

mayeulk avatar mayeulk commented on June 12, 2024 3

Here is a workaround and a blueprint for a fix:

.libPaths() # make sure to remove all dataverse in all places
remove.packages("dataverse")
# restart R
devtools::install_github("IQSS/dataverse-client-r")
#Installing package into ‘/home/mk/R/x86_64-pc-linux-gnu-library/3.6’
#(as ‘lib’ is unspecified)
#* installing *source* package ‘dataverse’
library("icews")
library("DBI")
library("dplyr")
library("usethis")
print(sessionInfo())
#loaded via a namespace (and not attached):
#[...]
#[25] glue_1.3.1             dataverse_0.2.1.9001   RSQLite_2.2.0

setup_icews(data_dir = "~/temp_icews", use_db = TRUE, keep_files = TRUE,
            r_profile = TRUE)
Sys.setenv("DATAVERSE_SERVER" = "dataverse.harvard.edu")
update_icews(dryrun = FALSE)

as per IQSS/dataverse-client-r#33 (comment)

Hope it helps! Cheers

from icews.

andybega avatar andybega commented on June 12, 2024 1

Hey @mayeulk, thanks! This works for me now as well:

devtools::install_github("IQSS/dataverse-client-r")

# restart R
library("icews")
library("dataverse")
Sys.setenv("DATAVERSE_SERVER" = "dataverse.harvard.edu")

# works now
foo = dataverse::get_file(2711073, dataset = get_doi()$historic)

I will keep this issue open until dataverse is updated on CRAN.

Also note that in the dry run there is no file for the past 6 months or so (last one is 20190625); maybe it's related:

ICEWS has stopped updating. I heard they managed to regain funding but I have no idea when they will resume.

from icews.

andybega avatar andybega commented on June 12, 2024

This is a simpler example:

This should download one of the documentation PDFs:

library("icews")
#> Options not set, consider running 'setup_icews()'
#> data_dir: NULL
#> use_db: NULL
#> keep_files: NULL
dataverse::get_file(2711073, dataset = get_doi()$historic)
#> Error in dataverse::get_file(2711073, dataset = get_doi()$historic): Not Found (HTTP 404).

Created on 2020-01-06 by the reprex package (v0.3.0)

from icews.

andybega avatar andybega commented on June 12, 2024

Looks like the problem is with either the R dataverse client or the dataverse API itself. The direct URL for the PDF file above is https://dataverse.harvard.edu/api/access/datafile/2711073, and it works.

However, in dataverse::get_file(), a query parameter for the desired format is set to "original" by default, leading to the URL https://dataverse.harvard.edu/api/access/datafile/2711073/?format=original. That breaks and leads to the 404 error.

This is the essential bit from the dataverse::get_file() internals:

library("dataverse")
library("httr")

key <- Sys.getenv("DATAVERSE_KEY")

u <- "https://dataverse.harvard.edu/api/access/datafile/2711073"
query <- list(format = "original")

r <- httr::GET(u, httr::add_headers(`X-Dataverse-key` = key))
# works
status_code(r)
#> [1] 200

# with format argument it does not work
r <- httr::GET(u, httr::add_headers(`X-Dataverse-key` = key), query = query)
status_code(r)
#> [1] 404

Created on 2020-01-06 by the reprex package (v0.3.0)

This looks like a relevant issue in the R dataverse client repo: IQSS/dataverse-client-r#33

from icews.

mayeulk avatar mayeulk commented on June 12, 2024

Hi, I have the same issue.
Also note that in the dry run there is no file for the past 6 months or so (last one is 20190625); maybe it's related:

Download            '20190622-icews-events.zip'
Ingest records from '20190622-icews-events.tab'
Download            '20190623-icews-events.zip'
Ingest records from '20190623-icews-events.tab'
Download            '20190624-icews-events.zip'
Ingest records from '20190624-icews-events.tab'
Download            '20190625-icews-events.zip'
Ingest records from '20190625-icews-events.tab'
> # Should list proposed downloads, ingests, etc.
> update_icews(dryrun = FALSE)
Downloading 'events.1995.20150313082510.tab.zip'
Error in dataverse::get_file(file = file_ref, dataset = get_doi()[[repo]]) : 
  Not Found (HTTP 404).

from icews.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.