Hi,

I notice that ots_create_tidy_data() returns identical export/import values for country pairs originating from the same "reporter," for example:

d <- ots_create_tidy_data(years = 2000, reporters = "usa", partners = c("afg", "can"), products=c("0101", "0705"), table="yrc", max_attempts=5)

which returns

d[, c(2, 10, 13:16)]
reporter_iso product_code export_value_usd export_rca import_value_usd import_rca
1: usa 0101 672770373 2.1196 485633089 1.1655
2: usa 0101 672770373 2.1196 485633089 1.1655
3: usa 0705 223787742 1.4500 43872463 0.2130
4: usa 0705 223787742 1.4500 43872463 0.2130

not sure what the reported values are supposed to be because they aren't aggregated values either.

Suppose I want to retrieve the export/import values for reporter-partner country pair i-j across k product codes, what exactly should I specify the ots_create_tidy_data() argument? Say, for example, reporter list c("usa", "can", "chl", "chn", "arg"), partners list c("usa", "can", "chl", "chn", "arg"), and product code c("0101", "0705"), such as

d <- ots_create_tidy_data(years = 2000, reporters = c("usa", "can", "chl", "chn", "arg"), partners = c("usa", "can", "chl", "chn", "arg"), products=c("0101", "0705"), table="yrc", max_attempts=5)

Also, when setting the "table" argument to "yrc", the output will not report partners' iso country code (as compared to when table is set to "yrp", yet this method will not report produce code). It will be a bit difficult to distinguish a reporter's export/import values of a particular product category to a particular "partner" from the output if I include a long list of partners in the function.
I am wondering if there's any way to get around this?

By all account, awesome and handy function!

The current branch seems to have an issue with requesting commodity data

e.g. ots_create_tidy_data(years = 2002, reporters = "chl", commodities = "apple", table = "yrc")

returns

Error in tradestatistics::ots_commodity_code(group = commodities_wm[x]) : 
  unused argument (group = commodities_wm[x])

I think this first error is caused by a typo where group should be section as ots_commodity_code has params commodity, section

This led to another error in ots_strings_processing where it looks as though lookup for sections has not been implemented.

I've forked your package and applied the bug fixes. I'm new to using github, so sorry for any breaches of etiquette. Thanks for maintaining this useful package

Hello. I get the next message when running this command:

yr <- ots_create_tidy_data(
years = 2019,
table = "yr"
)

Message:

Error in file(file, "rt") :
cannot open the connection to 'https://api.tradestatistics.io/year_range'
In addition: Warning message:
In file(file, "rt") :
cannot open URL 'https://api.tradestatistics.io/year_range': HTTP status was '502 Bad Gateway'
Error in year_range$year : $ operator is invalid for atomic vectors
Error in min(year_range):max(year_range) : NA/NaN argument
In addition: Warning messages:
1: In years %in% min(year_range):max(year_range) :
NAs introduced by coercion
2: In years %in% min(year_range):max(year_range) :
NAs introduced by coercion

Thanks

Hi guys,

first of all many thanks for the package - fantastic work.

I just realised, however, the server api.tradestatistics.io seems to be down, is this a temporary thing?

Many thanks!

Best,
Sven

Issue in R:

trade_data_bilateral <- ots_create_tidy_data(

• reporters = "all",
• partners = "all",
• years = 2002:2020,
• table = "yrp"
• )
  Error in file(file, "rt") :
  cannot open the connection to 'https://api.tradestatistics.io/year_range'
  In addition: Warning message:
  In file(file, "rt") :
  cannot open URL 'https://api.tradestatistics.io/year_range': HTTP status was '502 Bad Gateway'
  Error in year_range$year : $ operator is invalid for atomic vectors
  Error in min(year_range):max(year_range) : NA/NaN argument
  In addition: Warning messages:
  1: In years %in% min(year_range):max(year_range) :
  NAs introduced by coercion
  2: In years %in% min(year_range):max(year_range) :
  NAs introduced by coercion

I am working with the tradestatistics package to download international trade data. But I still get the same message when running this command:

ots_create_tidy_data(
years = 2019,
reporters = "all",
partners = "all",
commodities = "all",
table = "yr",
max_attempts = 5,
use_cache = FALSE,
file = NULL,
use_localhost = FALSE
)

Error in parse_con(txt, bigint_as_char) :
lexical error: invalid char in json text.
year 2002 2020
(right here) ------^

I really don't understand the origin of the problem..

Dear,

I needed a bulk bilateral data on ex/im, and wanted to install the tradestatistics package. The first problem I faced was the incompatibility of version of installed R and the package. Then I noticed it was archived, so I used remote library to install the package. However, after installation, when I tried to run the following basic command, it was not generating a data:

library(tradestatistics)

library(DT)

datatable(ots_create_tidy_data(years = 2016, reporters = "chl", partners = c("arg", "bol", "per")))

This is the returned output:

year reporter_iso reporter_fullname_english
1: 2016 chl Chile
2: 2016 chl Chile
3: 2016 chl Chile

But when I was extracting with the following code, it was working perfectly fine and generating the data:

read_from_ots <- function(p) {fromJSON(sprintf("https://api.tradestatistics.io/yrpc?y=2016&r=chl&p=%s", p))}

partners <- c("arg", "bol", "per")

ots_data <- map_df(partners, read_from_ots)

datatable(ots_data)

I think the tradestatistics package was not accessing an api properly. Thanks in advance for help!

Dr. Vargas and Team,

Firstly, let me tip my hat to your amazing work. I have some question about the 1980s product data. As I can understand, it is from the Feenstra et al. data set, but I don't see SITC4 codes, and when I querry 1989, which I assume is from this dataset, but I don't think your tables have the STIC4 codes, and when I try to query used tires (tyres) and retreads, I get zero results, but when I query rubber products I get an array of product flows from the 1980s, but not 4012.

yrpc1 <- ots_create_tidy_data(years = 1989, reporters = "all", partners = c("aia","atg","arb","bhs","brb","vgb","cym","cck","cub","dma","dom","glp","hti","jam","mtq","msr","ant","lca", "vct"), products = "4012", table = "yrpc")

I am also reading your documentation from here (https://docs.tradestatistics.io/data-processing.html#filling-gaps-in-our-data) in section 2.2 Filling gaps in our data, and was wondering if the tidy_data imports were already corrected, or if there were code. I looked through the manual but cold not find this (very possibly my own oversight or misunderstanding).

I was also wondering how you treated the stable unit problem in cases, e.g., Netherlands Antilles, where there have been admin changes in the ISO or 3 digit country codes.

Many thanks in advance.

Just out of curiosity, why is HS 2007 the HS revision of choice?

Hi, thanks for this super useful package. I noticed running ots_inflation_adjustment() shrank the original dataset and then realized this was due to the reference year being dropped. If this is desired behaviour, it would be good to add it to the function documentation. I would argue though that if the data for the reference year are found in the original data.frame, it should simply be left there unchanged.

Dear All,

I'm trying to use "tradestatistics" to pull data but, after successful installation, I keep getting an error: "Error in open.connection(con, "rb") :Could not resolve host: api.tradestatistics.io".
The code is simple: ots_create_tidy_data(years = 2002, reporters = "chl", partners = "chn")

I'm fairly new to R, but I would say that this error message means that either the server is down, or I've connectivity issues. I was trying to access it from three different computers with various security (firewall and antivirus softwares) but I keep getting the same error.

Does anyone else have the same issue?

Thank you!

Hello, thanks for the useful work with tradestatistics!

The following code used to work well:

jpn.ots <- ots_create_tidy_data(
  years = min_year:2020, # I would do "min_year:max_year", but unfortunately there are only data until 2020
  reporters = "jpn",
  #partners = "arg",
  table = "yrp" 
)

... however, now I get the error:

Downloading data for the combination 2008, jpn, all...
Error : Invalid: Parquet magic bytes not found in footer. Either the file is corrupted or this is not a parquet file.
Error in ots_read_from_api(table = table, max_attempts = max_attempts, :
It wasn't possible to obtain data. Provided this function tests your internet connection
you misspelled a reporter, partner or table, or there was a server problem. Please check and try again.

partners seem to not providing any function for now. reporters give us the total trade of the country, so no bilateral trade at the moment.

Hello,

I'm having an issue with getting tariff data using the tradestatistics package with the following code

test <- ots_create_tidy_data(years=2004,reporters="usa", partners = "can", commodities = "all", table = "tariffs", max_attempts = 30)

There are three issues in the outputs.

1. The code ran successfully, but the output includes every trade partner the US has instead of just Canada as specified.
2. In theory, reporter-partner-tariff should have about the same number product lines, but this is not the case. If I run table(test$partner_iso), it shows that each trade partner to the US have different number of product code observations -- I'm not sure if this is what the WITS API data provides or if it is something about the package.
3. The output dataset does not have a column(s) on tariff levels. The code I ran above yielded about 118141 obs of 8 variables, but none of the variables details the tariff for the product line per reporter-partner pair.

Thank you in advance for your help on this!

-Eric

I just installed the tradestatistics package on a new machine and, oddly enough, the ots_create_tidy_data() does not work. I tried getting data using the example in the help file, but I get the following error message:

ots_create_tidy_data(years = 1980, reporters = "chl", products = "0101", table = "yrc")
Error: lexical error: invalid char in json text.
                                       <html>  <head><title>502 Bad Ga
                     (right here) ------^

I even tried it with the development version, but the error persists.

On your workshop you use manual caching of results with code like this

yr_rda <- "yr.rda"
if (!file.exists(yr_rda)) {
  yr <- ots_create_tidy_data(years = 2000:2018, reporters = "arg", table = "yr")
  save(yr, file =  yr_rda, compress = "xz")
} else {
  load(yr_rda)
}
yr

How about adding cache logic inside {tradestatistics}, This would make it easier for user to cache data and lighten the load on your servers. I implemented an example of cache logic that works both in memory (using {memoise}) and on disk here.

Hello, I think this is a great project and wanted to inquire about it's status. The R package was removed in April this year. Are the packages and data still maintained / updated?

there seems to be a server side glitch. For example,

ots_create_tidy_data(years = 1975, reporters = "chl", partners = "arg", table = "yrp", max_attempts = 35)

produces the following error message:

Error in ots_read_from_api(year, reporter, partner, product_code_length, :

    It wasn't possible to obtain data.
    Provided this function tests your internet connection 
    you misspelled a reporter, partner or table, or there was 
    a server problem.
    Please check and try again.

(I tried with other tables, with the same result.) Any suggestion as to how solve this?

I just tried to play around with your package and there seems to be some issue in the API request:
> ots_create_tidy_data() Error in parse_con(txt, bigint_as_char) : lexical error: invalid char in json text. year 2002 2020 (right here) ------^

`sessionInfo()
R version 4.2.0 (2022-04-22 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22000)

Matrix products: default

locale:
[1] LC_COLLATE=German_Germany.utf8 LC_CTYPE=German_Germany.utf8
[3] LC_MONETARY=German_Germany.utf8 LC_NUMERIC=C
[5] LC_TIME=German_Germany.utf8

attached base packages:
[1] stats graphics grDevices utils datasets methods
[7] base

other attached packages:
[1] tradestatistics_3.0.2

loaded via a namespace (and not attached):
[1] Rcpp_1.0.8.3 lattice_0.20-45 here_1.0.1
[4] png_0.1-7 digest_0.6.29 rprojroot_2.0.3
[7] rappdirs_0.3.3 R6_2.5.1 grid_4.2.0
[10] crul_1.3 jsonlite_1.8.0 magrittr_2.0.3
[13] cachem_1.0.6 rlang_1.0.2 cli_3.2.0
[16] curl_4.3.2 data.table_1.14.2 Matrix_1.4-1
[19] reticulate_1.24 tools_4.2.0 purrr_0.3.4
[22] fastmap_1.1.0 compiler_4.2.0 memoise_2.0.1
[25] httpcode_0.3.0 `

Is there something wrong with the standard function call?

If I understand ots_tables correctly, one should be able to create a table covering "distances" with ots_create_tidy_data().

For instance, the following code should give me the distance between Japan and Albania, right?

library(tradestatistics)
dist <- ots_create_tidy_data(
  reporters = "jpn",
  partners = "Albania",
  table = "distances"
)

However, using that code, I receive the following messages in my console:

The requested table is included within the package.
Error in rbindlist(data, fill = TRUE) : 
  Item 1 of input is not a data.frame, data.table or list

Hello, I'm getting the below error in RStudio when trying to run a simple example:

ots_create_tidy_data(years = 2002, reporters = "chl", partners = "chn")

Error:

Error in parse_con(txt, bigint_as_char) : lexical error: invalid char in json text. year 2002 2020 (right here) ------^

Any idea how to resolve this?

I updated today (23 August) the package (from Github) and ran code that I originally wrote earlier in the year on I believe a February release of your package. I was surprised to see there were large discrepancies in the data obtained with the same function between the August and the February versions of the package. Here is an example for just one country pair but the issue is broader (I need data for EU countries):

I understand the differences can be partially explained by the new methodology for calculating the inflation adjustment. But that doesn't seem to explain everything - certainly not the missing data. Could you help me understand what's going on?

Here is the code that I ran to get the data:

ms_iso3 <- tradestatistics::ots_countries %>% 
  filter(eu28_member == TRUE) %>% 
  select(country_iso) %>%
  deframe()

trade_data <- tradestatistics::ots_create_tidy_data(reporters = ms_iso3,
                                                    partners = ms_iso3,
                                                    years = 1968:2020,
                                                    max_attempts = 10,
                                                    table = "yrp")

trade_adjusted <- trade_data %>%
  tradestatistics::ots_inflation_adjustment(reference_year = 2015) %>% 
  as_tibble() %>% 
  mutate(year = as.integer(year)) %>% 
  filter(reporter_iso!=partner_iso)

Hi,

Just to let you know that the server is (seemingly) down, and I'm getting an error message: "In file(file, "rt") :
cannot open URL 'https://api.tradestatistics.io/year_range': HTTP status was '502 Bad Gateway".

Cheers (and thanks)

Hello,
I tried the following code:

US.data.China <- ots_create_tidy_data(years = c(1980,1985,1990,1995,2000,2005,2010,2015,2018), reporters = 'usa', partners = 'chn')

US.data.Germany <- ots_create_tidy_data(years = c(1980,1985,1990,1995,2000,2005,2010,2015,2018), reporters = 'usa', partners = 'deu')

and the two tables are identical...am I doing something wrong here?

I made some changes in the API, which offer a 50% time reduction by reading parquet instead of JSON from the API

library(microbenchmark)

microbenchmark(
  ots_create_tidy_data(table = "yrpc-parquet", reporters = "can"),
  ots_create_tidy_data(table = "yrpc", reporters = "can"),
  times = 10L
)

Unit: seconds
                                                            expr      min       lq     mean   median       uq      max neval
 ots_create_tidy_data(table = "yrpc-parquet", reporters = "can") 1.390202 1.894805 2.006423 1.989092 2.309964 2.427866    10
         ots_create_tidy_data(table = "yrpc", reporters = "can") 3.548397 3.750299 3.891013 3.889045 3.950003 4.364702    10

I already sent the PR to allow Parquet serialization in the plumber package, api.tradestatistics.io is now using plumber development version

hi 👋

We want all rOpenSci pkgs to consistently keep track of changes. please do the below from now on:

• use a NEWS.md file, do not add this file to .Rbuildignore, as you want the file to be on the CRAN page for your pkg, a good example to follow: https://github.com/ropensci/drake/blob/master/NEWS.md
• git tag each CRAN version, to git tag commits for previous CRAN versions, you can find the commit, then do e.g., git tag -a v1.2 9fceb02 where 9fceb02 is the commit sha
• use the releases tab in this repo to include the associated NEWS items for each tag/version, e.g., see https://github.com/ropensci/taxize/releases/tag/v0.8.8