Giter VIP home page Giter VIP logo

tidyquant's Introduction

tidyquant

R-CMD-check Codecov CRAN_Status_Badge

Bringing financial and business analysis to the tidyverse

2-Minutes To tidyquant

Our short introduction to tidyquant on YouTube.

Anomalize

Features of tidyquant

tidyquant integrates the best resources for collecting and analyzing financial data, zoo, xts, quantmod, TTR, and PerformanceAnalytics, with the tidy data infrastructure of the tidyverse allowing for seamless interaction between each. You can now perform complete financial analyses in the tidyverse.

  • A few core functions with a lot of power
  • Integrates the quantitative analysis functionality of zoo, xts, quantmod, TTR, and now PerformanceAnalytics
  • Designed for modeling and scaling analyses using the tidyverse tools in R for Data Science
  • Implements ggplot2 functionality for beautiful and meaningful financial visualizations
  • User-friendly documentation to get you up to speed quickly!

New Excel Functionality in tidyquant

One-Stop Shop for Serious Financial Analysis

With tidyquant all the benefits add up to one thing: a one-stop shop for serious financial analysis!

Core Functions

  • Getting Financial Data from the web: tq_get(). This is a one-stop shop for getting web-based financial data in a “tidy” data frame format. Get data for daily stock prices (historical), key statistics (real-time), key ratios (historical), financial statements, dividends, splits, economic data from the FRED, FOREX rates from Oanda.

  • Manipulating Financial Data: tq_transmute() and tq_mutate(). Integration for many financial functions from xts, zoo, quantmod,TTR and PerformanceAnalytics packages. tq_mutate() is used to add a column to the data frame, and tq_transmute() is used to return a new data frame which is necessary for periodicity changes.

  • Performance Analysis and Portfolio Analysis: tq_performance() and tq_portfolio(). The newest additions to the tidyquant family integrate PerformanceAnalytics functions. tq_performance() converts investment returns into performance metrics. tq_portfolio() aggregates a group (or multiple groups) of asset returns into one or more portfolios.

Comparing Stock Prices

Visualizing the stock price volatility of four stocks side-by-side is quick and easy…

Evaluating Stock Performance

What about stock performance? Quickly visualize how a $10,000 investment in various stocks would perform.

Evaluating Portfolio Performance

Ok, stocks are too easy. What about portfolios? With the PerformanceAnalytics integration, visualizing blended portfolios are easy too!

  • Portfolio 1: 50% FB, 25% AMZN, 25% NFLX, 0% GOOG
  • Portfolio 2: 0% FB, 50% AMZN, 25% NFLX, 25% GOOG
  • Portfolio 3: 25% FB, 0% AMZN, 50% NFLX, 25% GOOG
  • Portfolio 4: 25% FB, 25% AMZN, 0% NFLX, 50% GOOG

This just scratches the surface of tidyquant. Here’s how to install to get started.

Installation

Development Version with Latest Features:

# install.packages("devtools")
devtools::install_github("business-science/tidyquant")

CRAN Approved Version:

install.packages("tidyquant")

Further Information

The tidyquant package includes several vignettes to help users get up to speed quickly:

Want to Learn tidyquant?

  • Learning Lab #9:

    • Performance Analysis & Portfolio Optimization with tidyquant - A 1-hour course on tidyquant in Learning Labs PRO
  • Learning Lab #10:

    • Building an API with plumber - Build a stock optimization API with plumber and tidyquant
  • Learning Lab #16:

    • Stock Portfolio Optimization and Nonlinear Programming - Use the ROI package with tidyquant to calculate optimal minimum variance portfolios and develop an efficient frontier.
  • Learning Lab #30:

    • Shiny Financial Analysis with Tidyquant API & Excel Pivot Tables - Learn how to use the new Excel Functionality to make Pivot Tables, VLOOKUPs, Sum-If’s, and more!

tidyquant's People

Contributors

andremueller avatar davisvaughan avatar evilgraham avatar hadley avatar jenswahl avatar mdancho84 avatar mgei avatar olivroy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tidyquant's Issues

design choice: why legacy wide format for some functions?

In your vignette, "Performance Analysis with tidyquant" you choose to make a table, "RaRb," to compare an asset to its benchmark. This is wide data. I would argue that the benchmark is an asset like the stock or portfolio and shoulc be part of the long format data set. This is how ggplot will take the data to compare the portfolio and it's benchmark. Tidyquant functions that reference Ra and Rb should, in effect, use filter() to get whatever named asset and benchmark that's relevant without the user having to go through step "3b" of the workflow described in the vignette. Be tidy all the way.

set.seed(12345)
RaRb_tibble<-tibble(date=as.Date(as.yearmon("2009-01-01")+seq(1/12,3,1/12),frac=1),
             Ra=rnorm(3*12,0.06/12,.01),
             Rb=rnorm(3*12,0.055/12,.01))


#current tidyquant design
RaRb_tibble %>%
  tq_performance(Ra = Ra, Rb = Rb, performance_fun = table.CAPM)

#It must be tidy to do common plots
RaRb_tidy<-RaRb_tibble %>% 
  gather(asset,return,-date) %>% 
  group_by(asset) %>% 
  mutate(wealth.index=cumprod(1+return))

ggplot(RaRb_tidy,aes(x=date,y=wealth.index,color=asset))+geom_line()

BTW, Tidyquant is awesome. I have been getting tidy-er lately and have been annoyed at switching back and forth with xts to do analysis. Thanks also for the detailed vignettes. The TLC and effort you put into them really shows!

Add Google Finance Data Source: diversify data sources (enhancement)

Library tidyquant is quite useful, but is hard-coded to Yahoo stock price data (see 'stockprice' in tq_get_util_1). Lately Yahoo stock price data has become somewhat unreliable.
A good enhancement would be to add Google as alternative data source, getSymbols.google is already available in quantmod, it would be just a matter of including it in code.

Date column coerced from "date" -> "dttm" when xts converts back to tibble.

When using a tq_mutate() and a function like periodReturn, the date column is coerced from a date to a dttm (POSIXct) class. Minor quirk, but hopefully is fixable.

From our email:
"The date class change is what happens during the conversion process from tibble to xts and then back to tibble. When xts goes back to tibble the date is POSIXct by default."

Reproducible example created using jennybc's reprex package:

library(tidyquant)
apple <- tq_get("AAPL")
apple %>% tq_transform(Ad, transform_fun = periodReturn, period = "monthly", col_rename = "monthly_ret")
#> # A tibble: 121 × 2
#>          date  monthly_ret
#>        <dttm>        <dbl>
#> 1  2007-01-31  0.023031022
#> 2  2007-02-28 -0.013064288
#> 3  2007-03-30  0.098097108
#> 4  2007-04-30  0.074157832
#> 5  2007-05-31  0.214328672
#> 6  2007-06-29  0.007013839
#> 7  2007-07-31  0.079645970
#> 8  2007-08-31  0.051001827
#> 9  2007-09-28  0.108246690
#> 10 2007-10-31  0.237701136
#> # ... with 111 more rows

feature:helper function to turn tidy data into RaRb format shown in vignette

You may think this is overly complicating matters. If so, please disregard. I do like to retrieve a full set of asset returns including mutual funds, their benchmarks and the risk free rate as a set when using the PerformanceAnalytics package. Long format makes it easy to compare arbitrary sets of series in ggplot. Yet many performance functions require wide data to compare the asset, its benchmark and the risk-free rate.

I took a stab at making a helper function to convert long tidy to wide RaRb format by matching a list of assets and their benchmarks. You can optionally specify the symbol for the time-varying risk free rate.

I am using the blp_bdh_tq data set I created in this issue note

# make RaRb helper function

#set what we want to pull from the long-style data frame
#match asset symbols to their benchmark symbols
RaRb_names<-tibble(Ra_name=c('AGTHX','FDEGX'),
                   Rb_name=c('SPX','SPX'))

#Rf could be a list matching number of rows in the RaRb_names table if 
# we are looking at assets in more than one currency,
# otherwise a single name value.
Rf_names<-'LD12TRUU' 

#names of the relevant columns in the data frame                   
name_col<-'symbol'
return_col<- 'return'

#the actual helper function
make_RaRbRf<-function(x,name_col,return_col,RaRb_names,Rf_names=NULL) {
  #function to take a list of portfolios/funds/stocks and 
  # merge with a list of benchmarks and, optionally, risk-free rate.
  # no error checking.
  #I am looping rather than applying. Shrug.
  #I really hate the squirrelly way we have to pass variable column names to filter and select 
  RaRbRf<-tibble()
  for (n in 1:nrow(RaRb_names)) {
    Ra<-x %>% 
      ungroup() %>% 
      filter_(paste(name_col," == '",RaRb_names$Ra_name[n],"'",sep='')) %>% 
      select(date,asset=get(name_col),Ra=get(return_col))
    Rb<-x %>% 
      ungroup() %>% 
      filter_(paste(name_col," == '",RaRb_names$Rb_name[n],"'",sep='')) %>% 
      select(date=date,benchmark=get(name_col),Rb=get(return_col))
    if (!(is.null(Rf_names))){
      if (length(Rf_names)<nrow(RaRb_names)){  #unmatched vectors. use only first Rf rate
        Rf<-x %>% 
          ungroup() %>% 
          filter_(paste(name_col," == '",Rf_names[1],"'",sep='')) %>% 
          select(date=date,Rf_rate=get(name_col),Rf=get(return_col))
      } else{ #multiple Rf rates
        Rf<-x %>% 
          ungroup() %>% 
          filter_(paste(name_col," == '",Rf_names[n],"'",sep='')) %>% 
          select(date=date,Rf_rate=get(name_col),Rf=get(return_col))
        
      }
      RaRbRf<-Ra %>% 
        left_join(Rb,by='date') %>% 
        left_join(Rf,by='date') %>% 
        bind_rows(RaRbRf)
    } else{
      RaRbRf<-Ra %>% 
        left_join(Rb,by='date') %>% 
        bind_rows(RaRbRf)
    }
  }
  return (group_by(RaRbRf,asset))
} #END FUNCTION

# test it out
make_RaRbRf(blp_bdh_tq,name_col,return_col,RaRb_names,Rf_names)
#Source: local data frame [252 x 7]
#Groups: asset [2]
#         date asset            Ra benchmark          Rb  Rf_rate          Rf
#       <date> <chr>         <dbl>     <chr>       <dbl>    <chr>       <dbl>
#1  2007-01-31 FDEGX  2.956467e-02       SPX  0.01614987 LD12TRUU 0.004266667
#2  2007-02-28 FDEGX -2.517091e-06       SPX -0.01956115 LD12TRUU 0.004300000
#3  2007-03-31 FDEGX  1.007866e-03       SPX  0.01118683 LD12TRUU 0.004200000
#4  2007-04-30 FDEGX  5.135944e-02       SPX  0.04429756 LD12TRUU 0.004091667
#5  2007-05-31 FDEGX  3.831427e-02       SPX  0.03489320 LD12TRUU 0.003941667
#6  2007-06-30 FDEGX  1.752771e-02       SPX -0.01661234 LD12TRUU 0.004016667
#7  2007-07-31 FDEGX  1.541267e-02       SPX -0.03100609 LD12TRUU 0.004133333
#8  2007-08-31 FDEGX -4.464409e-03       SPX  0.01498840 LD12TRUU 0.003341667
#9  2007-09-30 FDEGX  6.143493e-02       SPX  0.03740037 LD12TRUU 0.003183333
#10 2007-10-31 FDEGX  6.041405e-02       SPX  0.01590713 LD12TRUU 0.003283333
## ... with 242 more rows

#without Rf
make_RaRbRf(blp_bdh_tq,name_col,return_col,RaRb_names)

tidyquant with grouped data frame?

Love the idea of this package! I am not sure if the intention of tq_mutate was to closely mimic dplyr's mutate, but I am having the following issue with it and wanted to ask a quick question. Using the tidyquant example from github, I wanted to create scalable code where you use tidyquant to load in multiple stocks' stock.prices at once into a nested data frame, and then use tq_mutate() to run runSD() on each of them. I got that to work, but the code is quite long and kinda messy with the nested pipes.

I then unnest() the code, and use dplyr's group_by() function to create a large tidy grouped data frame. I can use mutate on the grouped data frame to add the running sd's to each, but tq_mutate fails. The mutate option is pretty clean looking, but I was wondering if you had wanted tq_mutate to work on something like this.

# Example from github, with unnest() removed to keep as list column
x <- tibble(symbol = c("FB", "AMZN", "NFLX", "GOOGL")) %>%
     mutate(stock.prices = map(.x = symbol,
                               .f = ~ tq_get(.x, get = "stock.prices")))

# For each tibble in the nested tibble, perform the function runSD on the Ad column
x %>% mutate(stock.prices = stock.prices %>% map(~tq_mutate(data = .x,
                                                           ohlc_fun = Ad, 
                                                           mutate_fun = runSD)))

# What about an easier way?

# In the unnested form
y <- unnest(x)

# First group by the symbols
y <- group_by(y, symbol)

# dplyr mutate seems to know what to do here
y %>% mutate(sd = runSD(adjusted))

# I thought tq_mutate should work, but it doesn't
y %>% tq_mutate(ohlc_fun = Ad, mutate_fun = runSD)

# Some debugging results
debugonce(as_xts_)
y %>% tq_mutate(ohlc_fun = Ad, mutate_fun = runSD)
#Stepping through, it seems to fail during the call to 'xts::xts[,-1]' because of the added groups
#select_() will add the grouping back to the single column of dates as the first column. Now the ordering is out of whack when converting to xts

"
Adding missing grouping variables: `symbol`
Error in sum(x[beg:(n + beg - 1)]) : 
  invalid type (character) of argument
"

tq_get(financials) not working with some stocks?

Hi, I have a list of 13 stocks, all listed in the US, and some but not all have issues getting financial statements (though none have issues getting stock prices)?

blah<-read.csv("tickers.csv",header=FALSE,colClasses=c("character"))

ffs <- tq_get(blah, get = "financials",complete_cases = TRUE)
Warning messages:
1: x = 'COP', get = 'financials': Error in download.file(paste(google.fin, Symbol, sep = ""), quiet = TRUE, : cannot download all files
Removing COP.
2: x = 'APC', get = 'financials': Error in download.file(paste(google.fin, Symbol, sep = ""), quiet = TRUE, : cannot download all files
Removing APC.
3: x = 'MRO', get = 'financials': Error in download.file(paste(google.fin, Symbol, sep = ""), quiet = TRUE, : cannot download all files
Removing MRO.
4: x = 'CRC', get = 'financials': Error in download.file(paste(google.fin, Symbol, sep = ""), quiet = TRUE, : cannot download all files
Removing CRC.
5: x = 'EOG', get = 'financials': Error in download.file(paste(google.fin, Symbol, sep = ""), quiet = TRUE, : cannot download all files
Removing EOG.

The full list of stocks is COP,APC,APA,HES,MRO,CLR,WLL,PXD,CRC,DVN,EOG,OAS,OXY,LPI

Thanks again

Quandl URL changed from *datatables* to *datasets*

When I execute this, I get the following error back.

tq_vix <- tidyquant::tq_get("CBOE/VIX", get  = "quandl.datatable")

Error

Warning message:
x = 'CBOE/VIX', get = 'quandl.datatable': Error: {"quandl_error":{"code":"QECx02","message":"The following datatable 'CBOE/VIX' does not exist."}}
 

Seen closer in this. Little data (only 97 bytes are returned).

debug: quandl.api.handl_errors(response)
Browse[3]> str(response)
List of 10
$ url        : chr "https://www.quandl.com/api/v3/datatables/CBOE/VIX?"
$ status_code: int 404
$ content    : raw [1:97] 7b 22 71 75 ...

When I do this

q_vix <- Quandl::Quandl("CBOE/VIX")

I get the following response back ( with a data.frame of data ( that is not shown here ) )

debug: quandl.api.handl_errors(response)
List of 10
 $ url        : chr "https://www.quandl.com/api/v3/datasets/CBOE/VIX?transform=&collapse=&order=desc"
 $ status_code: int 200

Then I change the tidyquant R code 'on the fly'

Browse[4]> debug(Quandl.datatable)

  path <- paste0("datatables/", code)
  TO
  path <- paste0("datatasets/", code)

And the response is good and the payload is now full.

Browse[4]> str(response)
List of 10
 $ url        : chr "https://www.quandl.com/api/v3/datasets/CBOE/VIX?"
 $ status_code: int 200
 $ content    : raw [1:131774] 7b 22 64 61 ...

--
Then I continue(c).
But then

> str(tq_vix, list.len = 999)
Classestbl_df’, ‘tbland 'data.frame':	0 obs. of  0 variables

So the output format may be different than before??

Incorrect candlestick charting with intraday data

Hi!
I've been trying to use tidyquant to chart intraday day and 've got problems with that.
For example, I have data in the following format:

x
A tibble: 465 × 6
date open high low close volume

1 2017-02-23 00:00:00 53.60 53.62 53.59 53.61 416
2 2017-02-23 00:05:00 53.60 53.61 53.58 53.58 206
3 2017-02-23 00:10:00 53.58 53.60 53.55 53.57 307
4 2017-02-23 00:15:00 53.57 53.58 53.55 53.57 221
5 2017-02-23 00:20:00 53.57 53.60 53.56 53.58 189
6 2017-02-23 00:25:00 53.58 53.58 53.52 53.53 420
7 2017-02-23 00:30:00 53.53 53.88 53.50 53.87 5636
8 2017-02-23 00:35:00 53.86 53.89 53.83 53.87 2205
9 2017-02-23 00:40:00 53.86 53.93 53.86 53.89 1774
10 2017-02-23 00:45:00 53.88 53.94 53.88 53.91 1468
... with 455 more rows

x[1:50,] %>% ggplot(aes(x = date, y = close)) + geom_candlestick(aes(open = open, high = high, low = low, close = close))

produces the following chart - bars do not have bodies just wigs.
For daily data it works just fine.
Do you have suggestions on it?

tq_performance cannot pass vector for Rf, PerformanceAnalytics does

In the performance analytics package, "Rf" is a variable that can be passed as a single value which tq_performance accepts, or a vector, which tq_performance does not.

set.seed(12345)
RaRf<-tibble(date=as.Date(as.yearmon("2009-01-01")+seq(1/12,3,1/12),frac=1),
       Ra=rnorm(3*12,.06/12,.01),
       Rf=rnorm(3*12,.02/12,.001))


RaRf_xts<-as_xts(RaRf,date_col = date)

#in tidyquant -------------------------
# vector for Rf NOT accepted
RaRf %>%
  tq_performance(Ra = Ra, Rf = Rf, performance_fun = SharpeRatio)

#Error in tq_performance_.tbl_df(data = data, Ra = lazyeval::expr_text(Ra),  : 
#object 'ret' not found
#In addition: Warning message:
#  In fun_performance(eval(parse(text = Ra)), ...) : object 'Rf' not found

#in PerformanceAnalytics -------------------------
# vector for Rf accepted
SharpeRatio(RaRf_xts$Ra,RaRF_xts$Rf)
                                       Ra
#StdDev Sharpe (Rf=0.2%, p=95%): 0.5956861
#VaR Sharpe (Rf=0.2%, p=95%):    0.6704678
#ES Sharpe (Rf=0.2%, p=95%):     0.4820222

tq_get(), get = "key.ratios": Returns NA with error message

This issue is related to issue #38 in which Yahoo! Finance changed URL. An example of the issue is shown below. We are working on the problem and intend to have a fix soon. We appreciate your patience while we work to resolve the issue.

> tq_get("AAPL", "key.ratios")
additional arguments ignored in warning()
[1] NA
Warning messages:
1: x = 'AAPL', get = 'stock.prices': Error in download.file(paste(yahoo.URL, "s=", Symbols.name, "&a=", from.m, : cannot open URL 'https://ichart.finance.yahoo.com/table.csv?s=AAPL&a=4&b=17&c=2005&d=4&e=17&f=2017&g=d&q=q&y=0&z=AAPL&x=.csv'
 
2: x = 'AAPL', get = 'key.ratios': Error in value[[3L]](cond): object 'key_ratios' not found

`tq_get()` Quandl data is returned newest to oldest

Most tq_get sources return data oldest to newest, and many functions rely on this format. Quandl data is not returned that way.

(Thanks to Rick for the example)

library(tidyquant)

CL_F1 = tq_get('CHRIS/CME_GC1', get = 'quandl')

ggplot(subset(CL_F1,date > '2016-01-01'),aes(x = date, y = settle)) +
  geom_line() +                  
  geom_ma(ma_fun = EMA, n = 126)

The EMA is thrown off and calculated backwards because of this. Arrange quandl data using something like:

CL_F1_multi = tq_get(x = c('CHRIS/CME_GC1', 'CHRIS/CME_C10'),
               get = 'quandl')
CL_F1_multi_sorted <- CL_F1_multi %>%
    mutate(symbol = forcats::as_factor(symbol)) %>%
    arrange(symbol, date) %>%
    mutate(symbol = as.character(symbol))

Converting to factor will take care of arrange attempting to order symbol alphabetically and ruining the user defined order.

EDIT: Going to overwrite the Quandl option order = "asc" instead.

install error on rstudio

Error when installing in RStudio, either through install.packages("tidyquant") or devtools::install_github("business-science/tidyquant").

Opened independent session of R and it installed on first try and then was available to load when I opened RStudio again.

tq_index(): Not Returning Symbols

tq_index() is returning an error when downloading one of the stock indexes.

> tq_index_options()
[1] "RUSSELL1000" "RUSSELL2000" "RUSSELL3000" "DOW"         "DOWGLOBAL"   "SP400"      
[7] "SP500"       "SP600"       "SP1000"     

> tq_index("DOW")
Getting holdings for DOW
# A tibble: 0 x 0
Warning message:
In tq_index("DOW") : Error at DOW during download. 
Error: NoSuchMethodError (Java): org.apache.poi.hssf.usermodel.HSSFWorkbook.getNumCellStyles()I

geom_candlestick OHLC same value

Hi,
I dont know if this is done intentionally but I have some data where open/high/low/close has the same value.

When I plot with geom_candlestick the plot renders empty.

Somebody told me the standard way to plot this is to have a horizontal dash/line (black)? Just a suggestion.

regards,
Lars Nygaard

marketvolume.com no longer working with tq_index()?

tq_index() doesn't seem to be able to retrieve market components from marketvolume.com any longer.

Visited the website (https://www.marketvolume.com/indexes_exchanges/sp500_components.asp?s=SPX&row=1) for the first time, and noticed there is a popup. Has this caused a problem with getting information from them?

I'll stick to using use_fallback = T for now.

example:

tq_index('SP500')
Getting data...

[1] NA
Warning message:
In value[[3L]](cond) :
  Could not access http://www.marketvolume.com/indexes_exchanges/sp500_components.asp?s=SPX&row=0. If problem persists, try setting `use_fallback = TRUE` to return the last dowloaded data set.

Thanks for a great package.

tq_get doesn't return multiple stocks on Linux Server

I'm running Ubuntu 16.04.2, and when I try to run:

tq_get(c("AAPL", "GOOGL")

I see the error:

# A tibble: 0 x 2
# ... with 2 variables: symbol <chr>, stock.prices <list>
Warning messages:
1: x = 'AAPL', get = 'stock.prices': Error in charToDate(x): character string is not in a standard unambiguous format
 Removing AAPL.
2: x = 'GOOGL', get = 'stock.prices': Error in charToDate(x): character string is not in a standard unambiguous format
 Removing GOOGL.
3: In value[[3L]](cond) : Returning as nested data frame.

As well, I've tried:

tq_get(tibble(symbol = c("AAPL", "GOOGL"))) 

to force the symbol column to always appear even if you're only checking a single ticker, and i get the same error.

Here's my sessionInfo too:

R version 3.4.1 (2017-06-30)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.2 LTS

Matrix products: default
BLAS: /usr/lib/libblas/libblas.so.3.6.0
LAPACK: /usr/lib/lapack/liblapack.so.3.6.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
 [1] bindrcpp_0.2                  tidyquant_0.5.2
 [3] dplyr_0.7.2                   purrr_0.2.3
 [5] readr_1.1.1                   tidyr_0.6.3
 [7] tibble_1.3.3                  ggplot2_2.2.1
 [9] tidyverse_1.1.1               quantmod_0.4-10
[11] TTR_0.23-2                    PerformanceAnalytics_1.4.3541
[13] xts_0.10-0                    zoo_1.8-0
[15] lubridate_1.6.0

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.12     cellranger_1.1.0 compiler_3.4.1   plyr_1.8.4
 [5] bindr_0.1        forcats_0.2.0    tools_3.4.1      jsonlite_1.5
 [9] nlme_3.1-131     gtable_0.2.0     lattice_0.20-35  pkgconfig_2.0.1
[13] rlang_0.1.1      psych_1.7.5      curl_2.8.1       parallel_3.4.1
[17] haven_1.1.0      xml2_1.1.1       httr_1.2.1       stringr_1.2.0
[21] hms_0.3          grid_3.4.1       glue_1.1.1       R6_2.2.2
[25] Quandl_2.8.0     readxl_1.0.0     foreign_0.8-69   modelr_0.1.0
[29] reshape2_1.4.2   magrittr_1.5     scales_0.4.1     rvest_0.3.2
[33] assertthat_0.2.0 mnormt_1.5-5     colorspace_1.3-2 stringi_1.1.5
[37] lazyeval_0.2.0   munsell_0.4.3    broom_0.4.2

error using tq_transmute function

Hi,

I have an issue using tq_transmute and xts functions.
I have followed the article post here : http://www.business-science.io/timeseries-analysis/2017/07/02/tidy-timeseries-analysis.html
but when I execute the following command :

mean_tidyverse_downloads_w <- tidyverse_downloads %>%
    tq_transmute(
        select     = count,
        mutate_fun = apply.weekly, 
        FUN        = mean,
        na.rm      = TRUE,
        col_rename = "mean_count"
    )

The result is the following error :
Error in mutate_impl(.data, dots) : Evaluation error: objet 'FUN' de mode 'function' introuvable.

I have reinstalled the tidyquant package and the other libraries but the issue remains.

Thanks for you help.

group_by(symbol) for tq_portfolio() to calc wealth index

I wanted to calculate the development of a 10000 Dollor for multiple markets to compare it finally with my portfolio. I am currently trying this:

#get sector prices
from <- today() - years(10)
sector_prices <- ticker %>% tq_get(get = "stock.prices", from = from, to = today())
sector_monthly_returns <- sector_prices %>% group_by(symbol) %>% tq_transmute(select=adjusted, periodReturn, period = "monthly", col_rename = "Rb") %>% ungroup()

sector_growth_monthly <-  sector_monthly_returns %>% group_by(symbol) %>%
    tq_portfolio(assets_col   = symbol, 
                 returns_col  = Rb, 
                 weights      = 1, 
                 col_rename   = "investment.growth",
                 wealth.index = TRUE) %>%
    mutate(investment.growth = investment.growth * 10000, symbol = rep(symbol, n()))

But that does not work. Do you know of the correct way to do that?

errors when coercing from tibble to xts

Hi,

following the vignette, I end up with the following error:
`

aapl_prices %>% as_xts()
Error in as_xts_(x, date_col, ...) :
date_col = NULL is invalid within names(x)
aapl_prices %>% select(-date) %>% as_xts(date_col = row.names)
Error in as_xts_(x, date_col, ...) :
date_col = row.names is invalid within names(x)
aapl_prices %>% select(-date) %>% as_xts(date_col = aapl_prices$date)
Error in as_xts_(x, date_col, ...) :
date_col = aapl_prices$date is invalid within names(x)
aapl_prices %>% select(open:close) %>% as_xts(date_col = aapl_prices$date)
Error in as_xts_(x, date_col, ...) :
date_col = aapl_prices$date is invalid within names(x)
`

So e.g. to display a candlestick chart in dygraph I need to do the rather unelegant:
dygraph(as.xts(aapl_prices[,c(2:5)], order.by = as.Date(sapply(aapl_prices[,1], function(x) {as.numeric(x)}), origin="1970-01-01"))) %>% dyCandlestick()

This is even true for just one line using the adjusted prices:
dygraph(as.xts(aapl_prices$adjusted, order.by = as.Date(sapply(aapl_prices$date, function(x) {as.numeric(x)}), origin="1970-01-01")))

You see that I am not yet fully immersed in the tidyverse so when things don't work I help myself with standard R. But I am sure this should be possible to do much more simple and elegant.

Errors generated inside functions wrapped by `tq_get` remain hidden

Right now, if you pass a valid get arg but otherwise-invalid args to tq_get, you are given very unhelpful errors, for example

> tq_get("WIKI/FOOBAR", get = "stock.prices")
[1] NA
Warning message:
In value[[3L]](cond) :
  Error at WIKI/FOOBAR during call to get = 'stock.prices'.
> tq_get("FOO/BAR", get = "quandl.data.table")
[1] NA
Warning message:
In value[[3L]](cond) :
  Error at FOO/BAR during call to get = 'quandl.datatable'.

I think the behavior of de-escalating errors to warnings might be the right move, and I see you catching the errors at https://github.com/business-science/tidyquant/blob/master/R/tq_get.R#L369, but I think ignoring e completely is leaving valuable information on the floor. At the very least, that error message should be output to the user.

Ideally, you'd find a way to get rid of In value[[3L]](cond) :, but that's less important and might require more careful refactoring of tq_get.

This might not be a problem for someone well-versed in the underlying packages, but to a newcomer this inability to debug will be greatly discouraging.

I'm on 0.4.9020.

Enable tq_get to accept multiple inputs

Feature Request:

Kan would like tq_get functionality expanded to accept a list of stocks as an input. Example:

tech_stock  <- tq_get(c(“AAPL”, “GOOG”, “TWTR"), get = "stock.prices", from = "2006-01-01") 

feature: convenient way to change all prices to a defined basis currency

Hi mdancho84,

I just started working with tidyquant mainly to analyze my own portfolio of ETFs. I wondered whether there is a simple method to convert all prices to a defined currency? Would be probably a nice feature for tidyquant, if there is not an easy way to do that.

Cheers,
Robby.

Conflicts with tidy packages

I am getting these strange messages.

Conflicts with tidy packages

Can those messages be ignored or SHOULD I do something?

W:\R-3.3._>W:\R-3.3._\App\R-Portable\bin\x64\R.exe

R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 14393)

locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base
> search()
[1] ".GlobalEnv"        "package:stats"     "package:graphics"
[4] "package:grDevices" "package:utils"     "package:datasets"
[7] "package:methods"   "Autoloads"         "package:base"

> update.packages()

Warning: package 'quantmod' in library 'W:/R-3.3._/App/R-Portable/library' will not be updated
Warning: package 'survival' in library 'W:/R-3.3._/App/R-Portable/library' will not be updated

> library(tidyquant)

Loading required package: lubridate

Attaching package: 'lubridate'

The following object is masked from 'package:base':

    date

Loading required package: quantmod
Loading required package: xts
Loading required package: zoo

Attaching package: 'zoo'

The following objects are masked from 'package:base':

    as.Date, as.Date.numeric

Loading required package: TTR
Version 0.4-0 included new data defaults. See ?getSymbols.
Loading required package: tidyverse
Loading tidyverse: ggplot2
Loading tidyverse: tibble
Loading tidyverse: tidyr
Loading tidyverse: readr
Loading tidyverse: purrr
Loading tidyverse: dplyr
Conflicts with tidy packages ---------------------------------------------------
as.difftime(): lubridate, base
date():        lubridate, base
filter():      dplyr, stats
first():       dplyr, xts
intersect():   lubridate, base
lag():         dplyr, stats
last():        dplyr, xts
setdiff():     lubridate, base
union():       lubridate, base

Attaching package: 'tidyquant'

The following object is masked from 'package:tibble':

    as_tibble

> sessionInfo()

R version 3.3.2 (2016-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 14393)

locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
 [1] tidyquant_0.2.0.9000 dplyr_0.5.0          purrr_0.2.2
 [4] readr_1.0.0          tidyr_0.6.0          tibble_1.2
 [7] ggplot2_2.2.1        tidyverse_1.0.0      quantmod_0.4-7
[10] TTR_0.23-1           xts_0.9-7            zoo_1.7-14
[13] lubridate_1.6.0

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.8      magrittr_1.5     munsell_0.4.3    colorspace_1.3-2
 [5] lattice_0.20-34  R6_2.2.0         stringr_1.1.0    plyr_1.8.4
 [9] tools_3.3.2      grid_3.3.2       gtable_0.2.0     DBI_0.5-1
[13] lazyeval_0.2.0   assertthat_0.1   stringi_1.1.2    scales_0.4.1

> search()
 [1] ".GlobalEnv"        "package:tidyquant" "package:dplyr"
 [4] "package:purrr"     "package:readr"     "package:tidyr"
 [7] "package:tibble"    "package:ggplot2"   "package:tidyverse"
[10] "package:quantmod"  "package:TTR"       "package:xts"
[13] "package:zoo"       "package:lubridate" "package:stats"
[16] "package:graphics"  "package:grDevices" "package:utils"
[19] "package:datasets"  "package:methods"   "Autoloads"
[22] "package:base"
>

Thanks.
Andre Mikulec

multiple portfolio doesn't seem to work

I get this error when I try to create a portfolio w/ different weights --

Warning messages:
1: In check_weights(weights, assets_col, map, x) :
The assets in weights does not match the assets in data.
2: Can't join on 'portfolio' x 'stock' because of incompatible types (integer / character)

Here are my weights :

head(w2)
Source: local data frame [6 x 3]
Groups: portfolio [1]

portfolio stocks weights

1 1 EMHY 0.27727790
2 1 DIVY 0.20706010
3 1 FXY 0.24806711
4 1 PSI 0.02861348
5 1 RSXJ 0.00000000
6 1 EUO 0.04888890

Here is my data :

head(d2)

A tibble: 6 × 3

    date stock         rtrn
  <date> <chr>        <dbl>

1 2014-12-22 BIZD 0.001358876
2 2014-12-31 BIZD -0.001092245
3 2015-01-02 BIZD 0.010304838
4 2015-01-09 BIZD -0.001115083
5 2015-01-21 BIZD -0.028958860
6 2015-01-23 BIZD 0.009974641

here is the command to join them :

port <- d2 %>% tq_portfolio(

  • assets_col = stock,
  • returns_col = rtrn,
  • weights = w2,
  • col_rename = FALSE,
  • wealth.index = TRUE
  • )
    Warning messages:
    1: In check_weights(weights, assets_col, map, x) :
    The assets in weights does not match the assets in data.
    2: Can't join on 'portfolio' x 'stock' because of incompatible types (integer / character)

It all looks correct to me.... not sure what I did wrong...

All the stock names in the weight exist in the data.

packageVersion("tidyquant")
[1] ‘0.4.0’

Depends vs. Imports vs. Suggests

What's the motivation for having dependencies in Depends? The modern convention is to have as much as possible in Imports instead, though there are exceptions, I'm curious why packages like quantmod have to be in Depends, while Quandl works fine in Imports.

Further, could some things be moved into Suggests, like Quandl and PerformanceAnalytics? I think there's plenty of use cases where one wouldn't need to touch functions that relate to those packages, so it would be nice to not force such users to install them in order to use tidyquant.

tq_get(get = "key.stats") fails with multiple x values

Problem:
tq_get(get = "key.stats") fails with multiple x during coercion if first value in a column is NA.

Example:

c("AAPL", "GOOG") %>%
    tq_get(get = "key.stats")
>>> Error in eval(substitute(expr), envir, enclos) : 
>>>   Can not automatically convert from character to integer in column "Last.Trade.Size".

Yahoo data source for dividends sometimes faulty

I have two ETFs in my portfolio for which yahoo has no dividend information and thus the adjusted price column is incorrect. If you do not figure out that and you work with the adjusted prices your analysis is not correct.
Maybe there is a possibility to check for it and give a warning?

feature: add Rblpapi package as a data source

Rblpapi, written by Dirk Eddelbuettel, et.al., interfaces to the Bloomberg terminal. This is, of course, a not-free data source and so is not going to be used by as many people as Yahoo, Quandl, et.al. Still, Rblpapi gets about 1500 downloads a month. It might be worth wrapping it. It's a small package with only a few functions. To whet your appetite I prototyped wrapping one function, Rblpapi::bdh() which would be wrapped by tq_get(). Easy peasy!

tq_get_blpapi<-function(secs,get="bdh",...){
  if (!("bdh" %in% get)) {
    stop("only bdh is implemented")
  }
  Rblpapi::blpConnect() #must have a valid blp session running
  blp_bdh  <-Rblpapi::bdh(secs,...=...)
 #tidy what Bloomberg gives us.
  blp_bdh_tq<-bind_rows(blp_bdh,.id='symbol')%>%
    separate(symbol,c('symbol','sector')) %>%
    select(date,symbol,everything())%>%
    group_by(sector,symbol)
  return(blp_bdh_tq)
}

The bit above is the TL;DR part. Here is the vignette with a dataset conformed to look like what the native Rblpapi::bdh() function returns.

# ----------------------------------------------------------
# What a Rblpapi::bdh function might look like in tidyquant

#form of the bdh function
START_DATE=as.Date("2007-01-01")
secs = c('SPX Index','AGTHX Equity','FDEGX Equity','LD12TRUU Index')
#BDH_OPTIONS = c("periodicitySelection"="MONTHLY")
#BBG_bdh  <-Rblpapi::bdh(secs,
#                  fields=c('RETURN','PRICE'), # not valid fields 
#                  start.date = START_DATE,
#                  end.date=Sys.Date(),
#                  options=BDH_OPTIONS)


#create an object that looks like what bdh will return using only public sources
#get from FRED
tbills_3mo <- tq_get("DGS3MO", get = "economic.data")
#convert to monthly rate
risk_free_rate <- tbills_3mo %>% transmute(date,Rf=price/100)
risk_free_rate_monthly<-risk_free_rate %>% 
  tq_transmute(select = Rf, 
               mutate_fun = to.monthly,
               indexAt='lastof',
               col_rename="return") %>% 
  mutate(return=return/12)

funds  <- tq_get(c("^SP500TR",'AGTHX','FDEGX'), 
                 get = "stock.prices", 
                 from = "2007-01-01") %>% 
  group_by(symbol)
#convert to monthly returns
funds_monthly<-funds %>% 
  tq_transmute(select     = adjusted, 
               mutate_fun = periodReturn,
               indexAt = 'lastof',
               period     = "monthly", 
               col_rename = "return") #%>% 

funds_monthly


#simulate what Rblpapi::bdh() will return
blp_bdh<-list(
  
  `LD12TRUU Index`=risk_free_rate_monthly %>% 
    select(date,return) %>% 
    mutate(wealth=cumprod(1+return)) %>% 
    as.data.frame() ,

`SPX Index`=funds_monthly %>% 
  filter(symbol=="^SP500TR") %>% 
  ungroup() %>% 
  select(date,return) %>% 
  mutate(wealth=cumprod(1+return)) %>% 
  as.data.frame(),

`AGTHX Equity`=funds_monthly %>% 
  filter(symbol=='AGTHX') %>% 
  ungroup() %>% 
  select(date,return) %>% 
  mutate(wealth=cumprod(1+return)) %>% 
  as.data.frame(),

`FDEGX Equity`=funds_monthly %>% 
  filter(symbol=='FDEGX') %>% 
  ungroup() %>% 
  select(date,return) %>% 
  mutate(wealth=cumprod(1+return)) %>% 
  as.data.frame()
)

blp_bdh

#Covert what Rblpapi returns to something like what tq_get returns
blp_bdh_tq<-bind_rows(blp_bdh,.id='symbol')%>%
  separate(symbol,c('symbol','sector')) %>%
  select(date,symbol,everything())%>%
  group_by(sector,symbol)
blp_bdh_tq

#a prototype tq_get for Rblpapi. Embed tidy coversion routine from above
tq_get_blpapi<-function(secs,get="bdh",...){
  if (!("bdh" %in% get)) {
    stop("only bdh is implemented")
  }
  Rblpapi::blpConnect() #must have a valid blp session running
  blp_bdh  <-Rblpapi::bdh(secs,...=...)
  blp_bdh_tq<-bind_rows(blp_bdh,.id='symbol')%>%
    separate(symbol,c('symbol','sector')) %>%
    select(date,symbol,everything())%>%
    group_by(sector,symbol)
  return(blp_bdh_tq)
}

# Working BBG Data get. Won't work if no blp session is running. Skip to 'fake it' below.
# CAUTION: The order of the returned series may not be the same as order of the inputs
# CAUTION: There is no guarantee that the all securities will have values for the same dates.
opts = c("periodicitySelection"="MONTHLY")
fields=c('DAY_TO_DAY_TOT_RETURN_GROSS_DVDS',"LAST_PRICE")

funds_and_bm<-tq_get_blpapi(secs,
              fields=fields,
              options=opts,
              start.date=as.Date("2015-01-01"),
              end.date=Sys.Date())

#fake it if no blpapi
#not using actual BBG field names which I usually rename anyway
funds_and_bm<-blp_bdh_tq

#looks okay?
funds_and_bm %>% 
  summarize(Avg_Return=mean(return),n=n())
## A tibble: 4 x 4
## Groups:   sector [?]
##  sector   symbol    Avg_Return     n
#  <chr>    <chr>        <dbl> <int>
#1 Equity    AGTHX 7.340296e-03   126
#2 Equity    FDEGX 6.885127e-03   126
#3  Index LD12TRUU 1.058918   126
#4  Index      SPX 7.043102e-03   126

ggplot(funds_and_bm,aes(x=date,y=wealth,color=symbol))+geom_line()

Thanks for considering this.
I am having a blast playing around with tidyquant and think it will help push the notion of the tidyverse as a meta language on top of R.

bind_rows: does not preserve zoo yearmon and yearqtr class

This issue is documented in the dplry github issue tidyverse/dplyr#2921

When doing grouped or binding operations in dplyr, the zoo::yearmon and yearqtr classes are not preserved. The error that occurs specifies that:

"Warning in bind_rows_(x, .id): Vectorizing 'yearmon' elements may not preserve their attributes"

library(dplyr)
library(zoo)

# Setup 
dates <- seq.Date(from = as.Date("2017-01-01"), to = as.Date("2017-06-01"), by = "month") %>%
    as.yearmon()

tib_1 <- tibble(dates, symbol = rep("ABC", 6), value = 1:6)
tib_1
#> # A tibble: 6 x 3
#>           dates symbol value
#>   <S3: yearmon>  <chr> <int>
#> 1      Jan 2017    ABC     1
#> 2      Feb 2017    ABC     2
#> 3      Mar 2017    ABC     3
#> 4      Apr 2017    ABC     4
#> 5      May 2017    ABC     5
#> 6      Jun 2017    ABC     6

tib_2 <- tibble(dates, symbol = rep("XYZ", 6), value = 6:1)
tib_2
#> # A tibble: 6 x 3
#>           dates symbol value
#>   <S3: yearmon>  <chr> <int>
#> 1      Jan 2017    XYZ     6
#> 2      Feb 2017    XYZ     5
#> 3      Mar 2017    XYZ     4
#> 4      Apr 2017    XYZ     3
#> 5      May 2017    XYZ     2
#> 6      Jun 2017    XYZ     1

# bind_rows() converts the yearmon class to numeric
bind_rows(tib_1, tib_2)
#> Warning in bind_rows_(x, .id): Vectorizing 'yearmon' elements may not
#> preserve their attributes

#> Warning in bind_rows_(x, .id): Vectorizing 'yearmon' elements may not
#> preserve their attributes
#> # A tibble: 12 x 3
#>       dates symbol value
#>       <dbl>  <chr> <int>
#>  1 2017.000    ABC     1
#>  2 2017.083    ABC     2
#>  3 2017.167    ABC     3
#>  4 2017.250    ABC     4
#>  5 2017.333    ABC     5
#>  6 2017.417    ABC     6
#>  7 2017.000    XYZ     6
#>  8 2017.083    XYZ     5
#>  9 2017.167    XYZ     4
#> 10 2017.250    XYZ     3
#> 11 2017.333    XYZ     2
#> 12 2017.417    XYZ     1

# what should happen
rbind(tib_1, tib_2)
#> # A tibble: 12 x 3
#>            dates symbol value
#>    <S3: yearmon>  <chr> <int>
#>  1      Jan 2017    ABC     1
#>  2      Feb 2017    ABC     2
#>  3      Mar 2017    ABC     3
#>  4      Apr 2017    ABC     4
#>  5      May 2017    ABC     5
#>  6      Jun 2017    ABC     6
#>  7      Jan 2017    XYZ     6
#>  8      Feb 2017    XYZ     5
#>  9      Mar 2017    XYZ     4
#> 10      Apr 2017    XYZ     3
#> 11      May 2017    XYZ     2
#> 12      Jun 2017    XYZ     1

Why can't you finance guys also think of others?

I read about your package on R-bloggers, and liked the idea of harmonizing xts with the tidyverse. But I hate that you always think of time series values to come in OHLC format.

There are millions of other data in univariate or multivariate time series format. Why do you require this damn OHLC format!?

Conflict with last function in xts package

When the tidyquant package is loaded there is a conflict with "last" function in xts.

When I do last(prices, 1) to get last row of xts object, I get the following error:

Error in x[[order(order_by)[n]]] : subscript out of bounds

Easy Column Renaming for tq_mutate and tq_transform

Feature Request:

@DavisVaughn would like an argument such as "col_rename" added to tq_mutate and tq_transform that allows for faster renaming of columns generated by the mutation/transformation. Plan to add in 0.3.0 release.

New function: tq_index(); potentially change "get" arg to "type"; allow the "get" arg to take a vector

Abstractly, the types of data that tq_get() retrieves are all about a specific asset (the price of the asset, the financials of the asset, the dividends, etc) EXCEPT for "stock.index". This is not information about a specific asset, but a list of the assets themselves.

With that in mind, maybe it would make sense to separate tq_index() as it's own function, taking exchange = as an argument or something similar.

Then, it would make sense to allow tq_get() to take in a vector in the get/type argument.

If, for example, you did the following:

tq_index("SP500") %>% tq_get(type = c("stock.prices", "financials"))

wouldn't it be cool if that returned a nested tibble of 500 rows, with columns of symbol, stock_price, and financials, where stock_price and financials are list-columns for that row?

#symbol  stock_price   financials
#AAPL    [tibble RxC]  [tibble RxC]

Foreign stocks key.stats not working although stock.prices are?

Hi, thanks for the great library. I'm doing some work with foreign stocks and though I've seen that people have had issues with key.ratios vs key.stats, etc (due to different ticker conventions between data sources Yahoo vs Morningstar), I seem to be having issues retrieving stock data even with the same source (Yahoo Finance).

Take FMG (Australian exchange) for instance, stock prices seem OK:

tq_get("FMG.AX", get = "stock.prices", from = "2017-05-01")
A tibble: 23 × 7
date open high low close volume adjusted

1 2017-05-01 5.26 5.330 5.160 5.25 13088483 5.25
2 2017-05-02 5.25 5.350 5.230 5.27 18140460 5.27

But I can't get most key.stats like shares outstanding?

tq_get("FMG.AX", get="key.stats",complete_cases=FALSE) %>% select(Shares.Outstanding)
A tibble: 1 × 1
Shares.Outstanding

1 NA

I understand both of these should be coming from Yahoo Finance.

Thanks much!

key.ratios: Issues with dowloading certain stocks

Issue with downloading certain stocks. Use the excel file, which contains the stock list that created the errors.

foobar.xlsx

library(tidyquant)
library(xlsx)
stock_list <- read.xlsx(file = "foobar.xlsx", sheetIndex = 1, ) %>%
    as_tibble() %>%
    mutate(value = as.character(value))
stock_list

stock_list %>%
    tq_get(get = "key.ratios")

# A tibble: 763 × 3
# value           section               data
# <chr>             <chr>             <list>
#     1    EFX        Financials <tibble [150 × 5]>
#     2    EFX     Profitability <tibble [170 × 5]>
#     3    EFX            Growth <tibble [160 × 5]>
#     4    EFX         Cash Flow  <tibble [50 × 5]>
#     5    EFX  Financial Health <tibble [240 × 5]>
#     6    EFX Efficiency Ratios  <tibble [80 × 5]>
#     7    EFX  Valuation Ratios  <tibble [40 × 5]>
#     8    TRU        Financials <tibble [150 × 5]>
#     9    TRU     Profitability <tibble [170 × 5]>
#     10   TRU            Growth <tibble [160 × 5]>
#     # ... with 753 more rows
#     There were 15 warnings (use warnings() to see them)

> warnings()
# Warning messages:
#     1: In download.file(url[1], destfile = tmp, quiet = TRUE) :
#     cannot open URL 'http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XNAS:WAL&region=usa&culture=en-US&cur=&order=asc': HTTP status was '500 Internal Server Error'
# 2: In value[[3L]](cond) :
#     Error at WAL during call to get = 'key.ratios'. Removing WAL.
# 3: In download.file(url[1], destfile = tmp, quiet = TRUE) :
#     cannot open URL 'http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XNAS:MS&region=usa&culture=en-US&cur=&order=asc': HTTP status was '500 Internal Server Error'
# 4: In value[[3L]](cond) :
#     Error at MS during call to get = 'key.ratios'. Removing MS.
# 5: In value[[3L]](cond) :
#     Error at CG during call to get = 'key.ratios'. Removing CG.
# 6: In readLines(tmp) :
#     incomplete final line found on 'C:\Users\mdanc\AppData\Local\Temp\RtmpotdUjr\file2d746b9320c3'
# 7: In value[[3L]](cond) :
#     Error at VNTV during call to get = 'key.ratios'. Removing VNTV.
# 8: In value[[3L]](cond) :
#     Error at BKFS during call to get = 'key.ratios'. Removing BKFS.
# 9: In value[[3L]](cond) :
#     Error at RMAX during call to get = 'key.ratios'. Removing RMAX.
# 10: In value[[3L]](cond) :
#     Error at APAM during call to get = 'key.ratios'. Removing APAM.
# 11: All formats failed to parse. No formats found.
# 12: In download.file(url[1], destfile = tmp, quiet = TRUE) :
#     cannot open URL 'http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XNAS:FLT&region=usa&culture=en-US&cur=&order=asc': HTTP status was '500 Internal Server Error'
# 13: In value[[3L]](cond) :
#     Error at FLT during call to get = 'key.ratios'. Removing FLT.
# 14: In download.file(url[1], destfile = tmp, quiet = TRUE) :
#     cannot open URL 'http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XNAS:AL&region=usa&culture=en-US&cur=&order=asc': HTTP status was '500 Internal Server Error'
# 15: In value[[3L]](cond) :
#     Error at AL during call to get = 'key.ratios'. Removing AL.

tq_index("SP500"): Returns NA with error message

Market Volume has changed the website to no longer offer the stock components for various stock indexes without user login. The error is as follows:

> tq_index("SP500")
Getting data...

[1] NA
Warning message:
In value[[3L]](cond) :
  Could not access http://www.marketvolume.com/indexes_exchanges/sp500_components.asp?s=SPX&row=0. If problem persists, try setting `use_fallback = TRUE` to return the last dowloaded data set.

A suitable alternative is to use the "fallback" dataset, which can be retrieved as follows:

> tq_index("SP500", use_fallback = TRUE)
Using fallback dataset last downloaded 2017-01-19.
# A tibble: 501 × 2
   symbol                   company
    <chr>                     <chr>
1     MMM                        3M
2     ABT       ABBOTT LABORATORIES
3    ABBV                ABBVIE INC
4     ACN                 ACCENTURE
5    ATVI       ACTIVISION BLIZZARD
6     AYI             ACUITY BRANDS
7    ADBE             ADOBE SYSTEMS
8     AAP        ADVANCE AUTO PARTS
9     AET                     AETNA
10    AMG AFFILIATED MANAGERS GROUP
# ... with 491 more rows

We apologize for any inconvenience, and we appreciate your patience while we fix this problem.

tq_get, get = "stock.prices": Yahoo! Finance returns NA with error message

Yahoo has changed the URL to the financial data download as discussed in Quantmod Issue 157. The error shows up as follow:

> tq_get("AAPL")
[1] NA
Warning message:
x = 'AAPL', get = 'stock.prices': Error in download.file(paste(yahoo.URL, "s=", Symbols.name, "&a=", from.m, : cannot open URL 'https://ichart.finance.yahoo.com/table.csv?s=AAPL&a=0&b=01&c=2007&d=4&e=17&f=2017&g=d&q=q&y=0&z=AAPL&x=.csv'

An alternative is to use Quandl's WIKI End Of Day database which has many of the same data available. Note that you will need an API key if you plan to make more than 50 API calls per day.

> tq_get("WIKI/AAPL", get = "quandl", from = "2007-01-01", to = "2017-01-01")
# A tibble: 2,518 × 13
         date  open  high   low close    volume ex.dividend split.ratio adj.open adj.high  adj.low adj.close
       <date> <dbl> <dbl> <dbl> <dbl>     <dbl>       <dbl>       <dbl>    <dbl>    <dbl>    <dbl>     <dbl>
1  2007-01-03 86.29 86.58 81.90 83.80  44225700           0           1 11.13446 11.17188 10.56800  10.81316
2  2007-01-04 84.05 85.95 83.82 85.66  30259300           0           1 10.84542 11.09059 10.81575  11.05317
3  2007-01-05 85.77 86.20 84.40 85.05  29812200           0           1 11.06736 11.12285 10.89059  10.97446
4  2007-01-08 85.96 86.53 85.28 85.47  28468100           0           1 11.09188 11.16543 11.00414  11.02865
5  2007-01-09 86.45 92.98 85.15 92.57 119617800           0           1 11.15511 11.99771 10.98736  11.94480
6  2007-01-10 94.75 97.80 93.45 97.00 105460000           0           1 12.22610 12.61966 12.05836  12.51643
7  2007-01-11 95.94 96.78 95.10 95.80  51437600           0           1 12.37965 12.48804 12.27126  12.36159
8  2007-01-12 94.59 95.06 93.23 94.62  46881800           0           1 12.20546 12.26610 12.02997  12.20933
9  2007-01-16 95.68 97.25 95.45 97.10  44431300           0           1 12.34610 12.54869 12.31643  12.52934
10 2007-01-17 97.56 97.60 94.82 94.95  58795000           0           1 12.58869 12.59385 12.23513  12.25191
# ... with 2,508 more rows, and 1 more variables: adj.volume <dbl>
Warning message:
No Quandl API key detected. Limited to 50 anonymous calls per day. Set key with 'quandl_api_key()'. 

We apologize for any inconvenience.

Ticker format for Non-US exchanges

Hi Matt,

I have a ticker formatting issue for non-us listed companies.

For China-A share, yahoo.finance/google.finance requires the "Ticker + Exchange" info in order to get the price data. For US listed companies, there is no such requirement. Moreover, for "key.ratios", Morningstar requires the original "Ticker" to retrieve the data.
So we may need a ticker formatting method to adjust for different exchange, different data request.

Here is an example: ticker "200725" in Shenzhen exchange

what works:
prices <- tidyquant::tq_get(x = "200725.SZ", get = "stock.prices", from = " 1990-01-01")
what doesn't work:
prices <- tidyquant::tq_get(x = "200725", get = "stock.prices", from = " 1990-01-01")

what works:
key-ratio url link:
http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XSHE:200725&region=chn&culture=en-US&cur=&order=asc

please let me know your thoughts.

thanks
andy

tq_performance does not accept yearmon for date values, PerformanceAnalytics does

When converting time series to monthly it makes sense to use the yearmon class. If you are comparing two assets from different data sources, the dates may not line up precisely. I found that happening using FRED with Yahoo. Similarly, trading days may not be equivalent across assets in different marketplaces, bonds and stocks, for example. Converting everything to yearmon when looking at monthly data forces alignment. In some cases it may introduce imprecision but for most uses it is fine. PerformanceAnalytics is fine with yearmon. tq_performance does not accept dates in yearmon form, which forces the user to convert to yearmon to ensure alignment, then convert back to date. It's not to tough to do, but it is an inconsistency.

set.seed(12345)
Ra_tibble<-tibble(date=as.yearmon("2009-01-01")+seq(1/12,3,1/12),
             Ra=rnorm(3*12,.06/12,.01))

Ra_xts<-as_xts(Ra_tibble,date_col = date)

#in tidyquant -------------------------
# yearmon for date NOT accepted
Ra_tibble %>%
  tq_performance(Ra = Ra, Rf = 0.02/12, performance_fun = SharpeRatio)

#Error in get_col_name_date_or_date_time(data) : 
#No date or POSIXct column found in `data`.

#fix it - easy enough!
Ra_tibble %>% 
  mutate(date=as.Date(date,frac=1)) %>%
  tq_performance(Ra = Ra, Rf = 0.02/12, performance_fun = SharpeRatio)
# A tibble: 1 × 3
#`ESSharpe(Rf=0.2%,p=95%)` `StdDevSharpe(Rf=0.2%,p=95%)` `VaRSharpe(Rf=0.2%,p=95%)`
#*                     <dbl>                         <dbl>                      <dbl>
#  1                 0.6415755                     0.6305925                  0.8175275

#in PerformanceAnalytics -------------------------
# yearmon for date accepted
SharpeRatio(Ra_xts$Ra,Rf=0.02/12)
#                                    Ra
#StdDev Sharpe (Rf=0.2%, p=95%): 0.6305925
#VaR Sharpe (Rf=0.2%, p=95%):    0.8175275
#ES Sharpe (Rf=0.2%, p=95%):     0.6415755

feature: get holdings of ETFs

Currently I am trying to figure out, how to automatically get the holdings of all all ETFs in my portfolio to analyze which stocks, countries and sectors proportions my hole portfolio actually has.
Do you know of an way to do that? From e.g. ishares I tried to get a csv, but it is not that easy as I thought.
Would also be a nice feature for tidyquant..

Cheers,
Robby.

Error in as.POSIXlt(x, tz = tz(x)) : error during cleanup

Got a very simple Rmd notebook (the demo offered when clicking on menu File > New File > Rnotebook. Just added code for loading tidyquant package.
When knitting to HTML Rstudio markdown pane gives me:

Output created: report.html
Error in as.POSIXlt(x, tz = tz(x)) : 
  argument "x" absent, with no default
Calls: .Last -> message -> date -> date.default -> as.POSIXlt
Execution interrupted

Tried to render in putty terminal accessing to server:

r -e "rmarkdown::render('/home/user/R/reporting/report.Rmd')"

Terminal gets cluttered with error and must kill process:


    Error in as.POSIXlt(x, tz = tz(x)) :
      el argumento "x" está ausente, sin valor por omisión
    Fatal error: error during cleanup
    
    Error in as.POSIXlt(x, tz = tz(x)) :
      el argumento "x" está ausente, sin valor por omisión
    Fatal error: error during cleanup
    
    Error in as.POSIXlt(x, tz = tz(x)) :
      el argumento "x" está ausente, sin valor por omisión
    Fatal error: error during cleanup
......
....
...

Had to comment library(tidyquant) to get sessionInfo

## R version 3.4.1 (2017-06-30)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 16.04.3 LTS
## 
## Matrix products: default
## BLAS: /usr/lib/libblas/libblas.so.3.6.0
## LAPACK: /usr/lib/lapack/liblapack.so.3.6.0
## 
## locale:
##  [1] LC_CTYPE=es_ES.UTF-8       LC_NUMERIC=C              
##  [3] LC_TIME=es_ES.UTF-8        LC_COLLATE=es_ES.UTF-8    
##  [5] LC_MONETARY=es_ES.UTF-8    LC_MESSAGES=es_ES.UTF-8   
##  [7] LC_PAPER=es_ES.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=es_ES.UTF-8 LC_IDENTIFICATION=C       
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## loaded via a namespace (and not attached):
##  [1] compiler_3.4.1  backports_1.0.5 magrittr_1.5    rprojroot_1.2  
##  [5] tools_3.4.1     htmltools_0.3.6 yaml_2.1.14     Rcpp_0.12.12   
##  [9] stringi_1.1.5   rmarkdown_1.6   knitr_1.17      stringr_1.2.0  
## [13] digest_0.6.12   evaluate_0.10.1

as_xts() doesn't play nicely with backticked columns (multiple words and numbers)

Backticked column names break when going from data frame / tibble to xts object. It seems that the names() function that is used under the hood when we set the not_date_names variable does not return the backticks in the names, but instead will just return the quoted column names. Then dplyr::select_() can't find the correct column names because there are no backticks.

Suggested fix: Either rework the way we create the not_date_names variable, or check for number or multi word columns after the not_date_names var is created, and add backticks manually.

suppressMessages(library(tidyquant))

# Create a data frame, notice how the `hello world` is a column with two words so it has to be surrounded with backticks
# This issue also applies to numbers like `22` when using them as column names
df <- tibble(
  date = c(Sys.Date(), Sys.Date()+1, Sys.Date()+2),
  x    = c(1,2,3),
  `hello world` = c(4,5,6)
)

# This breaks because under the hood we lose the backticks
as_xts(df, date_col = date)
#> Error in parse(text = x): <text>:1:7: unexpected symbol
#> 1: hello world
#>           ^

# A similar thing would happen if we renamed that column as a number
df %>%
  rename(`22` = `hello world`) %>%
  as_xts(date_col = date)
#> Error in combine_vars(vars, ind_list): Position must be between 0 and n

# See how it works without that backticked column
df %>%
  select(-`hello world`) %>%
  as_xts(date_col = date)
#>            x
#> 2017-06-13 1
#> 2017-06-14 2
#> 2017-06-15 3

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.