feature: add Rblpapi package as a data source about tidyquant HOT 37 OPEN

from/to works now

from tidyquant.

Great feature request. We are actually looking into this and other financial data sources at the moment. I agree 100% and thanks for the sample code. We'll certainly be evaluating this the second half of the year. 👍

from tidyquant.

Let me know if I can help, alpha-testing or whatever.

from tidyquant.

Will do. It may be a bit since we'd like to have some conversations with Bloomberg before jumping in.

from tidyquant.

Hey, we are getting ready to begin the Rblpapi integration. Wanted to see if you or others would be available for some alpha testing!

from tidyquant.

I am honored to help out any way I can. I will test code promptly and partner with a-player-to-be-named in my organization if I am indisposed.

from tidyquant.

Thanks Art. Davis is going to test in the next day or so. I am flying blind while I coded, so I suggest we let Davis do his thing first. Then once he and I agree, let's get you or someone to alpha test. Davis has access to a terminal.

from tidyquant.

Ok, we have good news and bad news. Good news is we have it working. Bad news is the integration is slower than bdh() because of how we are handling via iteration versus calling all at once (we will correct this). You can test it out, but just note that we will be reworking to improve performance shortly. You can try something like this...

my_bloomberg_data <- c('SPX Index','AGTHX Equity') %>%
    tq_get(get         = "rblpapi",
           rblpapi_fun = "bdh",
           fields      = c('RETURN','PRICE'),
           options     = c("periodicitySelection" = "MONTHLY"),
           from        = "2016-01-01",
           to          = "2016-12-31")

You can also test out other functions like bds() and bdp() by changing the rblpapi_fun argument (default is "bdh").

Let me know what you like/dislike about the API integration (besides performance) and any questions you may have.

from tidyquant.

Let's get started.
https://github.com/apsteinmetz/tq_testing.git

Not sure of the best way to share output. The html file in the repo shows it.

from tidyquant.

@apsteinmetz, looks good. As to your comments about from/to, you should be able to pass either:

• from/to as character vectors
• start.date/end.date as proper Dates

Internally we translate from/to into start.date/end.date if it is there. This is to maintain consistency with the other get functions that allow a date range!

from tidyquant.

I don't think from/to are getting translated:

Error in bdh(securities = "AGTHX Equity", fields = c("PX_LAST", "TOT_RETURN_INDEX_GROSS_DVDS": unused arguments (from = "2016-01-01", to = "2016-12-31")

from tidyquant.

Let me take a look.

from tidyquant.

Try now. I removed the from / to after being converted to start.date and end.date. Hopefully that works.

from tidyquant.

updated repo with more bdh variants. Seems pretty robust. Will look at bdp now and put tq_get in some of my production code.
Now in friendly github format! Look at the .md file.

from tidyquant.

Looks pretty good from the MD doc.

How is the speed? Slow?

Right now it’s setup for bds and bdp also. Let me know what else should be added.

from tidyquant.

tq_get(bdh) is prettty slow. See repo. In my use cases I wouldn't notice the difference, though. I grab the historical data once and then play with it. Still have to get to other functions. Day job, and all....

from tidyquant.

tq_get(bdp) tested. Looks good so far. see repo.

from tidyquant.

Haha, totally understand that open source software may not be your only thing going on. ;)

I checked the bdp results and it looks like the operation is working.

What are your thoughts on the API - Is it intuitive and how you would like to be able to interact with Bloomberg / Rblpapi? I'm interested in making it as easy and straightforward to use as possible.

Should speed be improved? Will there be demand for users to download a lot of Bloomberg data? My gut feel is it should, and I have a few ideas to improve speed.

from tidyquant.

'bds()' has problems. The parameter names for 'bds()' differ slightly from 'bdp()' and 'bdh()'. 'securities' becomes 'security' and that blows up piping in the ticker symbols and/or the default x parameter which seems to invariantly map to securities. 'fields'<> becomes 'field'. in bds() See error messages in repo. ...bds.md file

from tidyquant.

hat are your thoughts on the API - Is it intuitive and how you would like to be able to interact with Bloomberg / Rblpapi?

I think it's very easy to use! The syntax is the same as the native Rblpapi functions with just the two additional tq_get parameter. The key benefit of tq_get in all the APIs is returning a tidy formatted table and moving out of the xts/zoo domain into the lubridate/tibbletime (Ugh, that name!) world -which is where so many people are moving to.

On the speed issue, I don't know. Bloomberg limits how much data you can suck up daily and monthly (look at the PDF I put in the repo). They don't specifically tell you what it is, just that the hammer might come down. I don't think the BBG service is suited for 'big data' work. As such, speed won't be an issue for most users. I will ask around though.

from tidyquant.

Well that's a bummer about "bds". I checked the bds test report and it certainly looks like tq_get is struggling with the output. I'll have to investigate a bit more this weekend. Can you upload to GitHub an rds file of the output of the native "bds" call to Rblpapi? I want to use it to debug. I'd also like to do the same thing for the native call to "bdp" and "bdh" with multiple securities so I can test different methods for speed.

Regarding the feedback on the function, it sounds like we may be ready to move forward pending the speed concern and the BDS investigation. I have to get an update to CRAN soon because Yahoo Finance just took away the "key.stats" API, which is a bummer and now my tests are failing requiring a CRAN update. I'd like to include Bloomberg, Alpha Vantage, and Google Finance integrations.

from tidyquant.

added rds files with output from native calls.

The bds issue seems simple enough. You just have to recognize the default first parameter securities should become security No?

It is totally fine to release a version with just bdp() and bdh() working. That will cover 90% of use cases, I suspect. It covers 100% of mine.

from tidyquant.

I'm leaning towards releasing a first version soon. I just updated to change securities to security for "bds". I'd like to test a few things to improve speed, but not sure how long this will take. Let me know if "bds" is working now, and we'll call this issue closed if it checks out.

from tidyquant.

No bueno. Same errors. See repo.

from tidyquant.

This still fails even after devtools::install_github("business-science/tidyquant")?

my_bloomberg_data <- c('GOOG US Equity') %>%
    tq_get(get         = "rblpapi",
           rblpapi_fun = "bds",
           field       = c("TOP_20_HOLDERS_PUBLIC_FILINGS")
           )

from tidyquant.

No. That works. Piping in works. So does assigning the ticker to 'x'. Assigning the ticker to 'security' still fails with the complaint that x is missing.

from tidyquant.

Oh, yes. I override the "security" argument. Use "x" instead.

from tidyquant.

Is that the behavior you want? bds() takes the 'security' parameter. Not being able to name it seems a problem.
From the docs:
Usage:
bds(security, field, options = NULL, overrides = NULL, verbose = FALSE,
identity = NULL, con = defaultConnection())

from tidyquant.

Yes, I'd prefer to have the first parameter always be tq_get(x = symbols) so all API calls have the same format. Otherwise it becomes difficult for Matt the Developer to manage each combination, and ultimately makes it more difficult for Matt the User to remember which arguments to use.

It boils down to making tq_get() to be as consistent as possible for all APIs. For example, the Yahoo API call is very similar to the Quandl API call which in turn is very similar to the Bloomberg API call. We just set whatever base function arguments (e.g. "security" or "securities") to "x" internally.

# Yahoo
tq_get(x = "AAPL", get = "stock.prices")

# Quandl
tq_get(x = "WIKI/AAPL", get = "quandl")

#Bloomberg
tq_get(x = 'AAPL US Equity', get = "rblpapi", "bds")

from tidyquant.

I think that's a valid design choice. The tradeoff becomes you invalidate the reference docs for the native function that you ask users refer to for the details of the function. Some parameters DO use the native function assignments and some (well, one) doesn't. It's inconsistent.

BTW, I am re-writing a notebook in tidyverse/quant vernacular that analyzes a whole mutual fund family and I'm saving a lot of lines while making the code much clearer. Fun! I'll share when I'm done.

from tidyquant.

Thanks for agreeing. Really want to get it the way you want, but also need to keep the UI consistent with the other API calls.

Let me know about the mutual fund analysis. I'm really interested in seeing use cases of tidyquant in the wild. We have Business Science University in the works (you can actually enroll now but courses are not yet published). We will have 201 courses teaching high demand data science concepts and 301 courses teaching how to turn those concepts into interactive Shiny web apps that can be used as decision making tools. The use cases are key.

from tidyquant.

Happy New Year! I've completed a sort-of vignette for the blp stuff. You can see it HERE. The notebook is in the repo as well but obviously you can't run it without the Bloomberg.

Let me know what you think.

from tidyquant.

Hey Art, happy New Year! I’ll check it out. Also I want to send you an app we just did for Rstudio. It’s really cool. Look for an email shortly.

from tidyquant.

Is the "beqs" call from Rblpapi supported in tidyquant? I use beqs to download "point-in-time" historical data, but would like to do it all inside tidyquant if possible.

So, using my bloomberg screen "RIYM", I can download data using:

RIYM <- beqs("RIYM",screenType="PRIVATE",date=as.Date('12/29/2017',format='%m/%d/%Y'))

Using tidayquant, based on the above comments, I'm trying:

RIYM <- tq_get(x = '"RIYM",screenType="PRIVATE",date=as.Date('12/29/2017',format='%m/%d/%Y')', get = "rblpapi", "beqs")

...and getting the following error:

Error: unexpected numeric constant in "RIYM <- tq_get(x = '"RIYM",screenType="PRIVATE",date=as.Date('12"

I figure its either a dumb syntax error on my part or possibly that the beqs function from Rblpapi isn't supported. Any help is much appreciated!

from tidyquant.

Currently, only bdh, bds, and bdp options are supported. What is the output of the point in time typically? Just a value with a time stamp?

from tidyquant.

I'm actually downloading PIT data (monthly) for the Russell 1000 in order to backtest a model. According to Bloomberg, eqs is the only way to insure that we don't introduce survivorship bias. (If the stocks no longer trade then eqs (setting the index as the universe) is the only way to include all stocks that existed at the time.) I spent a lot of time manually chasing down random CUSIP's and FIGI codes before figuring this out...

In my case the output of the screen is a csv file containing tickers and factor data as it existed at the time. I run the screen iteratively, inserting the "as of" date each time.

from tidyquant.

@apsteinmetz Wanted to give you a heads up that I sent you an email regarding a presentation on Time Series Machine Learning and Deep Learning I'm giving at S&P Global on Thursday (4/18). You're invited if you'd like to join.

from tidyquant.