rblp / rblpapi Goto Github PK

First of all, I would like to say that as a former user of the Rbbg package, the new Rblpapi package is the best thing that has happened in years to the financial community working in R. Great thanks to the hardworking team behind this package and especially eddelbuettel and armstrtw!

Now to the issue at hand...

I found a strange bug which I believe may happens inside the "bdh_Impl" function within the bdh function. The order of the bdh output list does not always match the order of the securities vector. See examples below.

tickers <- c("000830 KS Equity", "003550 KS Equity", "005490 KS Equity",
             "005930 KS Equity", "006400 KS Equity", "0111145D US Equity",
             "0132533D CN Equity", "015760 KS Equity")

option.fields <- c("periodicitySelection", "nonTradingDayFillOption",
                   "nonTradingDayFillMethod", "periodicityAdjustment",
                   "adjustmentFollowDPDF", "pricingOption", "currency")

option.values <- c("MONTHLY", "NON_TRADING_WEEKDAYS",
"NIL_VALUE", "CALENDAR", "TRUE", "PRICING_OPTION_PRICE", "USD")

bbg.options <- structure(option.values, names = option.fields)

out1 <- bdh(securities = tickers,
            fields = "TOT_RETURN_INDEX_GROSS_DVDS",
            start.date = as.Date("1996-01-01"),
            end.date = "",
            options = bbg.options,
            overrides = NULL)

tickers
names(out1)
# the order of both character vectors match

The screenshot below shows that the order of the two vectors is the same

Now if we add another ticker the bdh shuffles the output so the order is no longer the same.

new.tickers <- c("000830 KS Equity", "003550 KS Equity", "005490 KS Equity",
                 "005930 KS Equity", "006400 KS Equity", "0111145D US Equity",
                 "0132533D CN Equity", "015760 KS Equity", "0202445Q US Equity")

option.fields <- c("periodicitySelection", "nonTradingDayFillOption",
                   "nonTradingDayFillMethod", "periodicityAdjustment",
                   "adjustmentFollowDPDF", "pricingOption", "currency")

option.values <- c("MONTHLY", "NON_TRADING_WEEKDAYS",
"NIL_VALUE", "CALENDAR", "TRUE", "PRICING_OPTION_PRICE", "USD")

bbg.options <- structure(option.values, names = option.fields)

out2 <- bdh(securities = new.tickers,
            fields = "TOT_RETURN_INDEX_GROSS_DVDS",
            start.date = as.Date("1996-01-01"),
            end.date = "",
            options = bbg.options,
            overrides = NULL)

new.tickers
names(out2)
# the order of both character vectors does no longer match

As the screenshot below shows the last ticker added is suddenly the first element in the list output from Bloomberg.

Is this unintended or a deliberate output choice?

The issue can easily be fixed after the bdh call by matching one's original ticker vector against the names vector on the output list to ensure that data get where it belong if one, as I do, want to manipulate the data into another data structure, than the Bloomberg output list, that fits better with one's data analysis desires.

If the authors can easily fix this it would be much appreciated. If not, I live without the fix and this issue thread will simply act as a help to future users of Rblpapi, if they run into the same issue, which they will likely do.

With the helper script to fetch the (Linux) tar.gz I'll update the headers as well.

Wonder if we could/should/... automate that. I presume nobody got bitten yet?

I keep thinking we should have something.

I tend to have enough environment variables around to autoconnect so I could basically test whether

• blpHost is set
• blpConnect is set
• I can ping the host
• I can connect successfully
  and if and only if all that is true run a unit test.

These would of course be off at Travis and CRAN as those have bare ~/.Rprofile files.

Am I making sense?

I was wondering if it was possible to load field infos via Rblpapi into R. Using Excel, this would work with the formula =BFieldInfo("MLI_OAS"). Up to now I have only been working with commands such as bdh and bdp. The difference now is that I do not have a certain Index. Have I overlooked the existence of such a command?
Thank you for your help!

The BEQS function fails under the following two circumstances:

• When the first 25 rows of the returned data is NA
• When the entire column returned is NA

We rely on xts to merge the bid, ask, trade, ... vectors but the timestamps are not unique. So the merged xts is not valid.

Hi,

Thank you for the package. I started using it for quant ML purposes in R and it looks great.

I was wondering if the time stamp of getTicks can be in higher resolution than seconds (mili seconds, even less?).
The problem is that I can get many ticks that are stamped for the same second. But I want higher resolution than that. Is it possible?

In ‘POSIXct’ I can define the format of %Y-%m-%d %H:%M:%OS or '%Y-%m-%d %H:%M:%OS1'
for a ms value. Is it possible to integrate inside the package?

Regards,
Ori

I set up the following screen called "tse_etf1".

Security universe:
Trading status: Active
Security attributes: show primary ticker only
Security types: ETF
Exchanges: Tokyo

Display only fields:
Relative index
Current market cap
Year to date total return
Current price earnings ratio (P/E)

This error was returned:

beqs("tse_etf1")
Error: could not convert using R function : as.data.frame

I noticed in the BBG terminal P/E is empty for all results. I deleted P/E and now it works as expected.

A workaround is to remove the columns known to return all blanks but the user may not always know a column in a screen would be all blanks.

@eddelbuettel I am thinking there should be some defaults for the getBars options vector besides NULL. What do you think?

If I added defaults to the options vector would something like this be acceptable?

getBars(security, eventType = "TRADE", barInterval = 60,  startTime = Sys.time() - 60 * 60 * 6,
endTime = Sys.time(), options = getOption("getBarsOptions", NULL),
verbose = FALSE, returnAs = getOption("blpType", "matrix"), tz = Sys.getenv("TZ", unset = "UTC"), 
con = defaultConnection())

Where getBarsOptions would be set as something like:

 options(getBarsOptions = c("gapFillInitialBar"="TRUE", "adjustmentFollowDPDF"="TRUE"))

BEQS (Bloomberg Equity Screening) request returns security data for a selected
screen created using the Bloomberg EQS function.

As suggested in Whit's comment here: #73 (comment)

If we are redoing the function interface by having a default con in the environment, can we also standardize other arguments? start.time vs startTime is just one example.

Consider the following lines of code

    library(Rblpapi)
    con <- blpConnect()
    opt <- c("START_DT"="19990101", "END_DT"="20160101")
    bds("CPI YOY Index","ECO_RELEASE_DT_LIST", options = opt, con = con)
    blpDisconnect(con)

I get the following error, "Error: Sub-element '(null)' does not exist." while executing the code. It has something to do with the options. I can get the data in excel with these options. Also note that without the options I can get a small chunk of data starting 2015. However, the goal is to get all data starting from a particular date.

Via email from Yang Lu (following chat at R/Finance)

> install_github("armstrtw/Rblpapi")                                                                                                                                                                   
Downloading github repo armstrtw/Rblpapi@master                                                                                                                                                        
Installing Rblpapi                                                                                                                                                                                     
'/usr/local/lib64/R/bin/R' --vanilla CMD INSTALL  \                                                                                                                                                    
  '/tmp/Rtmp0uBpmP/devtools520e26b00447/armstrtw-Rblpapi-f7ff208'  \                                                                                                                                   
  --library='/q5/hhq5/rlib' --install-tests                                                                                                                                                            

* installing *source* package  Rblpapi  ...                                                                                                                                                            
Setting up compilation for 64-bit system                                                                                                                                                               
** libs                                                                                                                                                                                                
g++ -std=c++0x -I/usr/local/lib64/R/include    -I../inst/include/ -I. -I/usr/local/include -I"/q5/hhq5/rlib/Rcpp/include" -I"/q5/hhq5/rlib/BH/include"   -fpic  -g -O2 -c RcppExports.cpp -o           
RcppExports.o                                                                                                                                                                                          
g++ -std=c++0x -I/usr/local/lib64/R/include    -I../inst/include/ -I. -I/usr/local/include -I"/q5/hhq5/rlib/Rcpp/include" -I"/q5/hhq5/rlib/BH/include"   -fpic  -g -O2 -c authenticate.cpp -o          
authenticate.o                                                                                                                                                                                         
g++ -std=c++0x -I/usr/local/lib64/R/include    -I../inst/include/ -I. -I/usr/local/include -I"/q5/hhq5/rlib/Rcpp/include" -I"/q5/hhq5/rlib/BH/include"   -fpic  -g -O2 -c bdh.cpp -o bdh.o             
g++ -std=c++0x -I/usr/local/lib64/R/include    -I../inst/include/ -I. -I/usr/local/include -I"/q5/hhq5/rlib/Rcpp/include" -I"/q5/hhq5/rlib/BH/include"   -fpic  -g -O2 -c bdp.cpp -o bdp.o             
g++ -std=c++0x -I/usr/local/lib64/R/include    -I../inst/include/ -I. -I/usr/local/include -I"/q5/hhq5/rlib/Rcpp/include" -I"/q5/hhq5/rlib/BH/include"   -fpic  -g -O2 -c bds.cpp -o bds.o             
g++ -std=c++0x -I/usr/local/lib64/R/include    -I../inst/include/ -I. -I/usr/local/include -I"/q5/hhq5/rlib/Rcpp/include" -I"/q5/hhq5/rlib/BH/include"   -fpic  -g -O2 -c blpConnect.cpp -o            
blpConnect.o                                                                                                                                                                                           
g++ -std=c++0x -I/usr/local/lib64/R/include    -I../inst/include/ -I. -I/usr/local/include -I"/q5/hhq5/rlib/Rcpp/include" -I"/q5/hhq5/rlib/BH/include"   -fpic  -g -O2 -c blpapi_utils.cpp -o          
blpapi_utils.o                                                                                                                                                                                         
blpapi_utils.cpp: In function  SEXPREC* buildDataFrame(LazyFrameT&, bool, bool) :                                                                                                                      
blpapi_utils.cpp:276: error: expected initializer before  :  token                                                                                                                                     
blpapi_utils.cpp:411: error: expected primary-expression at end of input                                                                                                                               
blpapi_utils.cpp:411: error: expected  ;  at end of input                                                                                                                                              
blpapi_utils.cpp:411: error: expected primary-expression at end of input                                                                                                                               
blpapi_utils.cpp:411: error: expected  )  at end of input                                                                                                                                              
blpapi_utils.cpp:411: error: expected statement at end of input                                                                                                                                        
blpapi_utils.cpp:411: error: expected  }  at end of input                                                                                                                                              
blpapi_utils.cpp:411: error: expected  }  at end of input                                                                                                                                              
make: *** [blpapi_utils.o] Error 1                                                                                                                                                                     
ERROR: compilation failed for package  Rblpapi                                                                                                                                                         
* removing  /q5/hhq5/rlib/Rblpapi                                                                                                                                                                      
Error: Command failed (1)  

Ok, I started a new branch with

• actually running compileAttributes() so that RcppExports.{cpp,R} is updated
• actually running roxygenize() so that the manual page is created

With that we can actually call beqs(), and I'll commit that shortly. That is the good news.

The bad news is that the returned object is, well, not really up to our standards:

!R> head(res <- beqs("Global Oil Companies YTD Return","GLOBAL"))
   Date Ticker             Short Name         Total Return YTD Market Cap          Price:D-1   P/E         Revenue T12M         EPS T12M
 1 NA   "002506 CH Equity" "GCL SYSTEM INT-A" "1078.202501"    "2513595136.000000" "1.001490"  "2.211897"  "833422051.073852"   "0.408008"
 2 NA   "S92 GR Equity"    "SMA SOLAR TECHNO" "152.222214"     "1339073024.000000" "38.590000" NA          "893484992.000000"   "-4.484000"
 3 NA   "PTG TB Equity"    "PTG ENERGY PCL"   "143.235624"     "590486336.000000"  "0.353585"  "48.333332" "1279731045.178220"  "0.006965"
 4 NA   "ALDW US Equity"   "ALON USA PARTNER" "131.713313"     "1379535744.000000" "22.069027" "7.233236"  "2321284053.504420"  "2.839346"
 5 NA   "SRS IM Equity"    "SARAS SPA"        "127.338135"     "1803096064.000000" "1.896000"  "34.100716" "9271717120.000000"  "0.055600"
 6 NA   "ES FP Equity"     "ESSO S.A.F."      "99.592984"      "819479360.000000"  "63.750000" NA          "14152000000.000000" "-13.029999"
 R> 

Returning everything as character is better than not having the function, but really not show this should be done.

@csrvermaak @wmorgan85 How can I know what the returned fields are going to? Is this is a fixed, or computable format? From the looks the above should be Date, char, char, numeric, int, numeric, numeric, int, numeric -- and I could agree to drop ints for all numeric if we must. But Date and character types are separate. This fits a data.frame to the tee, so let's build a proper one.

Hi, snippet below, seems like it should work and return valid condition codes along with price and quantity, but nothing is returned.

require(Rblpapi)
blpConnect()
symbol <- "WNH6 Comdty"
local.times <- as.POSIXct(c("2016-01-07 09:00:00", "2016-01-08 10:00:00"), format="%Y-%m-%d %H:%M:%S")

works

ticks <- getTicks(symbol, eventType = "TRADE", startTime = local.times[1], endTime = local.times[2],verbose = FALSE)

                times    value size

32658 2016-01-08 15:59:59 160.8125 1
32659 2016-01-08 15:59:59 160.8125 1
32660 2016-01-08 15:59:59 160.8125 2
32661 2016-01-08 15:59:59 160.8125 2
32662 2016-01-08 15:59:59 160.8125 1
32663 2016-01-08 16:00:00 160.8125 3

fails

ticks <- getTicks(symbol, eventType = "TRADE", startTime = local.times[1], endTime = local.times[2],verbose = FALSE, setCondCodes = TRUE)

Error in getTicks(symbol, eventType = "TRADE", startTime = local.times[1], :
unused argument (setCondCodes = TRUE)

Is the option named something different?

Thanks!

I'm using getTicks to get executed trade data from Bloomberg, for example as follows:

getTicks("3137G0EZ8@TRAC MTGE", eventType = "TRADE", startTime = as.POSIXct(as.Date("20151201", "%Y%m%d")), endTime = as.POSIXct(as.Date("20151231", "%Y%m%d")), tz = Sys.timezone())

This returns a matrix with trade data, similar to public TRACE data that can be found here: http://finra-markets.morningstar.com/BondCenter/BondTradeActivitySearchResult.jsp?ticker=FFMCC4239504&startdate=12/01/2015&enddate=12/31/2015

The matrix actually has both sides of one trade. On the website above, a column indicates which row corresponds to Buy, and which to Sell. (This detail is also available on the QR screen in Bloomberg, for users with sufficient access).

Would it be possible to add a column to the output containing the side? Or add an argument to the eventType for "BUY" and "SELL"?

Thank you for all your efforts!

code:

library(Rblpapi)
connect<-blpConnect() # connect to bloomberg terminal
indexName <- 'SPX INDEX'
queryFieldsImpliedVol <- c("60DAY_IMPVOL_100.0%MNY_DF","3MTH_IMPVOL_100.0%MNY_DF")
impliedVolSPX <- bdp(indexName,queryFieldsImpliedVol,con=connect)

output:

SPX INDEX                                          
60DAY IMPLIED VOL: 14.5635
90DAY IMPLIED VOL: 14.07569         

However, per Bloomberg Terminal with function call used in Excel:
=BDP(SPX INDEX,60DAY_IMPVOL_100.0%MNY_DF)

AND

=BDP(SPX INDEX,3MTH_IMPVOL_100.0%MNY_DF)

The output is:

SPX INDEX                                          
60DAY IMPLIED VOL: 14.07569
90DAY IMPLIED VOL: 14.5635

I believe there is an error for these two query fields, since they appear to be swapped. Please help is possible. Thank you.

In continuing to remove LazyFrame from the codebase, one issue that has come up is whether it is possible to query bloomberg for the fields corresponding to a BDS request vs having to loop over the output to discover the fields / fieldTypes (DVD_HIST for example).

@wmorgan85, it would be great if you could chime in on whether this is possible.

bdh documentation has the wrong enumeration: "periodicityAdjustment" s/b "periodicitySelection"

bdh("SPY US Equity", c("PX_LAST", "VOLUME"),

• start.date=Sys.Date()-31*6, options=c("periodicityAdjustment"="MONTHLY"))
  

  Error: Enumeration type could not be found for MONTHLY
  ..
  bdh("SPY US Equity", c("PX_LAST", "VOLUME"),

• start.date=Sys.Date()-31*6, options=c("periodicitySelection"="MONTHLY"))
  date PX_LAST     VOLUME
  

  1 2015-06-30 205.85 2535242753
  2 2015-07-31 210.50 2452757863

Hello,

  Could you please tell me how I can get all the ISIN list by this API? I see that I can get the details by /isin/XXXXXXXXX. But for now I want get all the ISIN first. 

Many thanks for you help.

BR,

Liang

This works

con = Rblpapi::blpConnect()
Rblpapi::bdp(securities = "DE US Equity", fields = "CRNCY", con = con)
Rblpapi::blpDisconnect(con = con)

but this returns an error:

con = Rblpapi::blpConnect()
Rblpapi::bdp(securities = "IP Index", fields = "CRNCY", con = con)
Rblpapi::blpDisconnect(con = con)

Error in bdp_Impl(con, securities, fields, options, overrides, identity) : 
   attempt to set an attribute on NULL

Maybe it is stupid to look for the CRNCY of IP Index, but I would have expected to get a NULL or an empty data.frame, not an error.

Thank you

packageVersion("Rblpapi")
[1] '0.2.2.6.2.3'


R.version                       
platform       i386-w64-mingw32            
arch           i386                        
os             mingw32                     
system         i386, mingw32               
status                                     
major          3                           
minor          2.0                         
year           2015                        
month          04                          
day            16                          
svn rev        68180                       
language       R                           
version.string R version 3.2.0 (2015-04-16)
nickname       Full of Ingredients

The Rblpapi fails check on my windows box due to missing curl program. I know this tool happens to be installed on the cran build server, but it is not part of the standard windows tool kit.

* installing *source* package 'Rblpapi' ...
** package 'Rblpapi' successfully unpacked and MD5 sums checked
** libs
make: curl: Command not found
make: *** [blpLibrary] Error 127
Warning: running command 'make -f "Makevars.win" -f "C:/PROGRA~1/R/R-EXPE~1/etc/x64/Makeconf" -f "C:/PROGRA~1/R/R-EXPE~1/share/make/winshlib.mk" CXX='$(CXX1X) $(CXX1XSTD)' CXXFLAGS='$(CXX1XFLAGS)' CXXPICFLAGS='$(CXX1XPICFLAGS)' SHLIB_LDFLAGS='$(SHLIB_CXX1XLDFLAGS)' SHLIB_LD='$(SHLIB_CXX1XLD)' SHLIB="Rblpapi.dll" WIN=64 TCLBIN=64 OBJECTS="RcppExports.o authenticate.o bdh.o bdp.o bds.o blpConnect.o blpapi_utils.o fieldsearch.o getBars.o getTicks.o"' had status 2

If you want this package to build on non-cran windows, it is safer to use R -e "download.file(.....)" or R -e "curl::curl_download(.....)" or similar. If you don't care, feel free to close.

When I try to run one of the example screens:

beqs("Global Oil Companies YTD Return","GLOBAL","ENGLISH","GENERAL","20150930")

This error is returned:

Error in format.default(date, "%Y%m%d") : invalid 'trim' argument

Suspect it might be in the wrapper function.

I have installed the Rblpapi package on Windows using

install.packages("drat", repos="http://cran.rstudio.com")
drat:::add("ghrr")
install.packages("Rblpapi")
bdh("SPY US Equity", c("PX_LAST", "VOLUME"), start.date=Sys.Date()-31)

Installation was successful, however I get the following error: Error: External pointer is NULL.

is it because I need to set the path to the blpapi dll?

In rbbg land, BDH/BDP support overrides AND options, I currently see only options supported in the code. How do I pass an override in to bdh/bdp?

Also when using nonTradingDayFillMethod "PREVIOUS_VALUE" should also be an option (as international portfolios may have securities with non synchronous dates).

Per a CRAN email:

I investigated

https://www.r-project.org/nosvn/R.check/r-devel-osx-x86_64-clang/Rblpapi-00check.html

which is fairly new. As you suppress error messages with curl -s and do
not check the return code you do not report

tawny% curl -k -L -O
https://github.com/Rblp/blp/raw/master/headers/osx/blpHeaders.tar.gz
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 145 100 145 0 0 134 0 0:00:01 0:00:01 --:--:--
134
curl: (35) Unknown SSL protocol error in connection to
raw.githubusercontent.com:-9805

So it seems github is now incompatible with OS X's curl (the URL works
in browsers).

[moving off email, per Dirk's request]

TL;DR: we should save a connection in the package environment and use it by default so that people don't have to deal with passing the same connection to every single function call.

I've made a few more changes and pushed a branch to my repo. Let me know what you think.

Following my previous issue, a second one, a bit less important, (although I am bit scared from Dirk's feedback :) Thank you guys for the good work on this project and I am also here to bring a positive contribution from my experience as tester.

When the daily or monthly limit is reached, Rblpapi returns a weird error. "Choice sub-element not found for name 'securityData'."

I have to catch it myself (see below. For clarification purpose, thatis NOT what I think is the best long term solution) , I suggest we catch it inside the code of Rblpapi. In excel the API returns a #N/A limit" error and I guess the C++ API is doing the same?

 # example with bdp but bdh and bds return the same error
 tryCatch({Rblpapi::bdp(securities = "IP Index", fields     = "PX_LAST",  con = con)
     },error=function(e){
      if (e$message=="Choice sub-element not found for name 'securityData'.") {
        stop("BBG limit is reached (daily or monthly)")
      } else {
        stop(e)
      }
    })

Firstly thanks for the initiative in supporting R and Bloomberg.

getTicks - is it possible to add the quote type (similar to verbose mode e.g. TRADE, BEST_BID) and the option to enable condition codes, similar to the Excel condcodes = S and QRM = S

I believe the following needs to be set:
request.Set("includeConditionCodes", true); // Excel eqv. CondCodes = S
request.Set("includeNonPlottableEvents", true); // Excel eqv. QRM = S

My C++ experience is quite limited, but in the meantime I have forked this project and will have a good go at this myself.

Thank you,
JD

In the following example, each security returns data from a call to bds. However, when the securities are combined in to a character vector, only data for the last security is returned.

> cn <- blpConnect()
> bds("AGL SJ Equity", "EQY_DVD_ADJUST_FACT")
  Adjustment Date Adjustment Factor Adjustment Factor Flag Adjustment Factor Operator Type
1      2007-07-02          0.910000                      3                               1
2      2007-07-02          0.937085                      1                               2
3      2007-07-02          1.000000                      1                               2
4      2001-05-08          4.000000                      3                               1

> bds("SOL SJ Equity", "EQY_DVD_ADJUST_FACT")
  Adjustment Date Adjustment Factor Adjustment Factor Flag Adjustment Factor Operator Type
1      1995-07-24          0.963111                      1                               2
2      1992-02-17          1.000000                      3                               2

> bds(c("AGL SJ Equity", "SOL SJ Equity"), "EQY_DVD_ADJUST_FACT")
  Adjustment Date Adjustment Factor Adjustment Factor Flag Adjustment Factor Operator Type
1      1995-07-24          0.963111                      1                               2
2      1992-02-17          1.000000                      3                               2

> blpDisconnect(cn)

Could getBars support returning the value traded in the bar? It seems from the BBG API Developer's guide and looking in BBG's Excel tools that 'value' is supported.

Ref: BBG API Developer's guide, http://www.bloomberglabs.com/content/uploads/sites/2/2014/07/blpapi-developers-guide-2.54.pdf (see p. 177)

library(Rblpapi)
blpConnect()
Error: Failed to start session.

I looked through the documents, but didn't find any idea how to troubleshoot this?

Hi,
Would somebody have time to implement BSRCH function? Example us case
BSRCH("COMDTY:NGFLOW")
It is commonly used for Fixed Income instrument screening.
Thank you for your time and help.
/Marek

From the 'crazy uncle Bob' department:

• Should be have a (finite, small) set of stub functions and headers to mock the Blp API which we cannot distribute?

If we did then the package would MUCH more widely used and CRAN-distributable. Worth it?

g++ -std=c++11 -I/usr/share/R/include    -I../inst/include/ -I.  -I"/usr/local/lib/R/site-library/Rcpp/include" -I"/usr/local/lib/R/site-library/BH/include"   -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c blpapi_utils.cpp -o blpapi_utils.o
In file included from /usr/local/lib/R/site-library/BH/include/boost/smart_ptr/shared_ptr.hpp:28:0,
                 from /usr/local/lib/R/site-library/BH/include/boost/shared_ptr.hpp:17,
                 from /usr/local/lib/R/site-library/BH/include/boost/date_time/gregorian/greg_month.hpp:14,
                 from /usr/local/lib/R/site-library/BH/include/boost/date_time/gregorian/greg_ymd.hpp:16,
                 from /usr/local/lib/R/site-library/BH/include/boost/date_time/gregorian/greg_calendar.hpp:16,
                 from /usr/local/lib/R/site-library/BH/include/boost/date_time/gregorian/gregorian_types.hpp:19,
                 from blpapi_utils.cpp:23:
/usr/local/lib/R/site-library/BH/include/boost/smart_ptr/detail/shared_count.hpp:395:33: warning: ‘template<class> class std::auto_ptr’ is deprecated [-Wdeprecated-declarations]
     explicit shared_count( std::auto_ptr<Y> & r ): pi_( new sp_counted_impl_p<Y>( r.get() ) )

And so on...

Hi

I've forked the code, to add the functionality for BEQS(), and checked it out of GitHub to C:\Rblpapi\ folder.

Before making any changes to the project, I would like to try to build the package on my machine. To build, I run the following command from my C:\ root folder:

R CMD build Rblpapi/trunk

These are my PATH settings to R:
C:\Program Files\RRO\R-3.2.2\bin\x64;

I get the following result/error:

C:>R CMD build Rblpapi/trunk

• checking for file 'Rblpapi/trunk/DESCRIPTION' ... OK

• preparing 'Rblpapi':

• checking DESCRIPTION meta-information ... OK

• cleaning src
  Warning in cleanup_pkg(pkgdir, Log) :
  unable to run 'make clean' in 'src'

• installing the package to build vignettes
  Warning: running command '"C:/PROGRA1/RRO/R-321.1/bin/x64/Rcmd.exe" INSTALL -l
  "C:\Users\Rvermaak\AppData\Local\Temp\Rtmp0OjXaB\Rinst34603e6b4550" --no-multia
  rch "C:/Users/Rvermaak/AppData/Local/Temp/Rtmp0OjXaB/Rbuild346041c57475/Rblpapi"
  ' had status 1

• installing source package 'Rblpapi' ...

  WARNING: this package has a configure script
  It probably needs manual configuration

** libs
Warning: running command 'make -f "Makevars.win" -f "C:/PROGRA1/RRO/R-321.1/et
c/x64/Makeconf" -f "C:/PROGRA1/RRO/R-321.1/share/make/winshlib.mk" CXX='$(CXX1
X) $(CXX1XSTD)' CXXFLAGS='$(CXX1XFLAGS)' CXXPICFLAGS='$(CXX1XPICFLAGS)' SHLIB_LD
FLAGS='$(SHLIB_CXX1XLDFLAGS)' SHLIB_LD='$(SHLIB_CXX1XLD)' SHLIB="Rblpapi.dll" WI
N=64 TCLBIN=64 OBJECTS="RcppExports.o authenticate.o bdh.o bdp.o bds.o blpConnec
t.o blpapi_utils.o fieldsearch.o getBars.o getTicks.o"' had status 127
ERROR: compilation failed for package 'Rblpapi'

• removing 'C:/Users/Rvermaak/AppData/Local/Temp/Rtmp0OjXaB/Rinst34603e6b4550/Rb
  lpapi'

  ERROR: package installation failed

Apologies for asking a basic question - I've written the code for EQS in C++ already, but it currently exports to csv. I don't have the bandwidth to learn package deployment right now, but I can build on the existing Rblpapi code to add the functionality. Please advise?

@eddelbuettel @johnlaing @armstrtw - first of all, excellent work on this package; think this is absolutely the way forward. I'm looking forward to making some pull requests in the future.

Quick question. Did you expect an increase in speed from Rbbg to Rblpapi?

This may well not be what you're aiming for, so please ignore the following if that's the case. I just wanted to give it a test before switching over and changing a fair bit of embedded code.

Using:

• Desktop API
• Win 8.1 VM
• Latest RStudio prod release

_Tl;dr_: Appears (at least on this setup) that Rbbg is quicker.

Ran the following "head to head" tests in two separate R instances in a Win VM

# Java Session Info

> sessionInfo()
R version 3.1.2 (2014-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] Rbbg_0.5.3                    rJava_0.9-6                   quantstrat_0.9.1657           foreach_1.4.2                
 [5] blotter_0.9.1666              PerformanceAnalytics_1.4.3541 FinancialInstrument_1.2.0     quantmod_0.4-4               
 [9] TTR_0.22-0                    xts_0.9-7                     zoo_1.7-11                   

loaded via a namespace (and not attached):
[1] codetools_0.2-11 grid_3.1.2       iterators_1.0.7  lattice_0.20-30  tools_3.1.2   

and

# C++ Session Info

> sessionInfo()
R version 3.1.2 (2014-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] Rblpapi_0.2.2.5               quantstrat_0.9.1657           foreach_1.4.2                 blotter_0.9.1666             
 [5] PerformanceAnalytics_1.4.3541 FinancialInstrument_1.2.0     quantmod_0.4-4                TTR_0.22-0                   
 [9] xts_0.9-7                     zoo_1.7-11                    RQuantLib_0.4.0              

loaded via a namespace (and not attached):
[1] codetools_0.2-11 grid_3.1.2       iterators_1.0.7  lattice_0.20-30  Rcpp_0.11.5      tools_3.1.2   

Test 1(a): Single Symbol BDH call using Rbbg. SPY US from 2001 and YTD

#2001 to Present

> system.time(
    bdh(conn,
           "SPY US Equity",
           c("OPEN", "HIGH", "LOW", "PX_LAST", "VOLUME"),
           as.POSIXct("2001-01-01)"),
           Sys.Date()))

   user  system elapsed 
   0.15    0.00    4.18 

#2015 YTD

> system.time(
    bdh(conn,
           "SPY US Equity",
           c("OPEN", "HIGH", "LOW", "PX_LAST", "VOLUME"),
           as.POSIXct("2015-01-01)"),
           Sys.Date()))

   user  system elapsed 
   0.05    0.00    0.30     

Test 1(b): Single Symbol BDH Call using Rblpapi. SPY US from 2001 and YTD

#2001 to Present

> system.time(bdh(securities = "SPY US Equity", fields = c("OPEN", "HIGH", "LOW", "PX_LAST", "VOLUME"), start.date =as.Date("2001-01-01"),end.date = Sys.Date()))

   user  system elapsed 
   0.02    0.01    5.22 

#2015 YTD

> system.time(bdh(securities = "SPY US Equity", fields = c("OPEN", "HIGH", "LOW", "PX_LAST", "VOLUME"), start.date =as.Date("2015-01-01"),end.date = Sys.Date()))

user  system elapsed 
   0.00    0.00    2.23 

Surprisingly Rbbg was much quicker on the YTD symbol pull.

So the next test was to pull 8 equity symbols and see if the difference was as pronounced

Test 2(a):Eight Symbol BDH Call using Rbbg. 2001 and YTD

#2001 to present

> system.time(
    bdh(conn,
    x,
    c("OPEN", "HIGH", "LOW", "PX_LAST", "VOLUME"),
    as.POSIXct("2001-01-01)"),
    Sys.Date()))


user  system elapsed 
   1.34    0.08   11.05

#2015 YTD

> system.time(
    bdh(conn,
    x,
    c("OPEN", "HIGH", "LOW", "PX_LAST", "VOLUME"),
    as.POSIXct("2015-01-01)"),
    Sys.Date()))

 user  system elapsed 
   0.27    0.00    2.61 

Test 2(b):Eight Symbol BDH Call using Rblpapi. 2001 and YTD

#2001 to present

> system.time(
    bdh(securities = x,
    fields = c("OPEN", "HIGH", "LOW", "PX_LAST", "VOLUME"),
    start.date =as.Date("2001-01-01"),
    end.date = Sys.Date()))

user  system elapsed 
   0.16    0.01   20.31

#2015 YTD

> system.time(
    bdh(securities = x,
    fields = c("OPEN", "HIGH", "LOW", "PX_LAST", "VOLUME"),
    start.date =as.Date("2015-01-01"),
    end.date = Sys.Date()))

   user  system elapsed 
   0.00    0.00    1.15  

In this case Rbbg _was nearly twice as fast pulling 14.5 years of 8 equity symbols from the API_ -- but Rblpapi was quicker on the YTD.

This could all be my VM setup or the way I ran these casual tests....but, otherwise, is this surprising?

The Rblpapi package is great. My colleagues and I make a lot of use of it so thank you for your time developing it.

I’m interested to know if your functions will work with Bloomberg’s Server API as opposed to the Desktop API that most of my colleagues use. I have successfully used the Rblpapi package on a server without Bloomberg installed by providing blpConnect with the correct host and port details of our SAPI server. This returns data under the Server Mode of the SAPI i.e. without the need to enter a Bloomberg UUID.

My question is can blpConnect be modified so that a Bloomberg UUID or a client IP Address can be passed to the SAPI so that the SAPI will operate in User Mode. This would allow a client (PC) to pass user details through the server (running R) to the SAPI and return data back to the client. I understand that the Bloomberg Data Centre would check that the IP Address of the client provided did have a logged in Bloomberg Anywhere Terminal thereby adhering to the Bloomberg data license and providing only the data that the user is entitled to.

An explanation of the differences between Server Mode and User Mode are given here in the Bloomberg API Developer’s Guide:

http://www.bloomberglabs.com/content/uploads/sites/2/2014/07/blpapi-developers-guide-2.54.pdf

Do you think that this is something that you could incorporate? It would add another very useful dimension to Rblpapi.

Thanks.

Dear All!

I have recently started to work with the great and helpful Rblpapi package to fetch data directly from Bloomberg. However, I have one question that could not be answered by searching the internet. So now I am asking you kindly if you might help me out in this matter.

Wherease
data1 <- bdh("EN00 INDEX", c("PX_LAST"), start.date=Sys.Date()-365)
works perfectly fine, I have troubles when it comes to other data that is provided by Merryll Lynch and available via Bloomberg. Concretely, I need the MLI_OAS mli date=20150901(or other date) of the index "EN00 INDEX", or, even better, the whole time series for" MLI_OAS" instead of "PX_LAST".

I have tried several attempts, but none of them did work. Such as
data1 <- bdh("EN00 INDEX", c("MLI_OAS"), start.date=Sys.Date()-365)
data2 <- bdp("EN00 INDEX", c("MLI_OAS"), MLI_Date=20150901)
data3 <- bdp("EN00 INDEX", c("MLI_OAS"),,"mli_date=20150901")
data4 <- bdp("EN00 INDEX", c("MLI_OAS"),,"mli date=20150901")
However, the only data point I can get is the most recent value (yesterday 09/14/2015: 123). I think I have to somehow name the override correctly. The error I get is "Request overrides must be named".

My question is how to address the override correctly. Is this even implemented in the package? Is it not possible, because it is Merryll Lynch data?

Thank you very much for your help.

It seems the getBars function never adjusts for corporate actions like stock splits. Eg. 4661 JT Equity had a 1:4 stock split on 3/27/2015. This means unadjusted prices should have a large gap between 3/26 and 3/27 where prices fall by a factor of 4, while adjusted prices should multiply pre-3/27 prices by 0.25 to remove this gap. I find it always returns unadjusted prices, even though my BBG settings in DPDF are set to adjust and IntradayBarRequest should follow this by default according to BBG.

I am trying to download some historical data using BDH. However, the calls to bdh hang (blocked waiting?) or fail with an error message "Failed to open //blp/refdata". When it hangs I eventually have to kill the R process. This has only started to happen recently and I can't pin point exactly what triggered it. A few more details

• Using a connection to localhost.
• There are multiple consecutive bdh invocations within the code. Each invocation downloads two years worth of daily historical data for c.25 data series. Code usually does not hang on the first invocation rather one of the subsequent invocations tends to hang. I have also tried "Sleep"ing the R code for upto 30 seconds between bdh invocations which does not help.
• An excel session is open with blp and bdh calls, the spreadsheet is not on AutoCalc but is recalculated periodically at 5 minute intervals. I have not noticed any causality between the blocks and excel re-calculation, other than the fact that the hanging problem happens a lot less frequently when no excel sessions are open.
• Bloomberg terminal app is also running on the desktop where the code is being run.
• Similar behaviour with Rbbg package
• Using the Bloomberg Java API directly does not seem to run into these issues.
• R version (3.1.0)
• Rblpapi version (0.3.2)
• OS (Windows 7)
• Killing the Bloomberg API processes helps, as does killing the excel spreadsheets.

Currently the header and library tarballs are download on each build and invocation of configure. This could be smarter -- and only download when not present. That would help for repeated builds from the source directory, ie R CMD build ..

First off all, I would like to extend my thanks for the significant effort expended in producing the Rblpapi package as an improved successor to the previous Java based efforts.

I have recently started to transition some scripts initially implemented using package RBloomberg/Rbbg to Rblpapi (version 0.3.2) and in the process encountered the following issues in conjunction with missing field values and treatment of non-trading days.

Option include.non.trading.days is a bit of a mystery to me as it seems to be doing the exact opposite of what it is purported to do. See output p1 of the included demonstration code which produces a value for a Sunday despite include.non.trading.days set to FALSE. I suspect that there is some interaction going on with API request option ALL_CALENDAR_DAYS. Because in the presence of that option setting include.non.trading.days TRUE actually suppresses output for non-trading days (output p2).

Output P2 demonstrates an additional issue as the unavailability of a field value is not indicated by inclusion of an NA value, but instead the field column is dropped from the function output. This is a deviation in semantics from package Rbbg, and I would assume that I am not the only one whose code broke due to this behavior.

Output p3 illustrates that the dropping of the field column only takes place if values are unavailable for all requested dates. Otherwise, as expected, NA is returned for those dates.

Output p4 demonstrates IMO broken behavior with a symbol for which the requested field does not contain any data for any period. Again, I would have expected NA to be returned for the PX_LAST field for the requested dates.

I would appreciate if the above issues could be addressed in an upcoming version of the package. Please let me know if there is anything else I can do to help track this down.

Thank you and best regards,
David

Code output:

p1:
        date PX_LAST
1 2000-01-30  1716.5

p2:
        date
1 2000-01-30

p3:
        date PX_LAST
1 2000-01-28  1716.5
2 2000-01-29      NA
3 2000-01-30      NA

p4:
        date
1 2000-05-28
2 2000-05-29
3 2000-05-30

Demonstration code:

require("Rblpapi")
require("xts")

fld <- "PX_LAST"
sdt <- as.Date("2000-01-30")
edt <- as.Date("2000-01-30")


p1 <- bdh(securities = "LAH00 Comdty", fields = fld,
         start.date = sdt, end.date = edt,
         include.non.trading.days = FALSE,
         options = c("periodicitySelection" = "DAILY",
                     "nonTradingDayFillOption" = "ALL_CALENDAR_DAYS",
                     "nonTradingDayFillMethod" = "PREVIOUS_VALUE"))
print(p1)


p2 <- bdh(securities = "LAH00 Comdty", fields = fld,
          start.date = sdt, end.date = edt,
          include.non.trading.days = TRUE,
          options = c("periodicitySelection" = "DAILY",
                      "nonTradingDayFillOption" = "ALL_CALENDAR_DAYS",
                      "nonTradingDayFillMethod" = "PREVIOUS_VALUE"))
print(p2)


sdt <- as.Date("2000-01-28")
edt <- as.Date("2000-01-30")

p3 <- bdh(securities = "LAH00 Comdty", fields = fld,
          start.date = sdt, end.date = edt,
          include.non.trading.days = TRUE,
          options = c("periodicitySelection" = "DAILY",
                      "nonTradingDayFillOption" = "ALL_CALENDAR_DAYS",
                      "nonTradingDayFillMethod" = "PREVIOUS_VALUE"))
print(p3)


sdt <- as.Date("2000-05-28")
edt <- as.Date("2000-05-30")

p4 <- bdh(securities = c("LLF01 Comdty"), fields = fld,
         start.date = sdt, end.date = edt,
         include.non.trading.days = FALSE,
         options = c("periodicitySelection" = "DAILY",
                     "nonTradingDayFillOption" = "ALL_CALENDAR_DAYS",
                     "nonTradingDayFillMethod" = "PREVIOUS_VALUE"))
print(p4)

The Portfolio Data Request enables retrieval of change information and portfolio positions with respect to a specific date in order to see how current market movements have affected user's portfolio's constituent weights.

Hi, is it possible to set nonTradingDayFillMethod to make NA's as opposed to blanks in r using Rblpapi?

This will make rewriting the list as dataframe easier, as at the moment it doesn't read any data points for tickers with no closing prices for a given period.

I'm not sure if you accept feature requests but this is one :) It'd be great to have Windows support.

I don't have admin privileges on the Bloomberg terminal/machine -- I doubt many Bloomberg users will. It makes it tricky for seamless Rtools and Rcpp integration.
I tried:

require( devtools )
devtools::install_github( "Rblp/Rblpapi" )

It seems I'm unable to edit/append to the system path (Error creating registry key when installing Rtools).

Any plans to support Windows?

I found another interesting error message, certainly linked to an incorrect conversion of data type, from string of characters to date.

Rblpapi::bdp(securities = "NDSS022 CMPT Index", fields = "LAST_UPDATE")
# returns Error: Year is out of valid range: 1400..10000

and as long as "NDSS022 CMPT Index","LAST_UPDATE" is part of the query, the entire query fails: compare the two following code:

Rblpapi::bdp(securities = c("IP Index"), fields = "LAST_UPDATE") # works
Rblpapi::bdp(securities = c("IP Index","NDSS022 CMPT Index"), fields = "LAST_UPDATE") # returns the error

This is the result from Excel BBG API:

as you can see, we cannot assume that the date is there. For NDSS022 CMPT Index, we get only the time...

I looked at the code of bdp.cpp but couldn't find if this conversion is done in Rcpp or in the bdp.cpp itself. Is there a way to prevent this date conversion to occur? If we keep the data as string of characters, we don't assume that the output of this field in in datetime format, and we should be able to prevent the case where the date is not in a recognisable format.

packageVersion("Rblpapi")
[1] '0.2.2.6.2.3'


R.version                       
platform       i386-w64-mingw32            
arch           i386                        
os             mingw32                     
system         i386, mingw32               
status                                     
major          3                           
minor          2.0                         
year           2015                        
month          04                          
day            16                          
svn rev        68180                       
language       R                           
version.string R version 3.2.0 (2015-04-16)
nickname       Full of Ingredients

For historical index data in excel for a given date, I use this
=BDS("Indu Index","INDX_MWEIGHT_HIST","END_DATE_OVERRIDE=20150914")

When I use bds from this package:
bds("INDU INDEX", "INDX_MWEIGHT_HIST",c("END_DATE_OVERRIDE"="20150914") ), I see this message:

Error: Sub-element '(null)' does not exist.

Any ideas?