merge spark dev branches about plyrmr HOT 2 CLOSED

merge is complete in spark-merge branch. Requires patches in rmr and SparkR.

from plyrmr.

the rmr patch has been merged. It reloads objects twice, so I am not too happy about it, but it was the simplest thing that worked. A pull request has been sent for the sparkR patch. The spark-merge branch is merged into the spark branch and this is published, and I also did a routine merge from 0.3.0 through dev. Open problems are:

• dependency on both rmr2 and SparkR. Can we use suggest to make them optional?
• keep the spark branch going in parallel or merge with dev and hide the spark changes for a while? On one hand there is no way we can support on the spark backend every feature, on the other a parallel branch is a maintenance burden and it has some features, like plyrmr options, that would be nice to have in dev independent of spark.

I am reorganizing the tests in the spark branch so that they can pass cleanly, with the ones that must be skipped in spark clearly marked. This will allow to perform testing according to standard procedures and also track progress in the support of features on spark (every time we exclude fewer tests, that's progress).

from plyrmr.