Use Spire's typeclasses. about algebird HOT 23 CLOSED

Hey, thanks for thinking of Spire! If you have features you need from those type classes please let me know.

from algebird.

One difference is that we (Spire) don't parameterize vectors on the scalar type in Module, VectorSpace, etc. Is this a problem?

from algebird.

Spire seems to require 2.10, which probably makes it a non-starter for us for a while. Is extracting a 2.9-compatible version of the alegbra typeclasses that we could both depend on infeasible?

from algebird.

It is definitely possible to extract the algebra part. I was thinking of backporting all of Spire to 2.9.x but if that fails (or is prohibitive) we could definitely extract spire.algebra.

from algebird.

If I was working on a 2.9 version, do I need to target 2.9.0? Could I target something more recent instead? (I'm not sure if there is any significant drift, maybe 2.9.0 will be just as easy. I'm just curious.)

from algebird.

AFAIK everyone using algebird is on either 2.9.1 or 2.9.2.

from algebird.

Great, thanks!

from algebird.

OK, so https://github.com/non/spire/tree/2.9-backport is building and passing tests. I need to take some more time to make sure specialization is working, and do some documentation clean up. But it should be in good enough shape for some initial integration work.

I'll comment as soon as I have a reasonable 2.9.x milestone published.

from algebird.

Thanks for doing this. When the 2.9.2 milestone is available, let us know.

from algebird.

@non one question: we have a lot of monoids/groups etc.. On built in collections. We resolve these implicitly by putting them in the Monoids object. Will you accept the same in spire so we can get easy implicit resolution for built in java and scala types?

from algebird.

Sure. As long as it has a well-defined identity/operator (which pretty much all collections do) there should be no problem. I'll take a look at what Algebird has and go about adding them.

from algebird.

@non thanks. PS: be careful with the map-(semigroup/ring). The means of joining the keys matters a lot for performance.

Also take a look at: https://github.com/twitter/algebird/blob/develop/src/main/scala/com/twitter/algebird/GeneratedAbstractAlgebra.scala

Those are generated by a script.

We use these pretty heavily in scalding.

from algebird.

I saw that and will make sure we have the necessary generated tuple-versions. Thanks!

from algebird.

I'll add benchmarks in Spire for Map semigroup op too and make sure there are no perf. regressions.

from algebird.

Hey, so we've implemented many more instances for Scala collections, including Map and tuples (the tuples are autogenerated on compile). We've made a tentative 0.3.0-RC1 release for 2.10 and will backport the changes shortly. Before we do the backport, would you mind taking a look and seeing if there is anything you feel we should change?

One thing that is different, is that we only do vector addition with Modules/VectorSpaces (so Vector(1, 2) + Vector(3, 4) === Vector(4, 6)). Default Monoid for all Seq-likes is concatenation (so Vector(1, 2) |+| Vector(3, 4) === Vector(1, 2, 3, 4)). You can use Module[List[Int], Int].additive to get a Monoid[List[Int]] that does vector addition. If you'd like to be able to do vector-style addition when the scalar is only in, eg, a Monoid, then we can implement an optional import for this.

The map implementation is like yours though, where, eg, Monoid[Map[K, V]] uses a Semigroup[V] to combine values.

https://github.com/non/spire/blob/2.10.0/src/main/scala/spire/algebra/map.scala
https://github.com/non/spire/blob/2.10.0/src/main/scala/spire/algebra/seq.scala
https://github.com/non/spire/blob/2.10.0/project/GenProductTypes.scala

from algebird.

I get 404 when I click the map/seq links above, and I'll take your word about the tuple generated ones (actually looking at the generated code is probably more fun).

About the Vector difference, I guess that's a fine approach too. The rule that we do element-wise operations on Vector but not on List was always a bit ad-hoc.

About the 2.9 backport... Honestly, my guess is that the internal cost of porting our code from algebird to spire will be significant enough that we probably won't get to it before we upgrade to 2.10. So, realistically, it might be better for you to save the effort of backporting if you are doing it on our behalf.

What do you think of some of the approximation structures? CountMinSketch, etc...?

Also, what is your policy about contributions? It looks like only 5 people have ever committed to the spire project. Do you welcome pull requests?

from algebird.

Now that I've (almost) completed the backport I will probably try to keep it up. There are places where Spire on 2.9 would be nice for me personally. But it's true that Spire is a lot nicer on 2.10.

I'll try to look at CountMinSketch soon, so I can say more about it. Maybe Tom has opinions?

About contributions: Spire is released under an MIT license. As long as the submitter is fine with that then we welcome pull requests from anyone.

P.S. The link for those source files actually changed to:

https://github.com/non/spire/blob/2.10.0/core/src/main/scala/spire/algebra/map.scala
https://github.com/non/spire/blob/2.10.0/core/src/main/scala/spire/algebra/seq.scala

from algebird.

To the extent that it makes sense to keep these projects separate (ie, assuming we aren't just trying to merge them), I think the approximation structures are definitely on the algebird side of the line... @tixxit and @non can comment, but it feels like Spire is more about setting up these typeclasses for existing Scala types, and Algebird is more about developing new, more exotic data structures that can also be tied into the same framework.

from algebird.

Yes, I agree with @avibryant -- Spire's focus is mostly on the type classes themselves and also new numeric types (ie. Complex, Rational, Real, FPFilter, Number, etc.), so more exotic/fun data structures may be a bit out of scope (though I am looking forward to Avi's talk at NEScala).

from algebird.

And, as Erik said, we definitely welcome PRs, as long as you are fine with the fairly liberal license.

from algebird.

So what happened? Everybody seemed in agreement, the Spire prep work was done but almost 2 years later the issue is stil open and there are no PRs.

from algebird.

So it's basically my fault. The algebra project is almost there but I never got everyone's complete agreement, and I also didn't do the Spire integration I needed.

It's on my radar and I'm hoping to get it working. Thanks for the nudge.

from algebird.

tackled by #523

from algebird.