This page is no longer maintained — Please continue to the home page at www.scala-lang.org

zipped is not enough: we need zipWith (well, I want zipWith, anyway)

7 replies
Chris Marshall
Joined: 2009-06-17,
User offline. Last seen 44 weeks 3 days ago.
Is there a reason why there is no zipWith (or perhaps zipBy, zipTo) method in the collections hierarchy whose use-case looks like this:
def zipWith[AA >: A, B](f : AA => B): CC[(A, B)]

It's extremely common for me to do these things:
1. Apply a map-to function to turn CC[A] and A => B into a Map[A, B]
This looks much nicer as:
(cc zipWith f).toMap

Than
(cc map (a => a -> f(b))).toMap

2. Zip all the elements of a collection to a single element
cc zipWith ( _ => b)

Is much, much safer than:
cc zipAll (Seq(b), null, b)

Ugh! Look, there's a null in there! The arguments aren't even by-name for me to provide a sys.error in case the "null" ever gets used accidentally (i.e. I cannot do  cc zipAll (Seq(b), sys.error("Ugh, null"), b))
Looking in JIRA, Martin suggests https://issues.scala-lang.org/browse/SI-1512 that the issue is "fixed" by Tuple2.zipped but this seems wrong to me as it addresses neither of the above points (and it pertains to a slightly different signature of zipWith, suggested by Dan Spiewak)
Chris
Tony Morris 2
Joined: 2009-03-20,
User offline. Last seen 42 years 45 weeks ago.
Re: zipped is not enough: we need zipWith (well, I want zipWith

On 16/02/12 23:15, Chris Marshall wrote:
> Is there a reason why there is no zipWith (or perhaps zipBy, zipTo) method in the collections hierarchy whose use-case looks like this:
> def zipWith[AA >: A, B](f : AA => B): CC[(A, B)]
> It's extremely common for me to do these things:
> 1. Apply a map-to function to turn CC[A] and A => B into a Map[A, B]
> This looks much nicer as:
> (cc zipWith f).toMap
> Than
> (cc map (a => a -> f(b))).toMap
> 2. Zip all the elements of a collection to a single element
> cc zipWith ( _ => b)
> Is much, much safer than:
> cc zipAll (Seq(b), null, b)
> Ugh! Look, there's a null in there! The arguments aren't even by-name for me to provide a sys.error in case the "null" ever gets used accidentally (i.e. I cannot do cc zipAll (Seq(b), sys.error("Ugh, null"), b))
> Looking in JIRA, Martin suggests https://issues.scala-lang.org/browse/SI-1512 that the issue is "fixed" by Tuple2.zipped but this seems wrong to me as it addresses neither of the above points (and it pertains to a slightly different signature of zipWith, suggested by Dan Spiewak)
> Chris
zipWith generalises to any applicative functor. That's what you really
need/want.

Chris Marshall
Joined: 2009-06-17,
User offline. Last seen 44 weeks 3 days ago.
RE: zipped is not enough: we need zipWith (well, I want zipWith

(cc <*> f.pure[CC]).toMap

and
cc <*> Seq( (a: A) => a -> b) 

Or for the latter
def pairTo[A, B](b: B): A => (A, B) = a => a -> b

And then 
cc <*> pairTo(b).pure[CC]

That's nice, but I still think they deserve addition into the standard library
Chris

> Date: Thu, 16 Feb 2012 23:16:22 +1000
> From: tonymorris [at] gmail [dot] com
> To: scala-user [at] googlegroups [dot] com
> Subject: Re: [scala-user] zipped is not enough: we need zipWith (well, I want zipWith, anyway)
>
> On 16/02/12 23:15, Chris Marshall wrote:
> > Is there a reason why there is no zipWith (or perhaps zipBy, zipTo) method in the collections hierarchy whose use-case looks like this:
> > def zipWith[AA >: A, B](f : AA => B): CC[(A, B)]
> > It's extremely common for me to do these things:
> > 1. Apply a map-to function to turn CC[A] and A => B into a Map[A, B]
> > This looks much nicer as:
> > (cc zipWith f).toMap
> > Than
> > (cc map (a => a -> f(b))).toMap
> > 2. Zip all the elements of a collection to a single element
> > cc zipWith ( _ => b)
> > Is much, much safer than:
> > cc zipAll (Seq(b), null, b)
> > Ugh! Look, there's a null in there! The arguments aren't even by-name for me to provide a sys.error in case the "null" ever gets used accidentally (i.e. I cannot do cc zipAll (Seq(b), sys.error("Ugh, null"), b))
> > Looking in JIRA, Martin suggests https://issues.scala-lang.org/browse/SI-1512 that the issue is "fixed" by Tuple2.zipped but this seems wrong to me as it addresses neither of the above points (and it pertains to a slightly different signature of zipWith, suggested by Dan Spiewak)
> > Chris
> zipWith generalises to any applicative functor. That's what you really
> need/want.
>
> --
> Tony Morris
> http://tmorris.net/
>
>
ichoran
Joined: 2009-08-14,
User offline. Last seen 2 years 3 weeks ago.
Re: zipped is not enough: we need zipWith (well, I want zipWith
On Thu, Feb 16, 2012 at 8:15 AM, Chris Marshall <oxbow_lakes [at] hotmail [dot] com> wrote:
Is there a reason why there is no zipWith (or perhaps zipBy, zipTo) method in the collections hierarchy whose use-case looks like this:
def zipWith[AA >: A, B](f : AA => B): CC[(A, B)]

It's extremely common for me to do these things:
1. Apply a map-to function to turn CC[A] and A => B into a Map[A, B]
This looks much nicer as:
(cc zipWith f).toMap

Than
(cc map (a => a -> f(b))).toMap

It does look nicer, but you can add your own zipWith if you really need it to look that nice.  Otherwise I think it's pretty clear what's happening.  (Actually, why not add your own mapTo?)


2. Zip all the elements of a collection to a single element
cc zipWith ( _ => b)

What's wrong with cc map (_ -> b)?

  --Rex

Chris Marshall
Joined: 2009-06-17,
User offline. Last seen 44 weeks 3 days ago.
RE: zipped is not enough: we need zipWith (well, I want zipWith

> It does look nicer, but you can add your own zipWith if you really need it to look that nice.  Otherwise I think it's pretty clear what's happening.  (Actually, why not add your own mapTo?)

Actually I did but then removed it again when 2.8 arrived because I'm not keen on pimping the collections library for the same reason that Yammer isn't

> What's wrong with cc map (_ -> b)?

Nothing at all: I'm an idiot - I have had zips and zippers on the brain for the last week and now feel rather foolish

>   --Rex

Thanks
ichoran
Joined: 2009-08-14,
User offline. Last seen 2 years 3 weeks ago.
Re: zipped is not enough: we need zipWith (well, I want zipWith
On Thu, Feb 16, 2012 at 1:26 PM, Chris Marshall <oxbow_lakes [at] hotmail [dot] com> wrote:

> It does look nicer, but you can add your own zipWith if you really need it to look that nice.  Otherwise I think it's pretty clear what's happening.  (Actually, why not add your own mapTo?)

Actually I did but then removed it again when 2.8 arrived because I'm not keen on pimping the collections library for the same reason that Yammer isn't

I thought Yammer's point was that they didn't use collections because they were not consistently as fast as Java collections.  I didn't recall them not pimping collections for any reason.  Did you mean Yang Zhang's "true Scala complexity" post?

Anyway, the entry barrier to enriching collections is high, but once you understand the three or four critical pieces and accept that there's no getting around a decent gob of boilerplate for a conceptually almost trivial task, it's not that hard to actually use.  Here:

import collection.generic.CanBuildFrom
class MappableCollection[A,C](ca: C)(implicit c2s: C => Iterable[A]) {
  def mapTo[B,D[_,_] <: Map[_,_]](f: A => B)(implicit cbf: CanBuildFrom[Nothing,(A,B),D[A,B]]): D[A,B] = {
    val mapped = cbf()
    var i = c2s(ca).iterator
    while (i.hasNext) {
      val a = i.next
      mapped += (a -> f(a))
    }
    mapped.result
  }
}
implicit def collections_can_be_mapped[A, C[A]](ca: C[A])(implicit c2s: C[A] => Iterable[A]) =
  new MappableCollection[A,C[A]](ca)(c2s)
implicit def strings_can_be_mapped(s: String)(implicit c2s: String => Iterable[Char]) =
  new MappableCollection[Char,String](s)(c2s)

Took just a couple minutes of alterations from something I already had.

Now:

scala> List(1,2,3).mapTo(_.toString*2)
res32: scala.collection.immutable.Map[Int,String] = Map(1 -> 11, 2 -> 22, 3 -> 33)

scala> "wish" mapTo(_.toInt)
res33: scala.collection.immutable.Map[Char,Int] = Map(w -> 119, i -> 105, s -> 115, h -> 104)

I'd really rather the Scala developers work on adding something to simplify collection extensions and creation for common use cases than add things like this which we have the capability to add ourselves.

  --Rex

Chris Marshall
Joined: 2009-06-17,
User offline. Last seen 44 weeks 3 days ago.
RE: zipped is not enough: we need zipWith (well, I want zipWith

>  Did you mean Yang Zhang's "true Scala complexity" post?

Ah yes; I meant that. I totally agree with your example - i.e. it's really not that hard. At the time I thought it was not worth that amount of code to save a few characters given the addition of the toMap method (previously I'd hated the fact that you had to do Map(xs map (x => x -> f(x)).toSeq : _ *) - ugh). Also given that "map" was starting to mean "Functor" to me, I didn't like using it as part of the "mapTo" name
> I'd really rather the Scala developers work on adding something to simplify collection extensions and creation for common use cases than add things like this which we have the capability to add ourselves.

That and specializing the library. I think a working, specialized collections library would get a lot of people to sit up and take notice. At the moment, specialization feels a lot like we brought a Lion to deal with a rogue mouse when there's a hyena in the bedroom terrorising the missus.
>  --Rex

ichoran
Joined: 2009-08-14,
User offline. Last seen 2 years 3 weeks ago.
Re: zipped is not enough: we need zipWith (well, I want zipWith
On Thu, Feb 16, 2012 at 3:26 PM, Chris Marshall <oxbow_lakes [at] hotmail [dot] com> wrote:

> I'd really rather the Scala developers work on adding something to simplify collection extensions and creation for common use cases than add things like this which we have the capability to add ourselves.

That and specializing the library. I think a working, specialized collections library would get a lot of people to sit up and take notice. At the moment, specialization feels a lot like we brought a Lion to deal with a rogue mouse when there's a hyena in the bedroom terrorising the missus.

Well, the value added to functions is quite high, even if right now it's really only functions that benefit.

But I agree--specialized collections would be a big deal, and I don't see how to get there without significant language enhancements (maybe scala-virtualized or macros will do the trick).  In addition to the existing bugs and limitations in the implementation of specialization, there is an issue with code bloat for things like map and fold, and an issue with multiple dispatch eating up much of the savings that you thought you had from lack of boxing.

I attempted to create a decent mutable primitive collections library, and even with writing code generators to produce lots of annoying boilerplate in the library, I couldn't reliably both improve performance significantly and keep the code clean.  I never was satisfied that it was a promising direction.  I'm going to try again when more specialization bugs are fixed and some other appropriate feature appears (any of MyType, scala-virtualized, or macros would change things enough to make it worth having another go).

  --Rex

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland