This page is no longer maintained — Please continue to the home page at www.scala-lang.org

2.9.0.RC1 strangeness

24 replies
Maxime Lévesque
Joined: 2009-08-18,
User offline. Last seen 42 years 45 weeks ago.

In the process of trying to build Squeryl against 2.9.9RC1 I encountered what seems to be a compiler bug.
The following expression which makes use of implicit conversions fails to compile, the complaint from scalac is that
more than one implicit conv. applies (see below), then lists the two ambiguous conversions, but clearly they don't apply :

val e1 =
    from(statementInvocations)((si) =>
      groupBy(si.statementHash, si.statementHashCollisionNumber)
      compute(avg(si.executeTime), count, sum(si.executeTime), nvl(avg(si.rowCount),0)) //the nvl func is what causes the compilation failure (see below)
    )

Now something even more strange, this transformation which changes nothing at the type level makes the compiler happy :
   
val e2   
    from(statementInvocations)((si) => {
      val gb = groupBy(si.statementHash, si.statementHashCollisionNumber)
      val z = nvl(avg(si.rowCount),0)
      gb.compute(avg(si.executeTime), count, sum(si.executeTime), z) // by merely using a temp val, the compiler is now happy !!!
      }
    )

The error with e1 is :
error: type mismatch;
found   : org.squeryl.dsl.NvlFunctionNumerical[?A,?B]
required: org.squeryl.dsl.ast.TypedExpressionNode[?]
Note that implicit conversions are not applicable because they are ambiguous:
both method binaryOpConv1 in trait TypeArithmetic of type
(op: org.squeryl.dsl.BinaryAMSOp[org.squeryl.PrimitiveTypeMode.ByteType,org.squeryl.PrimitiveTypeMode.ByteType])
 org.squeryl.dsl.NumericalTypeConversion[org.squeryl.PrimitiveTypeMode.ByteType]
and method binaryOpConv2 in trait TypeArithmetic of type
(op: org.squeryl.dsl.BinaryAMSOp[org.squeryl.PrimitiveTypeMode.ByteType,org.squeryl.PrimitiveTypeMode.IntType])
 org.squeryl.dsl.NumericalTypeConversion[org.squeryl.PrimitiveTypeMode.IntType]
are possible conversion functions from org.squeryl.dsl.NvlFunctionNumerical[?A,?B] to org.squeryl.dsl.ast.TypedExpressionNode[?]
compute(avg(si.executeTime), count, sum(si.executeTime), nvl(avg(si.rowCount),0))

Now I usually don't submit bugs to the mailing list without extracting a minimal reproduction case
with minimal dependencies, but before spending time to extract a standalone reproduction, I wanted
to ask if there were some serious issues with implicit convs in 2.9.0RC1.
So here goes : is there a known issue in 2.9.0RC1 with implicit conversions ?

Maxime Lévesque
Joined: 2009-08-18,
User offline. Last seen 42 years 45 weeks ago.
Re: 2.9.0.RC1 strangeness
It's probably worth mentioning that the code has been compiling fine since 2.8.0.
Alex Cruise
Joined: 2008-12-17,
User offline. Last seen 2 years 26 weeks ago.
Re: Re: 2.9.0.RC1 strangeness
2011/4/7 Maxime Lévesque <maxime [dot] levesque [at] gmail [dot] com>
It's probably worth mentioning that the code has been compiling fine since 2.8.0.

Your question immediately reminded me of "git bisect" (http://www.kernel.org/pub/software/scm/git/docs/git-bisect.html) but building a Scala distribution takes so long that it wouldn't be fun to do that as such.  However, we do have a history of prebuilt nightly releases available in a few places:
http://scala-tools.org/repo-snapshots/org/scala-lang/scala-compiler/2.9.0-SNAPSHOT/ goes back to Jan. 4th
http://www.scala-lang.org/node/212/distributions goes back to Mar. 17th unfortunately.  The older builds might be available somewhere, but they're not listed.
Nobody really likes paying for disk space, but are there any other places where old non-release builds are archived?  Sometimes Hudson/Jenkins keeps build artifacts around, but I can't see any from my unauthenticated perspective.
-0xe1a
extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Re: 2.9.0.RC1 strangeness

On 4/7/11 4:21 PM, Alex Cruise wrote:
> Nobody really likes paying for disk space, but are there any other
> places where old non-release builds are archived? Sometimes
> Hudson/Jenkins keeps build artifacts around, but I can't see any from my
> unauthenticated perspective.

I have thousands (every svn rev from 16000ish to 21000ish, and I'll
bring it up to date sooner or later.) Sitting on a disk. Would like to
make it available somehow if it can be easy and not bankrupt me.

Maxime Lévesque
Joined: 2009-08-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: 2.9.0.RC1 strangeness

Ok, so should I open a ticket ?

On Thu, Apr 7, 2011 at 7:30 PM, Paul Phillips <paulp [at] improving [dot] org> wrote:
On 4/7/11 4:21 PM, Alex Cruise wrote:
Nobody really likes paying for disk space, but are there any other
places where old non-release builds are archived?  Sometimes
Hudson/Jenkins keeps build artifacts around, but I can't see any from my
unauthenticated perspective.

I have thousands (every svn rev from 16000ish to 21000ish, and I'll bring it up to date sooner or later.) Sitting on a disk.  Would like to make it available somehow if it can be easy and not bankrupt me.

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Re: 2.9.0.RC1 strangeness

On 4/7/11 4:48 PM, Maxime Lévesque wrote:
> Ok, so should I open a ticket ?

Not unless it's going to be simplified from the email. Some of us are
slow and need to save our spare brainpower for setting alarm clocks and
traversing revolving doors.

Alex Cruise
Joined: 2008-12-17,
User offline. Last seen 2 years 26 weeks ago.
Re: Re: 2.9.0.RC1 strangeness
On Thu, Apr 7, 2011 at 4:30 PM, Paul Phillips <paulp [at] improving [dot] org> wrote:
I have thousands (every svn rev from 16000ish to 21000ish, and I'll bring it up to date sooner or later.) Sitting on a disk.  Would like to make it available somehow if it can be easy and not bankrupt me.

How big is it?   (That's a rather personal question, isn't it sir?) 
http://aws.amazon.com/free/  - free for a year, comes with a Linux box you can do what you like with. ~8GB "disk" and 5GB on S3.  $14/month to rent the box after the year is up, or just turn it off, because who needs another server to admin?  S3 storage is $0.14/GB/month, transfer is free up to 1GB, and $0.15/GB after that.
http://www.box.net/pricing - 5GB free, RSchulz might conceivably be able to pull some strings and get us more.
https://www.dropbox.com/plans - free for 2GB (up to 8GB if you can con your downline into signing up for free accounts too), $10/month or $100/year for 50GB.  Superb desktop and mobile clients.
-0xe1a
Maxime Lévesque
Joined: 2009-08-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: 2.9.0.RC1 strangeness

Of course, If I open a ticket, it will include an "as minimal as possible" reproduction.
I was just asking in case someone would respond : "don't bother, something is seriously broken in the implicit convs. and we already know about it"


2011/4/7 Paul Phillips <paulp [at] improving [dot] org>
On 4/7/11 4:48 PM, Maxime Lévesque wrote:
Ok, so should I open a ticket ?

Not unless it's going to be simplified from the email.  Some of us are slow and need to save our spare brainpower for setting alarm clocks and traversing revolving doors.

Maxime Lévesque
Joined: 2009-08-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: 2.9.0.RC1 strangeness

Ok, I managed to isolate a self containted reproduction, here it is, see the comments
and the compilation error message :

object Zaza {

  trait TypedExpressionNodeZ[T]
  class NumericalExpressionZ[T] extends TypedExpressionNodeZ[T]
  class NonNumericalExpressionZ[T] extends TypedExpressionNodeZ[T]

  implicit def conv1(i: Option[Float]) = new NumericalExpressionZ[Option[Float]]
  implicit def conv2(i: Int) = new NumericalExpressionZ[Int]

  implicit def binaryOpConv1Z(op: BinaryAMSOpZ[Byte,Byte]): TypedExpressionNodeZ[Byte] = null
  implicit def binaryOpConv2Z(op: BinaryAMSOpZ[Byte,Int]): TypedExpressionNodeZ[Int] = null
  implicit def binaryOpConv1Z1(op: BinaryAMSOpZ[Float,Int]): TypedExpressionNodeZ[Float] = null


  class BinaryAMSOpZ[A1,A2](a1: NumericalExpressionZ[A1], a2: NumericalExpressionZ[A2])

  class NvlFunctionNumericalZ[A1,A2](a1: NumericalExpressionZ[A1], a2: NumericalExpressionZ[A2]) extends BinaryAMSOpZ[A1,A2](a1,a2)

  def nvlZ[A,B](a: NumericalExpressionZ[Option[A]], b: NumericalExpressionZ[B]) = new NvlFunctionNumericalZ[A,B](a.asInstanceOf[NumericalExpressionZ[A]],b)

  //Strange : commenting this line "fixes" compilation (side effect !?!?!)
  def nvlZ[A](a: NonNumericalExpressionZ[Option[A]], b: NonNumericalExpressionZ[A]) = null

  def computeZ2[T1](e1: TypedExpressionNodeZ[T1]) = {}


  val q = nvlZ(Some(4F), 0)
  //Strange : this compiles :
  computeZ2(q)

  // while this doesn't :
  computeZ2(nvlZ(Some(4F), 0))
}



2011/4/7 Maxime Lévesque <maxime [dot] levesque [at] gmail [dot] com>

Of course, If I open a ticket, it will include an "as minimal as possible" reproduction.
I was just asking in case someone would respond : "don't bother, something is seriously broken in the implicit convs. and we already know about it"


2011/4/7 Paul Phillips <paulp [at] improving [dot] org>
On 4/7/11 4:48 PM, Maxime Lévesque wrote:
Ok, so should I open a ticket ?

Not unless it's going to be simplified from the email.  Some of us are slow and need to save our spare brainpower for setting alarm clocks and traversing revolving doors.


extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Re: 2.9.0.RC1 strangeness

On 4/7/11 6:55 PM, Maxime Lévesque wrote:
> def nvlZ[A,B](a: NumericalExpressionZ[Option[A]], b:
> NumericalExpressionZ[B]) = new
> NvlFunctionNumericalZ[A,B](a.asInstanceOf[NumericalExpressionZ[A]],b)
>
> //Strange : commenting this line "fixes" compilation (side effect !?!?!)
> def nvlZ[A](a: NonNumericalExpressionZ[Option[A]], b:
> NonNumericalExpressionZ[A]) = null

That's a big clue right there.

You're calling an overloaded method and the overloading resolution
depends on implicit conversions. Then to add a little zazz you're
making it infer the return types of the overloaded methods.

Commenting out that line fixes it because it removes the overload. As
it is, the inferred return type of that method will be "Null" which is
going to get you in trouble because it's more specific.

It compiles like this:

def nvlZ[A](a: NonNumericalExpressionZ[Option[A]], b:
NonNumericalExpressionZ[A]): AnyRef = null

Type inference doesn't perform an exhaustive search of the solution
space. I don't know for sure, but I don't think it's a bug, just
something which you used to get away with.

> val q = nvlZ(Some(4F), 0)
> //Strange : this compiles :
> computeZ2(q)
>
> // while this doesn't :
> computeZ2(nvlZ(Some(4F), 0))

Type inference is performed with an "expected type". When you assign to
a value like that it's easy to see a different result because the
expected type of the call to computeZ2 will be more precise. When it's
one big expression with an onion of inferred types and implicit
conversions and overload resolution, sometimes the ?s are going to come
out the other end.

If you give the second one the same advantage as the first one, namely
knowing what the type of q is:

computeZ2(nvlZ(Some(4F), 0): NvlFunctionNumericalZ[Float,Int])

then it will compile.

Also just FYI your code example is a good distance from being minimized.

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Re: 2.9.0.RC1 strangeness
As Paul suggests, I'd recommend removing the overload in nvlZ.   You might be able to get away with an implicit-based type dispatch here (hey, it works wonders at times), but that might make the "onions of implicits" larger and the likelihood of a ? explosion greater.
Implicit chaining works well when constructing a known type, but not so well when inferring an unknown type.
- Josh

2011/4/7 Maxime Lévesque <maxime [dot] levesque [at] gmail [dot] com>

Ok, I managed to isolate a self containted reproduction, here it is, see the comments
and the compilation error message :

object Zaza {

  trait TypedExpressionNodeZ[T]
  class NumericalExpressionZ[T] extends TypedExpressionNodeZ[T]
  class NonNumericalExpressionZ[T] extends TypedExpressionNodeZ[T]

  implicit def conv1(i: Option[Float]) = new NumericalExpressionZ[Option[Float]]
  implicit def conv2(i: Int) = new NumericalExpressionZ[Int]

  implicit def binaryOpConv1Z(op: BinaryAMSOpZ[Byte,Byte]): TypedExpressionNodeZ[Byte] = null
  implicit def binaryOpConv2Z(op: BinaryAMSOpZ[Byte,Int]): TypedExpressionNodeZ[Int] = null
  implicit def binaryOpConv1Z1(op: BinaryAMSOpZ[Float,Int]): TypedExpressionNodeZ[Float] = null


  class BinaryAMSOpZ[A1,A2](a1: NumericalExpressionZ[A1], a2: NumericalExpressionZ[A2])

  class NvlFunctionNumericalZ[A1,A2](a1: NumericalExpressionZ[A1], a2: NumericalExpressionZ[A2]) extends BinaryAMSOpZ[A1,A2](a1,a2)

  def nvlZ[A,B](a: NumericalExpressionZ[Option[A]], b: NumericalExpressionZ[B]) = new NvlFunctionNumericalZ[A,B](a.asInstanceOf[NumericalExpressionZ[A]],b)

  //Strange : commenting this line "fixes" compilation (side effect !?!?!)
  def nvlZ[A](a: NonNumericalExpressionZ[Option[A]], b: NonNumericalExpressionZ[A]) = null

  def computeZ2[T1](e1: TypedExpressionNodeZ[T1]) = {}


  val q = nvlZ(Some(4F), 0)
  //Strange : this compiles :
  computeZ2(q)

  // while this doesn't :
  computeZ2(nvlZ(Some(4F), 0))
}



2011/4/7 Maxime Lévesque <maxime [dot] levesque [at] gmail [dot] com>

Of course, If I open a ticket, it will include an "as minimal as possible" reproduction.
I was just asking in case someone would respond : "don't bother, something is seriously broken in the implicit convs. and we already know about it"


2011/4/7 Paul Phillips <paulp [at] improving [dot] org>
On 4/7/11 4:48 PM, Maxime Lévesque wrote:
Ok, so should I open a ticket ?

Not unless it's going to be simplified from the email.  Some of us are slow and need to save our spare brainpower for setting alarm clocks and traversing revolving doors.



Maxime Lévesque
Joined: 2009-08-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: 2.9.0.RC1 strangeness

The "real" code doesn't return null (the AnyRef is there for the sake of simplification) but a type that is not the same one as the other signature of nvlZ,
(and not a subtype) I'm trying to recreate the situation in my simplified example without having AnyRef as a return type.

but I don't understand why it is relevant, because at the call site of nlZ in this example, the overload resolution
is not ambiguous, since the params can *not* be (a: NonNumericalExpressionZ[Option[A]], b: NonNumericalExpressionZ[A])
since actual params are not of these types and there is no available implicit conv that will make them NonNumericalExpressionZ
so there is no way that the problematic signature of nvlZ can be considered a candidate for overload resolution.
Why I mean to say here is that the "wrong" nvlZ should get excluded and not cause problems, since it isn't a candidate based on the fact
that it's input args.

Now for this part :

  val q = nvlZ(Some(4F), 0)
  //Strange : this compiles :
  computeZ2(q)

  // while this doesn't :
  computeZ2(nvlZ(Some(4F), 0))

I fail to see how the 2 cases are different at the type level, I can totally see how the can differ in the implemetation of the type system
but maybe It's because I'm not knowledgeable enough of the language spec.

2011/4/8 Paul Phillips <paulp [at] improving [dot] org>
On 4/7/11 6:55 PM, Maxime Lévesque wrote:
  def nvlZ[A,B](a: NumericalExpressionZ[Option[A]], b:
NumericalExpressionZ[B]) = new
NvlFunctionNumericalZ[A,B](a.asInstanceOf[NumericalExpressionZ[A]],b)

  //Strange : commenting this line "fixes" compilation (side effect !?!?!)
  def nvlZ[A](a: NonNumericalExpressionZ[Option[A]], b:
NonNumericalExpressionZ[A]) = null

That's a big clue right there.

You're calling an overloaded method and the overloading resolution depends on implicit conversions.  Then to add a little zazz you're making it infer the return types of the overloaded methods.

Commenting out that line fixes it because it removes the overload.  As it is, the inferred return type of that method will be "Null" which is going to get you in trouble because it's more specific.

It compiles like this:

 def nvlZ[A](a: NonNumericalExpressionZ[Option[A]], b: NonNumericalExpressionZ[A]): AnyRef = null

Type inference doesn't perform an exhaustive search of the solution space.  I don't know for sure, but I don't think it's a bug, just something which you used to get away with.

  val q = nvlZ(Some(4F), 0)
  //Strange : this compiles :
  computeZ2(q)

  // while this doesn't :
  computeZ2(nvlZ(Some(4F), 0))

Type inference is performed with an "expected type".  When you assign to a value like that it's easy to see a different result because the expected type of the call to computeZ2 will be more precise.  When it's one big expression with an onion of inferred types and implicit conversions and overload resolution, sometimes the ?s are going to come out the other end.

If you give the second one the same advantage as the first one, namely knowing what the type of q is:

 computeZ2(nvlZ(Some(4F), 0): NvlFunctionNumericalZ[Float,Int])

then it will compile.

Also just FYI your code example is a good distance from being minimized.

Maxime Lévesque
Joined: 2009-08-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: 2.9.0.RC1 strangeness


I further simplified the thing, please bear with me Paul, I'll continue to simplify it, but
please let me know if you see something.

object ImplicitConvAmbiguity {

  class TypedExpressionNodeZ[T]
  class NumericalExpressionZ[T] extends TypedExpressionNodeZ[T]
  class NonNumericalExpressionZ[T] extends TypedExpressionNodeZ[T]

  implicit def conv1(i: Option[Float]) = new NumericalExpressionZ[Option[Float]]
  implicit def conv2(i: Int) = new NumericalExpressionZ[Int]
  implicit def conv3(op: BinaryAMSOpZ[Byte,Byte]) = new TypedExpressionNodeZ[Byte]
  implicit def conv4(op: BinaryAMSOpZ[Float,Int]) = new TypedExpressionNodeZ[Float]
  implicit def conv5(e: NvlFunctionNonNumericalZ[java.util.Date]) = new TypedExpressionNodeZ[java.util.Date] 

  class BinaryAMSOpZ[A1,A2]

  class NvlFunctionNumericalZ[A1,A2] extends BinaryAMSOpZ[A1,A2]

  class NvlFunctionNonNumericalZ[A]   
 
  def nvlZ[A,B](a: NumericalExpressionZ[Option[A]], b: NumericalExpressionZ[B]) = new NvlFunctionNumericalZ[A,B]

  def nvlZ[A](a: NonNumericalExpressionZ[Option[A]], b: NonNumericalExpressionZ[A]) = new NvlFunctionNonNumericalZ[A]

  def computeZ2[T1](e1: TypedExpressionNodeZ[T1]) = {}

  // compiler complains of ambiguity saying that conv3 and conv4 is applicable, for converting from NvlFunctionNumericalZ to TypedExpressionNodeZ
  // why is it even considering conv3 ?
  computeZ2(nvlZ(Some(4F), 0))
}


2011/4/8 Maxime Lévesque <maxime [dot] levesque [at] gmail [dot] com>

The "real" code doesn't return null (the AnyRef is there for the sake of simplification) but a type that is not the same one as the other signature of nvlZ,
(and not a subtype) I'm trying to recreate the situation in my simplified example without having AnyRef as a return type.

but I don't understand why it is relevant, because at the call site of nlZ in this example, the overload resolution
is not ambiguous, since the params can *not* be (a: NonNumericalExpressionZ[Option[A]], b: NonNumericalExpressionZ[A])
since actual params are not of these types and there is no available implicit conv that will make them NonNumericalExpressionZ
so there is no way that the problematic signature of nvlZ can be considered a candidate for overload resolution.
Why I mean to say here is that the "wrong" nvlZ should get excluded and not cause problems, since it isn't a candidate based on the fact
that it's input args.

Now for this part :

  val q = nvlZ(Some(4F), 0)
  //Strange : this compiles :
  computeZ2(q)

  // while this doesn't :
  computeZ2(nvlZ(Some(4F), 0))

I fail to see how the 2 cases are different at the type level, I can totally see how the can differ in the implemetation of the type system
but maybe It's because I'm not knowledgeable enough of the language spec.

2011/4/8 Paul Phillips <paulp [at] improving [dot] org>
On 4/7/11 6:55 PM, Maxime Lévesque wrote:
  def nvlZ[A,B](a: NumericalExpressionZ[Option[A]], b:
NumericalExpressionZ[B]) = new
NvlFunctionNumericalZ[A,B](a.asInstanceOf[NumericalExpressionZ[A]],b)

  //Strange : commenting this line "fixes" compilation (side effect !?!?!)
  def nvlZ[A](a: NonNumericalExpressionZ[Option[A]], b:
NonNumericalExpressionZ[A]) = null

That's a big clue right there.

You're calling an overloaded method and the overloading resolution depends on implicit conversions.  Then to add a little zazz you're making it infer the return types of the overloaded methods.

Commenting out that line fixes it because it removes the overload.  As it is, the inferred return type of that method will be "Null" which is going to get you in trouble because it's more specific.

It compiles like this:

 def nvlZ[A](a: NonNumericalExpressionZ[Option[A]], b: NonNumericalExpressionZ[A]): AnyRef = null

Type inference doesn't perform an exhaustive search of the solution space.  I don't know for sure, but I don't think it's a bug, just something which you used to get away with.

  val q = nvlZ(Some(4F), 0)
  //Strange : this compiles :
  computeZ2(q)

  // while this doesn't :
  computeZ2(nvlZ(Some(4F), 0))

Type inference is performed with an "expected type".  When you assign to a value like that it's easy to see a different result because the expected type of the call to computeZ2 will be more precise.  When it's one big expression with an onion of inferred types and implicit conversions and overload resolution, sometimes the ?s are going to come out the other end.

If you give the second one the same advantage as the first one, namely knowing what the type of q is:

 computeZ2(nvlZ(Some(4F), 0): NvlFunctionNumericalZ[Float,Int])

then it will compile.

Also just FYI your code example is a good distance from being minimized.


extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Re: 2.9.0.RC1 strangeness

On 4/8/11 8:30 AM, Maxime Lévesque wrote:
> I further simplified the thing, please bear with me Paul, I'll
> continue to simplify it, but please let me know if you see
> something.

One aspect of minimizing is not having really long identifiers which are
meaningless in the context of the alleged bug. Having to continually
use my vision parsing component to distinguish "NonNumericalExpressionZ"
vs "NumericalExpressionZ" and "NvlFunctionNumericalZ" and
"NvlFunctionNonNumericalZ" is taxing and makes me a lot less inclined to
get to the interesting part. (I am happy for those among us who see
right through these things.)

Maxime Lévesque
Joined: 2009-08-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: 2.9.0.RC1 strangeness

Paul, I totally understand your point, I changed the naming sheme.
Whatever I can do to make your life simpler I'll do it, I don't want
to waste your time, it is far too precious for the Scala community.

There might be some simplifications left to do, but maybe you can already
see something, here's my reasoning :

The compiler says that conv3 and conv4 are applicable to resolve the actual param of bFunc.
My first interogation is why conv3 is considered as applicable.
To further convince myself that conv3 is not applicable, if I commented out conv4,
then the compiler finds no applicable conversions at all, not even conv3 that
was considered a candidate implicit conv.


object ImplicitConvAmbiguity {

  class N[T]
  class NE[T] extends N[T]
  class NN[T] extends N[T]
  class AA[A1,A2]
  class BB[A]   

  implicit def conv1(i: Option[Float]) = new NE[Option[Float]]
  implicit def conv2(i: Int) = new NE[Int]
  implicit def conv3(op: AA[Byte,Byte]) = new N[Byte]
  implicit def conv4(op: AA[Float,Int]) = new N[Float]
  implicit def conv5(e: BB[java.util.Date]) = new N[java.util.Date] 

 
  def aFunc[A,B](a: NE[Option[A]], b: NE[B]) = new AA[A,B]

  def aFunc[A](a: NN[Option[A]], b: NN[A]) = new BB[A]

  def bFunc[T1](e1: N[T1]) = {}

  bFunc(aFunc(Some(4F), 0))
}


2011/4/8 Paul Phillips <paulp [at] improving [dot] org>
On 4/8/11 8:30 AM, Maxime Lévesque wrote:
I further simplified the thing, please bear with me Paul, I'll
continue to simplify it, but please let me know if you see
something.

One aspect of minimizing is not having really long identifiers which are meaningless in the context of the alleged bug.  Having to continually use my vision parsing component to distinguish "NonNumericalExpressionZ" vs "NumericalExpressionZ" and "NvlFunctionNumericalZ" and "NvlFunctionNonNumericalZ" is taxing and makes me a lot less inclined to get to the interesting part.  (I am happy for those among us who see right through these things.)

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Re: 2.9.0.RC1 strangeness

On 4/8/11 9:40 AM, Maxime Lévesque wrote:
> Paul, I totally understand your point, I changed the naming sheme.
> Whatever I can do to make your life simpler I'll do it, I don't want
> to waste your time, it is far too precious for the Scala community.

Always nice to see such a stellar attitude! I spent much of today trying
to make the voluminous typer debug output into something readable.
It'll be pretty cool when I'm done, but for now let me show you which
implicits it goes looking for in a variety of scenarios, and that'll get
the brain juices flowing.

object Test {
class N[T]
class NE[T] extends N[T]
class NN[T] extends N[T]
class AA[A1,A2]
class BB[A]

implicit def conv1(i: Option[Float]) = new NE[Option[Float]]
implicit def conv2(i: Int) = new NE[Int]
implicit def conv3(op: AA[Byte,Byte]) = new N[Byte]
implicit def conv4(op: AA[Float,Int]) = new N[Float]
implicit def conv5(e: BB[java.util.Date]) = new N[java.util.Date]

def aFunc[A,B](a: NE[Option[A]], b: NE[B]) = new AA[A,B]
def aFunc[A](a: NN[Option[A]], b: NN[A]): AnyRef = new BB[A]
def bFunc[T1](e1: N[T1]) = {}

def f = bFunc(aFunc(Some(4F), 0))
}

// As-is.
//
// infer: Test.BB[?A] => Test.N[?]
// infer: Test.AA[?A,?B] => Test.N[?]
// infer: Some[Float] => Test.NN[Option[?A]]
// infer: => Some[Float] => Test.NN[Option[?A]]
// infer: Some[Float] => Test.NE[Option[?A]]
// infer: Int => Test.NE[?B]
// infer: Test.AA[?A,?B] => Test.N[?]
// [fail]

// Second aFunc commented out.
//
// infer: Some[Float] => Test.NE[Option[?]]
// infer: Int(0) => Test.NE[?]
// infer: Test.AA[Float,Int] => Test.N[?]
// [success]

// Second aFunc annotated with :AnyRef return type.
// infer: AnyRef => Test.N[?]
// infer: => AnyRef => Test.N[?]
// infer: Test.AA[?A,?B] => Test.N[?]
// infer: Some[Float] => Test.NE[Option[?]]
// infer: Int(0) => Test.NE[?]
// infer: Test.AA[Float,Int] => Test.N[?]
// [success]

// conv5 commented out.
//
// infer: Test.BB[?A] => Test.N[?]
// infer: => Test.BB[?A] => Test.N[?]
// infer: Test.AA[?A,?B] => Test.N[?]
// infer: Some[Float] => Test.NE[Option[?]]
// infer: Int(0) => Test.NE[?]
// infer: Test.AA[Float,Int] => Test.N[?]
// [success]

Maxime Lévesque
Joined: 2009-08-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: 2.9.0.RC1 strangeness

So is it a compiler bug ?

The way I see it, the compiler is contradicting itself :
1) the compiler says there is conversion ambiguity because conv3 and conv4 applies
2) when removing conv4, all of a sudden, no conversion apply, not even conv3 which was causing the ambiguity since it apparently was applicable.

Is the reasoning correct ?

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Re: 2.9.0.RC1 strangeness

On 4/8/11 5:25 PM, Maxime Lévesque wrote:
> The way I see it, the compiler is contradicting itself :
> 1) the compiler says there is conversion ambiguity because conv3 and
> conv4 applies
> 2) when removing conv4, all of a sudden, no conversion apply, not even
> conv3 which was causing the ambiguity since it apparently was applicable.

The compiler is entitled to not make all possible inferences, so it's
unlikely to be a bug in the official sense. The question is where on
the "desirable enhancement" scale it will fall. Based on the early
guesses it makes, conv3 and conv4 appear ambiguous. That's not an
ironclad representation that either of them will be applicable; errors
arise at many different points during compilation and the earlier places
don't necessarily have any knowledge of the later places.

> Is the reasoning correct ?

Here's what I see happening. It sets out to type this expression:

bFunc(aFunc(Some(4F), 0))

And it doesn't know much. The parameter to bFunc is N[T] for any T, so
it says OK, it's N[?]. And at that point it looks like you are already
in trouble, because the successful inference avenues all seem to involve
it pinning something down up front. This guy:

implicit def conv5(e: BB[java.util.Date]) = new N[java.util.Date]

pushes the unknowns out farther than it presently solves for. Now given
where we see it ending up, it could rule some stuff out and take another
swing at it. But that's not how things are done at present.

BTW, you really need martin or adriaan to comment if you want remotely
reliable information. I'm more using this as an excuse to rewrite some
extremely messy pieces of the typechecker (which has been a lot of fun)
and hoping to stumble upon a fractional increase in understanding
somewhere along the way.

Maxime Lévesque
Joined: 2009-08-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: 2.9.0.RC1 strangeness

2011/4/8 Paul Phillips <paulp [at] improving [dot] org>
On 4/8/11 5:25 PM, Maxime Lévesque wrote:
The way I see it, the compiler is contradicting itself :
1) the compiler says there is conversion ambiguity because conv3 and
conv4 applies
2) when removing conv4, all of a sudden, no conversion apply, not even
conv3 which was causing the ambiguity since it apparently was applicable.

The compiler is entitled to not make all possible inferences, so it's unlikely to be a bug in the official sense.  The question is where on the "desirable enhancement" scale it will fall. Based on the early guesses it makes, conv3 and conv4 appear ambiguous.


If it's a "desirable enhancement", then it has been an undocumented feature all along in 2.8.x ;-)

So it seems that the question boils down to how far the compiler is required to investigate an implicit conversion candidate before putting it in the "candidate list",
given that growing this list beyond 1 will cause compilation failure.

Here we have a case where the compiler says conv3 and conv4 "seem applicable based on early investigation" and declares a dead end,
while the same compiler on complete investigation declares that conv3 was *not* applicable, which we know to be correct.


Also, doesn't it suck that this compiles :

 val x = aFunc(Some(4F), 0)
 bFunc(x)

while this doesn't :

 bFunc(aFunc(Some(4F), 0))

?

I'm *very* surprised that these two expressions differ at the type level.

I often go about decomposing an expression into smaller chunks, working them out, and them recomposing them,
I don't remember in 2.8.x having the compiler interpret decompositions or compositions differently at the type level.


 
That's not an ironclad representation that either of them will be applicable; errors arise at many different points during compilation and the earlier places don't necessarily have any knowledge of the later places.

Is the reasoning correct ?

Here's what I see happening.  It sets out to type this expression:

 bFunc(aFunc(Some(4F), 0))

And it doesn't know much.  The parameter to bFunc is N[T] for any T, so it says OK, it's N[?].  And at that point it looks like you are already in trouble, because the successful inference avenues all seem to involve it pinning something down up front.  This guy:

 implicit def conv5(e: BB[java.util.Date]) = new N[java.util.Date]

pushes the unknowns out farther than it presently solves for.  Now given where we see it ending up, it could rule some stuff out and take another swing at it. But that's not how things are done at present.

BTW, you really need martin or adriaan to comment if you want remotely reliable information.  I'm more using this as an excuse to rewrite some extremely messy pieces of the typechecker (which has been a lot of fun) and hoping to stumble upon a fractional increase in understanding somewhere along the way.

Seth Tisue
Joined: 2008-12-16,
User offline. Last seen 34 weeks 3 days ago.
Re: Re: 2.9.0.RC1 strangeness

>>>>> "Maxime" == Maxime Lévesque writes:

Maxime> Also, doesn't it suck that this compiles :
Maxime> val x = aFunc(Some(4F), 0) bFunc(x)
Maxime> while this doesn't :
Maxime> bFunc(aFunc(Some(4F), 0))

Maxime> I often go about decomposing an expression into smaller chunks,
Maxime> working them out, and them recomposing them, I don't remember
Maxime> in 2.8.x having the compiler interpret decompositions or
Maxime> compositions differently at the type level.

Only semi-related, but: here's an example where "val x = a.b; x.c"
compiles but "a.b.c" doesn't (in 2.7, 2.8, and 2.9):

http://stackoverflow.com/questions/5544536/type-inference-on-set-failing/

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Re: 2.9.0.RC1 strangeness

On 4/9/11 6:23 AM, Maxime Lévesque wrote:
> If it's a "desirable enhancement", then it has been an undocumented
> feature all along in 2.8.x ;-)

You can think of it that way if you like. Everything which isn't
specified is unspecified. You can call a behavior outside the spec a
feature, a bug, a shazbot, a bumblefritz...

> Also, doesn't it suck that this compiles :
>
> val x = aFunc(Some(4F), 0)
> bFunc(x)
>
> while this doesn't :
>
> bFunc(aFunc(Some(4F), 0))

Nobody is suggesting it is a desirable feature that these have different
behaviors. If it's important to you that this not happen, I can propose
a "strict mode" where you have to use type annotations everywhere. You
will get very consistent behavior. Because the available alternative at
the present time is not more type inference, but less.

Also, as seth points out this is not new. There have been expressions
which evaluate differently if broken into subexpressions for as long as
I've been around. If you think about how the collections for instance
work (in that they choose a builder based on the expected type of the
result), this is inevitable.

Maxime Lévesque
Joined: 2009-08-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: 2.9.0.RC1 strangeness

I further simplified the thing,
now if there's a way to keep the 2 signatures of aFunc, all the implicit convs, and not require a type annotation at the call site of bFunc,
I'll have a solution, but it looks like a lot to ask.

object ImplicitConvAmbiguity2 {

  class N[T]
  class NE[T] extends N[T]
  class NN[T] extends N[T]
  class AA[A]
  class BB[A]

  implicit def conv1(i: Float) = new NE[Float]
  implicit def conv3(op: AA[java.util.TooManyListenersException]) = new N[java.util.TooManyListenersException]
  implicit def conv4(op: AA[Float]) = new N[Float]
  implicit def conv5(e: BB[java.util.GregorianCalendar]) = new N[java.util.GregorianCalendar]

  def aFunc[A](a: NE[A]) = new AA[A]

  def aFunc[A](a: NN[A]) = new BB[A]

  def bFunc[T](e1: N[T]) = {}

  bFunc(aFunc(4F))
}


2011/4/9 Paul Phillips <paulp [at] improving [dot] org>
On 4/9/11 6:23 AM, Maxime Lévesque wrote:
If it's a "desirable enhancement", then it has been an undocumented
feature all along in 2.8.x ;-)

You can think of it that way if you like.  Everything which isn't specified is unspecified.  You can call a behavior outside the spec a feature, a bug, a shazbot, a bumblefritz...

Also, doesn't it suck that this compiles :

 val x = aFunc(Some(4F), 0)
 bFunc(x)

while this doesn't :

 bFunc(aFunc(Some(4F), 0))

Nobody is suggesting it is a desirable feature that these have different behaviors.  If it's important to you that this not happen, I can propose a "strict mode" where you have to use type annotations everywhere.  You will get very consistent behavior.  Because the available alternative at the present time is not more type inference, but less.

Also, as seth points out this is not new.  There have been expressions which evaluate differently if broken into subexpressions for as long as I've been around.  If you think about how the collections for instance work (in that they choose a builder based on the expected type of the result), this is inevitable.

Erik Bruchez
Joined: 2010-02-25,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: 2.9.0.RC1 strangeness

>> Also, doesn't it suck that this compiles :
>>
>>  val x = aFunc(Some(4F), 0)
>>  bFunc(x)
>>
>> while this doesn't :
>>
>>  bFunc(aFunc(Some(4F), 0))
>
> Nobody is suggesting it is a desirable feature that these have different
> behaviors.  If it's important to you that this not happen, I can propose a
> "strict mode" where you have to use type annotations everywhere.  You will
> get very consistent behavior.  Because the available alternative at the
> present time is not more type inference, but less.
>
> Also, as seth points out this is not new.  There have been expressions which
> evaluate differently if broken into subexpressions for as long as I've been
> around.  If you think about how the collections for instance work (in that
> they choose a builder based on the expected type of the result), this is
> inevitable.

I hit a similar case yesterday with 2.8.1 and spent an hour or two
trying to work around it. In the end of course there were multiple
possible workarounds: explicit result types, intermediate variables,
explicit conversions, but as Maxime said it "sucks" ;)

Now this is complicated stuff to get right, but what I am wondering is
whether type inference can be improved in such cases, at least to a
degree, of if there are fundamental reasons that it couldn't (besides
the man-hours needed to work on it).

-Erik

Sciss
Joined: 2008-12-17,
User offline. Last seen 28 weeks 5 days ago.
Re: Re: 2.9.0.RC1 strangeness

i agree with you. but i have seen this already in 2.8.0. or 2.8.1. something like

call( a, b, x( y ))

doesn't work, but

call( a, b, { val c = x( y ); c })

does work. i can hardly see this as a "feature" of the compiler.

best, -sciss-

On 9 Apr 2011, at 14:23, Maxime Lévesque wrote:

> Also, doesn't it suck that this compiles :
>
> val x = aFunc(Some(4F), 0)
> bFunc(x)
>
> while this doesn't :
>
> bFunc(aFunc(Some(4F), 0))
>
> ?
>
> I'm *very* surprised that these two expressions differ at the type level.

H-star Development
Joined: 2010-04-14,
User offline. Last seen 2 years 26 weeks ago.
Re: Re: 2.9.0.RC1 strangeness

i've seen something similar, just the other way round (which makes
perfectly sense)
list.filter(e => e+1) // type if e inferred

val aFunc = e => e+1
list.filter(aFunc) // whoopsy

might type inference mess up something in your example?

On 9 Apr 2011, at 14:23, Maxime Lévesque wrote:
>> Also, doesn't it suck that this compiles :
>>
>> val x = aFunc(Some(4F), 0)
>> bFunc(x)
>>
>> while this doesn't :
>>
>> bFunc(aFunc(Some(4F), 0))
>>
>> ?
>>
>> I'm *very* surprised that these two expressions differ at the type level.
>

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland