This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Phantom types on AnyVal's

17 replies
Jeff Olson
Joined: 2011-06-29,
User offline. Last seen 42 years 45 weeks ago.
I recently came across an interesting question on stackoverflow about mixing phantom types with AnyVal's [http://stackoverflow.com/questions/6358651/marking-primitive-types-with-phantom-types-in-scala]. It would be awesome if this was really allowed and if it worked it the way I would like it too. You could new strongly-typed AnyVal's without zero runtime overhead (such as boxing). Clearly the construction is allowed syntactically, but what the semantics should be are unclear. On the surface things seem to work as you might expect, but as you dig deeper things start to explode.

scala> trait IsEven
defined trait IsEven

scala> type EvenInt = Int with IsEven
defined type alias EvenInt

scala> def ensureEven(n: Int): EvenInt = { require(n % 2 == 0); n.asInstanceOf[EvenInt] }
ensureEven: (n: Int)EvenInt

scala> ensureEven(2)
res0: EvenInt = 2

scala> ensureEven(3)
java.lang.IllegalArgumentException: requirement failed
    at scala.Predef$.require(Predef.scala:145)
    at ...

So far so good. Although whether or not the asInstanceOf cast should work is dubious: we seem to creating an instance of a type which is simultaneously an AnyVal and an AnyRef.

scala> def prove[P](implicit proof: P) = true
prove: [P](implicit proof: P)Boolean

scala> prove[EvenInt <:< Int]
res1: Boolean = true

scala> prove[EvenInt <:< IsEven]
res2: Boolean = true

scala> prove[EvenInt <:< AnyVal]
res3: Boolean = true

scala> prove[EvenInt <:< AnyRef]
res4: Boolean = true

scala> prove[EvenInt =:= Int]
<console>:11: error: Cannot prove that EvenInt =:= Int.
              prove[EvenInt =:= Int]
                   ^

scala> prove[EvenInt =:= IsEven]
<console>:11: error: Cannot prove that EvenInt =:= IsEven.
              prove[EvenInt =:= IsEven]
                   ^

scala> prove[Nothing =:= EvenInt]
<console>:11: error: Cannot prove that Nothing =:= EvenInt.
              prove[Nothing =:= EvenInt]
                   ^
 Again the scala compiler seems to think we have created a type which is both an AnyVal and an AnyRef, but not Nothing. Playing with isInstanceOf gives different answers however:

scala> val n = ensureEven(8)
n: EvenInt = 8

scala> n.isInstanceOf[EvenInt]
res5: Boolean = true

scala> n.isInstanceOf[Int]
res6: Boolean = true

scala> n.isInstanceOf[IsEven]
res7: Boolean = false

Oops. That last on looks wrong, but asInstanceOf agrees:

scala> n.asInstanceOf[IsEven]
java.lang.ClassCastException: java.lang.Integer cannot be cast to IsEven

It appears to autobox n as a java.lang.Integer and then cast which is clearly not going to work. I get the same problem if I try to define any methods on my marker trait and then call them.

trait IsEven { self: Int =>
  def half: Int = self/2
}

type EvenInt = Int with IsEven

4.asInstanceOf[EvenInt].half // *BOOM* java.lang.ClassCastException: java.lang.Integer cannot be cast to IsEven

Okay, maybe I should have expected that one. More problems arise if I try to define a custom boxing function:

scala> class EvenInteger(val value: EvenInt)
defined class EvenInteger

scala> implicit def boxEven(n: EvenInt) = new EvenInteger(n)
boxEven: (n: EvenInt)EvenInteger

scala> n.getClass
res10: java.lang.Class[Int] = class java.lang.Integer

The compiler seems to prefer the standard boxing function (Predef.int2Integer) even though mine is more specific.

What about arrays? If n is an EvenInt, then Array(n) gives me an Array[Int]. Good but Array[EvenInt] would have been better. Alas:

scala> Array[EvenInt](n)
<console>:16: error: type mismatch;
 found   : scala.reflect.AnyValManifest[Int]
 required: scala.reflect.ClassManifest[EvenInt]
Note: Int >: EvenInt, but trait ClassManifest is invariant in type T.
You may wish to investigate a wildcard type such as `_ >: EvenInt`. (SLS 3.2.10)
              Array[EvenInt](n)
                            ^
What about lists? I can create a List[EvenInt] fine by doing n::Nil but the standard companion object construction explodes:

scala> List(n)
java.lang.ClassCastException: [I cannot be cast to [Ljava.lang.Object;

Alas. I could continue, but I think I've exposed enough issues to be getting on with. If phantom types are going to be allowed on AnyVal's then there clearly are many bugs to be worked out (not to mention semantics to be defined). On the other hand, if this construction is not going to be allowed, maybe we should teach the compiler that Int with Foo =:= Nothing and have 2.asInstanceOf[Int with Foo] throw an exception whenever Foo is an AnyRef.

Thoughts?

-Jeff




extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Phantom types on AnyVal's

On Wed, Oct 5, 2011 at 3:15 PM, Jeff Olson wrote:
> I recently came across an interesting question on stackoverflow about mixing
> phantom types with AnyVal's
> [http://stackoverflow.com/questions/6358651/marking-primitive-types-with-phantom-types-in-scala].
> It would be awesome if this was really allowed and if it worked it the way I
> would like it too. You could new strongly-typed AnyVal's without zero
> runtime overhead (such as boxing).

Coincidentally I spent a few hours trying to implement this a couple
days ago. It was harder than I hoped, but I don't see any fundamental
obstacles to it working. The visible obstacle is that it would take
me a long time to unmake the embedded assumptions about Holy AnyVal
and his Nine And Only Nine Disciples.

> On the other hand, if this construction is not going to be
> allowed, maybe we should teach the compiler that Int with Foo =:= Nothing
> and have 2.asInstanceOf[Int with Foo] throw an exception whenever Foo is an
> AnyRef.

Can't teach the compiler that because it's not true. (Maybe you meant
that less than literally.)

scala> def f[T >: Int with String] = new (T => T) { def apply(x: T) = x }
f: [T >: Int with String]=> Object with T => T

scala> f[Int](5)
res0: Int = 5

scala> f[String]("abc")
res1: String = abc

scala> f[Any]("abc")
res2: Any = abc

scala> f[List[Int]](Nil)
:9: error: type arguments [List[Int]] do not conform to
method f's type parameter bounds [T >: Int with String]
f[List[Int]](Nil)
^

Types still exist even if they're uninhabited. Let's see here...

scala> def f1[T1 >: Int with String, T2 >: String with Float](x: T1
with T2)(implicit ev: T1 =:= T2) = true
f1: [T1 >: Int with String, T2 >: String with Float](x: T1 with
T2)(implicit ev: =:=[T1,T2])Boolean

scala> f1(5)
:9: error: Cannot prove that Int =:= AnyVal.
f1(5)
^

scala> f1(5.0f)
:9: error: Cannot prove that AnyVal =:= Float.
f1(5.0f)
^

scala> f1("abc")
res11: Boolean = true

Anyway, there are indeeed plenty of bugs in this region (there are
also tickets with commentary) and yet-to-be-properly-specified bits
even in the parts you can get at now. But I don't think fixing the
bugs would be enough to get anywhere good, we'll need better language
and compiler support.

f.esser
Joined: 2009-11-23,
User offline. Last seen 42 years 45 weeks ago.
Re: Phantom types on AnyVal's
Hello,
this reminds me somehow of the talk of Miles Sablin
http://skillsmatter.com/podcast/scala/encoding-unboxed-union-types-in-scala
on union types.
best regardsesser

2011/10/6 Paul Phillips <paulp [at] improving [dot] org>
On Wed, Oct 5, 2011 at 3:15 PM, Jeff Olson <jdolson [at] rgmadvisors [dot] com> wrote:
> I recently came across an interesting question on stackoverflow about mixing
> phantom types with AnyVal's
> [http://stackoverflow.com/questions/6358651/marking-primitive-types-with-phantom-types-in-scala].
> It would be awesome if this was really allowed and if it worked it the way I
> would like it too. You could new strongly-typed AnyVal's without zero
> runtime overhead (such as boxing).

Coincidentally I spent a few hours trying to implement this a couple
days ago.  It was harder than I hoped, but I don't see any fundamental
obstacles to it working.  The visible obstacle is that it would take
me a long time to unmake the embedded assumptions about Holy AnyVal
and his Nine And Only Nine Disciples.

> On the other hand, if this construction is not going to be
> allowed, maybe we should teach the compiler that Int with Foo =:= Nothing
> and have 2.asInstanceOf[Int with Foo] throw an exception whenever Foo is an
> AnyRef.

Can't teach the compiler that because it's not true.  (Maybe you meant
that less than literally.)

scala> def f[T >: Int with String] = new (T => T) { def apply(x: T) = x }
f: [T >: Int with String]=> Object with T => T

scala> f[Int](5)
res0: Int = 5

scala> f[String]("abc")
res1: String = abc

scala> f[Any]("abc")
res2: Any = abc

scala> f[List[Int]](Nil)
<console>:9: error: type arguments [List[Int]] do not conform to
method f's type parameter bounds [T >: Int with String]
             f[List[Int]](Nil)
              ^

Types still exist even if they're uninhabited.  Let's see here...


scala> def f1[T1 >: Int with String, T2 >: String with Float](x: T1
with T2)(implicit ev: T1 =:= T2) = true
f1: [T1 >: Int with String, T2 >: String with Float](x: T1 with
T2)(implicit ev: =:=[T1,T2])Boolean

scala> f1(5)
<console>:9: error: Cannot prove that Int =:= AnyVal.
             f1(5)
               ^

scala> f1(5.0f)
<console>:9: error: Cannot prove that AnyVal =:= Float.
             f1(5.0f)
               ^

scala> f1("abc")
res11: Boolean = true


Anyway, there are indeeed plenty of bugs in this region (there are
also tickets with commentary) and yet-to-be-properly-specified bits
even in the parts you can get at now.  But I don't think fixing the
bugs would be enough to get anywhere good, we'll need better language
and compiler support.



--
Friedrich EsserHAW HamburgDepartment Informatik
milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Phantom types on AnyVal's

On Thu, Oct 6, 2011 at 9:36 AM, friedrich esser
wrote:
> this reminds me somehow of the talk of Miles Sablin
> http://skillsmatter.com/podcast/scala/encoding-unboxed-union-types-in-scala
> on union types.

Actually, this trick I discovered (unboxed "tagging" of types, value
types included),

https://gist.github.com/89c9b47a91017973a35f

is probably a little closer to where Jeff started from.

But certainly, like Paul, I discuss the possibility that a legal type
might be uninhabited in the unboxed union types talk.

Cheers,

Miles

Jeff Olson
Joined: 2011-06-29,
User offline. Last seen 42 years 45 weeks ago.
Re: Phantom types on AnyVal's

On Wednesday, October 5, 2011 8:04:02 PM UTC-5, Paul Phillips wrote:
Types still exist even if they're uninhabited.  Let's see here...

An excellent point. I had not considered that.

So the question remains: when, if ever, is the type 'Int with Foo' inhabited? Since Int is marked as final, I would think never. Even if we allow 'Int with Foo' to be inhabited, it's not clear to me how one would ever go about creating a value of this type. According to the spec (at least my reading of it): 1.asInstanceOf[Int with Foo] should always throw a CCE whenever Int does not conform to Foo.

(curiously I note that 1.asInstanceOf[Int with String] succeeds whereas 1.asInstanceOf[String with Int] throws a CCE)


 
extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Phantom types on AnyVal's

On Thu, Oct 6, 2011 at 2:16 PM, Jeff Olson wrote:
> So the question remains: when, if ever, is the type 'Int with Foo'
> inhabited? Since Int is marked as final, I would think never.

Nope. Here are some you know are inhabited:

Int with AnyVal
Int with Any
Int with Any with AnyVal
Int with T forSome { type T >: Int }

Here's a less obvious one, illustrative of why Int being final is only
worth so much:

Int with NotNull

> (curiously I note that 1.asInstanceOf[Int with String] succeeds whereas
> 1.asInstanceOf[String with Int] throws a CCE)

Int with String erases to Int. String with Int erases to String.

f.esser
Joined: 2009-11-23,
User offline. Last seen 42 years 45 weeks ago.
Re: Phantom types on AnyVal's
Nope.  Here are some you know are inhabited:
 Int with AnyVal
 Int with Any
 Int with Any with AnyVal
 Int with T forSome { type T >: Int }

 Int with NotNull

From a mathematical standpoint (conjunction) all these types are Int's, as Int <:AnyVal,(Null is not a bottom type of Int), etc. The only ugly part (as always) is erasure.
best regardsesser
extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Phantom types on AnyVal's

On Thu, Oct 6, 2011 at 3:02 PM, friedrich esser
wrote:
> From a mathematical standpoint (conjunction) all these types are Int's, as
> Int <:AnyVal,
> (Null is not a bottom type of Int), etc. The only ugly part (as always) is
> erasure.

Yes, "with" is not commutative. But if those examples do not satisfy
you, here is an intersection which is inhabited, which does not reduce
to Int, and which certainly can't be replaced with Nothing.

scala> def f(x: Int with Singleton) = "thank you for that very specific Int"
f: (x: Int with Singleton)String

scala> f(5)
res0: String = thank you for that very specific Int

scala> var x = 5
x: Int = 5

scala> f(x)
:10: error: type mismatch;
found : Int
required: Int with Singleton
f(x)
^

f.esser
Joined: 2009-11-23,
User offline. Last seen 42 years 45 weeks ago.
Re: Phantom types on AnyVal's
I don't like nitpicking, but f accepts all Int values. 
Something like a mutable var x= 5 is mathematically nonsense.But nevertheless, I did not know that neat trick/example with object Singleton.
Actually I only responded to this thread to support Miles (Sablin)idea to have a more orthogonal type system in Scala.
best regardsesser
2011/10/7 Paul Phillips <paulp [at] improving [dot] org>
On Thu, Oct 6, 2011 at 3:02 PM, friedrich esser
<esser [dot] friedrich [at] gmail [dot] com> wrote:
> From a mathematical standpoint (conjunction) all these types are Int's, as
> Int <:AnyVal,
> (Null is not a bottom type of Int), etc. The only ugly part (as always) is
> erasure.

Yes, "with" is not commutative.  But if those examples do not satisfy
you, here is an intersection which is inhabited, which does not reduce
to Int, and which certainly can't be replaced with Nothing.


scala> def f(x: Int with Singleton) = "thank you for that very specific Int"
f: (x: Int with Singleton)String

scala> f(5)
res0: String = thank you for that very specific Int

scala> var x = 5
x: Int = 5

scala> f(x)
<console>:10: error: type mismatch;
 found   : Int
 required: Int with Singleton
             f(x)
               ^



extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Phantom types on AnyVal's

On Fri, Oct 7, 2011 at 12:31 AM, friedrich esser
wrote:
> I don't like nitpicking, but f accepts all Int values.

You are not correct.

scala> def f(x: Int with Singleton) = "thank you for that very specific Int"
f: (x: Int with Singleton)String

scala> f(5: Int)
:9: error: type mismatch;
found : Int
required: Int with Singleton
f(5: Int)
^

Or is "5: Int" not an Int value.

> Something like a mutable var x= 5 is mathematically nonsense.

I don't know about that, but it was just an example.

Todd Vierling
Joined: 2011-04-27,
User offline. Last seen 42 years 45 weeks ago.
Re: Phantom types on AnyVal's
On Friday, October 7, 2011 3:31:33 AM UTC-4, Esser wrote:
I don't like nitpicking, but f accepts all Int values. 

Based on compiler inference rules. Whether this is a bug in inference or intentional leniency is up for debate.
Something like a mutable var x= 5 is mathematically nonsense.

Scala is a hybrid imperative/functional language. You can have variables that are mutable; that's irrelevant to this discussion (and I'm not sure why you brought this up).
However, the fact that it has different behavior, for a var versus a val, leads me to think this really is a bug. At call time for f(), its argument is effectlvely a val at that point (read-only), so whatever behavior is desired should be consistent.
f.esser
Joined: 2009-11-23,
User offline. Last seen 42 years 45 weeks ago.
Re: Phantom types on AnyVal's
Well, 
only immutable values are Int's in the mathematical sense. 
10 is 10 and cannot change to 11.This is expressed by
val x= 10
So let's play:
scala> object Singletondefined module Singleton
scala> def f(x: Int with Singleton) = "thank you for that very specific Int" f: (x: Int with Singleton)java.lang.String
scala> f(1)res0: java.lang.String = thank you for that very specific Int
scala> f(1000000)  // or any Int val res1: java.lang.String = thank you for that very specific Int
scala> val x= 1000000x: Int = 1000000
scala> f(x)res2: java.lang.String = thank you for that very specific Int
scala> var x= 1000000x: Int = 1000000
scala> f(x)<console>:10: error: type mismatch; found   : Int required: Int with Singleton               f(x)                ^
What is wrong with that. Behave like expected.
best regardsesser
dcsobral
Joined: 2009-04-23,
User offline. Last seen 38 weeks 5 days ago.
Re: Phantom types on AnyVal's

On Fri, Oct 7, 2011 at 17:08, friedrich esser wrote:
> Well,
> only immutable values are Int's in the mathematical sense.

Are you confusing what mathematics call integers with the type Int?
"var x" has the type Int, which makes perfect mathematical sense
(where the branch of mathematics is type theory), and which is not the
same as the type "Int with Singleton".

> 10 is 10 and cannot change to 11.
> This is expressed by
> val x= 10
> So let's play:
> scala> object Singleton
> defined module Singleton
> scala> def f(x: Int with Singleton) = "thank you for that very specific Int"
> f: (x: Int with Singleton)java.lang.String
> scala> f(1)
> res0: java.lang.String = thank you for that very specific Int
> scala> f(1000000)  // or any Int val
> res1: java.lang.String = thank you for that very specific Int
> scala> val x= 1000000
> x: Int = 1000000
> scala> f(x)
> res2: java.lang.String = thank you for that very specific Int
> scala> var x= 1000000
> x: Int = 1000000
> scala> f(x)
> :10: error: type mismatch;
>  found   : Int
>  required: Int with Singleton
>               f(x)
>                 ^
> What is wrong with that. Behave like expected.
> best regards
> esser
>

f.esser
Joined: 2009-11-23,
User offline. Last seen 42 years 45 weeks ago.
Re: Phantom types on AnyVal's
I am not. I know that Int is a concrete type ( as x= 1000000*1000000/1000000 is notwhat we learned at school).
But a disagree that var x: Int makes perfect mathematical sense as the result of x= x+1 is mathematical nonsense. So stay with your type theory and let me hope for better times.
best regardsEsser

2011/10/8 Daniel Sobral <dcsobral [at] gmail [dot] com>
On Fri, Oct 7, 2011 at 17:08, friedrich esser <esser [dot] friedrich [at] gmail [dot] com> wrote:
> Well,
> only immutable values are Int's in the mathematical sense.

Are you confusing what mathematics call integers with the type Int?
"var x" has the type Int, which makes perfect mathematical sense
(where the branch of mathematics is type theory), and which is not the
same as the type "Int with Singleton".

> 10 is 10 and cannot change to 11.
> This is expressed by
> val x= 10
> So let's play:
> scala> object Singleton
> defined module Singleton
> scala> def f(x: Int with Singleton) = "thank you for that very specific Int"
> f: (x: Int with Singleton)java.lang.String
> scala> f(1)
> res0: java.lang.String = thank you for that very specific Int
> scala> f(1000000)  // or any Int val
> res1: java.lang.String = thank you for that very specific Int
> scala> val x= 1000000
> x: Int = 1000000
> scala> f(x)
> res2: java.lang.String = thank you for that very specific Int
> scala> var x= 1000000
> x: Int = 1000000
> scala> f(x)
> <console>:10: error: type mismatch;
>  found   : Int
>  required: Int with Singleton
>               f(x)
>                 ^
> What is wrong with that. Behave like expected.
> best regards
> esser
>



--
Daniel C. Sobral

I travel to the future all the time.



--
Friedrich EsserHAW HamburgDepartment Informatik
vpatryshev
Joined: 2009-02-16,
User offline. Last seen 1 year 24 weeks ago.
Re: Phantom types on AnyVal's
Oh, how beautiful. And resolves the eternal problem (in Java and the like) of attaching a unit to a value.

Thanks,
-Vlad


On Thu, Oct 6, 2011 at 3:31 AM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Thu, Oct 6, 2011 at 9:36 AM, friedrich esser
<esser [dot] friedrich [at] gmail [dot] com> wrote:
> this reminds me somehow of the talk of Miles Sablin
> http://skillsmatter.com/podcast/scala/encoding-unboxed-union-types-in-scala
> on union types.

Actually, this trick I discovered (unboxed "tagging" of types, value
types included),

 https://gist.github.com/89c9b47a91017973a35f

is probably a little closer to where Jeff started from.

But certainly, like Paul, I discuss the possibility that a legal type
might be uninhabited in the unboxed union types talk.

Cheers,


Miles

--
Miles Sabin
tel: +44 7813 944 528
gtalk: miles [at] milessabin [dot] com
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Phantom types on AnyVal's

On Sun, Oct 9, 2011 at 7:32 PM, Vlad Patryshev wrote:
> Oh, how beautiful. And resolves the eternal problem (in Java and the like)
> of attaching a unit to a value.

Except that, as noted in one of the comments on that gist, it's really
pretty useless for units of measure.

A much better application (which Jason has picked up for scalaz) is to
use it as a tagging mechanism which can drive implicit resolution (ie.
typeclass instance selection).

Cheers,

Miles

Simon Ochsenreither
Joined: 2011-07-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Phantom types on AnyVal's
I think tha UoM problem can be solved in a better way. metascala does some pretty neat things which I tried to make more general and it looks pretty good now. (Except two problems which are potential bugs...)
Jeff Olson
Joined: 2011-06-29,
User offline. Last seen 42 years 45 weeks ago.
Re: Phantom types on AnyVal's
I had not thought about NotNull and Singleton--both are interesting. I note, however, that both traits are treated specially by the compiler. Which still leaves open the question: Is it possible for a user-defined trait (say trait Foo) to have a non-empty intersection with Int? If so, what are the restrictions on the trait and how does one go about creating values of this type?

With the present compiler the answers appear to be

1) yes
2) the trait must be empty
3) def tag(x: Int): Int with Foo = x.asInstanceOf[Int with Foo]

Although, I'm not convinced these are the correct answers.

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland