This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Re: An idea for reducing typeclass-related boilerplate

18 replies
Ben Hutchison 3
Joined: 2009-11-02,
User offline. Last seen 42 years 45 weeks ago.

(returning to scala-debate after accidentally straying off-list)

On Tue, Jan 31, 2012 at 2:49 AM, Erik Osheim wrote:
> On Tue, Jan 31, 2012 at 12:15:34AM +1100, Ben Hutchison wrote:
>> While it's possible to use the same typeclass twice in a method for
>> different types, I think its uncommon. I've not encountered code like
>> your example - why do you bring 2 numerics of unrelated type together
>> and want to operate on both?

> In the case I'm talking about there'd be conflicts around things like
> Numeric[A].fromInt(i:Int):A versus Numeric[B].fromInt(i:Int):B as well
> as Numeric[A].zero:A versus Numeric[B].zero:B. In fact, I have existing
> code that would be affected by this feature. So it's not a hypothetical
> or a thought experiment.
>
> To be clear, I'm not talking about situations where you want to call a
> method on "t" but rather where you need to call a method on "T".

Right. Generally, "constructors" provided by type classes.

I agree the the imports-of-parameters syntax doesn't have a neat
answer for ambiguous constructors. However, Pavel's proposal also has
problems with ambiguity when multiple typeclasses have a member with
the same name. Eg how could this (admittedly contrived) example be
resolved (using the syntax outlined by Pavel in the first post of the
thread):

trait Parser[A] {def parse(s: String): A}
trait Reader[A] {def parse(s: String): A}

def example[A: Parser, Reader](a: A) = {
//want to invoke both the 'parse' methods of Reader and Parser, but
can't distinguish
A.parse(a) +A.parse(a)
}

-Ben

Ben Hutchison 3
Joined: 2009-11-02,
User offline. Last seen 42 years 45 weeks ago.
Re: An idea for reducing typeclass-related boilerplate

On Tue, Jan 31, 2012 at 9:04 AM, Ben Hutchison wrote:
> trait Parser[A] {def parse(s: String): A}
> trait Reader[A] {def parse(s: String): A}
>
> def example[A: Parser, Reader](a: A) = {
>  //want to invoke both the 'parse' methods of Reader and Parser, but
> can't distinguish
>  A.parse(a) +A.parse(a)
> }

Sorry, corrected example below:

def example[A: Parser, Reader](s: String) = {
 //want to invoke both the 'parse' methods of Reader and Parser, but
can't distinguish
 (A.parse(a), A.parse(a))
}

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: An idea for reducing typeclass-related boilerplate
You should read: this proposal: http://docs.scala-lang.org/sips/pending/inline-classes.html before other suggestions.  If this works out, we might be able to inline the helper classes for type-traits...  (my initial goal for inline classes).
- Josh

On Mon, Jan 30, 2012 at 5:08 PM, Ben Hutchison <brhutchison [at] gmail [dot] com> wrote:
On Tue, Jan 31, 2012 at 9:04 AM, Ben Hutchison <brhutchison [at] gmail [dot] com> wrote:
> trait Parser[A] {def parse(s: String): A}
> trait Reader[A] {def parse(s: String): A}
>
> def example[A: Parser, Reader](a: A) = {
>  //want to invoke both the 'parse' methods of Reader and Parser, but
> can't distinguish
>  A.parse(a) +A.parse(a)
> }

Sorry, corrected example below:

def example[A: Parser, Reader](s: String) = {
 //want to invoke both the 'parse' methods of Reader and Parser, but
can't distinguish
 (A.parse(a), A.parse(a))
}

Pavel Pavlov
Joined: 2011-12-01,
User offline. Last seen 42 years 45 weeks ago.
Re: An idea for reducing typeclass-related boilerplate

31 января 2012 г. 7:04 пользователь Josh Suereth <joshua [dot] suereth [at] gmail [dot] com> написал:
You should read: this proposal: http://docs.scala-lang.org/sips/pending/inline-classes.html before other suggestions.  If this works out, we might be able to inline the helper classes for type-traits...  (my initial goal for inline classes).
- Josh

 Hi Josh,

Thank you for the link.

This is the greatest news about inline classes I've even heard of!
In fact, when reading the previous line of proposals I was constantly asking myself: "Why they are using this escape/inline-based approach instead of erasure-based one"? Good to know that finally the things gone to the right way.

I will make my comments on this proposal here.

1. While in previous proposals `@inline` was just a compiler hint which does not alter language semantics, now it does. I wonder if it is too much for an innocent-looking annotation?
All the properties of inline classes (such as final-ness, structural equality etc.) can be clearly thinked of, if we replace the historical name "inline class" with just "value class".
So,
  @inline class Meter(val underlying: Double)
becames
  class Meter(val underlying: Double) extends AnyVal

Value class definition here clearly expresses that objects have no identity and thus they are compared structurally etc.
Class Meter here is similar in many important aspects to predefined value types: it has no notion of reference equality (`ne` & `eq`), it hasn't all that Java stuff (clone/finalize/synchronized/wait/notify).
I believe they are similar enough to share one language concept.
What's wonderful here is that Scala already has the essential properties of our desired inline classes in its type system so why don't use it?

2. I believe that representation (and maybe implementation) of the proposal can be simplified if we introduce synthetic methods for boxing/unboxing in value classes. This will let us to have distiction between source-level and compiler-generated operations (which have different meanings, see below on this). So, at the erasure phase we will insert `Meter.box(e)` and `Meter.unbox(e’)` instead of `new C(e.asInstanceOf[U])` and `e’.u.asInstanceOf[C$unboxed]`.
As for peephole optimizations, all we need to do is eliminate `box(unbox)` and `unbox(box)` pairs and then inline the remaining box/unbox calls.

Going further, we can allow user to override such auto-generated box/unbox methods. This will give very simple and elegant way to user-defined boxing/unboxing logic, which in several cases is desirable to get acceptable performance characteristics (think for example for boxing performance of user-defined Uint8 type vs. one of predefined Byte).

3. With special boxing/unboxing functions we can drop the restrictions on the contents of constructors/initialization statements.
I think it's important because I came across this before quite often (well, frankly speaking, we have implemented our private extended Java with value classes and we use them a lot for performance-critical systems programming tasks in my company).
You can place there some asserts for example.

This can be supported if compiler will convert all the constructors into an extension methods returning underlying type and create one synthetic constructor for use with boxing.

4. I wonder if such generalized boxing/erasure logic in the compiler can be futher generalized to support other performance-problematic Scala features, such as structural types (I wrote about it earlier), or for unboxed Options (my sweetest dream about Scala).

WDYT?

Regards,
Pavel.

d_m
Joined: 2010-11-11,
User offline. Last seen 35 weeks 2 days ago.
Re: An idea for reducing typeclass-related boilerplate

On Mon, Jan 30, 2012 at 07:04:56PM -0500, Josh Suereth wrote:
> You should read: this proposal:
> http://docs.scala-lang.org/sips/pending/inline-classes.html before other
> suggestions. If this works out, we might be able to inline the helper
> classes for type-traits... (my initial goal for inline classes).

Just want to say that this looks like the best thing ever.

In the paper I submitted to ScalaDays 2012 I talk about the need for
such a feature to avoid performance costs associated with implicit
wrappers... but it seems like you folks were way ahead of me! :)

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
SIP 15 - WAS Re: An idea for reducing typeclass-related boilerp

This feedback is good, it needs to be on the sips mailing list though so it gets discussed by the authors.

On Jan 31, 2012 1:45 AM, "Pavel Pavlov" <pavel [dot] e [dot] pavlov [at] gmail [dot] com> wrote:

31 января 2012 г. 7:04 пользователь Josh Suereth <joshua [dot] suereth [at] gmail [dot] com> написал:
You should read: this proposal: http://docs.scala-lang.org/sips/pending/inline-classes.html before other suggestions.  If this works out, we might be able to inline the helper classes for type-traits...  (my initial goal for inline classes).
- Josh

 Hi Josh,

Thank you for the link.

This is the greatest news about inline classes I've even heard of!
In fact, when reading the previous line of proposals I was constantly asking myself: "Why they are using this escape/inline-based approach instead of erasure-based one"? Good to know that finally the things gone to the right way.

I will make my comments on this proposal here.

1. While in previous proposals `@inline` was just a compiler hint which does not alter language semantics, now it does. I wonder if it is too much for an innocent-looking annotation?
All the properties of inline classes (such as final-ness, structural equality etc.) can be clearly thinked of, if we replace the historical name "inline class" with just "value class".
So,
  @inline class Meter(val underlying: Double)
becames
  class Meter(val underlying: Double) extends AnyVal

Value class definition here clearly expresses that objects have no identity and thus they are compared structurally etc.
Class Meter here is similar in many important aspects to predefined value types: it has no notion of reference equality (`ne` & `eq`), it hasn't all that Java stuff (clone/finalize/synchronized/wait/notify).
I believe they are similar enough to share one language concept.
What's wonderful here is that Scala already has the essential properties of our desired inline classes in its type system so why don't use it?

2. I believe that representation (and maybe implementation) of the proposal can be simplified if we introduce synthetic methods for boxing/unboxing in value classes. This will let us to have distiction between source-level and compiler-generated operations (which have different meanings, see below on this). So, at the erasure phase we will insert `Meter.box(e)` and `Meter.unbox(e’)` instead of `new C(e.asInstanceOf[U])` and `e’.u.asInstanceOf[C$unboxed]`.
As for peephole optimizations, all we need to do is eliminate `box(unbox)` and `unbox(box)` pairs and then inline the remaining box/unbox calls.

Going further, we can allow user to override such auto-generated box/unbox methods. This will give very simple and elegant way to user-defined boxing/unboxing logic, which in several cases is desirable to get acceptable performance characteristics (think for example for boxing performance of user-defined Uint8 type vs. one of predefined Byte).

3. With special boxing/unboxing functions we can drop the restrictions on the contents of constructors/initialization statements.
I think it's important because I came across this before quite often (well, frankly speaking, we have implemented our private extended Java with value classes and we use them a lot for performance-critical systems programming tasks in my company).
You can place there some asserts for example.

This can be supported if compiler will convert all the constructors into an extension methods returning underlying type and create one synthetic constructor for use with boxing.

4. I wonder if such generalized boxing/erasure logic in the compiler can be futher generalized to support other performance-problematic Scala features, such as structural types (I wrote about it earlier), or for unboxed Options (my sweetest dream about Scala).

WDYT?

Regards,
Pavel.

Justin du coeur
Joined: 2009-03-04,
User offline. Last seen 42 years 45 weeks ago.
Re: An idea for reducing typeclass-related boilerplate
Speaking purely as a "consumer" (that is, from the viewpoint of a programmer using the language, not someone who is trying to implement the compiler), and not claiming to have spent a lot of time thinking this one through yet:
2012/1/31 Pavel Pavlov <pavel [dot] e [dot] pavlov [at] gmail [dot] com>
1. While in previous proposals `@inline` was just a compiler hint which does not alter language semantics, now it does. I wonder if it is too much for an innocent-looking annotation?
All the properties of inline classes (such as final-ness, structural equality etc.) can be clearly thinked of, if we replace the historical name "inline class" with just "value class".
So,
  @inline class Meter(val underlying: Double)
becames
  class Meter(val underlying: Double) extends AnyVal

+1 -- this seems like a delightfully clear way of thinking about the problem.  Specifically, I like the semantic focus.  As a programmer, I shouldn't be thinking so much about inlining most of the time -- that's an optimization that generations of compilers are likely to continue improving on.  But I *should* be thinking about the notion of this type being a pure, copyable value.
So this tweak seems to put the focus where it belongs.  It broadens the notion of "value type" more clearly and explicitly, leaving the inlining to be the nice side-effect that it should be.  (Potentially influenced by things like compiler flags of how much to favor inlining of derived value types in general.)  I like it...
extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: An idea for reducing typeclass-related boilerplate

On Tue, Jan 31, 2012 at 9:56 AM, Justin du coeur wrote:
>> So,
>>   @inline class Meter(val underlying: Double)
>> becames
>>   class Meter(val underlying: Double) extends AnyVal
>
> +1 -- this seems like a delightfully clear way of thinking about the
> problem.  Specifically, I like the semantic focus.  As a programmer, I
> shouldn't be thinking so much about inlining most of the time -- that's an
> optimization that generations of compilers are likely to continue improving
> on.  But I *should* be thinking about the notion of this type being a pure,
> copyable value.

Can you guys clarify for me what it is you are proposing. You know
the syntax is already "@inline class Meter(val underlying: Double)
extends AnyVal", right? Are you saying that it should extend AnyVal
even if the underlying type is a reference type? Are you saying you
should be able to omit the @inline annotation and just extend AnyVal?
Are you saying some third thing?

Justin du coeur
Joined: 2009-03-04,
User offline. Last seen 42 years 45 weeks ago.
Re: An idea for reducing typeclass-related boilerplate
On Tue, Jan 31, 2012 at 1:15 PM, Paul Phillips <paulp [at] improving [dot] org> wrote:
Can you guys clarify for me what it is you are proposing.  You know the syntax is already "@inline class Meter(val underlying: Double)
extends AnyVal", right? Are you saying that it should extend AnyVal
even if the underlying type is a reference type? Are you saying you
should be able to omit the @inline annotation and just extend AnyVal?
Are you saying some third thing?

Well, Pavel should probably say what he's thinking, but I'm mainly struck by the idea of being able to extend AnyVal, which is much more interesting to me than the @inline annotation.  I'm not likely to use @inline very often in my code, but I *am* likely to think in terms of extending AnyVal -- it makes intuitive sense that that be possible, and it seems like something I would actually use somewhat frequently.
And given something that is already marked as a subclass of AnyVal, I'd usually rather see inlining treated purely as an optimization.  (On the usual theory that the compiler is smarter than I about such things.)
In other words, extending AnyVal is a very clear semantic statement at the language level about what this class *is*, and all sorts of long-term possibilities fall out of it -- for example, while it may not currently be in scope to ensure that the class is immutable, it is reasonable to hope that that sort of enforcement might eventually happen.  It's a broader concept than inlining, but seems like some good inlining opportunities might fall out of it.
So basically, I'm suggesting your second point: being able to extend AnyVal is fascinating, even if it were tightly restricted to begin with, and I'd usually rather leave inlining up to the compiler...
Pavel Pavlov
Joined: 2011-12-01,
User offline. Last seen 42 years 45 weeks ago.
Re: An idea for reducing typeclass-related boilerplate
Hi Paul,

среда, 1 февраля 2012 г. 1:15:59 UTC+7 пользователь Paul Phillips написал:
Can you guys clarify for me what it is you are proposing.  You know

the syntax is already "@inline class Meter(val underlying: Double)
extends AnyVal", right?

Yeah.
 

Are you saying that it should extend AnyVal
even if the underlying type is a reference type?

Exactly.

The same way as `class A(val x: Int)` and `class B(val x: String)` both are reference types, no matter which type is type of `x` field.
 

Are you saying you
should be able to omit the @inline annotation and just extend AnyVal?


Well @inline anotation became no-op then, so why retain boilerplate?

Are you saying some third thing?


Regards,
Pavel.
 
extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: An idea for reducing typeclass-related boilerplate

On Tue, Jan 31, 2012 at 10:25 AM, Pavel Pavlov wrote:
> The same way as `class A(val x: Int)` and `class B(val x: String)` both are
> reference types, no matter which type is type of `x` field.

I tried to find some basis upon which to dismiss this idea but failed.
I think it is a good idea. The spec might already allow it,
depending on how one reads "represented as objects": "Value classes
are classes whose instances are not represented as objects by the
underlying host system. All value classes inherit from class AnyVal."

Regardless of what the spec says, AnyVal as a construct is pretty
useless in its current form. It's something I'm always fighting off,
nothing I'm ever driving toward.

// stupid anyval
scala> List(List(1), List(1L))
res0: List[List[AnyVal]] = List(List(1), List(1))

Point being that regardless of what the spec says, I think we should
feel free to respecify.

Pavel Pavlov
Joined: 2011-12-01,
User offline. Last seen 42 years 45 weeks ago.
Re: An idea for reducing typeclass-related boilerplate
Justin,

I agree with every your word here.
In addition, just want to say that if we introduce value classes then even inline classes optimization became so natural and semantically obvious for them that we can stop thinking about it as of "optimization" and start thinking of it as of just one of native properties of value classes.

среда, 1 февраля 2012 г. 1:28:03 UTC+7 пользователь Justin du Coeur написал:
On Tue, Jan 31, 2012 at 1:15 PM, Paul Phillips <pa [dot] [dot] [dot] [at] improving [dot] org> wrote:
Can you guys clarify for me what it is you are proposing.  You know the syntax is already "@inline class Meter(val underlying: Double)
extends AnyVal", right? Are you saying that it should extend AnyVal
even if the underlying type is a reference type? Are you saying you
should be able to omit the @inline annotation and just extend AnyVal?
Are you saying some third thing?

Well, Pavel should probably say what he's thinking, but I'm mainly struck by the idea of being able to extend AnyVal, which is much more interesting to me than the @inline annotation.  I'm not likely to use @inline very often in my code, but I *am* likely to think in terms of extending AnyVal -- it makes intuitive sense that that be possible, and it seems like something I would actually use somewhat frequently.
And given something that is already marked as a subclass of AnyVal, I'd usually rather see inlining treated purely as an optimization.  (On the usual theory that the compiler is smarter than I about such things.)
In other words, extending AnyVal is a very clear semantic statement at the language level about what this class *is*, and all sorts of long-term possibilities fall out of it -- for example, while it may not currently be in scope to ensure that the class is immutable, it is reasonable to hope that that sort of enforcement might eventually happen.  It's a broader concept than inlining, but seems like some good inlining opportunities might fall out of it.
So basically, I'm suggesting your second point: being able to extend AnyVal is fascinating, even if it were tightly restricted to begin with, and I'd usually rather leave inlining up to the compiler...
Pavel Pavlov
Joined: 2011-12-01,
User offline. Last seen 42 years 45 weeks ago.
Re: An idea for reducing typeclass-related boilerplate
Justin,

I think the point on guaranteed AnyVal immutability is very important, thank you for mentionning it.

среда, 1 февраля 2012 г. 1:28:03 UTC+7 пользователь Justin du Coeur написал:

In other words, extending AnyVal is a very clear semantic statement at the language level about what this class *is*, and all sorts of long-term possibilities fall out of it -- for example, while it may not currently be in scope to ensure that the class is immutable, it is reasonable to hope that that sort of enforcement might eventually happen.  It's a broader concept than inlining, but seems like some good inlining opportunities might fall out of it.
Pavel Pavlov
Joined: 2011-12-01,
User offline. Last seen 42 years 45 weeks ago.
Re: An idea for reducing typeclass-related boilerplate


In addition, just want to say that if we introduce value classes then even inline classes optimization became so natural and semantically obvious for them that we can stop thinking about it as of "optimization" and start thinking of it as of just one of native properties of value classes.


And of course, we already have all this stuff with AnyVals! You don't think about `val x = 1; val y = x` as of "compiler-specific optimization of AnyVal objects", you just see that the value is copied from x to y.
Pavel Pavlov
Joined: 2011-12-01,
User offline. Last seen 42 years 45 weeks ago.
Re: An idea for reducing typeclass-related boilerplate


среда, 1 февраля 2012 г. 1:41:30 UTC+7 пользователь Paul Phillips написал:

Regardless of what the spec says, AnyVal as a construct is pretty
useless in its current form.  It's something I'm always fighting off,
nothing I'm ever driving toward.

// stupid anyval
scala> List(List(1), List(1L))
res0: List[List[AnyVal]] = List(List(1), List(1))

Point being that regardless of what the spec says, I think we should
feel free to respecify.

 Have you an idea how to fix this?

Pavel Pavlov
Joined: 2011-12-01,
User offline. Last seen 42 years 45 weeks ago.
[SIP-15 / value classes] Was: Re: An idea for reducing typeclas
As the discussion already crawled into many threads, I'm collecting all the references here:

https://groups.google.com/forum/?pli=1#!topic/scala-debate/_R88lZcb-xo
https://groups.google.com/forum/?pli=1#!topic/scala-sips/rHJAbqzzdXM
https://groups.google.com/forum/?pli=1#!topic/scala-sips/Axn0xBpNA30

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: An idea for reducing typeclass-related boilerplate

On Tue, Jan 31, 2012 at 11:10 AM, Pavel Pavlov wrote:
> Have you an idea how to fix this?

a) Never infer AnyVal (or Any) and force people to write it if they
want it. Then that expression wouldn't typecheck until weak
conformance kicked in and the Int was widened to match the long. You
could write List[AnyVal] or List[Any] if you wanted those things. If
you're using "List[Any]" with any serious frequency, you're doing it
wrong, so we'd only be encouraging better programming practice. In
fact, when List[Any] is inferred when I'm programming, it is more
often than not mistake.

// I forgot and thought :+ was list append
scala> List(1, 2, 3) :+ List(4)
res0: List[Any] = List(1, 2, 3, List(4))

b) Unflatten the AnyVal hierarchy. There's one parent and nine
sibling children. This is what makes it so useless: how often do you
want to abstract over Boolean and Unit along with the other seven? Let
Boolean and Unit be direct children and the others form some modest
hierarchy, such that we end up with List[List[Something]] where
Something is some subset of AnyVals excluding Boolean and Unit:
NumericAnyVal, IntegralAnyVal, I dunno. Anything would be an
improvement.

They wouldn't be subtypes of one another, just subtypes of more
specific parents.

(For the record, I don't actually forget and think :+ is list append.)

d_m
Joined: 2010-11-11,
User offline. Last seen 35 weeks 2 days ago.
Re: An idea for reducing typeclass-related boilerplate

On Tue, Jan 31, 2012 at 01:02:57PM -0800, Paul Phillips wrote:
> b) Unflatten the AnyVal hierarchy. There's one parent and nine
> sibling children. This is what makes it so useless: how often do you
> want to abstract over Boolean and Unit along with the other seven? Let
> Boolean and Unit be direct children and the others form some modest
> hierarchy, such that we end up with List[List[Something]] where
> Something is some subset of AnyVals excluding Boolean and Unit:
> NumericAnyVal, IntegralAnyVal, I dunno. Anything would be an
> improvement.

Yes please!

I would love to replace the annoying phrase "numeric AnyVal types" with
an actual sub-hierarchy like NumericAnyVal. For bonus points I could
imagine being able to specialize on parts of the AnyVal hierarchy,
e.g.:

import scala.{specialized => spec}

// specialize ALL the things!
def foo[@spec(AnyVal) T](a:T) = ...

// only numbers please
def bar[@spec(NumericAnyVal) T])(a:T) = ...

// ok i'm a masochist
def duh[@spec(Int, Long, Float, Double) T])(a:T) = ...

Pavel Pavlov
Joined: 2011-12-01,
User offline. Last seen 42 years 45 weeks ago.
Re: An idea for reducing typeclass-related boilerplate


среда, 1 февраля 2012 г. 2:10:07 UTC+7 пользователь Pavel Pavlov написал:

среда, 1 февраля 2012 г. 1:41:30 UTC+7 пользователь Paul Phillips написал:

Regardless of what the spec says, AnyVal as a construct is pretty
useless in its current form.  It's something I'm always fighting off,
nothing I'm ever driving toward.

// stupid anyval
scala> List(List(1), List(1L))
res0: List[List[AnyVal]] = List(List(1), List(1))

Point being that regardless of what the spec says, I think we should
feel free to respecify.

 Have you an idea how to fix this?

 
Well, I think there is something we can do about this right now:
We can add a check in the compiler which will catch all such type convertions (* -> AnyVal) and issue warnings on them.
Then we can collect all such warnings on some huge Scala codebase (I heard you have one) and then we'll see if there exist any usages of these convertions which are not definitely bugs.


Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland