This page is no longer maintained — Please continue to the home page at www.scala-lang.org

annotations or traits to better scale scala up

8 replies
Walter Smith
Joined: 2009-05-13,
User offline. Last seen 1 year 39 weeks ago.

Hi!

[this is more or less a repost directly to Nabble, as the original post to
scala-lang.org didn't work properly, as I was told]

everybody will agree that Scala scales great from medium sized applications
to small scripts, much better than esp. Java does. I often hear that it
scales up to large systems just as well, but does it? I think it does, but
could it be even better?

[This became a long post. If you don't want to read all the blurb, skip
ahead to the [*]]

Scala has some very powerful features that alleviate you from a lot of
constantly considering best practices. For example: How many equals and
hashCode methods have you seen that are just plain wrong (with sometimes
very subtle effects)? Case classes do that for free (and a lot more)! How
many singletons have you seen that try to be, but are *not* thread safe?!?
Scala objects are so right out of the box!

Things like this help a lot in small and big systems alike; they only scale
"linearily", i.e. the amount of best practices you don't have to take into
account any more by using Scala, stays constant per line of code.

What really does make a difference, though, is that Scala allows you to take
best practices out of your business code and encapsulate them directly into
your frameworks... and this is especially important for big systems. A
seemingly trivial example is that you can move the best practice of using
the fully qualified class name for a logger into a very simple trait:

trait Logging {
import org.slf4j._
val logger: Logger = LoggerFactory.getLogger(this.getClass)
}

This is not possible in Java, and it makes a real difference!

Some tools and frameworks will just take some time to grow as the community
grows: Code formatters (more important to big systems that to small!), code
analysis tools, refactoring and other IDE support, to name just a few. BTW:
I don't think that the module system suggested by Martin in "the" book
replaces the need for a real dependency injection framework: It doesn't take
the different build/deploy/run stages into account properly; and it misses
one important point: 'inversion of control', i.e. changing from pull to
push. But it is an interesting starting point and time will show.

Maybe some things in the language and/or compiler deserve some more pushing
for Scala to better scale up. For example type inference is just great for
smaller projects and client code, but for frameworks I think it's a good
practice to annotate at least the return type of a method -- that additional
piece of directly available information can be of great help. Without it you
sometimes have to look several method calls deep and/or look very closely.
This best practice is adhered to in some Scala library classes (e.g. List),
but not in all (e.g. Either). I think it would be a great help, if you could
make the compiler issue a warning, if it infers the return type of a method
for you. An IDE could even be configured to automatically add it on save.

Another example for language things to scale up better: Some types are not
meant to be used in a certain position. For example a HashMap should not be
used as the type of a variable, just for construction! At least in the Java
world this is a best practice most programmers use.

[*]

To finally get to the point: There could be some annotations or traits or
even structural type that helped make some design aspects of a library
explicit. If you could make the compiler issue a warning or even an error
when violated, then the design of a library would not erode as quickly as it
often does and library evolution could become safer. [I don't like the names
I use here, they are just the first thing that came to my mind and not well
thought about, yet. Read them as placeholders only.] Here they are:

DontAssign: Like HashMap example above, this type is not meant for
variables, values, or return types. Just use it for construction. The
counterpart "DontInstantiate" is not required: Making a class abstract or
defining it as a trait is explicit enough.

DontExtend: This class is meant to just be called by client code; avoid the
much stronger coupling by implementing a trait or extending a class. This
privilege is left to the library itself.

DontCall: This class or trait is a callback for the client code to extend
and the library to call. Don't do that at home.

DontStore: You should not save instances of this class for a longer period
of time, be it to a database, to xml, or serialization to a file. Libraries
storing objects would have to somehow specify that they don't like such
types.

DontSerialize: Stronger than the above... the library is holding references
that would get lost (e.g. Hibernate). Libraries serializing objects would
have to somehow specify that they don't like such types. There generally
should be a way to decouple such objects.

Immutable: The class guarantees that it recursively has no variables and is
purely functional. (I think this is a must!)

Mutable (as in C++): An exception to the above, that is: A variable is
required for performance reasons (e.g. a cache), but the logical state of
the class remains unchanged.

Key: Instances of this class can be used e.g. as a key in a HashMap, i.e.
the class overloads == and hashCode.

More thinking would certainly turn up more, but what does the Scala
community think about the general point I tried to make? Do you agree that
it would relevantly improve the up-scaling of Scala?

Regards
Walter

Jorge Ortiz
Joined: 2008-12-16,
User offline. Last seen 29 weeks 3 days ago.
Re: annotations or traits to better scale scala up
You could write compiler plugins to enforce some of these type constraints.

For a sample compiler plugin that emits warnings when it detects division by zero, see here: http://www.scala-lang.org/node/140

--j

On Tue, Jun 2, 2009 at 10:07 PM, Walter Smith <walter [dot] smith [at] sinntr [dot] eu> wrote:

Hi!

[this is more or less a repost directly to Nabble, as the original post to
scala-lang.org didn't work properly, as I was told]

everybody will agree that Scala scales great from medium sized applications
to small scripts, much better than esp. Java does. I often hear that it
scales up to large systems just as well, but does it? I think it does, but
could it be even better?

[This became a long post. If you don't want to read all the blurb, skip
ahead to the [*]]

Scala has some very powerful features that alleviate you from a lot of
constantly considering best practices. For example: How many equals and
hashCode methods have you seen that are just plain wrong (with sometimes
very subtle effects)? Case classes do that for free (and a lot more)! How
many singletons have you seen that try to be, but are *not* thread safe?!?
Scala objects are so right out of the box!

Things like this help a lot in small and big systems alike; they only scale
"linearily", i.e. the amount of best practices you don't have to take into
account any more by using Scala, stays constant per line of code.

What really does make a difference, though, is that Scala allows you to take
best practices out of your business code and encapsulate them directly into
your frameworks... and this is especially important for big systems. A
seemingly trivial example is that you can move the best practice of using
the fully qualified class name for a logger into a very simple trait:

trait Logging {
 import org.slf4j._
 val logger: Logger = LoggerFactory.getLogger(this.getClass)
}

This is not possible in Java, and it makes a real difference!

Some tools and frameworks will just take some time to grow as the community
grows: Code formatters (more important to big systems that to small!), code
analysis tools, refactoring and other IDE support, to name just a few. BTW:
I don't think that the module system suggested by Martin in "the" book
replaces the need for a real dependency injection framework: It doesn't take
the different build/deploy/run stages into account properly; and it misses
one important point: 'inversion of control', i.e. changing from pull to
push. But it is an interesting starting point and time will show.

Maybe some things in the language and/or compiler deserve some more pushing
for Scala to better scale up. For example type inference is just great for
smaller projects and client code, but for frameworks I think it's a good
practice to annotate at least the return type of a method -- that additional
piece of directly available information can be of great help. Without it you
sometimes have to look several method calls deep and/or look very closely.
This best practice is adhered to in some Scala library classes (e.g. List),
but not in all (e.g. Either). I think it would be a great help, if you could
make the compiler issue a warning, if it infers the return type of a method
for you. An IDE could even be configured to automatically add it on save.

Another example for language things to scale up better: Some types are not
meant to be used in a certain position. For example a HashMap should not be
used as the type of a variable, just for construction! At least in the Java
world this is a best practice most programmers use.

[*]

To finally get to the point: There could be some annotations or traits or
even structural type that helped make some design aspects of a library
explicit. If you could make the compiler issue a warning or even an error
when violated, then the design of a library would not erode as quickly as it
often does and library evolution could become safer. [I don't like the names
I use here, they are just the first thing that came to my mind and not well
thought about, yet. Read them as placeholders only.] Here they are:

DontAssign: Like HashMap example above, this type is not meant for
variables, values, or return types. Just use it for construction. The
counterpart "DontInstantiate" is not required: Making a class abstract or
defining it as a trait is explicit enough.

DontExtend: This class is meant to just be called by client code; avoid the
much stronger coupling by implementing a trait or extending a class. This
privilege is left to the library itself.

DontCall: This class or trait is a callback for the client code to extend
and the library to call. Don't do that at home.

DontStore: You should not save instances of this class for a longer period
of time, be it to a database, to xml, or serialization to a file. Libraries
storing objects would have to somehow specify that they don't like such
types.

DontSerialize: Stronger than the above... the library is holding references
that would get lost (e.g. Hibernate). Libraries serializing objects would
have to somehow specify that they don't like such types. There generally
should be a way to decouple such objects.

Immutable: The class guarantees that it recursively has no variables and is
purely functional. (I think this is a must!)

Mutable (as in C++): An exception to the above, that is: A variable is
required for performance reasons (e.g. a cache), but the logical state of
the class remains unchanged.

Key: Instances of this class can be used e.g. as a key in a HashMap, i.e.
the class overloads == and hashCode.

More thinking would certainly turn up more, but what does the Scala
community think about the general point I tried to make? Do you agree that
it would relevantly improve the up-scaling of Scala?


Regards
Walter
--
View this message in context: http://www.nabble.com/annotations-or-traits-to-better-scale-scala-up-tp23845439p23845439.html
Sent from the Scala - Debate mailing list archive at Nabble.com.


sadie
Joined: 2008-12-21,
User offline. Last seen 42 years 45 weeks ago.
Re: annotations or traits to better scale scala up

Walter Smith wrote:
>
> DontAssign: Like HashMap example above, this type is not meant for
> variables, values, or return types. Just use it for construction.

I see two sides to this. The first is in issuing warnings should somebody
use the class manually, for example:

val map: HashMap
^ Cannot use scala.collection.mutable.HashMap as a value
type, suggest scala.collection.mutable.Map instead

This could get *really* annoying if used incorrectly. For example, if
HashMap implemented a method that Map does not.

Walter Smith wrote:
>
> DontExtend: This class is meant to just be called by client code; avoid
> the much stronger coupling by implementing a trait or extending a class.
> This privilege is left to the library itself.

Can't this be achieved with the sealed modifier? That prevents the class
being extended except in the same source file.

Walter Smith wrote:
>
> DontSerialize: Stronger than the above... the library is holding
> references that would get lost (e.g. Hibernate). Libraries serializing
> objects would have to somehow specify that they don't like such types.
> There generally should be a way to decouple such objects.

If you extend Externalizable and throw an exception on the methods, that
would prevent it from being stored at runtime. But presumably a compile-time
check is preferable.

Walter Smith wrote:
>
> Immutable: The class guarantees that it recursively has no variables and
> is purely functional. (I think this is a must!)

In most cases, this could be derived by the compiler rather than needing to
be specified manually.

What would you do with this metadata? How would it be treated differently?

Walter Smith wrote:
>
> Key: Instances of this class can be used e.g. as a key in a HashMap, i.e.
> the class overloads == and hashCode.

Are you suggesting that a HashMap would refuse to accept any non-valid keys?
This could be accomplished with a trait applied by the compiler, but any
class that wasn't compiled with this trait would then be invalid - including
String. You could define a conversion to RichString - including a
translation of HashMap[String, _] to HashMap[RichString, _], but this could
cause unexpected problems.

Erik Engbrecht
Joined: 2008-12-19,
User offline. Last seen 3 years 18 weeks ago.
Re: annotations or traits to better scale scala up
For example: How many equals and
hashCode methods have you seen that are just plain wrong (with sometimes
very subtle effects)?

I can think of a hashCode method I saw recently in a large OSS project with some subtle effects....

On Wed, Jun 3, 2009 at 1:07 AM, Walter Smith <walter [dot] smith [at] sinntr [dot] eu> wrote:

Hi!

[this is more or less a repost directly to Nabble, as the original post to
scala-lang.org didn't work properly, as I was told]

everybody will agree that Scala scales great from medium sized applications
to small scripts, much better than esp. Java does. I often hear that it
scales up to large systems just as well, but does it? I think it does, but
could it be even better?

[This became a long post. If you don't want to read all the blurb, skip
ahead to the [*]]

Scala has some very powerful features that alleviate you from a lot of
constantly considering best practices. For example: How many equals and
hashCode methods have you seen that are just plain wrong (with sometimes
very subtle effects)? Case classes do that for free (and a lot more)! How
many singletons have you seen that try to be, but are *not* thread safe?!?
Scala objects are so right out of the box!

Things like this help a lot in small and big systems alike; they only scale
"linearily", i.e. the amount of best practices you don't have to take into
account any more by using Scala, stays constant per line of code.

What really does make a difference, though, is that Scala allows you to take
best practices out of your business code and encapsulate them directly into
your frameworks... and this is especially important for big systems. A
seemingly trivial example is that you can move the best practice of using
the fully qualified class name for a logger into a very simple trait:

trait Logging {
 import org.slf4j._
 val logger: Logger = LoggerFactory.getLogger(this.getClass)
}

This is not possible in Java, and it makes a real difference!

Some tools and frameworks will just take some time to grow as the community
grows: Code formatters (more important to big systems that to small!), code
analysis tools, refactoring and other IDE support, to name just a few. BTW:
I don't think that the module system suggested by Martin in "the" book
replaces the need for a real dependency injection framework: It doesn't take
the different build/deploy/run stages into account properly; and it misses
one important point: 'inversion of control', i.e. changing from pull to
push. But it is an interesting starting point and time will show.

Maybe some things in the language and/or compiler deserve some more pushing
for Scala to better scale up. For example type inference is just great for
smaller projects and client code, but for frameworks I think it's a good
practice to annotate at least the return type of a method -- that additional
piece of directly available information can be of great help. Without it you
sometimes have to look several method calls deep and/or look very closely.
This best practice is adhered to in some Scala library classes (e.g. List),
but not in all (e.g. Either). I think it would be a great help, if you could
make the compiler issue a warning, if it infers the return type of a method
for you. An IDE could even be configured to automatically add it on save.

Another example for language things to scale up better: Some types are not
meant to be used in a certain position. For example a HashMap should not be
used as the type of a variable, just for construction! At least in the Java
world this is a best practice most programmers use.

[*]

To finally get to the point: There could be some annotations or traits or
even structural type that helped make some design aspects of a library
explicit. If you could make the compiler issue a warning or even an error
when violated, then the design of a library would not erode as quickly as it
often does and library evolution could become safer. [I don't like the names
I use here, they are just the first thing that came to my mind and not well
thought about, yet. Read them as placeholders only.] Here they are:

DontAssign: Like HashMap example above, this type is not meant for
variables, values, or return types. Just use it for construction. The
counterpart "DontInstantiate" is not required: Making a class abstract or
defining it as a trait is explicit enough.

DontExtend: This class is meant to just be called by client code; avoid the
much stronger coupling by implementing a trait or extending a class. This
privilege is left to the library itself.

DontCall: This class or trait is a callback for the client code to extend
and the library to call. Don't do that at home.

DontStore: You should not save instances of this class for a longer period
of time, be it to a database, to xml, or serialization to a file. Libraries
storing objects would have to somehow specify that they don't like such
types.

DontSerialize: Stronger than the above... the library is holding references
that would get lost (e.g. Hibernate). Libraries serializing objects would
have to somehow specify that they don't like such types. There generally
should be a way to decouple such objects.

Immutable: The class guarantees that it recursively has no variables and is
purely functional. (I think this is a must!)

Mutable (as in C++): An exception to the above, that is: A variable is
required for performance reasons (e.g. a cache), but the logical state of
the class remains unchanged.

Key: Instances of this class can be used e.g. as a key in a HashMap, i.e.
the class overloads == and hashCode.

More thinking would certainly turn up more, but what does the Scala
community think about the general point I tried to make? Do you agree that
it would relevantly improve the up-scaling of Scala?


Regards
Walter
--
View this message in context: http://www.nabble.com/annotations-or-traits-to-better-scale-scala-up-tp23845439p23845439.html
Sent from the Scala - Debate mailing list archive at Nabble.com.




--
http://erikengbrecht.blogspot.com/
sadie
Joined: 2008-12-21,
User offline. Last seen 42 years 45 weeks ago.
Re: annotations or traits to better scale scala up

Marcus Downing wrote:
>
> I see two sides to this...

Ahem.

The second is that assignments to an inferred type would jump down to the
next valid type.

val map = new HashMap()
// map: Map

In cases where this is ambiguous or insufficient, it may prove necessary to
specify the preferred type for the val or var.

@InferenceDelegate{Map}
class HashMap...

dcsobral
Joined: 2009-04-23,
User offline. Last seen 38 weeks 5 days ago.
Re: annotations or traits to better scale scala up

On Wed, Jun 3, 2009 at 3:47 PM, Marcus Downing wrote:
>
>
> Walter Smith wrote:
>>
>> Immutable: The class guarantees that it recursively has no variables and
>> is purely functional. (I think this is a must!)
>
> In most cases, this could be derived by the compiler rather than needing to
> be specified manually.
>
> What would you do with this metadata? How would it be treated differently?

I don't know what Walter has in mind, but to me it seems obvious that
the compiler ensures at compile time that your class is, in fact,
purely functional, by checking "recursively" all its component
classes. Useful against changes in external classes.

ewilligers
Joined: 2008-08-20,
User offline. Last seen 3 years 17 weeks ago.
Re: annotations or traits to better scale scala up

Walter Smith wrote:
> DontExtend: This class is meant to just be called by client code; avoid the
> much stronger coupling by implementing a trait or extending a class. This
> privilege is left to the library itself.

package mylibrary

// Only library code may extend
class DontExtend private[mylibrary](args: List[Any]) {

private[mylibrary] def this() { this(Nil) }
}

> DontCall: This class or trait is a callback for the client code to extend
> and the library to call. Don't do that at home.

package mylibrary

abstract class DontCall {

// Abstract. User code implements
def implementation(): Unit

// Only library code may call
private[mylibrary] final def call() { implementation() }
}

Walter Smith
Joined: 2009-05-13,
User offline. Last seen 1 year 39 weeks ago.
Re: annotations or traits to better scale scala up

Daniel Sobral wrote:
>
> On Wed, Jun 3, 2009 at 3:47 PM, Marcus Downing wrote:
>>> Immutable: The class guarantees that it recursively has no variables and
>>> is purely functional. (I think this is a must!)
>>
>> In most cases, this could be derived by the compiler rather than needing
>> to
>> be specified manually.
>>
>> What would you do with this metadata? How would it be treated
>> differently?
>
> I don't know what Walter has in mind, but to me it seems obvious that
> the compiler ensures at compile time that your class is, in fact,
> purely functional, by checking "recursively" all its component
> classes. Useful against changes in external classes.
>

If I want a class A to be purely functional, I still may happen to use
within it's definition some other class B that isn't. Right now the compiler
accepts that, because there is no explicit distinction for it to derive the
intention from. I'd like to specify that I want it to remain pure!

Walter

Walter Smith
Joined: 2009-05-13,
User offline. Last seen 1 year 39 weeks ago.
Re: annotations or traits to better scale scala up

Marcus Downing wrote:
>
>
> Walter Smith wrote:
>>
>> DontAssign: Like HashMap example above, this type is not meant for
>> variables, values, or return types. Just use it for construction.
>
> I see two sides to this. The first is in issuing warnings should somebody
> use the class manually, for example:
>
> val map: HashMap
> ^ Cannot use scala.collection.mutable.HashMap as a value
> type, suggest scala.collection.mutable.Map instead
>
> This could get *really* annoying if used incorrectly. For example, if
> HashMap implemented a method that Map does not.
>

True, but I think for bigger systems the possibility to use it correctly
outweighs the danger of annoyingly bad design.

Marcus Downing wrote:
>
>
> Walter Smith wrote:
>>
>> DontExtend: This class is meant to just be called by client code; avoid
>> the much stronger coupling by implementing a trait or extending a class.
>> This privilege is left to the library itself.
>
> Can't this be achieved with the sealed modifier? That prevents the class
> being extended except in the same source file.
>

Yes it can, but there may be just too many extensions to reasonably put into
one file!

Marcus Downing wrote:
>
>
> Walter Smith wrote:
>>
>> Key: Instances of this class can be used e.g. as a key in a HashMap, i.e.
>> the class overloads == and hashCode.
>
> Are you suggesting that a HashMap would refuse to accept any non-valid
> keys? This could be accomplished with a trait applied by the compiler, but
> any class that wasn't compiled with this trait would then be invalid -
> including String.
>

I *do* suggest that HashMaps don't accept non-valid keys. But Strings *are*
valid keys. An annotation or a common trait would not work at all, but some
sort of structural type could; it does not work, yet, as you can't tell the
compiler that the structural type has to overload these methods instead of
inheriting them from Any; maybe an annotation on the structural type...
something like:

type Key = {
@overriding def equals(that: Any): Boolean
}

class HashMap[K <: Key, V]() { ... }

class Invalid()
class Valid() { def equals(that: Any): Boolean = ... }

new HashMap[Invalid, String] // should not compile
new HashMap[Valid, String] // does compile
new HashMap[String, String] // does compile as well

Maybe allowing the override keyword in a structural type would be an even
better option.

Thanks for the feedback and kind regards
Walter

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland