This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Language stability (was Re: Corporate scala levels)

9 replies
Justin du coeur
Joined: 2009-03-04,
User offline. Last seen 42 years 45 weeks ago.
I'm going to spin off Bernd's thread, with a related question that's been much on my mind the past couple of months.

I *adore* Scala -- IMO, it's the best language currently going for serious down-and-dirty programming.  It has everything I might want in a language, comes closest to the DWIM principle for me, and while it's pretty complex, that complexity scales nicely.  (That is, you can get a great deal done in it even if you only think of it as "Java++", and know little about the stranger and more interesting bits.)

That said, I haven't been able to argue for it seriously in a work environment.  Partly that's because it's still a little new and unproven, but even more it's because the language is still a little too much in flux.  As I understand it, 2.8 has a fair number of slight but non-trivial breaking changes in it, and I gather that's not all that unusual.

So here's a question, more for thought by the group than looking for a hard and fast answer: when is the language likely to settle down to the point where it is practical for a corporate environment?  One key aspect of this is accepting that, sooner or later, the language is probably going to need to stop having significant breaking changes, even of the "this is deprecated and will be removed in two releases" sort, if it's going to achieve widespread acceptance, because corporate engineering departments are, quite reasonably, leery of anything that isn't promising backward compatibility for a *long* time to come.  That doesn't mean eliminating language evolution, but it does mean that it becomes harder.

This isn't an easy question (which is why I'm posting it to -debate), and strikes to the heart of what the language is about.  I've often heard it said that Scala is trying to be a bit "realer" than some of the more academic languages, more focused on day-to-day needs of the engineer working on ordinary real-world problems.  This meta-issue of language stability is one of the *most* important day-to-day needs, but it implies that the language will eventually start getting a bit grungy: stabilizing a language usually means that eventually, that language will become obsolete.  I think that's a good thing, and part of the natural evolution of the field -- languages grow, stabilize, gradually age and then get replaced by their successors.  (Personally, I think of Scala as the much-needed successor to Java.)

Anyway, I toss it out for thought and discussion; forgive me if it's old hat, but I haven't seen it discussed yet in my limited time with the group.  I would very much *like* to be an advocate for Scala in my workplaces, but this is the key reason why I've been reluctant to do so heretofore: until I can promise a VP of Engineering that the language is stable enough for them, I don't think I can ever win that argument...
Ricky Clarkson
Joined: 2008-12-19,
User offline. Last seen 3 years 2 weeks ago.
Re: Language stability (was Re: Corporate scala levels)

For reasons like this, I have only bothered to argue for Scala in
small projects and for automated tests. The reasons that are like it
are:

* Poor IDE support. This has changed since we last discussed it at work.
* Poor documentation. The Scala libraries we use are fantastic (only
Specs, ScalaCheck and paulp's optional) but the core library is
sparsely documented; I normally read the source instead of the
scaladocs, and I can't really suggest that for, let's say, programmers
who only code in work hours.

However, the kind of companies that are scared of breaking changes
tend to be so scared that they run the same version for years anyway
(I bet you know someone stuck on Java 1.4 or 5). Our company isn't
perfect, but instead of being scared of change, we'll happily upgrade
when we want to, but respond quickly when customers have problems (and
catch most problems before customers see them). Luckily, most Scala
changes will manifest themselves as compile errors, not runtime
errors, making the risk that we release buggy code because of Scala
changes quite tiny.

2009/10/23 Justin du coeur :
> I'm going to spin off Bernd's thread, with a related question that's been
> much on my mind the past couple of months.
>
> I *adore* Scala -- IMO, it's the best language currently going for serious
> down-and-dirty programming.  It has everything I might want in a language,
> comes closest to the DWIM principle for me, and while it's pretty complex,
> that complexity scales nicely.  (That is, you can get a great deal done in
> it even if you only think of it as "Java++", and know little about the
> stranger and more interesting bits.)
>
> That said, I haven't been able to argue for it seriously in a work
> environment.  Partly that's because it's still a little new and unproven,
> but even more it's because the language is still a little too much in flux.
> As I understand it, 2.8 has a fair number of slight but non-trivial breaking
> changes in it, and I gather that's not all that unusual.
>
> So here's a question, more for thought by the group than looking for a hard
> and fast answer: when is the language likely to settle down to the point
> where it is practical for a corporate environment?  One key aspect of this
> is accepting that, sooner or later, the language is probably going to need
> to stop having significant breaking changes, even of the "this is deprecated
> and will be removed in two releases" sort, if it's going to achieve
> widespread acceptance, because corporate engineering departments are, quite
> reasonably, leery of anything that isn't promising backward compatibility
> for a *long* time to come.  That doesn't mean eliminating language
> evolution, but it does mean that it becomes harder.
>
> This isn't an easy question (which is why I'm posting it to -debate), and
> strikes to the heart of what the language is about.  I've often heard it
> said that Scala is trying to be a bit "realer" than some of the more
> academic languages, more focused on day-to-day needs of the engineer working
> on ordinary real-world problems.  This meta-issue of language stability is
> one of the *most* important day-to-day needs, but it implies that the
> language will eventually start getting a bit grungy: stabilizing a language
> usually means that eventually, that language will become obsolete.  I think
> that's a good thing, and part of the natural evolution of the field --
> languages grow, stabilize, gradually age and then get replaced by their
> successors.  (Personally, I think of Scala as the much-needed successor to
> Java.)
>
> Anyway, I toss it out for thought and discussion; forgive me if it's old
> hat, but I haven't seen it discussed yet in my limited time with the group.
> I would very much *like* to be an advocate for Scala in my workplaces, but
> this is the key reason why I've been reluctant to do so heretofore: until I
> can promise a VP of Engineering that the language is stable enough for them,
> I don't think I can ever win that argument...
>

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Language stability (was Re: Corporate scala levels)

On Fri, Oct 23, 2009 at 9:45 PM, Justin du coeur wrote:
> I'm going to spin off Bernd's thread, with a related question that's been
> much on my mind the past couple of months.
>
> I *adore* Scala -- IMO, it's the best language currently going for serious
> down-and-dirty programming.  It has everything I might want in a language,
> comes closest to the DWIM principle for me, and while it's pretty complex,
> that complexity scales nicely.  (That is, you can get a great deal done in
> it even if you only think of it as "Java++", and know little about the
> stranger and more interesting bits.)
>
> That said, I haven't been able to argue for it seriously in a work
> environment.  Partly that's because it's still a little new and unproven,
> but even more it's because the language is still a little too much in flux.
> As I understand it, 2.8 has a fair number of slight but non-trivial breaking
> changes in it, and I gather that's not all that unusual.
>
Depends what you call unusual. The way I see it over the last two
years any breaking changes can be regarded as bug fixes, and indeed I
can't remember having seen a single report where these changes upset
anybody's code.

We certainly don't like to break code, and we won't do it on a whim.
2.8 pays down a lot of technical debt. In the rush of previous
releases certain things crept into the distribution which in
retrospect should not be there. One example is the design of arrays,
which leaked at the edges. Another was some of the additions to
collections (projections, jcl) which were rather baroque and had
surprising behavior. Originally, 2.8 was meant as a quick code
breaking fix to collections before too much code relied on them (in
particular projections). It took much longer than expected to come up
with a good alternative collections design. In the meantime other
problems were discovered and we decided to roll fixes for them into
2.8. Better break code once than on every iteration.

So, yes, 2.8 will be a difficult release in that it will break some
code. We believe this was necessary to correct certain mistakes that
crept into the distribution. We do not want to make a habit of
breaking code. And we hope that there will be enough new useful things
in 2.8 to entice people to switch to it.

We also asked our users (including the larger corporate users) whether
we should do these breaking changes and the answer we got from
everyone was to go ahead and do it now rather than later (or regret
forever not having done it).

Cheers

David Pollak
Joined: 2008-12-16,
User offline. Last seen 42 years 45 weeks ago.
Re: Language stability (was Re: Corporate scala levels)


On Fri, Oct 23, 2009 at 1:41 PM, martin odersky <martin [dot] odersky [at] epfl [dot] ch> wrote:
On Fri, Oct 23, 2009 at 9:45 PM, Justin du coeur <jducoeur [at] gmail [dot] com> wrote:
> I'm going to spin off Bernd's thread, with a related question that's been
> much on my mind the past couple of months.
>
> I *adore* Scala -- IMO, it's the best language currently going for serious
> down-and-dirty programming.  It has everything I might want in a language,
> comes closest to the DWIM principle for me, and while it's pretty complex,
> that complexity scales nicely.  (That is, you can get a great deal done in
> it even if you only think of it as "Java++", and know little about the
> stranger and more interesting bits.)
>
> That said, I haven't been able to argue for it seriously in a work
> environment.  Partly that's because it's still a little new and unproven,
> but even more it's because the language is still a little too much in flux.
> As I understand it, 2.8 has a fair number of slight but non-trivial breaking
> changes in it, and I gather that's not all that unusual.
>
Depends what you call unusual. The way I see it over the last two
years any breaking changes can be regarded as bug fixes, and indeed I
can't remember having seen a single report where these changes upset
anybody's code.

We certainly don't like to break code, and we won't do it on a whim.
2.8 pays down a lot of technical debt. In the rush of previous
releases certain things crept into the distribution which in
retrospect should not be there. One example is the design of arrays,
which leaked at the edges. Another was some of the additions to
collections (projections, jcl) which were rather baroque and had
surprising behavior. Originally, 2.8 was meant as a quick code
breaking fix to collections before too much code relied on them (in
particular projections). It took much longer than expected to come up
with a good alternative collections design. In the meantime other
problems were discovered and we decided to roll fixes for them into
2.8. Better break code once than on every iteration.

So, yes, 2.8 will be a difficult release in that it will break some
code. We believe this was necessary to correct certain mistakes that
crept into the distribution. We do not want to make a habit of
breaking code. And we hope that there will be enough new useful things
in 2.8 to entice people to switch to it.

We also asked our users (including the larger corporate users) whether
we should do these breaking changes and the answer we got from
everyone was to go ahead and do it now rather than later (or regret
forever not having done it).

I've been around Scala for 3 years (well, 2 weeks shy of 3 years).  The pace of Scala language/library change has radically slowed in that period of time.

While the 2.8 shift is the largest change in the last 2 years, I heartily endorse and support it.  What it is really doing is avoiding a lot of dead-ends in the language.  If Java had had a similar change in the 1.2-1.3 timeframe, Java would still be a language that could evolve.  Instead, Java is syntactically and semantically boxed in and cannot evolve.

I am working to make sure that Lift migrates to 2.8 smoothly.  I expect that between some of the changes to how packages work and the excellent support for building one's own collections, Scala 2.8+ will have a lot of room to evolve while preserving backward (at least source) compatibility.

Thanks,

David
 

Cheers

Dave Griffith
Joined: 2009-01-14,
User offline. Last seen 42 years 45 weeks ago.
Re: Language stability (was Re: Corporate scala levels)

The usual way this problem is dealt with at this point in a language's
evolution is for one or more for-profit consultancies to arise capable of
doing maintenance on the core tools of some fixed versions the language.
That way crucial bug fixes to non-tip versions of the language
implementations can be guaranteed, which is basically all it takes to make
CTOs happy. These consultancies will often have close-but-not-too-close
relations to the language implementers, and supplement their income with
other consulting or training on projects in the language, easing the way for
early adopters. Even if the consultancies rarely have to make tool fixes,
just knowing that such fixes are possible and that someone exists with the
capability and inclination to do them opens a lot of doors.

I'm hoping that 2.8 is going to be stable enough for that sort of ecosystem
to emerge (it's certainly featureful enough). That seems to be the intent
of this release. You've correctly identified a key problem (indeed, I
believe, the key remaining problem) preventing Scala from supplanting Java
in the enterprise. Hopefully, a path to a solution can be found.

(Yes, this is a naked invitation for someone on this list to start such a
consultancy. I'm sadly otherwise booked for a while.)

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Language stability (was Re: Corporate scala levels)

On Fri, Oct 23, 2009 at 10:16 PM, Dave Griffith
wrote:
> (Yes, this is a naked invitation for someone on this list to start such a
> consultancy.  I'm sadly otherwise booked for a while.)

You've just given a very good description of what I do.

Cheers,

Miles

Viktor Klang
Joined: 2008-12-17,
User offline. Last seen 1 year 27 weeks ago.
Re: Language stability (was Re: Corporate scala levels)
Also, as long as bugfixes are backported I fail to see what the problem is. If you want to stay on a particular version, please do so.
Noone's forcing anyone to download new binaries...

Doesn't matter if you break backwards or not, I still know shops stuck on Java 1.3
On Fri, Oct 23, 2009 at 11:32 PM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Fri, Oct 23, 2009 at 10:16 PM, Dave  Griffith
<dave [dot] l [dot] griffith [at] gmail [dot] com> wrote:
> (Yes, this is a naked invitation for someone on this list to start such a
> consultancy.  I'm sadly otherwise booked for a while.)

You've just given a very good description of what I do.

Cheers,


Miles

--
Miles Sabin
tel: +44 (0)7813 944 528
skype:  milessabin
http://www.chuusai.com/
http://twitter.com/milessabin



--
Viktor Klang
| "A complex system that works is invariably
| found to have evolved from a simple system
| that worked." - John Gall

Blog: klangism.blogspot.com
Twttr: viktorklang
Code: github.com/viktorklang
Ken Bloom
Joined: 2009-09-22,
User offline. Last seen 42 years 45 weeks ago.
Re: Language stability (was Re: Corporate scala levels)

On Fri, 23 Oct 2009 21:00:55 +0100, Ricky Clarkson wrote:
> However, the kind of companies that are scared of breaking changes tend
> to be so scared that they run the same version for years anyway (I bet
> you know someone stuck on Java 1.4 or 5).

Consider what happens in the Python community: they release a new version
of Python every few years, and it contains incompatible changes with the
previous version. People in the community seem to accept this, and Linux
distributions understand. In Debian there are two stable versions of
Python packaged (currently) and two more in the experimental distribution
for things that aren't yet considered release worthy. People target
particular versions of the language, and indicate that their code should
be run using that version of the language.

Heck even C and C++ compilers need to be treated this way, Ruby is
treated this way, and Perl is about to accumulate a lot of breaking
changes with the advent of Perl 6.

Targeting a particular version of a language is an even easier model to
use in the Java community where platform support for sharing jars between
two programs is so abysmal that everyone just packages up all the jars
their code depends on and ships them with their code. (And even so, any
principled way to support dependencies on the Java platform is likely to
take into account the fact that there will be incompatibilities between
versions of a library.) You can look at the overall stability and
consistency of Scala 2.7 or Scala 2.8, and its long term support
prognosis if strange bugs are found whose fixes don't break source
compatibility (like the finally-block fix in Scala 2.7.7) and decide
based on that which version you want to target. Then the worry about
breaking changes becomes much more academic.

bjohanns
Joined: 2009-10-23,
User offline. Last seen 1 year 37 weeks ago.
Re: Language stability (was Re: Corporate scala levels)

Martin Odersky wrote:
>
> On Fri, Oct 23, 2009 at 9:45 PM, Justin du coeur
> wrote:
>> I'm going to spin off Bernd's thread, with a related question that's been
>> much on my mind the past couple of months.
>>
>> I *adore* Scala -- IMO, it's the best language currently going for
>> serious
>> down-and-dirty programming.  It has everything I might want in a
>> language,
>> comes closest to the DWIM principle for me, and while it's pretty
>> complex,
>> that complexity scales nicely.  (That is, you can get a great deal done
>> in
>> it even if you only think of it as "Java++", and know little about the
>> stranger and more interesting bits.)
>>
> [...]
>>
> [...]
> We certainly don't like to break code, and we won't do it on a whim.
> [...]
> So, yes, 2.8 will be a difficult release in that it will break some
> code. We believe this was necessary to correct certain mistakes that
> crept into the distribution. We do not want to make a habit of
> breaking code. And we hope that there will be enough new useful things
> in 2.8 to entice people to switch to it.
>
> [...]
> Cheers
>

Ian Clarke
Joined: 2008-12-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Language stability (was Re: Corporate scala levels)

On Fri, Oct 23, 2009 at 2:45 PM, Justin du coeur wrote:
> but even more it's because the language is still a little too much in flux.
> As I understand it, 2.8 has a fair number of slight but non-trivial breaking
> changes in it, and I gather that's not all that unusual.

Clearly there is a trade-off. We can have a language with some
irritating an unfixable flaws, or we can tolerate the occasional
backwards-incompatible change.

I think Java is the former, and Scala should be the latter. At some
point Scala must make the leap, but I think it would be a real wasted
opportunity to lock things in now.

Frankly, if your's is the kind of organization that requires an API
that won't change for the remainder of the lifetime of the language -
stick with Java for the time being and let the Scala team get it
right.

Ian.

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland