Parallel collections binary compatibility

Hi Martin,

On Thu, Apr 21, 2011 at 11:29 AM, martin odersky wrote:
> This means we will see major binary compatibility problems, because existing
> calls to these methods use the old signatures.

Is this any worse than what was there previously? A "small" binary
compatibility break is no better (and perhaps worse) than a "large"
binary compatibility break. Nothing worse than having code that looks
to be working and then throws a NoSuchMethodError or similar at
runtime. Given Scala's current reality, it seems to me that source
compatibility is the best we can aim for in feature releases. At least
until the Migration Manager or new JVM features show up.

> I just thought of a way to address this problem: Let's generate for each of
> these 130 methods a bridge method that has the old signature and forwards to
> the method with the new signature. Being bridges, these won't show up in
> ScalaDoc and the compiler will not generate calls to them. It's just the JVM
> runtime that will see them.

Unless this fixes all binary compatibility issues (and now just the
ones introduced by the parallel collections change), I am not sure
it's worth all the work (and testing).

Best,
Ismael

On Thu, Apr 21, 2011 at 11:49 AM, Ismael Juma wrote:
> Unless this fixes all binary compatibility issues (and now just the
> ones introduced by the parallel collections change)

Sorry, I meant to say "and not just the ones introduced...".

Ismael

On Thu, Apr 21, 2011 at 12:49 PM, Ismael Juma <ismael [at] juma [dot] me [dot] uk> wrote:

Hi Martin,

On Thu, Apr 21, 2011 at 11:29 AM, martin odersky <martin [dot] odersky [at] epfl [dot] ch> wrote:
> This means we will see major binary compatibility problems, because existing
> calls to these methods use the old signatures.

Is this any worse than what was there previously? A "small" binary
compatibility break is no better (and perhaps worse) than a "large"
binary compatibility break. Nothing worse than having code that looks
to be working and then throws a NoSuchMethodError or similar at
runtime. Given Scala's current reality, it seems to me that source
compatibility is the best we can aim for in feature releases. At least
until the Migration Manager or new JVM features show up.

In short, the previous compatibility problems between 2.8 and 2.9 showed up if you combined 2.8 implementations of collections with 2.9 clients. This does not seem very common, probably because almost everyone uses the standard implementations of collections which have been updated for 2.9. What's more, these problems can be fixed by the migration manager.

The new compatibility problems show up for every client of a collection that uses a method like zip, containsSeq, patch, etc which takes a collection parameter. That is, virtually everyone.
Also, we do not know how to fix this class of problems automatically with tools.

Cheers
 
 -- Martin

On Thu, Apr 21, 2011 at 12:03 PM, martin odersky wrote:
> In short, the previous compatibility problems between 2.8 and 2.9 showed up
> if you combined 2.8 implementations of collections with 2.9 clients. This
> does not seem very common, probably because almost everyone uses the
> standard implementations of collections which have been updated for 2.9.

Is this really true? I would guess that for the typical non-toy
project, once you include all the dependencies, it's likely that there
are some implementations of collections. This doesn't mean much, but I
have a few collection implementations in my project.

> What's more, these problems can be fixed by the migration manager.

This is a stronger argument in my opinion (if we expect the migration
manager to be here before everyone gets around to recompiling with
2.9.0 anyway).

Best,
Ismael

Just to add a related use case: one often needs one or even more
alternative signatures to call the very same function what I am using
the nickname @synonym for. Because there exists no such annotation I
just use @inline. Okay, the intention for @bridge is to make calls to
deprecated and removed functions transparent while @synonym encourages
to call any of the signature variants. But from my perspective they
are related to some extent...

Hi!

> Now that we got a handle on the compile times, there's one other problem
> opened by the new parallel collections design: Binary compatibility.
>
> There are about 130 methods in the collection hierarchy that changed
> signature from a X collection parameter to a GenX parameter, where X is one
> of Seq, Iterable, Traversable, TraversableOnce, Map, Set.
>
> This means we will see major binary compatibility problems, because existing
> calls to these methods use the old signatures.

I don't know if anyone really expects compatibility between these
large feature releases. Imho it would make more sense to release
updates to 2.8.x if customers really need it.

The "I hate Scala because of no binary compatibility"-crowd has
already moved on to the more general "Scala is too complex" whining
... so I wonder if it is wise to invest time and resources in it. I
suspect there is no real demand except Source compatibility, but I'm
far from an expert here.

> I just thought of a way to address this problem: Let's generate for each of
> these 130 methods a bridge method that has the old signature and forwards to
> the method with the new signature. Being bridges, these won't show up in
> ScalaDoc and the compiler will not generate calls to them. It's just the JVM
> runtime that will see them.

Won't this bloat every single collection class with these bridge methods?
What's the effect on .class size?

> To generate these bridges will need a little bit of help of the compiler. I
> was thinking of a new annotation, @bridge, which the compiler would
> translate to a bridge method at erasure time. Furthermore, @bridge methods
> always lose out to other methods for overloading resolution. Since this is
> dicey stuff I was thinking of making the annotations.bridge class
> private[scala], so the method would initiially not be accessible to user
> code (we can open it up later once we have gained enough experience with the
> feature).

I think there are more drawbacks than benefits from that approach.

Quite recently, while adding version information to @deprecated I
found a method which was deprecated since Scala 2.3.2 but was
forgotten to be removed in every single release until today ... If
things like this still manage to slip through for years, I don't want
to imagine the chaos if there are an additional _invisible_ 130
methods sitting around in the dark.

What's the deprecation strategy for these bridge methods? When will
they be removed? Will they be carried around until the end of time?

I just think it is a short-sighted way which _helps_ people accumulate
technical debt.

Scala should be honest: There are certain efforts necessary to upgrade
to newer feature releases and there is no sensible way to migitate
them (considering the fact that Scala - unlike Java - actually
removes old/deprectaed/broken things).

> What do you think? Is that something we should do? In any case, we should
> hold off with the next Scala 2.9 RC until we have decided this.

Scala's 2.9 release was announced for the beginning of the year ... I
don't want to imagine how many months this feature would further delay
the final release. :-/

Just my thoughts ...

Thanks and bye!

Simon

I‘m with you.

On Apr 21, 8:22 pm, Simon Ochsenreither
wrote:
> Hi!
>
> > Now that we got a handle on the compile times, there's one other problem
> > opened by the new parallel collections design: Binary compatibility.
>
> > There are about 130 methods in the collection hierarchy that changed
> > signature from a X collection parameter to a GenX parameter, where X is one
> > of Seq, Iterable, Traversable, TraversableOnce, Map, Set.
>
> > This means we will see major binary compatibility problems, because existing
> > calls to these methods use the old signatures.
>
> I don't know if anyone really expects compatibility between these  
> large feature releases. Imho it would make more sense to release  
> updates to 2.8.x if customers really need it.
>
> The "I hate Scala because of no binary compatibility"-crowd has  
> already moved on to the more general "Scala is too complex" whining  
> ... so I wonder if it is wise to invest time and resources in it. I  
> suspect there is no real demand except Source compatibility, but I'm  
> far from an expert here.

Hi,

What's the deprecation strategy for these bridge methods? When will they be removed? Will they be carried around until the end of time?

This was something I was wondering too. I can see the rationale for backwards-binary-compatible classes. However, for how long would they be backwards compattible? My 2c would be that if the compiler is producing class files that scalac won't link during compilation then all existing @bridge methods can go away.  

I just think it is a short-sighted way which _helps_ people accumulate technical debt.

I'm reminded of the horror of certain APIs from an OS vendor, who kludges, hacks and shims support for legacy APIs into their newest OSes in an attempt to get a reasonable coverage of existing applications to compile/run on the next 'improvement'. Sometimes the new version just *is* incompatible with the old one.  

What do you think? Is that something we should do? In any case, we should
hold off with the next Scala 2.9 RC until we have decided this.

I see how @bridge would help, but am not convinced that it is fixing a problem that should be fixed. If libs are essentially source-compattible between 2.8.1 and 2.9, then perhaps the man-hours that would go into implementing and maintaining the @bridge support would be better spent making it trivial to do that recompilation and encouraging as many existing projects as possible to do that rebuild so as to offer 2.8.1 and 2.9 jars.  

Scala's 2.9 release was announced for the beginning of the year ... I don't want to imagine how many months this feature would further delay the final release. :-/


Just my thoughts ...


Thanks and bye!


Simon

--
Matthew Pocockmailto: turingatemyhamster [at] gmail [dot] comgchat: turingatemyhamster [at] gmail [dot] com msn: matthew_pocock [at] yahoo [dot] co [dot] ukirc.freenode.net: drdozer(0191) 2566550

Just some clarifications:

We probably won't need a bridge method for each of the 130 affected methods, as some of them are overrides of others. So, probably less than 100 bridge methods, which in the overall picture of things is negligible.

I believe it would take about 2 days to implement this.

I would keep the bridge methods in place for at least one major revision.

Cheers

 -- Martin

>>>>> "Simon" == Simon Ochsenreither writes:

Simon> I don't know if anyone really expects compatibility between
Simon> these large feature releases. Imho it would make more sense to
Simon> release updates to 2.8.x if customers really need it.

Yeah, I don't care if 2.8 and 2.9 are binary compatible or not, and
honestly, even if it was claimed they were, I wouldn't trust them to be,
or recommend others trust in it. (Am I wrong? I'm open to being
convinced otherwise.)

Have the people who have been testing 2.9 so far been testing it with
2.8 libraries? I certainly haven't.

Simon> What's the deprecation strategy for these bridge methods? When
Simon> will they be removed? Will they be carried around until the end
Simon> of time?

...exactly. The pain is postponed, and multiplied. I'd rather get it
over with.

Simon> Scala - unlike Java - actually removes old/deprectaed/broken
Simon> things).

Yeah, and I consider this very much a feature of Scala, not a bug. Any
pain in the present is more than compensated for by the brighter future
this enables.

Anyway, this is all just my own personal perspective -- I can certainly
imagine other kinds of users having other points of view on these issues.

On Thu, Apr 21, 2011 at 11:33 AM, Seth Tisue <seth [at] tisue [dot] net> wrote:

>>>>> "Simon" == Simon Ochsenreither <simon [at] ochsenreither [dot] de> writes:

 Simon> I don't know if anyone really expects compatibility between
 Simon> these large feature releases. Imho it would make more sense to
 Simon> release updates to 2.8.x if customers really need it.

Yeah, I don't care if 2.8 and 2.9 are binary compatible or not, and
honestly, even if it was claimed they were, I wouldn't trust them to be,
or recommend others trust in it.  (Am I wrong?  I'm open to being
convinced otherwise.)

No, I feel the same way.

I think it's a broader issue than just between releases. Either Scala guarantees binary compatibility between all releases always (like Java), but since we already know that that's not going to happen, I don't see the big reason to expend resources on it between individual releases, particularly not if it only requires a recompile. The community has to handle the option anyway due to the overall policy.
 

Have the people who have been testing 2.9 so far been testing it with
2.8 libraries?  I certainly haven't.

I have, but only in terms of compilation, not runtime.

On 21/04/2011 18:33, Seth Tisue wrote:
>>>>>> "Simon" == Simon Ochsenreither writes:
> Simon> Scala - unlike Java - actually removes old/deprectaed/broken
> Simon> things).
>
> Yeah, and I consider this very much a feature of Scala, not a bug. Any
> pain in the present is more than compensated for by the brighter future
> this enables.

I completely agree with that for major release.
But for minor releases, I would really love to have binary compatibility
and so forget forever 2.7.x ages.

On Thu, Apr 21, 2011 at 6:38 PM, Nils Kilden-Pedersen <nilskp [at] gmail [dot] com> wrote:

On Thu, Apr 21, 2011 at 11:33 AM, Seth Tisue <seth [at] tisue [dot] net> wrote:

>>>>> "Simon" == Simon Ochsenreither <simon [at] ochsenreither [dot] de> writes:

 Simon> I don't know if anyone really expects compatibility between
 Simon> these large feature releases. Imho it would make more sense to
 Simon> release updates to 2.8.x if customers really need it.

Yeah, I don't care if 2.8 and 2.9 are binary compatible or not, and
honestly, even if it was claimed they were, I wouldn't trust them to be,
or recommend others trust in it.  (Am I wrong?  I'm open to being
convinced otherwise.)

No, I feel the same way.

I think it's a broader issue than just between releases. Either Scala guarantees binary compatibility between all releases always (like Java), but since we already know that that's not going to happen, I don't see the big reason to expend resources on it between individual releases, particularly not if it only requires a recompile. The community has to handle the option anyway due to the overall policy.
 

Have the people who have been testing 2.9 so far been testing it with
2.8 libraries?  I certainly haven't.

I have, but only in terms of compilation, not runtime.

One more clarification: The kind of binary compatibility problem posed by parallel collection would be just the same in Java. If you widen the parameter of a method, you become binary incompatible, no matter whether you are in java or in Scala.

Cheers

 -- Martin

Hi,

I think this might maybe implemented by some commercial entity (Scala
Solutions?) as a special purpose build if there is special customer
demand, but I have the feeling that this is not a feature most people
are waiting for.

While it might be useful in very rare use-cases, it makes accumulating
technical debt much more easier... I can already predict that those
bridge methods will start to accumulate over time and will not be
removed, because more and more people start depending on them instead of
fixing their code.

I just don't see where the huge testing team will magically appear to do
the testing and long-term support for that feature.

Martin, could you clarify if there is some customer request behind your
idea, or is it just hypothetical thinking?

Most businesses needing such features are neither using Scala 2.7 nor
2.8, they are firmly in the Java 1.3/1.4 camp, because "it works" for
them, so I can't imagine benefits catering to them, they won't be using
Scala anyways.

Generally, I think it is OK to break binary compatibility between major
releases, but please don't tell anybody about it ... the angry "expert
bloggers" are by far the most annoying thing overall, because they keep
spreading FUD with their irrational complaints.

(Remember that the various shitstorms were a result of Scala devs saying
"we will fix IDE support/look into compatibility/... for the next
release", not the result of those "expert bloggers" actually testing Scala.)

It would be nice if updates to older releases would not break
compatibility (afaik there are some guys waiting for a backport of some
fixes to 2.8.x already ...).

Bye,

Simon

On Thu, Apr 21, 2011 at 5:33 PM, Seth Tisue wrote:
> Yeah, I don't care if 2.8 and 2.9 are binary compatible or not, and
> honestly, even if it was claimed they were, I wouldn't trust them to be,
> or recommend others trust in it.

This is how I feel too. And if the community doesn't trust or test for
binary compatibility, the chance of it being really binary compatible
is small in my opinion. Also, it sounds to me like Martin is
suggesting that it still won't be fully binary compatible (e.g. if you
implement Scala collections or use other libraries that do), but
please correct me if I misunderstood.

Best,
Ismael

I'd actually agree with this.  I think the convention of 2.x being binary incompatible with 2.y is 'ok' for Scala, as long as 2.x.y and 2.x.z *are* binary compatible.   Makes a much smoother dev process for us all!
Also, the company I work for (somewhat) monolithically updates/recompiles code when upgrading Scala, so it's not a terrible burden for us.  Hopefully for everyone else there's SBT.
- Josh

On Thu, Apr 21, 2011 at 1:59 PM, Ismael Juma <ismael [at] juma [dot] me [dot] uk> wrote:

On Thu, Apr 21, 2011 at 5:33 PM, Seth Tisue <seth [at] tisue [dot] net> wrote:
> Yeah, I don't care if 2.8 and 2.9 are binary compatible or not, and
> honestly, even if it was claimed they were, I wouldn't trust them to be,
> or recommend others trust in it.

This is how I feel too. And if the community doesn't trust or test for
binary compatibility, the chance of it being really binary compatible
is small in my opinion. Also, it sounds to me like Martin is
suggesting that it still won't be fully binary compatible (e.g. if you
implement Scala collections or use other libraries that do), but
please correct me if I misunderstood.

Best,
Ismael

On Thu, Apr 21, 2011 at 7:44 PM, Simon Ochsenreither <simon [at] ochsenreither [dot] de> wrote:

Hi,

I think this might maybe implemented by some commercial entity (Scala Solutions?) as a special purpose build if there is special customer demand, but I have the feeling that this is not a feature most people are waiting for.

While it might be useful in very rare use-cases, it makes accumulating technical debt much more easier... I can already predict that those bridge methods will start to accumulate over time and will not be removed, because more and more people start depending on them instead of fixing their code.
 

I just don't see where the huge testing team will magically appear to do the testing and long-term support for that feature.
 

Martin, could you clarify if there is some customer request behind your idea, or is it just hypothetical thinking?

I don't have a specific request, but know that some large Scala users care deeply about the issue.
 
I have promised we will take care of binary compatibility from 2.8 on. It was mainly that promise that made people quiet down quickly last fall. So I feel bad turning around and opening a huge binary compatibility hole that would make the transition from 2.8 to 2.9 just as hard as the transition from 2.7 to 2.8 was. No matter that this is technically not a Scala problem; people will react negatively anyway.

I don't buy the technical debt argument. The beauty of bridge methods is that they are invisible to user code. So it will only be old binaries, but never freshly compiled code, that calls a bridge method.

Cheers

 -- Martin

Hi,
> I don't have a specific request, but know that some large Scala users
> care deeply about the issue.
>
> I have promised we will take care of binary compatibility from 2.8 on.
> It was mainly that promise that made people quiet down quickly last
> fall. So I feel bad turning around and opening a huge binary
> compatibility hole that would make the transition from 2.8 to 2.9 just
> as hard as the transition from 2.7 to 2.8 was. No matter that this is
> technically not a Scala problem; people will react negatively anyway.
I think most people think that compatibility should not be broken within
2.8.x or 2.9.x ... the complaints were mostly about incompatibility
within the 2.7.x series as far as I remember.

Even if there would be @bridge methods, how would you fix those methods
which changed behavior between 2.8 and 2.9 (like scanRight, or the
cleanups Paul did to reduce dozens of different implementation of span(?)?
> I don't buy the technical debt argument. The beauty of bridge methods
> is that they are invisible to user code. So it will only be old
> binaries, but never freshly compiled code, that calls a bridge method.
I think it is clearly technical debt, because people will now delay
fixing their code until they are really forced to and will start arguing
for not removing the bridges "just yet", which will force us to carry it
around forever.

It is also a problem of support:
I don't think that many will have the remembrance to answer questions
about moving code from 2.8 to 2.9 when most people on IRC are already
running 2.11 trunk.
In my opinion migration problems should be fail-fast and shouldn't crop
up after everyone has started working on code targeting two versions later.

(Maybe I'm just a bit against those compatibility bridges, because I saw
what happened to OpenGL a few years ago, where the major stakeholders
like CAD companies basically grew such powerful that they were able to
shoot down the cleaned up API, which was planned for years. Not because
of valid concerns, but just because they feared that the GPU
manufacturers would invest less time making the legacy code of the CAD
companies work if they had a new API to support. Meanwhile, most CAD
companies switched to Direct3D, citing the lack of development in the
OpenGL standard...)

Thanks and bye!

Simon

On Thu, Apr 21, 2011 at 9:40 PM, Simon Ochsenreither <simon [at] ochsenreither [dot] de> wrote:

Hi,

I don't have a specific request, but know that some large Scala users care deeply about the issue.

I have promised we will take care of binary compatibility from 2.8 on. It was mainly that promise that made people quiet down quickly last fall. So I feel bad turning around and opening a huge binary compatibility hole that would make the transition from 2.8 to 2.9 just as hard as the transition from 2.7 to 2.8 was. No matter that this is technically not a Scala problem; people will react negatively anyway.

I think most people think that compatibility should not be broken within 2.8.x or 2.9.x ... the complaints were mostly about incompatibility within the 2.7.x series as far as I remember.

No. They were 2.7 to 2.8, with a beta in-between.
 

Even if there would be @bridge methods, how would you fix those methods which changed behavior between 2.8 and 2.9 (like scanRight, or the cleanups Paul did to reduce dozens of different implementation of span(?)?

I don't buy the technical debt argument. The beauty of bridge methods is that they are invisible to user code. So it will only be old binaries, but never freshly compiled code, that calls a bridge method.

I think it is clearly technical debt, because people will now delay fixing their code until they are really forced to and will start arguing for not removing the bridges "just yet", which will force us to carry it around forever.

No. Nobody needs to fix their code. Just recompile. The question is do we force a synchronous recompile to 2.9 on the world, or do we have a staged way where most old code continues to work for at least one major release without a recompile. The latter seems like a win to me.
 

It is also a problem of support:
I don't think that many will have the remembrance to answer questions about moving code from 2.8 to 2.9 when most people on IRC are already running 2.11 trunk.
In my opinion migration problems should be fail-fast and shouldn't crop up after everyone has started working on code targeting two versions later.

As I wrote, it does not affect source code in any way.

 -- Martin

Hi Martin,
Sorry if I am stating the obvious - perhaps I read things too quickly. Is your intention with @bridge that:
a) It enables code compiled against the 2.8.* collections API to run against the 2.9.* collections implementation without recompilation - run-time binary backwards compatibility
b) no @bridge method is available via scalac, so you can't compile code that binds these bridge methods - compile-time backwards incompatibility forces people to fix their code when they try to rebuild it.
c) Methods tagged with @bridge may be removed at any time, and will be removed in practice. In particular, they may be removed if it is no longer practical to implement the old method with calls to the new API, so we should never be in the situation where new code is hobbled for supporting legacy logic in a @bridge method.
I predict that just as people ask for a non-specialised core for android and other small-footprint targets, I can see them asking for a non-bridge core. Perhaps if much more 'code bloat' compiler magic goes on, there would be an argument for producing blessed minimal builds - perhaps with a scalac -minimal flag.
I'd hope that if I try to compile 2.8.* code against 2.9.* and I hit a method now implemented with @bind that scalac would give me an informative message - perhaps printing out the body of the @bind method to show me one way to fix the problem. If something like this was in place, then @bind has a second use as a documentation of how an API has evolved. I expect that some functions currently annotated with @deprecated could be shuffled over to @bind if this kind of support was in place, providing a more graceful and gradated path for API evolution.
Assuming I've understood what you're suggesting, my worry is the maintenance cost. However, if you think it's a couple of days work and you have the time (or minions) then I guess this is moot.
Matthew
--
Matthew Pocockmailto: turingatemyhamster [at] gmail [dot] comgchat: turingatemyhamster [at] gmail [dot] com msn: matthew_pocock [at] yahoo [dot] co [dot] ukirc.freenode.net: drdozer(0191) 2566550

On Apr 21, 2011, at 10:59 AM, Ismael Juma wrote:

> On Thu, Apr 21, 2011 at 5:33 PM, Seth Tisue wrote:
>> Yeah, I don't care if 2.8 and 2.9 are binary compatible or not, and
>> honestly, even if it was claimed they were, I wouldn't trust them to be,
>> or recommend others trust in it.
>
> This is how I feel too. And if the community doesn't trust or test for
> binary compatibility, the chance of it being really binary compatible
> is small in my opinion. Also, it sounds to me like Martin is
> suggesting that it still won't be fully binary compatible (e.g. if you
> implement Scala collections or use other libraries that do), but
> please correct me if I misunderstood.

As a commercial user of Scala, I don't mind if 2.8 and 2.9 are binary incompatible. In fact, I would probably insist everything is recompiled with 2.9 to avoid the possibility of running into an issue in production (not saying we don't unit test and deploy onto QA systems before production, but you know what I mean :) )

Blair

On Thu, Apr 21, 2011 at 10:06 PM, Matthew Pocock <turingatemyhamster [at] gmail [dot] com> wrote:

Hi Martin,
Sorry if I am stating the obvious - perhaps I read things too quickly. Is your intention with @bridge that:
a) It enables code compiled against the 2.8.* collections API to run against the 2.9.* collections implementation without recompilation - run-time binary backwards compatibility

Correct.

b) no @bridge method is available via scalac, so you can't compile code that binds these bridge methods - compile-time backwards incompatibility forces people to fix their code when they try to rebuild it.

There is no source incompatibility. Methods that took a Seq parameter before now take a supertype.
 

c) Methods tagged with @bridge may be removed at any time, and will be removed in practice. In particular, they may be removed if it is no longer practical to implement the old method with calls to the new API, so we should never be in the situation where new code is hobbled for supporting legacy logic in a @bridge method.
I predict that just as people ask for a non-specialised core for android and other small-footprint targets, I can see them asking for a non-bridge core. Perhaps if much more 'code bloat' compiler magic goes on, there would be an argument for producing blessed minimal builds - perhaps with a scalac -minimal flag.

The bridge methods will need less than 0.1% of code of the collections library. I really do not see a problem here.

 

I'd hope that if I try to compile 2.8.* code against 2.9.* and I hit a method now implemented with @bind that scalac would give me an informative message - perhaps printing out the body of the @bind method to show me one way to fix the problem.

There is no problem. Scalac will just compile your code so that it calls the new method, not the old. As I wrote before, bridge methods are purely for binary compatibility, no source code is ever affected.

Cheers

 -- Martin

On Thu, Apr 21, 2011 at 7:09 PM, martin odersky wrote:
> I have promised we will take care of binary compatibility from 2.8 on. It
> was mainly that promise that made people quiet down quickly last fall. So I
> feel bad turning around and opening a huge binary compatibility hole

I understand.

> that would make the transition from 2.8 to 2.9 just as hard as the transition
> from 2.7 to 2.8 was.

I don't think this is the case though. In 2.8, there were more source
incompatible changes (including some that required compilation with
@migration to spot). Also, due to the transition to 2.8.0, the tools
and ecosystem are much better prepared for it. Once a given Scala
release is out, the important libraries publish a new binary for it in
a short amount of time since SBT makes it so easy.

Best,
Ismael

>>>>> "Ismael" == Ismael Juma writes:

>> that would make the transition from 2.8 to 2.9 just as hard as the
>> transition from 2.7 to 2.8 was.

Ismael> I don't think this is the case though. In 2.8, there were more
Ismael> source incompatible changes (including some that required
Ismael> compilation with @migration to spot).

Agree. When we moved to 2.8 my codebase required lots of changes
(little ones mostly, but lots of them), 2.9 so far very few.

Ismael> Also, due to the transition to 2.8.0, the tools and ecosystem
Ismael> are much better prepared for it. Once a given Scala release is
Ismael> out, the important libraries publish a new binary for it in a
Ismael> short amount of time since SBT makes it so easy.

Yup.

ticket today:
http://lampsvn.epfl.ch/trac/scala/ticket/4517

--
Seth Tisue | Northwestern University | http://tisue.net
lead developer, NetLogo: http://ccl.northwestern.edu/netlogo/

Clearly, most respondents on this list have indicated they don't see
maintaining 2.8 -> 2.9 binary compatibility as a priority.

However, I suspect there is huge "sample bias" among the respondents:
people who subscribe to scala internals are expert scala users & devs,
the *least* likely users to be troubled by binary compatibility
problems.

There's probably alot of Scala newbies out there largely unaware of
(a) exactly what Scala versions they are using, and (b) the whole
binary compatibility issue. All they would perceive is that "scala 2.9
doesn't work".

While I personally would find 95% or 99% binary-compatibility
annoyingly incomplete, a scala novice playing with the language would
probably prefer that since they are much less likely to encounter the
1-5% that breaks.

-Ben

On Thu, Apr 28, 2011 at 3:30 AM, Ben Hutchison wrote:
> Clearly, most respondents on this list have indicated they don't see
> maintaining 2.8 -> 2.9 binary compatibility as a priority.

To make it clear, I think binary compatibility would be great. The
form of incomplete binary compatibility that 2.9.0 will have is not
useful in my opinion and I believe it may even be harmful if the wrong
message is given to the users. I can already predict users showing up
on IRC, mailing lists and Trac with weird errors caused by running
some libraries compiled against 2.8.1 and others compiled against
2.9.0. It has the potential to be a support nightmare, so I hope the
right message is delivered. In terms of the bridge code, that decision
is made and Martin has already committed the code.

> There's probably alot of Scala newbies out there largely unaware of
> (a) exactly what Scala versions they are using, and (b) the whole
> binary compatibility issue. All they would perceive is that "scala 2.9
> doesn't work".

These are the users I care about to be honest. They are much more
likely to think Scala is unreliable if it works for simple examples
and then fails in more complex ones. Fail-fast is a good thing.

> While I personally would find 95% or 99% binary-compatibility
> annoyingly incomplete, a scala novice playing with the language would
> probably prefer that since they are much less likely to encounter the
> 1-5% that breaks.

We like the compiler to tell us about problems, so this attitude
puzzles me. Fail-fast is a good thing.

Best,
Ismael

I'll respond to scala-users to clarify where we are.

 -- Martin


On Thu, Apr 28, 2011 at 9:45 AM, Ismael Juma <ismael [at] juma [dot] me [dot] uk> wrote:

On Thu, Apr 28, 2011 at 3:30 AM, Ben Hutchison <ben [at] playscapegames [dot] com> wrote:
> Clearly, most respondents on this list have indicated they don't see
> maintaining 2.8 -> 2.9 binary compatibility as a priority.

To make it clear, I think binary compatibility would be great. The
form of incomplete binary compatibility that 2.9.0 will have is not
useful in my opinion and I believe it may even be harmful if the wrong
message is given to the users. I can already predict users showing up
on IRC, mailing lists and Trac with weird errors caused by running
some libraries compiled against 2.8.1 and others compiled against
2.9.0. It has the potential to be a support nightmare, so I hope the
right message is delivered. In terms of the bridge code, that decision
is made and Martin has already committed the code.

> There's probably alot of Scala newbies out there largely unaware of
> (a) exactly what Scala versions they are using, and (b) the whole
> binary compatibility issue. All they would perceive is that "scala 2.9
> doesn't work".

These are the users I care about to be honest. They are much more
likely to think Scala is unreliable if it works for simple examples
and then fails in more complex ones. Fail-fast is a good thing.

> While I personally would find 95% or 99% binary-compatibility
> annoyingly incomplete, a scala novice playing with the language would
> probably prefer that since they are much less likely to encounter the
> 1-5% that breaks.

We like the compiler to tell us about problems, so this attitude
puzzles me. Fail-fast is a good thing.

Best,
Ismael

--
----------------------------------------------
Martin Odersky
Prof., EPFL and CEO, Scala Solutions
PSED, 1015 Lausanne, Switzerland