This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Eclipse plugin health?

40 replies
Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Hi,

I've been pretty busy at work. I was just wondering how the Eclipse plugin was holding on after 2.7.1/2 was released. I should pretend that little activity is probably good news. However being the pessimist I am, I would take a lack of complaints about the plugin as a sign of hopelessness and despair :)

Its a bit depressing not being able to program in Scala. I haven't even fired up the plugin myself for over a couple of months now.

Sean
daniel
Joined: 2008-08-20,
User offline. Last seen 44 weeks 14 hours ago.
Re: Eclipse plugin health?
Well, to be honest, the lack of comments is more toward despair than contentment on my part.  I do use the plugin now and again, but it's so unreliable (particularly with lots of case classes) that I just can't apply it to anything serious.  Don't get me wrong, I was impressed with the 2.7.2 release!  The stability you guys achieved was light-years ahead of what things had been in some of the snapshot builds, but I still don't see it as a tool that I can use for anything non-trivial.  My silence has been mostly due to the fact that previous remarks on the instability have been mostly ineffective.

What I think the plugin really needs is a couple people to really "own" it.  Developers who can sit down and hammer out why the editor is so flaky and how it can be repaired.  Miles has been great (is he gone now?), but he was always focused on bringing the plugin's features to a level worthy of recognition.  I think that SDT has reached the point where it is useful feature-wise, it just needs a lot more work to make it a reliable tool.  For starters, a working forward-delete key might be a nice "feature".  :-)  Random loss of highlighting and those ever-disturbing erroneous error highlights would also be a couple problems to fix.

Daniel

On Wed, Dec 17, 2008 at 12:29 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
Hi,

I've been pretty busy at work. I was just wondering how the Eclipse plugin was holding on after 2.7.1/2 was released. I should pretend that little activity is probably good news. However being the pessimist I am, I would take a lack of complaints about the plugin as a sign of hopelessness and despair :)

Its a bit depressing not being able to program in Scala. I haven't even fired up the plugin myself for over a couple of months now.

Sean

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Eclipse plugin health?

On Wed, Dec 17, 2008 at 6:29 PM, Sean McDirmid wrote:
> I've been pretty busy at work. I was just wondering how the Eclipse plugin
> was holding on after 2.7.1/2 was released. I should pretend that little
> activity is probably good news. However being the pessimist I am, I would
> take a lack of complaints about the plugin as a sign of hopelessness and
> despair :)

Well, you could, you know ... try it and see?

Cheers,

Miles

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Eclipse plugin health?

On Wed, Dec 17, 2008 at 6:41 PM, Daniel Spiewak wrote:
> For starters, a working forward-delete key might be a nice "feature". :-)

Works for me and I don't see your ticket for this in Trac ...

Cheers,

Miles

Samuel Robert Reid
Joined: 2008-12-17,
User offline. Last seen 1 year 22 weeks ago.
Re: Eclipse plugin health?
>>a working forward-delete key might be a nice "feature".  :-)

I presume that by "forward-delete key" you mean, pressing the "delete" key should delete the character to the right of the caret, leaving the caret in the same place.

This works properly for me in the scala 2.7.2final eclipse editor.  Please let me know if you meant something else.

Sam Reid

Miles Sabin wrote:

30961e500812171044w4a1beebuf4a4614f040069d [at] mail [dot] gmail [dot] com" type="cite">
On Wed, Dec 17, 2008 at 6:41 PM, Daniel Spiewak djspiewak [at] gmail [dot] com (<djspiewak [at] gmail [dot] com>) wrote:
  
For starters, a working forward-delete key might be a nice "feature".  :-)
    
Works for me and I don't see your ticket for this in Trac ...

Cheers,


Miles

  
daniel
Joined: 2008-08-20,
User offline. Last seen 44 weeks 14 hours ago.
Re: Eclipse plugin health?
Assumed it was a known issue that just hadn't been reported (single-dev syndrome) and so I didn't want to spam your ticket queue.  :-)

Here ya go!  https://lampsvn.epfl.ch/trac/scala/ticket/1588

Daniel

On Wed, Dec 17, 2008 at 12:44 PM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Wed, Dec 17, 2008 at 6:41 PM, Daniel Spiewak <djspiewak [at] gmail [dot] com> wrote:
> For starters, a working forward-delete key might be a nice "feature".  :-)

Works for me and I don't see your ticket for this in Trac ...

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

Samuel Robert Reid
Joined: 2008-12-17,
User offline. Last seen 1 year 22 weeks ago.
Re: Eclipse plugin health?

I tried out the Eclipse plugin as of 12-15-2008, known as "scala eclipse
plugin 2.7.2.final". I'm new to scala and to eclipse, but here are the
features I noticed:

1. Able to handle mixed scala/java projects.
2. Good syntax highlighting
3. 'Open declaration' works for both Java and Scala source (though this
only worked through the context menu, not through F3 keystroke)
4. Easy to set up and run a scala application
5. Error highlighting
6. Debugger looks like it's working properly

High priority features that aren't there or I couldn't figure out:
1. Automatic imports.
2. Rename/refactor
3. Find usages
4. Autocomplete for some scenarios
5. Autocomplete by type

In a few cases, the compiler and the editor became out of sync; the code
would compile, but the editor complained and wouldn't continue. When I
got into this scenario, restarting eclipse seemed to sync up the editor.

Let me know how it goes with you,
Sam Reid

Sean McDirmid wrote:
> Hi,
>
> I've been pretty busy at work. I was just wondering how the Eclipse
> plugin was holding on after 2.7.1/2 was released. I should pretend
> that little activity is probably good news. However being the
> pessimist I am, I would take a lack of complaints about the plugin as
> a sign of hopelessness and despair :)
>
> Its a bit depressing not being able to program in Scala. I haven't
> even fired up the plugin myself for over a couple of months now.
>
> Sean

Naftoli Gugenheim
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?

I thought I read that there was some behind the scenes getting help
from the AspectJ developers?

On 12/17/08, Daniel Spiewak wrote:
> Assumed it was a known issue that just hadn't been reported (single-dev
> syndrome) and so I didn't want to spam your ticket queue. :-)
>
> Here ya go! https://lampsvn.epfl.ch/trac/scala/ticket/1588
>
> Daniel
>
> On Wed, Dec 17, 2008 at 12:44 PM, Miles Sabin wrote:
>
>> On Wed, Dec 17, 2008 at 6:41 PM, Daniel Spiewak
>> wrote:
>> > For starters, a working forward-delete key might be a nice "feature".
>> :-)
>>
>> Works for me and I don't see your ticket for this in Trac ...
>>
>> Cheers,
>>
>>
>> Miles
>>
>> --
>> Miles Sabin
>> tel: +44 (0)1273 720 779
>> mobile: +44 (0)7813 944 528
>> skype: milessabin
>>
>

imaier
Joined: 2008-07-01,
User offline. Last seen 23 weeks 2 days ago.
Re: Eclipse plugin health?

Well, I don't have the feeling it has become more usable. Maybe it's my
projects, but I have to close and reopen and clean build my projects
more than ever. I cannot open editors, eclipse throws errors at me when
I want to open a context menu of a Scala file, the editor screws up my
source code (not on disk, just the display), renaming a file (from the
Scala submenu) doesn't really work when it's currently open (getting a
nice smearing effect) and incremental compilation too often doesn't work
so I end up cleaning projects more than necessary. Unfortunately, I
don't have much time to investigate eclipse bugs and they seem so
obvious to me that I actually believe that someone must have noticed
them before.

Ingo

Sean McDirmid wrote:
> Hi,
>
> I've been pretty busy at work. I was just wondering how the Eclipse
> plugin was holding on after 2.7.1/2 was released. I should pretend that
> little activity is probably good news. However being the pessimist I am,
> I would take a lack of complaints about the plugin as a sign of
> hopelessness and despair :)
>
> Its a bit depressing not being able to program in Scala. I haven't even
> fired up the plugin myself for over a couple of months now.
>
> Sean

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Eclipse plugin health?

On Wed, Dec 17, 2008 at 6:56 PM, Naftoli Gugenheim wrote:
> I thought I read that there was some behind the scenes getting help
> from the AspectJ developers?

Yes, that's correct. As of 2.8.0 the plugin's JDT integration will be
using the JDT weaving feature developed by the AJDT people,

http://wiki.eclipse.org/JDT_weaving_features

This will dramatically improve the depth and robustness of the JDT
integration over 2.7.2 (and the upcoming 2.7.3). This should be
landing on trunk by the end of the week.

Cheers,

Miles

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Eclipse plugin health?

This is more my bad. I've been slacking on my goal of stabilizing the
plugin on windows. Just use Linux from now on :)

I have been able to use the plugin with a maven plugin successfully on
windows. My issues are more with trying to develop/bugfix the plugin
itself on windows.

I'm of the opinion that unless I care enough to look at the issue,
then I shouldn't complain too loudly. That being said, I have a lot
of confidence in miles ability to work out bugs. I just wish more
people would help out..

On Dec 17, 2008, at 1:44 PM, "Miles Sabin" wrote:

> On Wed, Dec 17, 2008 at 6:41 PM, Daniel Spiewak
> wrote:
>> For starters, a working forward-delete key might be a nice
>> "feature". :-)
>
> Works for me and I don't see your ticket for this in Trac ...
>
> Cheers,
>
>
> Miles
>

Chris Twiner
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?

I just wanted to add some comments, I've been using the plugin fairly
frequently for almost a year. I have, I assume, a fairly unique setup
of using OSGi / PDE exclusively.

Sam's list is a great starting point for my comments

On Wed, Dec 17, 2008 at 7:44 PM, Samuel Robert Reid wrote:
> I tried out the Eclipse plugin as of 12-15-2008, known as "scala eclipse
> plugin 2.7.2.final". I'm new to scala and to eclipse, but here are the
> features I noticed:
>
> 1. Able to handle mixed scala/java projects.
> 2. Good syntax highlighting
> 3. 'Open declaration' works for both Java and Scala source (though this only
> worked through the context menu, not through F3 keystroke)
> 4. Easy to set up and run a scala application
> 5. Error highlighting

can't agree more

> 6. Debugger looks like it's working properly

it works up to the point of the expressions view. That would be the
best possible addition to the plugin debug experience imo.

> High priority features that aren't there or I couldn't figure out:
> 1. Automatic imports.

I don't really mind an absence of this, I think that due to my
approach of "import where you need to" it probably can't help much.

> 2. Rename/refactor

I don't use this nearly as much working with scala as I do in java but
the times I've needed rename, search replace have worked fine. The
only other refactoring items I'd like is extract function / constant.
Those would be great.

> 3. Find usages

Total godsend if this was introduced. Features like the outline (and
resulting ctrl-o usage) were absolutely excellent, find usages I
thought would have derived from this, but seems not to have been.

> 4. Autocomplete for some scenarios
> 5. Autocomplete by type

for some reason the autocomplete seems patchy, but often its a simple
close editor, clean and reopen.

What I'd add to the wish list is that under pde / osgi usage, if I add
classes, or functions to classes etc to one project they aren't
visible in the other projects until I do a clean / rebuild on the
providing project. This alone would save me 60% of the rebuilds.

Having said that the plugin is 20 times better (arbritary number
pulled from thin air) over vim/nedit/emacs/ant combinations, but thats
the reason I'm using eclipse in the first place.

Great work from Miles (and of course Sean), they have both been
extremely helpful in adding small fixes and helping with problems I've
had. I have high hopes for the future of the plugin.

many thanks,
Chris

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Eclipse plugin health?

On Wed, Dec 17, 2008 at 6:50 PM, Daniel Spiewak wrote:
> Assumed it was a known issue that just hadn't been reported (single-dev
> syndrome) and so I didn't want to spam your ticket queue. :-)
>
> Here ya go! https://lampsvn.epfl.ch/trac/scala/ticket/1588

Ahh ... you're on a Mac ...

I'm sure I've mentioned this before, but I don't have a Mac to test
the plugin on, so contributions (of bug reports, patches) from Mac
users are especially important.

Cheers,

Miles

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?
Case classes are a  sore point with me, as they got changed in a way in 2.7 that broke the plugin in a huge way that I couldn't hack around very well. I'm still waiting for Martin to do something about this or tell us what we should do about it.

The editor needs to be completely revamped as I think I'm using the Eclipse editor interface wrong. The problem is, there is no documentation on how to use it right...its all trial and error.

I agree we need more people to own features, especially now that I think I'm officially semi-retired from the project (at least until something decent comes out for .NET).

On Thu, Dec 18, 2008 at 2:41 AM, Daniel Spiewak <djspiewak [at] gmail [dot] com> wrote:
Well, to be honest, the lack of comments is more toward despair than contentment on my part.  I do use the plugin now and again, but it's so unreliable (particularly with lots of case classes) that I just can't apply it to anything serious.  Don't get me wrong, I was impressed with the 2.7.2 release!  The stability you guys achieved was light-years ahead of what things had been in some of the snapshot builds, but I still don't see it as a tool that I can use for anything non-trivial.  My silence has been mostly due to the fact that previous remarks on the instability have been mostly ineffective.

What I think the plugin really needs is a couple people to really "own" it.  Developers who can sit down and hammer out why the editor is so flaky and how it can be repaired.  Miles has been great (is he gone now?), but he was always focused on bringing the plugin's features to a level worthy of recognition.  I think that SDT has reached the point where it is useful feature-wise, it just needs a lot more work to make it a reliable tool.  For starters, a working forward-delete key might be a nice "feature".  :-)  Random loss of highlighting and those ever-disturbing erroneous error highlights would also be a couple problems to fix.

Daniel

On Wed, Dec 17, 2008 at 12:29 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
Hi,

I've been pretty busy at work. I was just wondering how the Eclipse plugin was holding on after 2.7.1/2 was released. I should pretend that little activity is probably good news. However being the pessimist I am, I would take a lack of complaints about the plugin as a sign of hopelessness and despair :)

Its a bit depressing not being able to program in Scala. I haven't even fired up the plugin myself for over a couple of months now.

Sean


Chris Twiner
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?

On Wed, Dec 17, 2008 at 8:04 PM, Miles Sabin wrote:
> On Wed, Dec 17, 2008 at 6:56 PM, Naftoli Gugenheim wrote:
>> I thought I read that there was some behind the scenes getting help
>> from the AspectJ developers?
>
> Yes, that's correct. As of 2.8.0 the plugin's JDT integration will be
> using the JDT weaving feature developed by the AJDT people,
>
> http://wiki.eclipse.org/JDT_weaving_features
>
> This will dramatically improve the depth and robustness of the JDT
> integration over 2.7.2 (and the upcoming 2.7.3). This should be
> landing on trunk by the end of the week.
>
Hi Miles,

just to clarify, as an Xmas present the plugin should get another leap
in functionality? Please drop an email on scala-tools and let us know
when its out. I for one would love to help testing that out.

yours droolingly,
Chris

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?
What would I do with it? :p

On Thu, Dec 18, 2008 at 2:40 AM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Wed, Dec 17, 2008 at 6:29 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> I've been pretty busy at work. I was just wondering how the Eclipse plugin
> was holding on after 2.7.1/2 was released. I should pretend that little
> activity is probably good news. However being the pessimist I am, I would
> take a lack of complaints about the plugin as a sign of hopelessness and
> despair :)

Well, you could, you know ... try it and see?

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

Sciss
Joined: 2008-12-17,
User offline. Last seen 28 weeks 5 days ago.
Re: Eclipse plugin health?

hi sean, hi miles,

first of all i must say i think the plug-in stability has greately
improved with 2.7.2, but then your assumption about the "silence" is
right, i got a little disappears with the all the remaining problems
in everyday work. i think they are very obvious and there are a lot
(maybe just a few bugs but they are omnipresent) so i didn't yet talk
back....

priority wise, i think the most severe problems are
- fixing mistakes usually produces an avalange of weird unrelated
compiler errors, so the incremental compiling doesn't work at all. i
basically (as i have auto-build enabled) hit "clean" for every error
i fix, this costs a lot of time and energy.
- the editor usually gets screwed up after a while, often in
combination with copy and paste:
- code completion disappears usually pretty quickly (i think i have
a lot of .scala files where code completion isn't even offered even
if i open the editor from scratch)
- the outline either doesn't show the complete outline, or it shows
it but clicking on the items won't bring me to the correct line (but
to like the top of the source file)
- indentation has a lot of problems; i'm not sure but my theory is
that the plug-in tries to look at lines _below_ the line you want to
indent (hitting the tab key), so the line gets shifted to a weird
position, where it should in the first place consider the previous
line. i can have for example three lines of line comments // and
while indention is correct for like the first two lines, the third
line (for example) gets indented completely wrong. also if the
previous line contains more than one statement with a semicolon, like
"gaga.make; gugu.make" the identation of the next line is wrong. this
is a very annoying issue
- missing features: toggle line comments (Cmd+Shift+7 (german
keyboard) in the java plug-in)
- refactor -> rename doesn't work properly (it always ends with an
error dialog, although it starts to convert files i think)

other annoying things
- the debugger is very cool and usefull. however, when you hit the go-
into button (to branch into a method), you often end up in
ClassLoader.class for some reason, this should be skipped i think

minor issues
- sometimes .scala files appear duplicate in the project browser (at
some point the duplicates disappear again i think)

cool features would be
- a preferences of compiler warnings, like in the java-plug in, like
to get warnings about unused local variables, shadowing of class
fields by local variables etc.

thanks, -sciss-

Am 17.12.2008 um 19:29 schrieb Sean McDirmid:

> Hi,
>
> I've been pretty busy at work. I was just wondering how the Eclipse
> plugin was holding on after 2.7.1/2 was released. I should pretend
> that little activity is probably good news. However being the
> pessimist I am, I would take a lack of complaints about the plugin
> as a sign of hopelessness and despair :)
>
> Its a bit depressing not being able to program in Scala. I haven't
> even fired up the plugin myself for over a couple of months now.
>
> Sean

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Eclipse plugin health?

On Wed, Dec 17, 2008 at 8:22 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> Case classes are a sore point with me, as they got changed in a way in 2.7
> that broke the plugin in a huge way that I couldn't hack around very well.
> I'm still waiting for Martin to do something about this or tell us what we
> should do about it.
>
Frankly, I think this a strong indication that the plugin is tied too
closely to the compiler. If it's going to break everytime we change
some datastructure in the compiler we have tough times ahead of us. I
don't think I can do anything about it; it's up to the Eclipse plugin
developers to make it more robust wrt compiler changes.

Cheers

Michael Neale
Joined: 2008-12-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?

I believe this is (possibly) what prompted eclipse looong ago to come
up with JDT - something they could control deeply.

I have been using IntelliJ for a while - its plugin seems to be
reasonably stable (those who have tried it on larger apps - please
jump in - my experience is really quite shallow with it) - but the
core things seem to work well (when it crashes - it tends to happen on
startup and can be fixed) - I generally can always do what I need to
do in it, and the usual tools work quite well.

I am not sure how brittle it is with compiler versions though - I have
noticed that sometimes the content awareness is not inline with scalac
- but that doesn't actually stop things from working.

On Thu, Dec 18, 2008 at 9:47 AM, martin odersky <martin [dot] odersky [at] epfl [dot] ch> wrote:
> On Wed, Dec 17, 2008 at 8:22 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
>> Case classes are a sore point with me, as they got changed in a way in 2.7
>> that broke the plugin in a huge way that I couldn't hack around very well.
>> I'm still waiting for Martin to do something about this or tell us what we
>> should do about it.
>>
> Frankly, I think this a strong indication that the plugin is tied too
> closely to the compiler. If it's going to break everytime we change
> some datastructure in the compiler we have tough times ahead of us. I
> don't think I can do anything about it; it's up to the Eclipse plugin
> developers to make it more robust wrt compiler changes.
>
> Cheers
>

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?
Yes, its the huge tradeoff that we talked about at the beginning of this project. On the one hand, if we implement our own compiler (like the IntelliJ effort), we have to re-implement the Scala language and we'll have to worry about language drift. On the one hand, if we integrate too closely with Scalac, then we are at the mercy of the compiler writer to not change the architecture too much. We are taking an approach similar to the JDT team, but then the team in France knows they really have to support the team in Zurich, otherwise they don't have a product. They had a lot of arguments when the effort first started because the compiler people in France couldn't really understand the design requirements of the IDE people in Zurich.

In my opinion, the case class apply/unapply method issue is very unique:  it really breaks the mostly clean compiler model since it reaches out across trees and does a "if the method doesn't exist in that tree over there, make it exist in that tree over there" test that just can't be rationalized when trees are being compiled separately. Nothing else in the compiler is really written in such a non-modular way (with respect to modularity in the tree data structure): tree data structures are mostly processed by themselves way where inter-tree communication between trees occurs through scopes in a way that we can track and manage. I can deal with that model, I can't deal with code that reaches out to another top-level tree directly and by completely passes namers/type completion/scopes.

The problem isn't really changes to data structures in the compiler, its changes to the underlying design principles that the compiler was written under; i.e., huge hacks to support new features that are written in such weird ways because the existing compiler infrastructure doesn't support the feature very well (also known as compiler rot) are going to kill the IDE.

Anyways, overall, I'm surprised how well the technique works. Of course, when you make changes to the compiler someone who understands the compiler really well (me) has to make sure that they haven't broken the IDE, but this hasn't been a huge time draw. I am worried that no one can really replace me in this regard, as learning scalac internals takes a long time and a lot of discussion/working with Martin.


On Thu, Dec 18, 2008 at 6:47 AM, martin odersky <martin [dot] odersky [at] epfl [dot] ch> wrote:
On Wed, Dec 17, 2008 at 8:22 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> Case classes are a  sore point with me, as they got changed in a way in 2.7
> that broke the plugin in a huge way that I couldn't hack around very well.
> I'm still waiting for Martin to do something about this or tell us what we
> should do about it.
>
Frankly, I think this a strong indication that the plugin is tied too
closely to the compiler. If it's going to break everytime we change
some datastructure in the compiler we have tough times ahead of us. I
don't think I can do anything about it; it's up to the Eclipse plugin
developers to make it more robust wrt compiler changes.

Cheers

 -- Martin

Michael Neale
Joined: 2008-12-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?

Sean - I have endless admiration for people who do what you do. I have
a bit of experience with an evolving language and an eclipse IDE tool
to support it - to this day we still can't/won't separate our IDE
release cycles from the language cause its just really hard work !

Is it possible or desirable to have IDE tooling that *mostly* works
and is useful in almost all case? (ie when it comes across a case it
can't deal with, pun intended, it just soldiers on?) - is that a
realistic goal? I guess I haven't done enough to know how much the
short comings of the IDEs would let me down.

On Thu, Dec 18, 2008 at 1:39 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> Yes, its the huge tradeoff that we talked about at the beginning of this
> project. On the one hand, if we implement our own compiler (like the
> IntelliJ effort), we have to re-implement the Scala language and we'll have
> to worry about language drift. On the one hand, if we integrate too closely
> with Scalac, then we are at the mercy of the compiler writer to not change
> the architecture too much. We are taking an approach similar to the JDT
> team, but then the team in France knows they really have to support the team
> in Zurich, otherwise they don't have a product. They had a lot of arguments
> when the effort first started because the compiler people in France couldn't
> really understand the design requirements of the IDE people in Zurich.
>
> In my opinion, the case class apply/unapply method issue is very unique: it
> really breaks the mostly clean compiler model since it reaches out across
> trees and does a "if the method doesn't exist in that tree over there, make
> it exist in that tree over there" test that just can't be rationalized when
> trees are being compiled separately. Nothing else in the compiler is really
> written in such a non-modular way (with respect to modularity in the tree
> data structure): tree data structures are mostly processed by themselves way
> where inter-tree communication between trees occurs through scopes in a way
> that we can track and manage. I can deal with that model, I can't deal with
> code that reaches out to another top-level tree directly and by completely
> passes namers/type completion/scopes.
>
> The problem isn't really changes to data structures in the compiler, its
> changes to the underlying design principles that the compiler was written
> under; i.e., huge hacks to support new features that are written in such
> weird ways because the existing compiler infrastructure doesn't support the
> feature very well (also known as compiler rot) are going to kill the IDE.
>
> Anyways, overall, I'm surprised how well the technique works. Of course,
> when you make changes to the compiler someone who understands the compiler
> really well (me) has to make sure that they haven't broken the IDE, but this
> hasn't been a huge time draw. I am worried that no one can really replace me
> in this regard, as learning scalac internals takes a long time and a lot of
> discussion/working with Martin.
>
>
> On Thu, Dec 18, 2008 at 6:47 AM, martin odersky <martin [dot] odersky [at] epfl [dot] ch>
> wrote:
>>
>> On Wed, Dec 17, 2008 at 8:22 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com>
>> wrote:
>> > Case classes are a sore point with me, as they got changed in a way in
>> > 2.7
>> > that broke the plugin in a huge way that I couldn't hack around very
>> > well.
>> > I'm still waiting for Martin to do something about this or tell us what
>> > we
>> > should do about it.
>> >
>> Frankly, I think this a strong indication that the plugin is tied too
>> closely to the compiler. If it's going to break everytime we change
>> some datastructure in the compiler we have tough times ahead of us. I
>> don't think I can do anything about it; it's up to the Eclipse plugin
>> developers to make it more robust wrt compiler changes.
>>
>> Cheers
>>
>> -- Martin
>
>

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?
The IDE is supposed to work like this: we've hardened it to the point that when it begins to fail it shouldn't bring down the entire user experience. There are a few places where this doesn't work so well:

* When the IDE get's confused about what symbols exist and don't, the type checker degrades and begins to indicate false errors and can't provide certain services on identifiers. The failure will then cascade and you'll have large parts of code that aren't processed correctly by the IDE. This is especially true in Scala because of type inference and its heavy use of nested expressions.

* Sometimes the editor state gets really screwed up. This isn't the compiler or anything, its really the internal Eclipse data structures that are getting corrupted.

* Certain bugs in the compiler (when they exist) can become exacerbated in the IDE. E.g., if there is a infinite/long computation in the IDE, this will manifest itself as something that won't let you save your file (because we want the compiler to finish before we save, even if its running in a background thread).

The architecture is partly to blame for our vulnerability to the compiler and Eclipse, I just didn't isolate the IDE components well enough from them. In particular, we integrate very closely with the compiler more so than the JDT does: we chose a tree-incremental approach vs. a background file-incremental compiler backed up by a heuristic compiler for real-time services. I'm not sure anymore if this was the right choice, the JDT approach works well and is robust, even if it might not work on very large files (Java files tend to be simple, while the JDT compiler tends to be very fast).

There are lots of things we can do to improve user experience. One thing I've tried to do is ensure that the crashes I know about don't bring down the IDE and just fill up the error log. This is kind of worked and now users can ignore more problems in the IDE. We still have a long way to go, but...I simply can't work on this now given that I've sold my soul to the other side of the force. I'm hoping people like Miles and Caoyuan can step in, and once I can incorporate Scala back into my work, I'd like to continue work on this also. 

On Thu, Dec 18, 2008 at 1:29 PM, Michael Neale <michael [dot] neale [at] gmail [dot] com> wrote:
Sean - I have endless admiration for people who do what you do. I have
a bit of experience with an evolving language and an eclipse IDE tool
to support it - to this day we still can't/won't separate our IDE
release cycles from the language cause its just really hard work !

Is it possible or desirable to have IDE tooling that *mostly* works
and is useful in almost all case? (ie when it comes across a case it
can't deal with, pun intended, it just soldiers on?) - is that a
realistic goal? I guess I haven't done enough to know how much the
short comings of the IDEs would let me down.



On Thu, Dec 18, 2008 at 1:39 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> Yes, its the huge tradeoff that we talked about at the beginning of this
> project. On the one hand, if we implement our own compiler (like the
> IntelliJ effort), we have to re-implement the Scala language and we'll have
> to worry about language drift. On the one hand, if we integrate too closely
> with Scalac, then we are at the mercy of the compiler writer to not change
> the architecture too much. We are taking an approach similar to the JDT
> team, but then the team in France knows they really have to support the team
> in Zurich, otherwise they don't have a product. They had a lot of arguments
> when the effort first started because the compiler people in France couldn't
> really understand the design requirements of the IDE people in Zurich.
>
> In my opinion, the case class apply/unapply method issue is very unique:  it
> really breaks the mostly clean compiler model since it reaches out across
> trees and does a "if the method doesn't exist in that tree over there, make
> it exist in that tree over there" test that just can't be rationalized when
> trees are being compiled separately. Nothing else in the compiler is really
> written in such a non-modular way (with respect to modularity in the tree
> data structure): tree data structures are mostly processed by themselves way
> where inter-tree communication between trees occurs through scopes in a way
> that we can track and manage. I can deal with that model, I can't deal with
> code that reaches out to another top-level tree directly and by completely
> passes namers/type completion/scopes.
>
> The problem isn't really changes to data structures in the compiler, its
> changes to the underlying design principles that the compiler was written
> under; i.e., huge hacks to support new features that are written in such
> weird ways because the existing compiler infrastructure doesn't support the
> feature very well (also known as compiler rot) are going to kill the IDE.
>
> Anyways, overall, I'm surprised how well the technique works. Of course,
> when you make changes to the compiler someone who understands the compiler
> really well (me) has to make sure that they haven't broken the IDE, but this
> hasn't been a huge time draw. I am worried that no one can really replace me
> in this regard, as learning scalac internals takes a long time and a lot of
> discussion/working with Martin.
>
>
> On Thu, Dec 18, 2008 at 6:47 AM, martin odersky <martin [dot] odersky [at] epfl [dot] ch>
> wrote:
>>
>> On Wed, Dec 17, 2008 at 8:22 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com>
>> wrote:
>> > Case classes are a  sore point with me, as they got changed in a way in
>> > 2.7
>> > that broke the plugin in a huge way that I couldn't hack around very
>> > well.
>> > I'm still waiting for Martin to do something about this or tell us what
>> > we
>> > should do about it.
>> >
>> Frankly, I think this a strong indication that the plugin is tied too
>> closely to the compiler. If it's going to break everytime we change
>> some datastructure in the compiler we have tough times ahead of us. I
>> don't think I can do anything about it; it's up to the Eclipse plugin
>> developers to make it more robust wrt compiler changes.
>>
>> Cheers
>>
>>  -- Martin
>
>



--
Michael D Neale
home: www.michaelneale.net
blog: michaelneale.blogspot.com

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Eclipse plugin health?

Hi Sean,

OK let's see how we can get case classes unstuck. I'll describe the
current scheme first:

For setters and getters, namer creates new symbols that refer to the
original field definition, but that have special type completers so
that we know we are dealing with a setter/getter, not a field. Then,
in typerphase, when we see a field, we also generate setter and getter
trees for it and patch them so their symbol fields point to the
peviously generated symbols.

Pre-2.7 we did analogous things with case-class factories. The
disadvantage of this scheme is that it's rather ad hoc and smears work
over namers and typers.

For case class related things in 2.7, I put a new scheme in place
which I thought was more systematic. The idea was that we would then
also migrate setters and getters to the new scheme.

The scheme is to have a `synthetics' collection in CompilationUnit.
When namer finds it has to create a new symbol (for instance for
companion object of a case class, or its apply or unapply method), it
immediately creates an unattributed tree for it. The tree is then
processed as normal, which leads to the correct symbol to be created.
The symbol is entered into the right scope automatically. Then, during
typerphase all symbols in a scope are traversed. If a synthetic symbol
is found, its corresponding tree is incorporated into the result tree.
The big advantage is that it's one localized solution rather than a
dozen hacks, one for each class of synthetic symbols.

Because of this advantage I am reluctant to give the new scheme up.
Can we analyze what needs to be done to make the plugin happy? I mean,
could we maybe create the symbols as described above but have their
positions point to the originating tree instead of the synthetic tree,
much as we do with setters and getters now? Would that already be
enough?

Cheers

Erkki Lindpere
Joined: 2008-12-19,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?

Yes, I have the same opinion. Lots of strange things happen like moving
a file will break undo/redo for ALL Scala editors etc. The bugs are so
numerous and feel so obvious that I haven't bothered reporting them. Of
course, as a software developer, I shouldn't assume their obviousness...
So I'll try to report at least one bug a week from now on :)

I still use the plugin though and just try to ignore the pain, mostly
because I strongly prefer Eclipse (and/or don't want to cough up the
dough for IDEA).

Erkki

Ingo Maier wrote:
> Well, I don't have the feeling it has become more usable. Maybe it's
> my projects, but I have to close and reopen and clean build my
> projects more than ever. I cannot open editors, eclipse throws errors
> at me when I want to open a context menu of a Scala file, the editor
> screws up my source code (not on disk, just the display), renaming a
> file (from the Scala submenu) doesn't really work when it's currently
> open (getting a nice smearing effect) and incremental compilation too
> often doesn't work so I end up cleaning projects more than necessary.
> Unfortunately, I don't have much time to investigate eclipse bugs and
> they seem so obvious to me that I actually believe that someone must
> have noticed them before.
>
> Ingo
>
> Sean McDirmid wrote:
>> Hi,
>>
>> I've been pretty busy at work. I was just wondering how the Eclipse
>> plugin was holding on after 2.7.1/2 was released. I should pretend
>> that little activity is probably good news. However being the
>> pessimist I am, I would take a lack of complaints about the plugin as
>> a sign of hopelessness and despair :)
>>
>> Its a bit depressing not being able to program in Scala. I haven't
>> even fired up the plugin myself for over a couple of months now.
>>
>> Sean
>

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?
You are kind of right: these bugs are obvious and the only way we could fix them is through a redesign of the plugin/editor interface. We should make that a priority, I mean, I'm willing to help out but my time is basically gone now.

I think you can use IntelliJ Scala plugin for free or something. They might know on this list. The IntelliJ/NetBeans plugins are actually much more stable than the Eclipse plugin given the approach they take is more stable and they interface with the editor in a much better way. But also given the approach, the Eclipse plugin wins on features and the ability to handle big files.

Sean

On Sat, Dec 20, 2008 at 6:28 AM, Erkki Lindpere <erkki [at] lap [dot] ee> wrote:
Yes, I have the same opinion. Lots of strange things happen like moving a file will break undo/redo for ALL Scala editors etc. The bugs are so numerous and feel so obvious that I haven't bothered reporting them. Of course, as a software developer, I shouldn't assume their obviousness... So I'll try to report at least one bug a week from now on :)

I still use the plugin though and just try to ignore the pain, mostly because I strongly prefer Eclipse (and/or don't want to cough up the dough for IDEA).

Erkki


Ingo Maier wrote:
Well, I don't have the feeling it has become more usable. Maybe it's my projects, but I have to close and reopen and clean build my projects more than ever. I cannot open editors, eclipse throws errors at me when I want to open a context menu of a Scala file, the editor screws up my source code (not on disk, just the display), renaming a file (from the Scala submenu) doesn't really work when it's currently open (getting a nice smearing effect) and incremental compilation too often doesn't work so I end up cleaning projects more than necessary. Unfortunately, I don't have much time to investigate eclipse bugs and they seem so obvious to me that I actually believe that someone must have noticed them before.

Ingo

Sean McDirmid wrote:
Hi,

I've been pretty busy at work. I was just wondering how the Eclipse plugin was holding on after 2.7.1/2 was released. I should pretend that little activity is probably good news. However being the pessimist I am, I would take a lack of complaints about the plugin as a sign of hopelessness and despair :)

Its a bit depressing not being able to program in Scala. I haven't even fired up the plugin myself for over a couple of months now.

Sean


Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?
Hi Martin,

I'm pretty immersed in the compiler so I understand the scheme. The getters/setters fields can be kind of difficult to deal with (it took some code to figure out) but they only involve one Scala tree, so we can process the logic in one tree This is important: if we have to process something in two places we are screwed as the trees will eventually not be processed together.

The current scheme is simply unmanageable in the plugin no matter how much I hack. Its easy to break once people start changing the name of a case class, and heck it will even break the resident compiler if you add a companion object to another file after the case class was compiled.  The older scheme was manageable because it basically disallowed a companion object, and the method created could be completely owned/managed by the case class. The current scheme would also be completely managebale if we just disallowed case classes from having companion objects, because it would basically degenerate into the older scheme. The current scheme absolutely requires that the companion object header and case class header are compiled together, but the only way for me to ensure that is to not memoize class and object definitions, which would destroy the plugin's performance.

The synthetic apply/unapply methods need to be owned by the case class and not by the companion object, at least during type checking. Here is a hacky idea: for case class Foo create a new synthetic object FooXX (that can't be seen) and add the apply/unapply methods to FooXX. In method resolution, if an apply/unapply method cannot be found via normal lookup, and the target is of Foo type, check FooXX for the methods. After type checking, move the apply/unapply methods into a fresh or existing Foo companion object unless they already exist: if the Foo companion already re-defines apply or unapply, the apply/unapply method is never used and doesn't need to be moved over.

All my other ideas basically resolve around getting apply/unapply out of the companion object during type checking since this is the part that requires the big reach across tree boundaries, even simply causing the companion object to exists is kind of a problem. On my side, I guess I could try not memoizing class/object headers and instead memoize their templates. In this way, class/object headers at the same level are always compiled together, but I don't think your parser/type checker code could be easily changed to handle memoized templates. Option C would be to memoize the definitions list of a class, while option D is to just memoize method bodies and possibly case clauses. However, I think this would be too slow..

On Fri, Dec 19, 2008 at 11:35 PM, martin odersky <martin [dot] odersky [at] epfl [dot] ch> wrote:
Hi Sean,

OK let's see how we can get case classes unstuck. I'll describe the
current scheme first:

For setters and getters, namer creates new symbols that refer to the
original field definition, but that have special type completers so
that we know we are dealing with a setter/getter, not a field. Then,
in typerphase, when we see a field, we also generate setter and getter
trees for it and patch them so their symbol fields point to the
peviously generated symbols.

Pre-2.7 we did analogous things with case-class factories. The
disadvantage of this scheme is that it's rather ad hoc and smears work
over namers and typers.

For case class related things in 2.7, I put a new scheme in place
which I thought was more systematic. The idea was that we would then
also migrate setters and getters to the new scheme.

The scheme is to have a `synthetics' collection in CompilationUnit.
When namer finds it has to create a new symbol (for instance for
companion object of a case class, or its apply or unapply method), it
immediately creates an unattributed tree for it. The tree is then
processed as normal, which leads to the correct symbol to be created.
The symbol is entered into the right scope automatically. Then, during
typerphase all symbols in a scope are traversed. If a synthetic symbol
is found, its corresponding tree is incorporated into the result tree.
The big advantage is that it's one localized solution rather than a
dozen hacks, one for each class of synthetic symbols.

Because of this advantage I am reluctant to give the new scheme up.
Can we analyze what needs to be done to make the plugin happy? I mean,
could we maybe create the symbols as described above but have their
positions point to the originating tree instead of the synthetic tree,
much as we do with setters and getters now? Would that already be
enough?

Cheers

 -- Martin

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Eclipse plugin health?

Hi Sean

I got seriously behind my e-mail over the holidays; that's why I could
respond only now.

> I'm pretty immersed in the compiler so I understand the scheme. The
> getters/setters fields can be kind of difficult to deal with (it took some
> code to figure out) but they only involve one Scala tree, so we can process
> the logic in one tree This is important: if we have to process something in
> two places we are screwed as the trees will eventually not be processed
> together.
>
Hmm. But I don't think it's a good idea to hack the type system or the
compiler to satisfy that criterion. Scala is subtle enough as it is.
So we should try our utmost to make type checking clearer not to make
it more opaque. I think we need to keep apply/unapply in the companion
object.
We can play with lots of things around it: Like, we can let their
completers point to the original tree, or we can even have special
marker symbols for the sake of the Eclipse plugin. But I resist the
suggestion to compromise the integrity of the compiler for the plugins
sake.

Any idea is welcome!

Cheers

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Eclipse plugin health?

I talked to Erik Gamma shortly before Christmas, and complained about
the closedness of the JDT.
His answer was that we should just do a cp -r of it and hack it as we
like. It seems many other projects did the same thing. Is this
something we have considered? Opinions?

Cheers

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Eclipse plugin health?

On Sun, Jan 4, 2009 at 10:49 PM, martin odersky wrote:
> I talked to Erik Gamma shortly before Christmas, and complained about
> the closedness of the JDT.
> His answer was that we should just do a cp -r of it and hack it as we
> like. It seems many other projects did the same thing. Is this
> something we have considered? Opinions?

It's something I've considered, and something the AspectJ people did
at one point.

Aside from the pain and effort involved in maintaining a fork of the
JDT we'd then have to worry about interoperability with all the other
projects which depend on the JDT in some way or another.

Fortunately the AspectJ people came up with something better than a
fork. From the next release of the Eclipse AspectJ tooling they'll be
using Equinox Aspects[1] to retrofit the kind of API that we need to
JDT without having to fork or otherwise modify the JDT.

I've been working with Andrew Eisenberg on using this mechanism in the
Scala plugin for a little while now and I can report that it works
extremely well.

I'd hoped to have it committed to trunk before Xmas but unfortunately
got bogged down with other things ... expect to see something RSN ...

[1] http://www.eclipse.org/equinox/incubator/aspects/

Cheers,

Miles

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Eclipse plugin health?

On Mon, Jan 5, 2009 at 12:16 AM, Miles Sabin wrote:
> On Sun, Jan 4, 2009 at 10:49 PM, martin odersky wrote:
>> I talked to Erik Gamma shortly before Christmas, and complained about
>> the closedness of the JDT.
>> His answer was that we should just do a cp -r of it and hack it as we
>> like. It seems many other projects did the same thing. Is this
>> something we have considered? Opinions?
>
> It's something I've considered, and something the AspectJ people did
> at one point.
>
> Aside from the pain and effort involved in maintaining a fork of the
> JDT we'd then have to worry about interoperability with all the other
> projects which depend on the JDT in some way or another.
>
> Fortunately the AspectJ people came up with something better than a
> fork. From the next release of the Eclipse AspectJ tooling they'll be
> using Equinox Aspects[1] to retrofit the kind of API that we need to
> JDT without having to fork or otherwise modify the JDT.
>
> I've been working with Andrew Eisenberg on using this mechanism in the
> Scala plugin for a little while now and I can report that it works
> extremely well.
>
> I'd hoped to have it committed to trunk before Xmas but unfortunately
> got bogged down with other things ... expect to see something RSN ...
>
Cool! I can't wait. -- Martin

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?
I don't know if this is an exclusive IDE issue. I think case classes are already kind of messed up implementation wise, where a cleaner implementation would probably benefit everyone. By this, I mean, there are so many things that can go wrong with the current implementation; e.g., what if someone puts a case class and companion object into separate source files (I think there is a bug report opened on this). Problems like these are going to fester with or without the IDE. 
I don't think the solution is fork or aspect weave. If one compiler isn't an option, the next best solution is a clean implementation of Scala where de-sugaring doesn't happen until after type checking. This would make tools easier to write. We could de-emphasize performance and reuse the existing lexer/parser (just throw out namers and typers), we wouldn't have to go beyond type checking. This is still a lot of stuff to implement, and there are always drift problems. I'd like to avoid it but I know you'd like to de-sugaring as early as possible to make the code generation easier to implement.


On Mon, Jan 5, 2009 at 6:46 AM, martin odersky <martin [dot] odersky [at] epfl [dot] ch> wrote:
Hi Sean

Hmm. But I don't think it's a good idea to hack the type system or the
compiler to satisfy that criterion. Scala is subtle enough as it is.
So we should try our utmost to make type checking clearer not to make
it more opaque. I think we need to keep apply/unapply in the companion
object.
We can play with lots of things around it: Like, we can let their
completers point to the original tree, or we can even have special
marker symbols for the sake of the Eclipse plugin. But I resist the
suggestion to compromise the integrity of the compiler for the plugins
sake.

Any idea is welcome!

Cheers

 -- Martin

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Eclipse plugin health?

On Tue, Jan 6, 2009 at 6:54 AM, Sean McDirmid wrote:
> I don't know if this is an exclusive IDE issue. I think case classes are
> already kind of messed up implementation wise, where a cleaner
> implementation would probably benefit everyone. By this, I mean, there are
> so many things that can go wrong with the current implementation; e.g., what
> if someone puts a case class and companion object into separate source files
> (I think there is a bug report opened on this). Problems like these are
> going to fester with or without the IDE.

But, Sean, the current design is the result of a focussed effort to
*clean up* case classes.
Maybe we need to put in a couple more guards against mis-use (such as
declaring object and class in different source files) but that's a
separate issue. I maintain that the current design is sound and cannot
think of a better one, unless somebody steps forward and shows me one.
In fact, I'd like to roll setters and getters over to the new scheme.

> I don't think the solution is fork or aspect weave. If one compiler isn't an
> option, the next best solution is a clean implementation of Scala where
> de-sugaring doesn't happen until after type checking. This would make tools
> easier to write. We could de-emphasize performance and reuse the existing
> lexer/parser (just throw out namers and typers), we wouldn't have to go
> beyond type checking. This is still a lot of stuff to implement, and there
> are always drift problems. I'd like to avoid it but I know you'd like to
> de-sugaring as early as possible to make the code generation easier to
> implement.
>
I think forking and aspect weaving are considered for JDT integration,
not for compiler integration.

Cheers

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Eclipse plugin health?
The clean up was a hack that reached across trees to break separate compilation. I'm not sure how you'll fix things with a few extra checks, the problem occurs when things are compiled separately so the checks won't know something else exists and the user will just scratch their head wondering what the heck is going on. Anyways, you are backing yourself into a corner with this design and it will be interesting to see how you get out of it, but its a scalac issue and not an IDE issue. 
Anyways, all the synthetic code you add during type checking is only their to support code generation. If we just suppressed adding the apply method to the companion object and add special logic to the type checker to otherwise "see" the apply method, the problem would probably go away. 

I'm afraid of what you will do to getters and setters, if its of the style of "if (!overthere.exists) overthere.make_it_exists", or involves some kind of global hack map to track info across trees, the IDE is screwed. If not, there is probably a work around. We already have a lot of logic to work around the way getters/setters are handled, but it works pretty well. 
On Tue, Jan 6, 2009 at 6:51 PM, martin odersky <martin [dot] odersky [at] epfl [dot] ch> wrote:
On Tue, Jan 6, 2009 at 6:54 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> I don't know if this is an exclusive IDE issue. I think case classes are
> already kind of messed up implementation wise, where a cleaner
> implementation would probably benefit everyone. By this, I mean, there are
> so many things that can go wrong with the current implementation; e.g., what
> if someone puts a case class and companion object into separate source files
> (I think there is a bug report opened on this). Problems like these are
> going to fester with or without the IDE.

But, Sean, the current design is the result of a focussed effort to
*clean up* case classes.
Maybe we need to put in a couple more guards against mis-use (such as
declaring object and class in different source files) but that's a
separate issue. I maintain that the current design is sound and cannot
think of a better one, unless somebody steps forward and shows me one.
In fact, I'd like to roll setters and getters over to the new scheme.

> I don't think the solution is fork or aspect weave. If one compiler isn't an
> option, the next best solution is a clean implementation of Scala where
> de-sugaring doesn't happen until after type checking. This would make tools
> easier to write. We could de-emphasize performance and reuse the existing
> lexer/parser (just throw out namers and typers), we wouldn't have to go
> beyond type checking. This is still a lot of stuff to implement, and there
> are always drift problems. I'd like to avoid it but I know you'd like to
> de-sugaring as early as possible to make the code generation easier to
> implement.
>
I think forking and aspect weaving are considered for JDT integration,
not for compiler integration.

Cheers

 -- Martin

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Re: Eclipse plugin health?

On Wed, Jan 7, 2009 at 1:13 AM, Sean McDirmid wrote:
> I'm afraid of what you will do to getters and setters, if its of the style
> of "if (!overthere.exists) overthere.make_it_exists", or involves some kind
> of global hack map to track info across trees, the IDE is screwed.

I really don't buy that.

I've asked you a couple of times before ... can you explain _why_ the
IDE is screwed in this scenario.

Cheers,

Miles

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Eclipse plugin health?
The IDE compiles some trees separately, just like scalac can compile files separately. Typically, information between trees during type checking in scalac is passed via an input context object and a symbol table that then points to types, and usually not much more than that. This restricted communication pattern is basically a form of run-time encapsulation even if its not enforced in Scala's static type system. I can then take advantage of this form of run-time modularity to memoize and process trees separately, so when you change one tree in the editor it only re-processes that one tree (in a file of 50K trees, when you only reprocess 1 tree, its very fast).  
Now, when processing one tree if the compiler reaches out and looks at another tree directly, it violates the above modularity. If the compiler tries to transmit information between trees using a hack map, it also violates this modularity. Basically, the context of the tree is enhanced in some way to include something else, and if I can't trace this state, I'm screwed. I have to try and trace this context somehow, but its not always easy, especially if the computations are order dependent (depend on the order that trees are processed). 
In the case of case classes, the compiler will reach out and create a companion object + an apply method, which is ok by itself. However, if a companion object exists, then either it exists before the case class and the apply method is added after its own methods are processed, or it exists before the case class and the apply method is added after its own method are processed. Now, we have to clear the methods when we process the tree, but unfortunately this means clearing the apply method as well. Does it get added back? If the case class is reprocessed it does, otherwise it doesn't. I've tried hacking around this so that the apply method is always added/removed as needed, but its pretty easy to get to a state where the IDE "loses" the apply method and large swathes of the editor go dark (I've solved this in my own code by using new for case class definition, but this isn't what we want users to use).

On Wed, Jan 7, 2009 at 9:22 AM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Wed, Jan 7, 2009 at 1:13 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> I'm afraid of what you will do to getters and setters, if its of the style
> of "if (!overthere.exists) overthere.make_it_exists", or involves some kind
> of global hack map to track info across trees, the IDE is screwed.

I really don't buy that.

I've asked you a couple of times before ... can you explain _why_ the
IDE is screwed in this scenario.

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Re: Eclipse plugin health?

On Wed, Jan 7, 2009 at 1:43 AM, Sean McDirmid wrote:
> The IDE compiles some trees separately, just like scalac can compile
> files separately.

OK, so what's to prevent the IDE from always attempting to compile
classes and their companion objects together?

Cheers,

Miles

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Eclipse plugin health?
The IDE already tries to do this. If the companion object is processed, it will invalidate the case class so the apply method will be added back. However, in some cases it seems that the case class is being processed before the companion object and then this invalidation is ignored, hence the missing apply method. 
Basically, I want to fuse the companion object and case class into one tree to avoid this mess, but I can't as scalac currently doesn't require any connection between the companion object/class beyond that they are in the same file/scope; e.g., you can write this kind of crap:
case class Acase class Bobject Aobject B

If it was instead
object Acase class Aobject B case class B or
case class Aobject Acase class Bobject B
then I would simply fuse the trees together for memoization purposes. 
On Wed, Jan 7, 2009 at 9:48 AM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Wed, Jan 7, 2009 at 1:43 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> The IDE compiles some trees separately, just like scalac can compile
> files separately.

OK, so what's to prevent the IDE from always attempting to compile
classes and their companion objects together?

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Re: Eclipse plugin health?

On Wed, Jan 7, 2009 at 1:54 AM, Sean McDirmid wrote:
> The IDE already tries to do this. If the companion object is processed, it
> will invalidate the case class so the apply method will be added back.
> However, in some cases it seems that the case class is being processed
> before the companion object and then this invalidation is ignored, hence the
> missing apply method.

This is basically just a bug then, no?

BTW, some of your comments suggest you're planning to do some work on
the compiler or the plugin in the not too distant future ... is that
actually the case?

Cheers,

Miles

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Eclipse plugin health?
Yes, its a bug with no easy resolution.
I'm willing to do maintenance work on the plugin/compiler if its going to survive, but I'm not very encouraged right now. If Martin insists that scalac will not consider the IDE in its design, its basically dead.  
I'd like to get an IDE going on .NET, but the .NET backend is unusable right now for that purpose. Hopefully that changes soon. Right now I don't even know if I can remember how to write in Scala. 
On Wed, Jan 7, 2009 at 9:59 AM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Wed, Jan 7, 2009 at 1:54 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> The IDE already tries to do this. If the companion object is processed, it
> will invalidate the case class so the apply method will be added back.
> However, in some cases it seems that the case class is being processed
> before the companion object and then this invalidation is ignored, hence the
> missing apply method.

This is basically just a bug then, no?

BTW, some of your comments suggest you're planning to do some work on
the compiler or the plugin in the not too distant future ... is that
actually the case?

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Re: Eclipse plugin health?

On Wed, Jan 7, 2009 at 2:17 AM, Sean McDirmid wrote:
> Yes, its a bug with no easy resolution.

I'm still not clear on this. So far you've not flagged up anything as
being especially difficult rather than being a matter of time and
effort. I appreciate that you don't have the time right now, but I do,
so I don't see this as too much of a problem. If there's something
else about this particular bug which you've not mentioned, please let
us know, otherwise I really don't see it as something which should
affect the design of the language.

> I'm willing to do maintenance work on the plugin/compiler if its going to
> survive, but I'm not very encouraged right now.

That's fine. Any bug fixing you can do would be very welcome, however
the thing which would help me and the community the most would be some
documentation, even rough notes, of the interface between the compiler
and the plugin, esp. the maintenance of the incremental ASTs.

Cheers,

Miles

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Re: Eclipse plugin health?

On Wed, Jan 7, 2009 at 2:13 AM, Sean McDirmid wrote:
> The clean up was a hack that reached across trees to break separate
> compilation. I'm not sure how you'll fix things with a few extra checks, the
> problem occurs when things are compiled separately so the checks won't know
> something else exists and the user will just scratch their head wondering
> what the heck is going on. Anyways, you are backing yourself into a corner
> with this design and it will be interesting to see how you get out of it,
> but its a scalac issue and not an IDE issue.
> Anyways, all the synthetic code you add during type checking is only their
> to support code generation. If we just suppressed adding the apply method to
> the companion object and add special logic to the type checker to otherwise
> "see" the apply method, the problem would probably go away.

Are you sure? I mean the reason the method tree is added is so that
the compiler will produce a symbol to play with during typechecking.
The tree generation migth be delayed, but not the symbol generation.
In any case, even if we generate the tree in Namers, it would be
fairly straightforward to mark it to be invisible for the Eclipse
plugin. But would that solve the probem?

Also, it would be fairly straightfroward to always typecheck companion
object or class in some order if that's what's required.

Cheers

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland