This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Re: Re: Tracking changed files for recompilation

28 replies
milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.

On Tue, Feb 10, 2009 at 1:47 AM, Sean McDirmid wrote:
> It does. Miles, if you are so sure that dependency tracking is broken in the
> plugin, just provide me with a bug report.

Do you actually intend to do any more work on the plugin? If not I'd
rather focus on David's work which is will be maintained going
forward.

Cheers,

Miles

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation
Bug report Miles. If there is a bug in build dependency tracking, I can fix it fairly quickly. If you want me to redo the editor to compiler interface, that is something I can't do quickly. If you want me to rewrite traits out of the plugin, that is something I won't do willingly. 
If people just want to say that dependency tracking is broken in the plugin without providing any examples, you can understand why I'm getting very agitated by that. 

On Tue, Feb 10, 2009 at 9:54 AM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Tue, Feb 10, 2009 at 1:47 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> It does. Miles, if you are so sure that dependency tracking is broken in the
> plugin, just provide me with a bug report.

Do you actually intend to do any more work on the plugin? If not I'd
rather focus on David's work which is will be maintained going
forward.

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Re: Tracking changed files for recompilation

On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid wrote:
> If there is a bug in build dependency tracking, I can fix it fairly quickly.

That really doesn't matter.

* The current code is impenetrable and buggy.

* Nobody other than you wants to support it.

* You don't appear to be willing to do the work of maintaining it yourself.

* David is in the process of putting together a better, maintained alternative.

I can't really see why I should waste my time on this ...

Cheers,

Miles

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Re: Tracking changed files for recompilation
Because email is fun!!!!!ONE!!!!!!!  

(This probably makes no sense to anyone who didn't read my earlier email on the emoticon subject)


Anyway to take this conversation off-line or, Sean, would you mind joining scala-debate?  I really believe we've entered debate territory, and I'd rather leave this channel open for more "How does this tool work, or Should this tool be created?" questions.   I beleive we've left "This tool is useful" territory and are now arguing details over whose code to use.

On Mon, Feb 9, 2009 at 9:01 PM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> If there is a bug in build dependency tracking, I can fix it fairly quickly.

That really doesn't matter.

* The current code is impenetrable and buggy.

* Nobody other than you wants to support it.

* You don't appear to be willing to do the work of maintaining it yourself.

* David is in the process of putting together a better, maintained alternative.

I can't really see why I should waste my time on this ...

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Re: Tracking changed files for recompilation

On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid wrote:
> If there is a bug in build dependency tracking, I can fix it fairly quickly.

That really doesn't matter.

* The current code is impenetrable and buggy.

* Nobody other than you wants to support it.

* You don't appear to be willing to do the work of maintaining it yourself.

* David is in the process of putting together a better, maintained alternative.

I can't really see why I should waste my time on this ...

Cheers,

Miles

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Re: Tracking changed files for recompilation

On Tue, Feb 10, 2009 at 2:05 AM, Sean McDirmid wrote:
> This is BS and you are just avoiding the question. Put up or shut up.

Pot, kettle calling ...

Cheers,

Miles

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Re: Tracking changed files for recompilation
Because email is fun!!!!!ONE!!!!!!!  

(This probably makes no sense to anyone who didn't read my earlier email on the emoticon subject)


Anyway to take this conversation off-line or, Sean, would you mind joining scala-debate?  I really believe we've entered debate territory, and I'd rather leave this channel open for more "How does this tool work, or Should this tool be created?" questions.   I beleive we've left "This tool is useful" territory and are now arguing details over whose code to use.

On Mon, Feb 9, 2009 at 9:01 PM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> If there is a bug in build dependency tracking, I can fix it fairly quickly.

That really doesn't matter.

* The current code is impenetrable and buggy.

* Nobody other than you wants to support it.

* You don't appear to be willing to do the work of maintaining it yourself.

* David is in the process of putting together a better, maintained alternative.

I can't really see why I should waste my time on this ...

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation
This is BS and you are just avoiding the question.  Put up or shut up.
On Tue, Feb 10, 2009 at 10:01 AM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> If there is a bug in build dependency tracking, I can fix it fairly quickly.

That really doesn't matter.

* The current code is impenetrable and buggy.

* Nobody other than you wants to support it.

* You don't appear to be willing to do the work of maintaining it yourself.

* David is in the process of putting together a better, maintained alternative.

I can't really see why I should waste my time on this ...

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Re: Tracking changed files for recompilation

Sean, you're useful to to project if you're still willing and able to
make concrete contributions. Code would be nice (there are plenty of
open bugs in areas where you're the best authority) and documentation
would be wonderful (the scalac-IDE interface and incremental AST
maintenance, hint, hint).

But if you're not going to do that and instead just respond negatively
to efforts like the one David is making then it might be helpful for
you to take a little break, at least until you're able to adopt a
slightly more detached attitude to the project.

Cheers,

Miles

--
Miles Sabin
tel: +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype: milessabin

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation
I'd rather not join scala-debate, I don't have much time, and I've got no vested interest in Scala right now given my work commitments :(
Miles is correct for feeling that the code is unapproachable, but then I never got much help from Martin on scalac either (same arguments can be made there). He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised, and no one thinks its even worth filing a decent bug report--even Miles would rather bash the plugin than go into what the problem is. 
In that case, I should just as well resign from the project/scala and I can move on to promoting Bling full time. I don't really see a better option right now, I don't even get to write Scala code anymore, and won't unless the mythical Scala.NET ever gets to a point where it is usable. Its nothing against anyone, I really want to see David write a decent dependency calculator, and I want to see a great IDE, I don't care if its Cao Yuan's NetBeans or IntelliJ, or whatever. I just don't think I'm very useful to the project anymore. 

On Tue, Feb 10, 2009 at 10:08 AM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com> wrote:
Because email is fun!!!!!ONE!!!!!!!  

(This probably makes no sense to anyone who didn't read my earlier email on the emoticon subject)


Anyway to take this conversation off-line or, Sean, would you mind joining scala-debate?  I really believe we've entered debate territory, and I'd rather leave this channel open for more "How does this tool work, or Should this tool be created?" questions.   I beleive we've left "This tool is useful" territory and are now arguing details over whose code to use.

On Mon, Feb 9, 2009 at 9:01 PM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> If there is a bug in build dependency tracking, I can fix it fairly quickly.

That really doesn't matter.

* The current code is impenetrable and buggy.

* Nobody other than you wants to support it.

* You don't appear to be willing to do the work of maintaining it yourself.

* David is in the process of putting together a better, maintained alternative.

I can't really see why I should waste my time on this ...

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin


Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Re: Tracking changed files for recompilation
Sean, for future reference:
(09:29:37 PM) tupshin: I took a 20 class java project and converted it to scala one class at a time using eclipse. Was an exercise in frustration. I probably hit "clean project" a few hundred times while doing the conversion.
(09:29:58 PM) tupshin: Just to see what the *real* remaining compilation errors were.

I've seen the same myself, and If you want I'll make a bug report, however it won't be very helpful.   By the time I diagnose what the problem was, there's no context for me to figure out what's in my .log *and* it's not very reproducable (that doesn't mean it doesn't exist, just that we can't figure out how to reproduce it).

The easiest way for scalac to get freaked out is to use it with java.  If your dependency analysis doesn't handle this case, then I'd rather go with David's solution, as I live in the Java world.

Also note that a lot of people never report bugs against OS projects (90% of my coworkers for instance).

-Josh

On Mon, Feb 9, 2009 at 9:37 PM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
Sean, you're useful to to project if you're still willing and able to
make concrete contributions. Code would be nice (there are plenty of
open bugs in areas where you're the best authority) and documentation
would be wonderful (the scalac-IDE interface and incremental AST
maintenance, hint, hint).

But if you're not going to do that and instead just respond negatively
to efforts like the one David is making then it might be helpful for
you to take a little break, at least until you're able to adopt a
slightly more detached attitude to the project.

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Re: Tracking changed files for recompilation
Sean, for future reference:
(09:29:37 PM) tupshin: I took a 20 class java project and converted it to scala one class at a time using eclipse. Was an exercise in frustration. I probably hit "clean project" a few hundred times while doing the conversion.
(09:29:58 PM) tupshin: Just to see what the *real* remaining compilation errors were.

I've seen the same myself, and If you want I'll make a bug report, however it won't be very helpful.   By the time I diagnose what the problem was, there's no context for me to figure out what's in my .log *and* it's not very reproducable (that doesn't mean it doesn't exist, just that we can't figure out how to reproduce it).

The easiest way for scalac to get freaked out is to use it with java.  If your dependency analysis doesn't handle this case, then I'd rather go with David's solution, as I live in the Java world.

Also note that a lot of people never report bugs against OS projects (90% of my coworkers for instance).

-Josh

On Mon, Feb 9, 2009 at 9:37 PM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
Sean, you're useful to to project if you're still willing and able to
make concrete contributions. Code would be nice (there are plenty of
open bugs in areas where you're the best authority) and documentation
would be wonderful (the scalac-IDE interface and incremental AST
maintenance, hint, hint).

But if you're not going to do that and instead just respond negatively
to efforts like the one David is making then it might be helpful for
you to take a little break, at least until you're able to adopt a
slightly more detached attitude to the project.

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Re: Tracking changed files for recompilation

Sean, you're useful to to project if you're still willing and able to
make concrete contributions. Code would be nice (there are plenty of
open bugs in areas where you're the best authority) and documentation
would be wonderful (the scalac-IDE interface and incremental AST
maintenance, hint, hint).

But if you're not going to do that and instead just respond negatively
to efforts like the one David is making then it might be helpful for
you to take a little break, at least until you're able to adopt a
slightly more detached attitude to the project.

Cheers,

Miles

--
Miles Sabin
tel: +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype: milessabin

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation
But we are not talking about users, we are talking about the developers of the project deciding not to report bugs! This is really serious and basically unworkable for a project. 
Mixed scala/java is a big question: it has nothing to do with the Scala-on-scala dependency tracker. Before, with separate Java/Scala projects we just reused the classfile invalidation events provided by the JDT, very very simple but we didn't support mixed Scala/Java projects then. I'm not sure what the deal is now, I did nothing with mixed Scala/Java projects, its definitely quite possible that it doesn't work. The plugin is always a moving target :(

Why did we ever decide that it was a good idea to mix Scala and Java code in the same project? Is having two separate projects too much inconvenience? 
On Tue, Feb 10, 2009 at 10:40 AM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com> wrote:
Sean, for future reference:
(09:29:37 PM) tupshin: I took a 20 class java project and converted it to scala one class at a time using eclipse. Was an exercise in frustration. I probably hit "clean project" a few hundred times while doing the conversion.
(09:29:58 PM) tupshin: Just to see what the *real* remaining compilation errors were.

I've seen the same myself, and If you want I'll make a bug report, however it won't be very helpful.   By the time I diagnose what the problem was, there's no context for me to figure out what's in my .log *and* it's not very reproducable (that doesn't mean it doesn't exist, just that we can't figure out how to reproduce it).

The easiest way for scalac to get freaked out is to use it with java.  If your dependency analysis doesn't handle this case, then I'd rather go with David's solution, as I live in the Java world.

Also note that a lot of people never report bugs against OS projects (90% of my coworkers for instance).

-Josh

On Mon, Feb 9, 2009 at 9:37 PM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
Sean, you're useful to to project if you're still willing and able to
make concrete contributions. Code would be nice (there are plenty of
open bugs in areas where you're the best authority) and documentation
would be wonderful (the scalac-IDE interface and incremental AST
maintenance, hint, hint).

But if you're not going to do that and instead just respond negatively
to efforts like the one David is making then it might be helpful for
you to take a little break, at least until you're able to adopt a
slightly more detached attitude to the project.

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin


Tupshin Harper
Joined: 2009-02-10,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation

After converting every last line to scala and getting rid of the last
trace of Java in that same project, if
a) there is any single compilation error in any file (even a standalone
one that has no interdependencies on any other file in the project) and
b) I deliberately introduce new compilation errors by renaming a class
(not doing file rename, just changing the name of the class inside the file)
then a cascading series of 44 errors show up in my project "not found:
type X", "nof found: value y", etc etc etc etc etc.
And when I rename the class back to what it should be and save it, all
of those errors stay until I do a "clean project", at which point I am
back to the one original (and correct) error.

Then when I tried to verify this test case and delete the file with an
error in it, I get:
Can't delete file...TestListerner.scala [in com.foo.bar[in src [in
Bar]]] does not exist
Which I couldn't resolve except by quitting eclipse and restarting it
(closing and reopening the project didn't work).

I then tried to reproduce the same problem with no existing errors in
the project, and this time, had an even worse (IMO) result, which was a
failure to detect *any* error after renaming the class until I cleaned
the project at which point it succesfully detected the missing class.

I then went to see whether this problem could have to do with my project
still having a Java nature in addition to the scala one, so I created a
new scala project and copied my source tree from hybrid project to scala
project.
Problems:
1) There are immediately two representation of every scala file, one
with a "J" icon and one with a "S" icon.
2) Most of the package icons have an error symbol in them despite there
being no classes with errors in those packages. (this resolves itself
after closing and reopening the project
3) I then deliberately reintroduced one isolated compilation error in
this pristine scala-only project, renamed one class to introduce a
second error, and had it fail to detect the compilation problems that
should occur when that class is missing until I cleaned the project.

In the meantime, the error log shows a whole progression of fun things like:
While loading class "lampion.presentation.Presentations$", thread
"Thread[worker-Foo,6,main]" timed out waiting (5000ms) for thread
"Thread[main,6,main]" to finish starting bundle
"reference:file:plugins\ch.epfl.lamp.sdt.core_2.8.0.r17010-b20090131023503.jar
[920]". To avoid deadlock, thread "Thread[worker-LBDone,6,main]" is
proceeding but "lampion.presentation.Presentations$" may not be fully
initialized.
org.eclipse.core.internal.resources.ResourceException: Resource '/Bar'
is not open. (After I deliberately closed the project)
File not found: G:\Users\tupshin\AppData\Local\Temp\7zE7E36.tmp\Foo.jar
"Paste" did not complete normally. Please see the log for more
information. An exception stack trace is not available. (when trying to
copy a very short string into a scala file which worked fine when I
typed it manuall)
etc.
etc.

An exercise in frustration indeed, and none of this associated with
mixed Scala/Java project.

-Tupshin

Sean McDirmid wrote:
> But we are not talking about users, we are talking about the
> developers of the project deciding not to report bugs! This is really
> serious and basically unworkable for a project.
>
> Mixed scala/java is a big question: it has nothing to do with the
> Scala-on-scala dependency tracker. Before, with separate Java/Scala
> projects we just reused the classfile invalidation events provided by
> the JDT, very very simple but we didn't support mixed Scala/Java
> projects then. I'm not sure what the deal is now, I did nothing with
> mixed Scala/Java projects, its definitely quite possible that it
> doesn't work. The plugin is always a moving target :(
>
> Why did we ever decide that it was a good idea to mix Scala and Java
> code in the same project? Is having two separate projects too much
> inconvenience?
>
> On Tue, Feb 10, 2009 at 10:40 AM, Josh Suereth
> > wrote:
>
> Sean, for future reference:
> (09:29:37 PM) tupshin: I took a 20 class java project and
> converted it to scala one class at a time using eclipse. Was an
> exercise in frustration. I probably hit "clean project" a few
> hundred times while doing the conversion.
> (09:29:58 PM) tupshin: Just to see what the *real* remaining
> compilation errors were.
>
> I've seen the same myself, and If you want I'll make a bug report,
> however it won't be very helpful. By the time I diagnose what
> the problem was, there's no context for me to figure out what's in
> my .log *and* it's not very reproducable (that doesn't mean it
> doesn't exist, just that we can't figure out how to reproduce it).
>
> The easiest way for scalac to get freaked out is to use it with
> java. If your dependency analysis doesn't handle this case, then
> I'd rather go with David's solution, as I live in the Java world.
>
> Also note that a lot of people never report bugs against OS
> projects (90% of my coworkers for instance).
>
> -Josh
>
> On Mon, Feb 9, 2009 at 9:37 PM, Miles Sabin > wrote:
>
> Sean, you're useful to to project if you're still willing and
> able to
> make concrete contributions. Code would be nice (there are
> plenty of
> open bugs in areas where you're the best authority) and
> documentation
> would be wonderful (the scalac-IDE interface and incremental AST
> maintenance, hint, hint).
>
> But if you're not going to do that and instead just respond
> negatively
> to efforts like the one David is making then it might be
> helpful for
> you to take a little break, at least until you're able to adopt a
> slightly more detached attitude to the project.
>
> Cheers,
>
>
> Miles
>
> --
> Miles Sabin
> tel: +44 (0)1273 720 779
> mobile: +44 (0)7813 944 528
> skype: milessabin
>
>
>

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation
Could you send me a copy of your project? This sounds like it is easy to reproduce. 

On Tue, Feb 10, 2009 at 11:29 AM, Tupshin Harper <tupshin [at] tupshin [dot] com> wrote:
After converting every last line to scala and getting rid of the last trace of Java in that same project, if
a) there is any single compilation error in any file (even a standalone one that has no interdependencies on any other file in the project) and
b) I deliberately introduce new compilation errors by renaming a class (not doing file rename, just changing the name of the class inside the file)
then a cascading series of 44 errors show up in my project "not found: type X", "nof found: value y", etc etc etc etc etc.
And when I rename the class back to what it should be and save it, all of those errors stay until I do a "clean project", at which point I am back to the one original (and correct) error.

Then when I tried to verify this test case and delete the file with an error in it, I get:
Can't delete file...TestListerner.scala [in com.foo.bar[in src [in Bar]]] does not exist
Which I couldn't resolve except by quitting eclipse and restarting it (closing and reopening the project didn't work).

I then tried to reproduce the same problem with no existing errors in the project, and this time, had an even worse (IMO) result, which was a failure to detect *any* error after renaming the class until I cleaned the project at which point it succesfully detected the missing class.

I then went to see whether this problem could have to do with my project still having a Java nature in addition to the scala one, so I created a new scala project and copied my source tree from hybrid project to scala project.
Problems:
1) There are immediately two representation of every scala file, one with a "J" icon and one with a "S" icon.
2) Most of the package icons have an error symbol in them despite there being no classes with errors in those packages. (this resolves itself after closing and reopening the project
3) I then deliberately reintroduced one isolated compilation error in this pristine scala-only project, renamed one class to introduce a second error, and had it fail to detect the compilation problems that should occur when that class is missing until I cleaned the project.

In the meantime, the error log shows a whole progression of fun things like:
While loading class "lampion.presentation.Presentations$", thread "Thread[worker-Foo,6,main]" timed out waiting (5000ms) for thread "Thread[main,6,main]" to finish starting bundle "reference:file:plugins\ch.epfl.lamp.sdt.core_2.8.0.r17010-b20090131023503.jar [920]". To avoid deadlock, thread "Thread[worker-LBDone,6,main]" is proceeding but "lampion.presentation.Presentations$" may not be fully initialized.
org.eclipse.core.internal.resources.ResourceException: Resource '/Bar' is not open. (After I deliberately closed the project)
File not found: G:\Users\tupshin\AppData\Local\Temp\7zE7E36.tmp\Foo.jar
"Paste" did not complete normally.  Please see the log for more information. An exception stack trace is not available. (when trying to copy a very short string into a scala file which worked fine when I typed it manuall)
etc.
etc.

An exercise in frustration indeed, and none of this associated with mixed Scala/Java project.

-Tupshin


Sean McDirmid wrote:
But we are not talking about users, we are talking about the developers of the project deciding not to report bugs! This is really serious and basically unworkable for a project.
Mixed scala/java is a big question: it has nothing to do with the Scala-on-scala dependency tracker. Before, with separate Java/Scala projects we just reused the classfile invalidation events provided by the JDT, very very simple but we didn't support mixed Scala/Java projects then. I'm not sure what the deal is now, I did nothing with mixed Scala/Java projects, its definitely quite possible that it doesn't work. The plugin is always a moving target :(

Why did we ever decide that it was a good idea to mix Scala and Java code in the same project? Is having two separate projects too much inconvenience?
On Tue, Feb 10, 2009 at 10:40 AM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com <mailto:joshua [dot] suereth [at] gmail [dot] com>> wrote:

   Sean, for future reference:
   (09:29:37 PM) tupshin: I took a 20 class java project and
   converted it to scala one class at a time using eclipse. Was an
   exercise in frustration. I probably hit "clean project" a few
   hundred times while doing the conversion.
   (09:29:58 PM) tupshin: Just to see what the *real* remaining
   compilation errors were.

   I've seen the same myself, and If you want I'll make a bug report,
   however it won't be very helpful.   By the time I diagnose what
   the problem was, there's no context for me to figure out what's in
   my .log *and* it's not very reproducable (that doesn't mean it
   doesn't exist, just that we can't figure out how to reproduce it).

   The easiest way for scalac to get freaked out is to use it with
   java.  If your dependency analysis doesn't handle this case, then
   I'd rather go with David's solution, as I live in the Java world.

   Also note that a lot of people never report bugs against OS
   projects (90% of my coworkers for instance).

   -Josh

   On Mon, Feb 9, 2009 at 9:37 PM, Miles Sabin <miles [at] milessabin [dot] com
   <mailto:miles [at] milessabin [dot] com>> wrote:

       Sean, you're useful to to project if you're still willing and
       able to
       make concrete contributions. Code would be nice (there are
       plenty of
       open bugs in areas where you're the best authority) and
       documentation
       would be wonderful (the scalac-IDE interface and incremental AST
       maintenance, hint, hint).

       But if you're not going to do that and instead just respond
       negatively
       to efforts like the one David is making then it might be
       helpful for
       you to take a little break, at least until you're able to adopt a
       slightly more detached attitude to the project.

       Cheers,


       Miles

       --
       Miles Sabin
       tel:    +44 (0)1273 720 779
       mobile: +44 (0)7813 944 528
       skype:  milessabin





Tupshin Harper
Joined: 2009-02-10,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation

I could, but I really really don't need to as you can reproduce it
yourself very quickly.

Test scenario:
1) I just updated to scala plugin 2.8.0.r17068-b20090210023355, but
identical problem reproduced with a version from a couple of weeks ago
2) Eclipse if version 3.4.1
3) Created brand new workspace and did a fresh launch of eclipse.
4) System is 64 bit Windows Vista. If you are unable to reproduce, I
will try it on a Debian Linux system, but that will take some time to
set up.
5) Unlike my more complex scenarios, there are no errors in the Error
log, only a couple of non-threatening warnings.

Steps to reproduce:
1) Create scala project "TestProject"
2) Say yes when given the chance to switch to scala perspective.
3) create new package in src called "test"
4) create new class in test called "Foo"
5) create new class in test called "Bar"
6) Edit Foo.scala and change "class Foo {" to be "class Foo2 {" and save.
Observation 1:
a) The "Problems" view briefly changes its text to Italic, indicating
that it is compiling something
b) No problems show up.
c) workspace/TestProject/bin/test contain files Bar.class, Foo.class,
and Foo2.class
Conclusion:
Renaming a class in a scala file fails to delete the orignal .class file
even though it creates a new one. (But see below)

Further testing:
1) If I "clean project", I correctly get "not found: type Foo"
2) If I then rename Foo2 back to Foo, the error correctly goes away.
3) Somewhat surprisingly, bin directory only has Bar.class and Foo.class
this time. It does not still have Foo2.class
Conclusion:
Contrary to above, class files are sometimes deleted when the class is
renamed but possibly not when anything depends on them. Confirming this
last hypothesis hard to do.

-Tupshin

Sean McDirmid wrote:
> Could you send me a copy of your project? This sounds like it is easy
> to reproduce.
>
> On Tue, Feb 10, 2009 at 11:29 AM, Tupshin Harper > wrote:
>
> After converting every last line to scala and getting rid of the
> last trace of Java in that same project, if
> a) there is any single compilation error in any file (even a
> standalone one that has no interdependencies on any other file in
> the project) and
> b) I deliberately introduce new compilation errors by renaming a
> class (not doing file rename, just changing the name of the class
> inside the file)
> then a cascading series of 44 errors show up in my project "not
> found: type X", "nof found: value y", etc etc etc etc etc.
> And when I rename the class back to what it should be and save it,
> all of those errors stay until I do a "clean project", at which
> point I am back to the one original (and correct) error.
>
> Then when I tried to verify this test case and delete the file
> with an error in it, I get:
> Can't delete file...TestListerner.scala [in com.foo.bar[in src [in
> Bar]]] does not exist
> Which I couldn't resolve except by quitting eclipse and restarting
> it (closing and reopening the project didn't work).
>
> I then tried to reproduce the same problem with no existing errors
> in the project, and this time, had an even worse (IMO) result,
> which was a failure to detect *any* error after renaming the class
> until I cleaned the project at which point it succesfully detected
> the missing class.
>
> I then went to see whether this problem could have to do with my
> project still having a Java nature in addition to the scala one,
> so I created a new scala project and copied my source tree from
> hybrid project to scala project.
> Problems:
> 1) There are immediately two representation of every scala file,
> one with a "J" icon and one with a "S" icon.
> 2) Most of the package icons have an error symbol in them despite
> there being no classes with errors in those packages. (this
> resolves itself after closing and reopening the project
> 3) I then deliberately reintroduced one isolated compilation error
> in this pristine scala-only project, renamed one class to
> introduce a second error, and had it fail to detect the
> compilation problems that should occur when that class is missing
> until I cleaned the project.
>
> In the meantime, the error log shows a whole progression of fun
> things like:
> While loading class "lampion.presentation.Presentations$", thread
> "Thread[worker-Foo,6,main]" timed out waiting (5000ms) for thread
> "Thread[main,6,main]" to finish starting bundle
> "reference:file:plugins\ch.epfl.lamp.sdt.core_2.8.0.r17010-b20090131023503.jar
> [920]". To avoid deadlock, thread "Thread[worker-LBDone,6,main]"
> is proceeding but "lampion.presentation.Presentations$" may not be
> fully initialized.
> org.eclipse.core.internal.resources.ResourceException: Resource
> '/Bar' is not open. (After I deliberately closed the project)
> File not found:
> G:\Users\tupshin\AppData\Local\Temp\7zE7E36.tmp\Foo.jar
> "Paste" did not complete normally. Please see the log for more
> information. An exception stack trace is not available. (when
> trying to copy a very short string into a scala file which worked
> fine when I typed it manuall)
> etc.
> etc.
>
> An exercise in frustration indeed, and none of this associated
> with mixed Scala/Java project.
>
> -Tupshin
>
>
> Sean McDirmid wrote:
>
> But we are not talking about users, we are talking about the
> developers of the project deciding not to report bugs! This is
> really serious and basically unworkable for a project.
> Mixed scala/java is a big question: it has nothing to do with
> the Scala-on-scala dependency tracker. Before, with separate
> Java/Scala projects we just reused the classfile invalidation
> events provided by the JDT, very very simple but we didn't
> support mixed Scala/Java projects then. I'm not sure what the
> deal is now, I did nothing with mixed Scala/Java projects, its
> definitely quite possible that it doesn't work. The plugin is
> always a moving target :(
>
> Why did we ever decide that it was a good idea to mix Scala
> and Java code in the same project? Is having two separate
> projects too much inconvenience?
> On Tue, Feb 10, 2009 at 10:40 AM, Josh Suereth
>
> >> wrote:
>
> Sean, for future reference:
> (09:29:37 PM) tupshin: I took a 20 class java project and
> converted it to scala one class at a time using eclipse. Was an
> exercise in frustration. I probably hit "clean project" a few
> hundred times while doing the conversion.
> (09:29:58 PM) tupshin: Just to see what the *real* remaining
> compilation errors were.
>
> I've seen the same myself, and If you want I'll make a bug
> report,
> however it won't be very helpful. By the time I diagnose what
> the problem was, there's no context for me to figure out
> what's in
> my .log *and* it's not very reproducable (that doesn't mean it
> doesn't exist, just that we can't figure out how to
> reproduce it).
>
> The easiest way for scalac to get freaked out is to use it with
> java. If your dependency analysis doesn't handle this
> case, then
> I'd rather go with David's solution, as I live in the Java
> world.
>
> Also note that a lot of people never report bugs against OS
> projects (90% of my coworkers for instance).
>
> -Josh
>
> On Mon, Feb 9, 2009 at 9:37 PM, Miles Sabin
>
> >> wrote:
>
> Sean, you're useful to to project if you're still
> willing and
> able to
> make concrete contributions. Code would be nice (there are
> plenty of
> open bugs in areas where you're the best authority) and
> documentation
> would be wonderful (the scalac-IDE interface and
> incremental AST
> maintenance, hint, hint).
>
> But if you're not going to do that and instead just respond
> negatively
> to efforts like the one David is making then it might be
> helpful for
> you to take a little break, at least until you're able
> to adopt a
> slightly more detached attitude to the project.
>
> Cheers,
>
>
> Miles
>
> --
> Miles Sabin
> tel: +44 (0)1273 720 779
> mobile: +44 (0)7813 944 528
> skype: milessabin
>
>
>
>
>

Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation
Ah, ok. I understand, everything is working as expected. We aren't cleaning up stale classes in between recompiles. Its easy enough to delete the stale class files, but it is much more difficult to evict the stale class symbols from symbol loaders. 
scalac doesn't generate class files on a bad build, so that explains why Foo2.class is never generated (in either case) while it doesn't delete class files either after a rebuild (which is why Foo.class hangs around). 
In scalac non-resident, we can fix this by deleting all class files generated from the source file on its previous compile before it is recompiled. I'll let David and Martin decide if they want scalac to do that. For scalac resident, we have to flush out all symbols of a class before the source file is recompiled (rather than just see if we can reuse symbols as is the case now). Martin should decide if he wants to do that or not, if Martin does that for FSC we can piggy back on the change for the plugin. 
FYI, in Java, the ejc does the right thing when rebuilding source files. I don't believe javac does (i.e., scalac is mimicking javac's behavior).

On Tue, Feb 10, 2009 at 1:34 PM, Tupshin Harper <tupshin [at] tupshin [dot] com> wrote:
I could, but I really really don't need to as you can reproduce it yourself very quickly.

Test scenario:
1) I just updated to scala plugin 2.8.0.r17068-b20090210023355, but identical problem reproduced with a version from a couple of weeks ago
2) Eclipse if version 3.4.1
3) Created brand new workspace and did a fresh launch of eclipse.
4) System is 64 bit Windows Vista. If you are unable to reproduce, I will try it on a Debian Linux system, but that will take some time to set up.void
5) Unlike my more complex scenarios, there are no errors in the Error log, only a couple of non-threatening warnings.

Steps to reproduce:
1) Create scala project "TestProject"
2) Say yes when given the chance to switch to scala perspective.
3) create new package in src called "test"
4) create new class in test called "Foo"
5) create new class in test called "Bar"
6) Edit Foo.scala and change "class Foo {" to be "class Foo2 {" and save.
Observation 1:
a) The "Problems" view briefly changes its text to Italic, indicating that it is compiling something
b) No problems show up.
c) workspace/TestProject/bin/test contain files Bar.class, Foo.class, and Foo2.class
Conclusion:
Renaming a class in a scala file fails to delete the orignal .class file even though it creates a new one. (But see below)

Further testing:
1) If I "clean project", I correctly get "not found: type Foo"
2) If I then rename Foo2 back to Foo, the error correctly goes away.
3) Somewhat surprisingly, bin directory only has Bar.class and Foo.class this time. It does not still have Foo2.class
Conclusion:
Contrary to above, class files are sometimes deleted when the class is renamed but possibly not when anything depends on them. Confirming this last hypothesis hard to do.

-Tupshin


Sean McDirmid wrote:
Could you send me a copy of your project? This sounds like it is easy to reproduce.
On Tue, Feb 10, 2009 at 11:29 AM, Tupshin Harper <tupshin [at] tupshin [dot] com <mailto:tupshin [at] tupshin [dot] com>> wrote:

   After converting every last line to scala and getting rid of the
   last trace of Java in that same project, if
   a) there is any single compilation error in any file (even a
   standalone one that has no interdependencies on any other file in
   the project) and
   b) I deliberately introduce new compilation errors by renaming a
   class (not doing file rename, just changing the name of the class
   inside the file)
   then a cascading series of 44 errors show up in my project "not
   found: type X", "nof found: value y", etc etc etc etc etc.
   And when I rename the class back to what it should be and save it,
   all of those errors stay until I do a "clean project", at which
   point I am back to the one original (and correct) error.

   Then when I tried to verify this test case and delete the file
   with an error in it, I get:
   Can't delete file...TestListerner.scala [in com.foo.bar[in src [in
   Bar]]] does not exist
   Which I couldn't resolve except by quitting eclipse and restarting
   it (closing and reopening the project didn't work).

   I then tried to reproduce the same problem with no existing errors
   in the project, and this time, had an even worse (IMO) result,
   which was a failure to detect *any* error after renaming the class
   until I cleaned the project at which point it succesfully detected
   the missing class.

   I then went to see whether this problem could have to do with my
   project still having a Java nature in addition to the scala one,
   so I created a new scala project and copied my source tree from
   hybrid project to scala project.
   Problems:
   1) There are immediately two representation of every scala file,
   one with a "J" icon and one with a "S" icon.
   2) Most of the package icons have an error symbol in them despite
   there being no classes with errors in those packages. (this
   resolves itself after closing and reopening the project
   3) I then deliberately reintroduced one isolated compilation error
   in this pristine scala-only project, renamed one class to
   introduce a second error, and had it fail to detect the
   compilation problems that should occur when that class is missing
   until I cleaned the project.

   In the meantime, the error log shows a whole progression of fun
   things like:
   While loading class "lampion.presentation.Presentations$", thread
   "Thread[worker-Foo,6,main]" timed out waiting (5000ms) for thread
   "Thread[main,6,main]" to finish starting bundle
   "reference:file:plugins\ch.epfl.lamp.sdt.core_2.8.0.r17010-b20090131023503.jar
   [920]". To avoid deadlock, thread "Thread[worker-LBDone,6,main]"
   is proceeding but "lampion.presentation.Presentations$" may not be
   fully initialized.
   org.eclipse.core.internal.resources.ResourceException: Resource
   '/Bar' is not open. (After I deliberately closed the project)
   File not found:
   G:\Users\tupshin\AppData\Local\Temp\7zE7E36.tmp\Foo.jar
   "Paste" did not complete normally.  Please see the log for more
   information. An exception stack trace is not available. (when
   trying to copy a very short string into a scala file which worked
   fine when I typed it manuall)
   etc.
   etc.

   An exercise in frustration indeed, and none of this associated
   with mixed Scala/Java project.

   -Tupshin


   Sean McDirmid wrote:

       But we are not talking about users, we are talking about the
       developers of the project deciding not to report bugs! This is
       really serious and basically unworkable for a project.
       Mixed scala/java is a big question: it has nothing to do with
       the Scala-on-scala dependency tracker. Before, with separate
       Java/Scala projects we just reused the classfile invalidation
       events provided by the JDT, very very simple but we didn't
       support mixed Scala/Java projects then. I'm not sure what the
       deal is now, I did nothing with mixed Scala/Java projects, its
       definitely quite possible that it doesn't work. The plugin is
       always a moving target :(

       Why did we ever decide that it was a good idea to mix Scala
       and Java code in the same project? Is having two separate
       projects too much inconvenience?
       On Tue, Feb 10, 2009 at 10:40 AM, Josh Suereth
       <joshua [dot] suereth [at] gmail [dot] com <mailto:joshua [dot] suereth [at] gmail [dot] com>
       <mailto:joshua [dot] suereth [at] gmail [dot] com
       <mailto:joshua [dot] suereth [at] gmail [dot] com>>> wrote:

          Sean, for future reference:
          (09:29:37 PM) tupshin: I took a 20 class java project and
          converted it to scala one class at a time using eclipse. Was an
          exercise in frustration. I probably hit "clean project" a few
          hundred times while doing the conversion.
          (09:29:58 PM) tupshin: Just to see what the *real* remaining
          compilation errors were.

          I've seen the same myself, and If you want I'll make a bug
       report,
          however it won't be very helpful.   By the time I diagnose what
          the problem was, there's no context for me to figure out
       what's in
          my .log *and* it's not very reproducable (that doesn't mean it
          doesn't exist, just that we can't figure out how to
       reproduce it).

          The easiest way for scalac to get freaked out is to use it with
          java.  If your dependency analysis doesn't handle this
       case, then
          I'd rather go with David's solution, as I live in the Java
       world.

          Also note that a lot of people never report bugs against OS
          projects (90% of my coworkers for instance).

          -Josh

          On Mon, Feb 9, 2009 at 9:37 PM, Miles Sabin
       <miles [at] milessabin [dot] com <mailto:miles [at] milessabin [dot] com>
          <mailto:miles [at] milessabin [dot] com
       <mailto:miles [at] milessabin [dot] com>>> wrote:

              Sean, you're useful to to project if you're still
       willing and
              able to
              make concrete contributions. Code would be nice (there are
              plenty of
              open bugs in areas where you're the best authority) and
              documentation
              would be wonderful (the scalac-IDE interface and
       incremental AST
              maintenance, hint, hint).

              But if you're not going to do that and instead just respond
              negatively
              to efforts like the one David is making then it might be
              helpful for
              you to take a little break, at least until you're able
       to adopt a
              slightly more detached attitude to the project.

              Cheers,


              Miles

              --
              Miles Sabin
              tel:    +44 (0)1273 720 779
              mobile: +44 (0)7813 944 528
              skype:  milessabin







Tupshin Harper
Joined: 2009-02-10,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation

OK. Good to know that it's an understood problem. I'll weigh in as
completely neutral on *how* it gets fixed, but unless it does get fixed,
then doing any significant amount of refactoring is an exceedingly
painful process.

In the interests of correctness, it would seem essential to flush the
symbols of a class before recompiling until and unless there existed a
more sophisticated approach that did a first pass to see if symbols were
reusable and a second pass to do a full compile. The current approach is
proven incorrect and therefore must change.

-Tupshin

Sean McDirmid wrote:
> Ah, ok. I understand, everything is working as expected. We aren't
> cleaning up stale classes in between recompiles. Its easy enough to
> delete the stale class files, but it is much more difficult to evict
> the stale class symbols from symbol loaders.
>
> scalac doesn't generate class files on a bad build, so that explains
> why Foo2.class is never generated (in either case) while it doesn't
> delete class files either after a rebuild (which is why Foo.class
> hangs around).
>
> In scalac non-resident, we can fix this by deleting all class files
> generated from the source file on its previous compile before it is
> recompiled. I'll let David and Martin decide if they want scalac to do
> that. For scalac resident, we have to flush out all symbols of a class
> before the source file is recompiled (rather than just see if we can
> reuse symbols as is the case now). Martin should decide if he wants to
> do that or not, if Martin does that for FSC we can piggy back on the
> change for the plugin.
>
> FYI, in Java, the ejc does the right thing when rebuilding source
> files. I don't believe javac does (i.e., scalac is mimicking javac's
> behavior).
>
> On Tue, Feb 10, 2009 at 1:34 PM, Tupshin Harper > wrote:
>
> I could, but I really really don't need to as you can reproduce it
> yourself very quickly.
>
> Test scenario:
> 1) I just updated to scala plugin 2.8.0.r17068-b20090210023355,
> but identical problem reproduced with a version from a couple of
> weeks ago
> 2) Eclipse if version 3.4.1
> 3) Created brand new workspace and did a fresh launch of eclipse.
> 4) System is 64 bit Windows Vista. If you are unable to reproduce,
> I will try it on a Debian Linux system, but that will take some
> time to set up.void
> 5) Unlike my more complex scenarios, there are no errors in the
> Error log, only a couple of non-threatening warnings.
>
> Steps to reproduce:
> 1) Create scala project "TestProject"
> 2) Say yes when given the chance to switch to scala perspective.
> 3) create new package in src called "test"
> 4) create new class in test called "Foo"
> 5) create new class in test called "Bar"
> 6) Edit Foo.scala and change "class Foo {" to be "class Foo2 {"
> and save.
> Observation 1:
> a) The "Problems" view briefly changes its text to Italic,
> indicating that it is compiling something
> b) No problems show up.
> c) workspace/TestProject/bin/test contain files Bar.class,
> Foo.class, and Foo2.class
> Conclusion:
> Renaming a class in a scala file fails to delete the orignal
> .class file even though it creates a new one. (But see below)
>
> Further testing:
> 1) If I "clean project", I correctly get "not found: type Foo"
> 2) If I then rename Foo2 back to Foo, the error correctly goes away.
> 3) Somewhat surprisingly, bin directory only has Bar.class and
> Foo.class this time. It does not still have Foo2.class
> Conclusion:
> Contrary to above, class files are sometimes deleted when the
> class is renamed but possibly not when anything depends on them.
> Confirming this last hypothesis hard to do.
>
> -Tupshin
>
>
> Sean McDirmid wrote:
>
> Could you send me a copy of your project? This sounds like it
> is easy to reproduce.
> On Tue, Feb 10, 2009 at 11:29 AM, Tupshin Harper
>
> >> wrote:
>
> After converting every last line to scala and getting rid
> of the
> last trace of Java in that same project, if
> a) there is any single compilation error in any file (even a
> standalone one that has no interdependencies on any other
> file in
> the project) and
> b) I deliberately introduce new compilation errors by
> renaming a
> class (not doing file rename, just changing the name of the
> class
> inside the file)
> then a cascading series of 44 errors show up in my project "not
> found: type X", "nof found: value y", etc etc etc etc etc.
> And when I rename the class back to what it should be and
> save it,
> all of those errors stay until I do a "clean project", at which
> point I am back to the one original (and correct) error.
>
> Then when I tried to verify this test case and delete the file
> with an error in it, I get:
> Can't delete file...TestListerner.scala [in com.foo.bar[in
> src [in
> Bar]]] does not exist
> Which I couldn't resolve except by quitting eclipse and
> restarting
> it (closing and reopening the project didn't work).
>
> I then tried to reproduce the same problem with no existing
> errors
> in the project, and this time, had an even worse (IMO) result,
> which was a failure to detect *any* error after renaming
> the class
> until I cleaned the project at which point it succesfully
> detected
> the missing class.
>
> I then went to see whether this problem could have to do
> with my
> project still having a Java nature in addition to the scala
> one,
> so I created a new scala project and copied my source tree from
> hybrid project to scala project.
> Problems:
> 1) There are immediately two representation of every scala
> file,
> one with a "J" icon and one with a "S" icon.
> 2) Most of the package icons have an error symbol in them
> despite
> there being no classes with errors in those packages. (this
> resolves itself after closing and reopening the project
> 3) I then deliberately reintroduced one isolated
> compilation error
> in this pristine scala-only project, renamed one class to
> introduce a second error, and had it fail to detect the
> compilation problems that should occur when that class is
> missing
> until I cleaned the project.
>
> In the meantime, the error log shows a whole progression of fun
> things like:
> While loading class "lampion.presentation.Presentations$",
> thread
> "Thread[worker-Foo,6,main]" timed out waiting (5000ms) for
> thread
> "Thread[main,6,main]" to finish starting bundle
>
> "reference:file:plugins\ch.epfl.lamp.sdt.core_2.8.0.r17010-b20090131023503.jar
> [920]". To avoid deadlock, thread
> "Thread[worker-LBDone,6,main]"
> is proceeding but "lampion.presentation.Presentations$" may
> not be
> fully initialized.
> org.eclipse.core.internal.resources.ResourceException: Resource
> '/Bar' is not open. (After I deliberately closed the project)
> File not found:
> G:\Users\tupshin\AppData\Local\Temp\7zE7E36.tmp\Foo.jar
> "Paste" did not complete normally. Please see the log for more
> information. An exception stack trace is not available. (when
> trying to copy a very short string into a scala file which
> worked
> fine when I typed it manuall)
> etc.
> etc.
>
> An exercise in frustration indeed, and none of this associated
> with mixed Scala/Java project.
>
> -Tupshin
>
>
> Sean McDirmid wrote:
>
> But we are not talking about users, we are talking
> about the
> developers of the project deciding not to report bugs!
> This is
> really serious and basically unworkable for a project.
> Mixed scala/java is a big question: it has nothing to
> do with
> the Scala-on-scala dependency tracker. Before, with
> separate
> Java/Scala projects we just reused the classfile
> invalidation
> events provided by the JDT, very very simple but we didn't
> support mixed Scala/Java projects then. I'm not sure
> what the
> deal is now, I did nothing with mixed Scala/Java
> projects, its
> definitely quite possible that it doesn't work. The
> plugin is
> always a moving target :(
>
> Why did we ever decide that it was a good idea to mix Scala
> and Java code in the same project? Is having two separate
> projects too much inconvenience?
> On Tue, Feb 10, 2009 at 10:40 AM, Josh Suereth
>
> >
>
> >>> wrote:
>
> Sean, for future reference:
> (09:29:37 PM) tupshin: I took a 20 class java
> project and
> converted it to scala one class at a time using
> eclipse. Was an
> exercise in frustration. I probably hit "clean
> project" a few
> hundred times while doing the conversion.
> (09:29:58 PM) tupshin: Just to see what the *real*
> remaining
> compilation errors were.
>
> I've seen the same myself, and If you want I'll make
> a bug
> report,
> however it won't be very helpful. By the time I
> diagnose what
> the problem was, there's no context for me to figure out
> what's in
> my .log *and* it's not very reproducable (that
> doesn't mean it
> doesn't exist, just that we can't figure out how to
> reproduce it).
>
> The easiest way for scalac to get freaked out is to
> use it with
> java. If your dependency analysis doesn't handle this
> case, then
> I'd rather go with David's solution, as I live in
> the Java
> world.
>
> Also note that a lot of people never report bugs
> against OS
> projects (90% of my coworkers for instance).
>
> -Josh
>
> On Mon, Feb 9, 2009 at 9:37 PM, Miles Sabin
>
> >
>
>
> >>> wrote:
>
> Sean, you're useful to to project if you're still
> willing and
> able to
> make concrete contributions. Code would be nice
> (there are
> plenty of
> open bugs in areas where you're the best
> authority) and
> documentation
> would be wonderful (the scalac-IDE interface and
> incremental AST
> maintenance, hint, hint).
>
> But if you're not going to do that and instead
> just respond
> negatively
> to efforts like the one David is making then it
> might be
> helpful for
> you to take a little break, at least until
> you're able
> to adopt a
> slightly more detached attitude to the project.
>
> Cheers,
>
>
> Miles
>
> --
> Miles Sabin
> tel: +44 (0)1273 720 779
> mobile: +44 (0)7813 944 528
> skype: milessabin
>
>
>
>
>
>
>

Andreas W
Joined: 2009-01-10,
User offline. Last seen 1 year 14 weeks ago.
Re: Re: Tracking changed files for recompilation

> He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised,

Let me assert here that the plugin is not universally despised. I feel the plugin, warts and all, is tremendously more useful than any mere editor can be, and from what I know about the direction that Miles is working in, I see a glorious future for it. Please keep up the good work.

Andreas



From: Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com>
To: Josh Suereth <joshua [dot] suereth [at] gmail [dot] com>
Cc: Miles Sabin <miles [at] milessabin [dot] com>; Mark Harrah <harrah [at] bu [dot] edu>; scala-tools [at] listes [dot] epfl [dot] ch; scala-debate <scala-debate [at] listes [dot] epfl [dot] ch>
Sent: Monday, February 9, 2009 9:22:22 PM
Subject: Re: [scala-debate] Re: [scala-tools] Tracking changed files for recompilation

I'd rather not join scala-debate, I don't have much time, and I've got no vested interest in Scala right now given my work commitments :(
Miles is correct for feeling that the code is unapproachable, but then I never got much help from Martin on scalac either (same arguments can be made there). He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised, and no one thinks its even worth filing a decent bug report--even Miles would rather bash the plugin than go into what the problem is. 
In that case, I should just as well resign from the project/scala and I can move on to promoting Bling full time. I don't really see a better option right now, I don't even get to write Scala code anymore, and won't unless the mythical Scala.NET ever gets to a point where it is usable. Its nothing against anyone, I really want to see David write a decent dependency calculator, and I want to see a great IDE, I don't care if its Cao Yuan's NetBeans or IntelliJ, or whatever. I just don't think I'm very useful to the project anymore. 

On Tue, Feb 10, 2009 at 10:08 AM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com" target="_blank" href="mailto:joshua [dot] suereth [at] gmail [dot] com" rel="nofollow">joshua [dot] suereth [at] gmail [dot] com> wrote:
Because email is fun!!!!!ONE!!!!!!!  

(This probably makes no sense to anyone who didn't read my earlier email on the emoticon subject)


Anyway to take this conversation off-line or, Sean, would you mind joining scala-debate?  I really believe we've entered debate territory, and I'd rather leave this channel open for more "How does this tool work, or Should this tool be created?" questions.   I beleive we've left "This tool is useful" territory and are now arguing details over whose code to use.

On Mon, Feb 9, 2009 at 9:01 PM, Miles Sabin <miles [at] milessabin [dot] com" target="_blank" href="mailto:miles [at] milessabin [dot] com" rel="nofollow">miles [at] milessabin [dot] com> wrote:
On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com" target="_blank" href="mailto:sean [dot] mcdirmid [at] gmail [dot] com" rel="nofollow">sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> If there is a bug in build dependency tracking, I can fix it fairly quickly.

That really doesn't matter.

* The current code is impenetrable and buggy.

* Nobody other than you wants to support it.

* You don't appear to be willing to do the work of maintaining it yourself.

* David is in the process of putting together a better, maintained alternative.

I can't really see why I should waste my time on this ...

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin



Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation
I'm waiting for a description of how dependency tracking is broken in the plugin. I won't think of doing anything else on the plugin until that happens. Miles claims its broken, but he wants to keep it a secret in how its broken. And that really irks me off, how can a project be run like that??? 
Really, its not fun working on this project anymore. 
On Tue, Feb 10, 2009 at 11:24 PM, Windemuth Andreas <windemut [at] yahoo [dot] com> wrote:

> He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised,

Let me assert here that the plugin is not universally despised. I feel the plugin, warts and all, is tremendously more useful than any mere editor can be, and from what I know about the direction that Miles is working in, I see a glorious future for it. Please keep up the good work.

Andreas



From: Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com>
To: Josh Suereth <joshua [dot] suereth [at] gmail [dot] com>
Cc: Miles Sabin <miles [at] milessabin [dot] com>; Mark Harrah <harrah [at] bu [dot] edu>; scala-tools [at] listes [dot] epfl [dot] ch; scala-debate <scala-debate [at] listes [dot] epfl [dot] ch>
Sent: Monday, February 9, 2009 9:22:22 PM
Subject: Re: [scala-debate] Re: [scala-tools] Tracking changed files for recompilation

I'd rather not join scala-debate, I don't have much time, and I've got no vested interest in Scala right now given my work commitments :(
Miles is correct for feeling that the code is unapproachable, but then I never got much help from Martin on scalac either (same arguments can be made there). He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised, and no one thinks its even worth filing a decent bug report--even Miles would rather bash the plugin than go into what the problem is. 
In that case, I should just as well resign from the project/scala and I can move on to promoting Bling full time. I don't really see a better option right now, I don't even get to write Scala code anymore, and won't unless the mythical Scala.NET ever gets to a point where it is usable. Its nothing against anyone, I really want to see David write a decent dependency calculator, and I want to see a great IDE, I don't care if its Cao Yuan's NetBeans or IntelliJ, or whatever. I just don't think I'm very useful to the project anymore. 

On Tue, Feb 10, 2009 at 10:08 AM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com> wrote:
Because email is fun!!!!!ONE!!!!!!!  

(This probably makes no sense to anyone who didn't read my earlier email on the emoticon subject)


Anyway to take this conversation off-line or, Sean, would you mind joining scala-debate?  I really believe we've entered debate territory, and I'd rather leave this channel open for more "How does this tool work, or Should this tool be created?" questions.   I beleive we've left "This tool is useful" territory and are now arguing details over whose code to use.

On Mon, Feb 9, 2009 at 9:01 PM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> If there is a bug in build dependency tracking, I can fix it fairly quickly.

That really doesn't matter.

* The current code is impenetrable and buggy.

* Nobody other than you wants to support it.

* You don't appear to be willing to do the work of maintaining it yourself.

* David is in the process of putting together a better, maintained alternative.

I can't really see why I should waste my time on this ...

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin




Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation
I'm waiting for a description of how dependency tracking is broken in the plugin. I won't think of doing anything else on the plugin until that happens. Miles claims its broken, but he wants to keep it a secret in how its broken. And that really irks me off, how can a project be run like that??? 
Really, its not fun working on this project anymore. 
On Tue, Feb 10, 2009 at 11:24 PM, Windemuth Andreas <windemut [at] yahoo [dot] com> wrote:

> He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised,

Let me assert here that the plugin is not universally despised. I feel the plugin, warts and all, is tremendously more useful than any mere editor can be, and from what I know about the direction that Miles is working in, I see a glorious future for it. Please keep up the good work.

Andreas



From: Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com>
To: Josh Suereth <joshua [dot] suereth [at] gmail [dot] com>
Cc: Miles Sabin <miles [at] milessabin [dot] com>; Mark Harrah <harrah [at] bu [dot] edu>; scala-tools [at] listes [dot] epfl [dot] ch; scala-debate <scala-debate [at] listes [dot] epfl [dot] ch>
Sent: Monday, February 9, 2009 9:22:22 PM
Subject: Re: [scala-debate] Re: [scala-tools] Tracking changed files for recompilation

I'd rather not join scala-debate, I don't have much time, and I've got no vested interest in Scala right now given my work commitments :(
Miles is correct for feeling that the code is unapproachable, but then I never got much help from Martin on scalac either (same arguments can be made there). He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised, and no one thinks its even worth filing a decent bug report--even Miles would rather bash the plugin than go into what the problem is. 
In that case, I should just as well resign from the project/scala and I can move on to promoting Bling full time. I don't really see a better option right now, I don't even get to write Scala code anymore, and won't unless the mythical Scala.NET ever gets to a point where it is usable. Its nothing against anyone, I really want to see David write a decent dependency calculator, and I want to see a great IDE, I don't care if its Cao Yuan's NetBeans or IntelliJ, or whatever. I just don't think I'm very useful to the project anymore. 

On Tue, Feb 10, 2009 at 10:08 AM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com> wrote:
Because email is fun!!!!!ONE!!!!!!!  

(This probably makes no sense to anyone who didn't read my earlier email on the emoticon subject)


Anyway to take this conversation off-line or, Sean, would you mind joining scala-debate?  I really believe we've entered debate territory, and I'd rather leave this channel open for more "How does this tool work, or Should this tool be created?" questions.   I beleive we've left "This tool is useful" territory and are now arguing details over whose code to use.

On Mon, Feb 9, 2009 at 9:01 PM, Miles Sabin <miles [at] milessabin [dot] com> wrote:
On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:
> If there is a bug in build dependency tracking, I can fix it fairly quickly.

That really doesn't matter.

* The current code is impenetrable and buggy.

* Nobody other than you wants to support it.

* You don't appear to be willing to do the work of maintaining it yourself.

* David is in the process of putting together a better, maintained alternative.

I can't really see why I should waste my time on this ...

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin




Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Re: Tracking changed files for recompilation
Sean,
You reported that the bug described is "expected behavior" as scalac (or fsc?) does not flush symbols.  We either need a workaround, or a fix to scalac.
Until then it will be hard to figure out which bugs you'll accept as dependency analysis bugs, as they have the same symptoms to the user.


On Feb 10, 2009, at 6:38 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:

I'm waiting for a description of how dependency tracking is broken in the plugin. I won't think of doing anything else on the plugin until that happens. Miles claims its broken, but he wants to keep it a secret in how its broken. And that really irks me off, how can a project be run like that??? 
Really, its not fun working on this project anymore. 
On Tue, Feb 10, 2009 at 11:24 PM, Windemuth Andreas <windemut [at] yahoo [dot] com (windemut [at] yahoo [dot] com" rel="nofollow">windemut [at] yahoo [dot] com)> wrote:

> He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised,

Let me assert here that the plugin is not universally despised. I feel the plugin, warts and all, is tremendously more useful than any mere editor can be, and from what I know about the direction that Miles is working in, I see a glorious future for it. Please keep up the good work.

Andreas



From: Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com (sean [dot] mcdirmid [at] gmail [dot] com" rel="nofollow">sean [dot] mcdirmid [at] gmail [dot] com)>
To: Josh Suereth <joshua [dot] suereth [at] gmail [dot] com (joshua [dot] suereth [at] gmail [dot] com" rel="nofollow">joshua [dot] suereth [at] gmail [dot] com)>
Cc: Miles Sabin <miles [at] milessabin [dot] com (miles [at] milessabin [dot] com" rel="nofollow">miles [at] milessabin [dot] com)>; Mark Harrah <harrah [at] bu [dot] edu (harrah [at] bu [dot] edu" rel="nofollow">harrah [at] bu [dot] edu)>; scala-tools [at] listes [dot] epfl [dot] ch (scala-tools [at] listes [dot] epfl [dot] ch" rel="nofollow">scala-tools [at] listes [dot] epfl [dot] ch); scala-debate <scala-debate [at] listes [dot] epfl [dot] ch (scala-debate [at] listes [dot] epfl [dot] ch" rel="nofollow">scala-debate [at] listes [dot] epfl [dot] ch)>
Sent: Monday, February 9, 2009 9:22:22 PM
Subject: Re: [scala-debate] Re: [scala-tools] Tracking changed files for recompilation

I'd rather not join scala-debate, I don't have much time, and I've got no vested interest in Scala right now given my work commitments :(
Miles is correct for feeling that the code is unapproachable, but then I never got much help from Martin on scalac either (same arguments can be made there). He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised, and no one thinks its even worth filing a decent bug report--even Miles would rather bash the plugin than go into what the problem is. 
In that case, I should just as well resign from the project/scala and I can move on to promoting Bling full time. I don't really see a better option right now, I don't even get to write Scala code anymore, and won't unless the mythical Scala.NET ever gets to a point where it is usable. Its nothing against anyone, I really want to see David write a decent dependency calculator, and I want to see a great IDE, I don't care if its Cao Yuan's NetBeans or IntelliJ, or whatever. I just don't think I'm very useful to the project anymore. 

On Tue, Feb 10, 2009 at 10:08 AM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com (joshua [dot] suereth [at] gmail [dot] com" rel="nofollow">joshua [dot] suereth [at] gmail [dot] com)> wrote:
Because email is fun!!!!!ONE!!!!!!!   <349.gif><343.gif><364.gif><35D.gif><1A5.gif><347.gif><322.gif><35E.gif><33E.gif><360.gif>

(This probably makes no sense to anyone who didn't read my earlier email on the emoticon subject)


Anyway to take this conversation off-line or, Sean, would you mind joining scala-debate?  I really believe we've entered debate territory, and I'd rather leave this channel open for more "How does this tool work, or Should this tool be created?" questions.   I beleive we've left "This tool is useful" territory and are now arguing details over whose code to use.

On Mon, Feb 9, 2009 at 9:01 PM, Miles Sabin <miles [at] milessabin [dot] com (miles [at] milessabin [dot] com" rel="nofollow">miles [at] milessabin [dot] com)> wrote:
On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com (sean [dot] mcdirmid [at] gmail [dot] com" rel="nofollow">sean [dot] mcdirmid [at] gmail [dot] com)> wrote:
> If there is a bug in build dependency tracking, I can fix it fairly quickly.

That really doesn't matter.

* The current code is impenetrable and buggy.

* Nobody other than you wants to support it.

* You don't appear to be willing to do the work of maintaining it yourself.

* David is in the process of putting together a better, maintained alternative.

I can't really see why I should waste my time on this ...

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin




Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Re: Tracking changed files for recompilation
Sean,
You reported that the bug described is "expected behavior" as scalac (or fsc?) does not flush symbols.  We either need a workaround, or a fix to scalac.
Until then it will be hard to figure out which bugs you'll accept as dependency analysis bugs, as they have the same symptoms to the user.


On Feb 10, 2009, at 6:38 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:

I'm waiting for a description of how dependency tracking is broken in the plugin. I won't think of doing anything else on the plugin until that happens. Miles claims its broken, but he wants to keep it a secret in how its broken. And that really irks me off, how can a project be run like that??? 
Really, its not fun working on this project anymore. 
On Tue, Feb 10, 2009 at 11:24 PM, Windemuth Andreas <windemut [at] yahoo [dot] com (windemut [at] yahoo [dot] com" rel="nofollow">windemut [at] yahoo [dot] com)> wrote:

> He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised,

Let me assert here that the plugin is not universally despised. I feel the plugin, warts and all, is tremendously more useful than any mere editor can be, and from what I know about the direction that Miles is working in, I see a glorious future for it. Please keep up the good work.

Andreas



From: Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com (sean [dot] mcdirmid [at] gmail [dot] com" rel="nofollow">sean [dot] mcdirmid [at] gmail [dot] com)>
To: Josh Suereth <joshua [dot] suereth [at] gmail [dot] com (joshua [dot] suereth [at] gmail [dot] com" rel="nofollow">joshua [dot] suereth [at] gmail [dot] com)>
Cc: Miles Sabin <miles [at] milessabin [dot] com (miles [at] milessabin [dot] com" rel="nofollow">miles [at] milessabin [dot] com)>; Mark Harrah <harrah [at] bu [dot] edu (harrah [at] bu [dot] edu" rel="nofollow">harrah [at] bu [dot] edu)>; scala-tools [at] listes [dot] epfl [dot] ch (scala-tools [at] listes [dot] epfl [dot] ch" rel="nofollow">scala-tools [at] listes [dot] epfl [dot] ch); scala-debate <scala-debate [at] listes [dot] epfl [dot] ch (scala-debate [at] listes [dot] epfl [dot] ch" rel="nofollow">scala-debate [at] listes [dot] epfl [dot] ch)>
Sent: Monday, February 9, 2009 9:22:22 PM
Subject: Re: [scala-debate] Re: [scala-tools] Tracking changed files for recompilation

I'd rather not join scala-debate, I don't have much time, and I've got no vested interest in Scala right now given my work commitments :(
Miles is correct for feeling that the code is unapproachable, but then I never got much help from Martin on scalac either (same arguments can be made there). He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised, and no one thinks its even worth filing a decent bug report--even Miles would rather bash the plugin than go into what the problem is. 
In that case, I should just as well resign from the project/scala and I can move on to promoting Bling full time. I don't really see a better option right now, I don't even get to write Scala code anymore, and won't unless the mythical Scala.NET ever gets to a point where it is usable. Its nothing against anyone, I really want to see David write a decent dependency calculator, and I want to see a great IDE, I don't care if its Cao Yuan's NetBeans or IntelliJ, or whatever. I just don't think I'm very useful to the project anymore. 

On Tue, Feb 10, 2009 at 10:08 AM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com (joshua [dot] suereth [at] gmail [dot] com" rel="nofollow">joshua [dot] suereth [at] gmail [dot] com)> wrote:
Because email is fun!!!!!ONE!!!!!!!   <349.gif><343.gif><364.gif><35D.gif><1A5.gif><347.gif><322.gif><35E.gif><33E.gif><360.gif>

(This probably makes no sense to anyone who didn't read my earlier email on the emoticon subject)


Anyway to take this conversation off-line or, Sean, would you mind joining scala-debate?  I really believe we've entered debate territory, and I'd rather leave this channel open for more "How does this tool work, or Should this tool be created?" questions.   I beleive we've left "This tool is useful" territory and are now arguing details over whose code to use.

On Mon, Feb 9, 2009 at 9:01 PM, Miles Sabin <miles [at] milessabin [dot] com (miles [at] milessabin [dot] com" rel="nofollow">miles [at] milessabin [dot] com)> wrote:
On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com (sean [dot] mcdirmid [at] gmail [dot] com" rel="nofollow">sean [dot] mcdirmid [at] gmail [dot] com)> wrote:
> If there is a bug in build dependency tracking, I can fix it fairly quickly.

That really doesn't matter.

* The current code is impenetrable and buggy.

* Nobody other than you wants to support it.

* You don't appear to be willing to do the work of maintaining it yourself.

* David is in the process of putting together a better, maintained alternative.

I can't really see why I should waste my time on this ...

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin




Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation
Understood and I mostly agree.
The bug described will effect everyone in the scala-verse, not just the plugin, its something built into scalac; e.g., if or if not David's dependency tracker is used, it doesn't matter. Miles' accusation is that plugin dependency tracking was especially broken and that somehow David's component could only be an improvement.  The problem here is that there are bazillion ways that the plugin can fail: it could be a problem in scalac, it could be a problem in the new scala/java interop, the build compiler could be crashing, we might not be getting change delta's from Eclipse correctly, or it could be because of a full moon (well, hopefully not that). The user of course probably doesn't know this, they just see that things aren't working, and that's fine. But after they report problem, the maintainers have to do a good job in triaging the problem. Right now, there is no real triaging going on, current maintainers feel free to bash components without knowing whats going on, alienating and agitating other contributors. So then we have a project management problem, not really a technical problem. 
It could very well be that dependency tracking is broken in the plugin, or maybe it isn't and something else is broken. Without knowing, we have no idea what needs to be fixed or replaced. If its not a problem with the plugin's dependency tracking, then replacing it with David's will substantially slow down the plugin (transitive closure recompile is close to recompile all on many projects) for no good reason. Right now, Miles is either proposing to act now and figure out the problem later, or he knows what the problems are and is just refusing to share that with me. I'm disagreeing with either option.  
On Wed, Feb 11, 2009 at 7:49 AM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com> wrote:
Sean,
You reported that the bug described is "expected behavior" as scalac (or fsc?) does not flush symbols.  We either need a workaround, or a fix to scalac.
Until then it will be hard to figure out which bugs you'll accept as dependency analysis bugs, as they have the same symptoms to the user.


On Feb 10, 2009, at 6:38 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:

I'm waiting for a description of how dependency tracking is broken in the plugin. I won't think of doing anything else on the plugin until that happens. Miles claims its broken, but he wants to keep it a secret in how its broken. And that really irks me off, how can a project be run like that??? 
Really, its not fun working on this project anymore. 
On Tue, Feb 10, 2009 at 11:24 PM, Windemuth Andreas <windemut [at] yahoo [dot] comwindemut [at] yahoo [dot] com> wrote:

> He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised,

Let me assert here that the plugin is not universally despised. I feel the plugin, warts and all, is tremendously more useful than any mere editor can be, and from what I know about the direction that Miles is working in, I see a glorious future for it. Please keep up the good work.

Andreas



From: Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] comsean [dot] mcdirmid [at] gmail [dot] com>
To: Josh Suereth <joshua [dot] suereth [at] gmail [dot] comjoshua [dot] suereth [at] gmail [dot] com>
Cc: Miles Sabin <miles [at] milessabin [dot] commiles [at] milessabin [dot] com>; Mark Harrah <harrah [at] bu [dot] eduharrah [at] bu [dot] edu>; scala-tools [at] listes [dot] epfl [dot] chscala-tools [at] listes [dot] epfl [dot] ch; scala-debate <scala-debate [at] listes [dot] epfl [dot] chscala-debate [at] listes [dot] epfl [dot] ch>
Sent: Monday, February 9, 2009 9:22:22 PM
Subject: Re: [scala-debate] Re: [scala-tools] Tracking changed files for recompilation

I'd rather not join scala-debate, I don't have much time, and I've got no vested interest in Scala right now given my work commitments :(
Miles is correct for feeling that the code is unapproachable, but then I never got much help from Martin on scalac either (same arguments can be made there). He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised, and no one thinks its even worth filing a decent bug report--even Miles would rather bash the plugin than go into what the problem is. 
In that case, I should just as well resign from the project/scala and I can move on to promoting Bling full time. I don't really see a better option right now, I don't even get to write Scala code anymore, and won't unless the mythical Scala.NET ever gets to a point where it is usable. Its nothing against anyone, I really want to see David write a decent dependency calculator, and I want to see a great IDE, I don't care if its Cao Yuan's NetBeans or IntelliJ, or whatever. I just don't think I'm very useful to the project anymore. 

On Tue, Feb 10, 2009 at 10:08 AM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] comjoshua [dot] suereth [at] gmail [dot] com> wrote:
Because email is fun!!!!!ONE!!!!!!!   <349.gif><343.gif><364.gif><35D.gif><1A5.gif><347.gif><322.gif><35E.gif><33E.gif><360.gif>

(This probably makes no sense to anyone who didn't read my earlier email on the emoticon subject)


Anyway to take this conversation off-line or, Sean, would you mind joining scala-debate?  I really believe we've entered debate territory, and I'd rather leave this channel open for more "How does this tool work, or Should this tool be created?" questions.   I beleive we've left "This tool is useful" territory and are now arguing details over whose code to use.

On Mon, Feb 9, 2009 at 9:01 PM, Miles Sabin <miles [at] milessabin [dot] commiles [at] milessabin [dot] com> wrote:
On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] comsean [dot] mcdirmid [at] gmail [dot] com> wrote:
> If there is a bug in build dependency tracking, I can fix it fairly quickly.

That really doesn't matter.

* The current code is impenetrable and buggy.

* Nobody other than you wants to support it.

* You don't appear to be willing to do the work of maintaining it yourself.

* David is in the process of putting together a better, maintained alternative.

I can't really see why I should waste my time on this ...

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin





Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation
Understood and I mostly agree.
The bug described will effect everyone in the scala-verse, not just the plugin, its something built into scalac; e.g., if or if not David's dependency tracker is used, it doesn't matter. Miles' accusation is that plugin dependency tracking was especially broken and that somehow David's component could only be an improvement.  The problem here is that there are bazillion ways that the plugin can fail: it could be a problem in scalac, it could be a problem in the new scala/java interop, the build compiler could be crashing, we might not be getting change delta's from Eclipse correctly, or it could be because of a full moon (well, hopefully not that). The user of course probably doesn't know this, they just see that things aren't working, and that's fine. But after they report problem, the maintainers have to do a good job in triaging the problem. Right now, there is no real triaging going on, current maintainers feel free to bash components without knowing whats going on, alienating and agitating other contributors. So then we have a project management problem, not really a technical problem. 
It could very well be that dependency tracking is broken in the plugin, or maybe it isn't and something else is broken. Without knowing, we have no idea what needs to be fixed or replaced. If its not a problem with the plugin's dependency tracking, then replacing it with David's will substantially slow down the plugin (transitive closure recompile is close to recompile all on many projects) for no good reason. Right now, Miles is either proposing to act now and figure out the problem later, or he knows what the problems are and is just refusing to share that with me. I'm disagreeing with either option.  
On Wed, Feb 11, 2009 at 7:49 AM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com> wrote:
Sean,
You reported that the bug described is "expected behavior" as scalac (or fsc?) does not flush symbols.  We either need a workaround, or a fix to scalac.
Until then it will be hard to figure out which bugs you'll accept as dependency analysis bugs, as they have the same symptoms to the user.


On Feb 10, 2009, at 6:38 PM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com> wrote:

I'm waiting for a description of how dependency tracking is broken in the plugin. I won't think of doing anything else on the plugin until that happens. Miles claims its broken, but he wants to keep it a secret in how its broken. And that really irks me off, how can a project be run like that??? 
Really, its not fun working on this project anymore. 
On Tue, Feb 10, 2009 at 11:24 PM, Windemuth Andreas <windemut [at] yahoo [dot] comwindemut [at] yahoo [dot] com> wrote:

> He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised,

Let me assert here that the plugin is not universally despised. I feel the plugin, warts and all, is tremendously more useful than any mere editor can be, and from what I know about the direction that Miles is working in, I see a glorious future for it. Please keep up the good work.

Andreas



From: Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] comsean [dot] mcdirmid [at] gmail [dot] com>
To: Josh Suereth <joshua [dot] suereth [at] gmail [dot] comjoshua [dot] suereth [at] gmail [dot] com>
Cc: Miles Sabin <miles [at] milessabin [dot] commiles [at] milessabin [dot] com>; Mark Harrah <harrah [at] bu [dot] eduharrah [at] bu [dot] edu>; scala-tools [at] listes [dot] epfl [dot] chscala-tools [at] listes [dot] epfl [dot] ch; scala-debate <scala-debate [at] listes [dot] epfl [dot] chscala-debate [at] listes [dot] epfl [dot] ch>
Sent: Monday, February 9, 2009 9:22:22 PM
Subject: Re: [scala-debate] Re: [scala-tools] Tracking changed files for recompilation

I'd rather not join scala-debate, I don't have much time, and I've got no vested interest in Scala right now given my work commitments :(
Miles is correct for feeling that the code is unapproachable, but then I never got much help from Martin on scalac either (same arguments can be made there). He's right that I don't have any feeling about maintaining the plugin anymore, since it seems to be universally despised, and no one thinks its even worth filing a decent bug report--even Miles would rather bash the plugin than go into what the problem is. 
In that case, I should just as well resign from the project/scala and I can move on to promoting Bling full time. I don't really see a better option right now, I don't even get to write Scala code anymore, and won't unless the mythical Scala.NET ever gets to a point where it is usable. Its nothing against anyone, I really want to see David write a decent dependency calculator, and I want to see a great IDE, I don't care if its Cao Yuan's NetBeans or IntelliJ, or whatever. I just don't think I'm very useful to the project anymore. 

On Tue, Feb 10, 2009 at 10:08 AM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] comjoshua [dot] suereth [at] gmail [dot] com> wrote:
Because email is fun!!!!!ONE!!!!!!!   <349.gif><343.gif><364.gif><35D.gif><1A5.gif><347.gif><322.gif><35E.gif><33E.gif><360.gif>

(This probably makes no sense to anyone who didn't read my earlier email on the emoticon subject)


Anyway to take this conversation off-line or, Sean, would you mind joining scala-debate?  I really believe we've entered debate territory, and I'd rather leave this channel open for more "How does this tool work, or Should this tool be created?" questions.   I beleive we've left "This tool is useful" territory and are now arguing details over whose code to use.

On Mon, Feb 9, 2009 at 9:01 PM, Miles Sabin <miles [at] milessabin [dot] commiles [at] milessabin [dot] com> wrote:
On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] comsean [dot] mcdirmid [at] gmail [dot] com> wrote:
> If there is a bug in build dependency tracking, I can fix it fairly quickly.

That really doesn't matter.

* The current code is impenetrable and buggy.

* Nobody other than you wants to support it.

* You don't appear to be willing to do the work of maintaining it yourself.

* David is in the process of putting together a better, maintained alternative.

I can't really see why I should waste my time on this ...

Cheers,


Miles

--
Miles Sabin
tel:    +44 (0)1273 720 779
mobile: +44 (0)7813 944 528
skype:  milessabin





Sean McDirmid
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation
Tupshin, thanks the problem report, every issue like this should probably be in Track. 
I'm not sure what the plan is on fixing this problem, the last time we discussed it (more than a year and a half ago?), the decision was to not fix. It is definitely something that can be fixed when the resident compiler gets some attention, but I don't know what the plan is now. 

On Wed, Feb 11, 2009 at 9:18 AM, Tupshin Harper <tharper [at] livejournalinc [dot] com> wrote:
After this problem with evicting stale class symbols is fixed I promise to diligently report all problems I can that look at all like they could be related to incorrect dependency tracking.

-Tupshin

Sean McDirmid wrote:
Understood and I mostly agree.

The bug described will effect everyone in the scala-verse, not just the plugin, its something built into scalac; e.g., if or if not David's dependency tracker is used, it doesn't matter. Miles' accusation is that plugin dependency tracking was especially broken and that somehow David's component could only be an improvement. The problem here is that there are bazillion ways that the plugin can fail: it could be a problem in scalac, it could be a problem in the new scala/java interop, the build compiler could be crashing, we might not be getting change delta's from Eclipse correctly, or it could be because of a full moon (well, hopefully not that). The user of course probably doesn't know this, they just see that things aren't working, and that's fine. But after they report problem, the maintainers have to do a good job in triaging the problem. Right now, there is no real triaging going on, current maintainers feel free to bash components without knowing whats going on, alienating and agitating other contributors. So then we have a project management problem, not really a technical problem.
It could very well be that dependency tracking is broken in the plugin, or maybe it isn't and something else is broken. Without knowing, we have no idea what needs to be fixed or replaced. If its not a problem with the plugin's dependency tracking, then replacing it with David's will substantially slow down the plugin (transitive closure recompile is close to recompile all on many projects) for no good reason. Right now, Miles is either proposing to act now and figure out the problem later, or he knows what the problems are and is just refusing to share that with me. I'm disagreeing with either option.  
On Wed, Feb 11, 2009 at 7:49 AM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com <mailto:joshua [dot] suereth [at] gmail [dot] com>> wrote:

   Sean,

   You reported that the bug described is "expected behavior" as
   scalac (or fsc?) does not flush symbols.  We either need a
   workaround, or a fix to scalac.

   Until then it will be hard to figure out which bugs you'll accept
   as dependency analysis bugs, as they have the same symptoms to the
   user.



   On Feb 10, 2009, at 6:38 PM, Sean McDirmid
   <sean [dot] mcdirmid [at] gmail [dot] com <mailto:sean [dot] mcdirmid [at] gmail [dot] com>> wrote:

   I'm waiting for a description of how dependency tracking is
   broken in the plugin. I won't think of doing anything else on the
   plugin until that happens. Miles claims its broken, but he wants
   to keep it a secret in how its broken. And that really irks me
   off, how can a project be run like that???
   Really, its not fun working on this project anymore.
   On Tue, Feb 10, 2009 at 11:24 PM, Windemuth Andreas
   <windemut [at] yahoo [dot] com <mailto:windemut [at] yahoo [dot] com>> wrote:


       > He's right that I don't have any feeling about maintaining
       the plugin anymore, since it seems to be universally despised,

       Let me assert here that the plugin is not universally
       despised. I feel the plugin, warts and all, is tremendously
       more useful than any mere editor can be, and from what I know
       about the direction that Miles is working in, I see a
       glorious future for it. Please keep up the good work.

       Andreas



       ------------------------------------------------------------------------
       *From:* Sean McDirmid <sean [dot] mcdirmid [at] gmail [dot] com
       <mailto:sean [dot] mcdirmid [at] gmail [dot] com>>
       *To:* Josh Suereth <joshua [dot] suereth [at] gmail [dot] com
       <mailto:joshua [dot] suereth [at] gmail [dot] com>>
       *Cc:* Miles Sabin <miles [at] milessabin [dot] com
       <mailto:miles [at] milessabin [dot] com>>; Mark Harrah <harrah [at] bu [dot] edu
       <mailto:harrah [at] bu [dot] edu>>; scala-tools [at] listes [dot] epfl [dot] ch
       <mailto:scala-tools [at] listes [dot] epfl [dot] ch>; scala-debate
       <scala-debate [at] listes [dot] epfl [dot] ch
       <mailto:scala-debate [at] listes [dot] epfl [dot] ch>>
       *Sent:* Monday, February 9, 2009 9:22:22 PM

       *Subject:* Re: [scala-debate] Re: [scala-tools] Tracking
       changed files for recompilation

       I'd rather not join scala-debate, I don't have much time, and
       I've got no vested interest in Scala right now given my work
       commitments :(

       Miles is correct for feeling that the code is unapproachable,
       but then I never got much help from Martin on scalac either
       (same arguments can be made there). He's right that I don't
       have any feeling about maintaining the plugin anymore, since
       it seems to be universally despised, and no one thinks its
       even worth filing a decent bug report--even Miles would
       rather bash the plugin than go into what the problem is.
       In that case, I should just as well resign from the
       project/scala and I can move on to promoting Bling full time.
       I don't really see a better option right now, I don't even
       get to write Scala code anymore, and won't unless the
       mythical Scala.NET <http://Scala.NET> ever gets to a point
       where it is usable. Its nothing against anyone, I really want
       to see David write a decent dependency calculator, and I want
       to see a great IDE, I don't care if its Cao Yuan's NetBeans
       or IntelliJ, or whatever. I just don't think I'm very useful
       to the project anymore.
       On Tue, Feb 10, 2009 at 10:08 AM, Josh Suereth
       <joshua [dot] suereth [at] gmail [dot] com <mailto:joshua [dot] suereth [at] gmail [dot] com>>
       wrote:

           Because email is fun!!!!!ONE!!!!!!!              <349.gif><343.gif><364.gif><35D.gif><1A5.gif><347.gif><322.gif><35E.gif><33E.gif><360.gif>



           (This probably makes no sense to anyone who didn't read
           my earlier email on the emoticon subject)


           Anyway to take this conversation off-line or, Sean, would
           you mind joining scala-debate?  I really believe we've
           entered debate territory, and I'd rather leave this
           channel open for more "How does this tool work, or Should
           this tool be created?" questions.   I beleive we've left
           "This tool is useful" territory and are now arguing
           details over whose code to use.


           On Mon, Feb 9, 2009 at 9:01 PM, Miles Sabin
           <miles [at] milessabin [dot] com <mailto:miles [at] milessabin [dot] com>> wrote:

               On Tue, Feb 10, 2009 at 1:58 AM, Sean McDirmid
               <sean [dot] mcdirmid [at] gmail [dot] com
               <mailto:sean [dot] mcdirmid [at] gmail [dot] com>> wrote:
               > If there is a bug in build dependency tracking, I
               can fix it fairly quickly.

               That really doesn't matter.

               * The current code is impenetrable and buggy.

               * Nobody other than you wants to support it.

               * You don't appear to be willing to do the work of
               maintaining it yourself.

               * David is in the process of putting together a
               better, maintained alternative.

               I can't really see why I should waste my time on this ...

               Cheers,


               Miles

               --
               Miles Sabin
               tel:    +44 (0)1273 720 779
               mobile: +44 (0)7813 944 528
               skype:  milessabin








Naftoli Gugenheim
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation
One "solution" (for eclipse) would be to use the refactor command, (when it works -- does it yet?) instead of manually renaming, and put the logic of deleting the old class files in the refactor component, not the build component.

On Tue, Feb 10, 2009 at 2:48 AM, Tupshin Harper <tupshin [at] tupshin [dot] com> wrote:
OK. Good to know that it's an understood problem. I'll weigh in as completely neutral on *how* it gets fixed, but unless it does get fixed, then doing any significant amount of refactoring is an exceedingly painful process.

In the interests of correctness, it would seem essential to flush the symbols of a class before recompiling until and unless there existed a more sophisticated approach that did a first pass to see if symbols were reusable and a second pass to do a full compile. The current approach is proven incorrect and therefore must change.

-Tupshin

Sean McDirmid wrote:
Ah, ok. I understand, everything is working as expected. We aren't cleaning up stale classes in between recompiles. Its easy enough to delete the stale class files, but it is much more difficult to evict the stale class symbols from symbol loaders.
scalac doesn't generate class files on a bad build, so that explains why Foo2.class is never generated (in either case) while it doesn't delete class files either after a rebuild (which is why Foo.class hangs around).
In scalac non-resident, we can fix this by deleting all class files generated from the source file on its previous compile before it is recompiled. I'll let David and Martin decide if they want scalac to do that. For scalac resident, we have to flush out all symbols of a class before the source file is recompiled (rather than just see if we can reuse symbols as is the case now). Martin should decide if he wants to do that or not, if Martin does that for FSC we can piggy back on the change for the plugin.
FYI, in Java, the ejc does the right thing when rebuilding source files. I don't believe javac does (i.e., scalac is mimicking javac's behavior).

On Tue, Feb 10, 2009 at 1:34 PM, Tupshin Harper <tupshin [at] tupshin [dot] com <mailto:tupshin [at] tupshin [dot] com>> wrote:

   I could, but I really really don't need to as you can reproduce it
   yourself very quickly.

   Test scenario:
   1) I just updated to scala plugin 2.8.0.r17068-b20090210023355,
   but identical problem reproduced with a version from a couple of
   weeks ago
   2) Eclipse if version 3.4.1
   3) Created brand new workspace and did a fresh launch of eclipse.
   4) System is 64 bit Windows Vista. If you are unable to reproduce,
   I will try it on a Debian Linux system, but that will take some
   time to set up.void
   5) Unlike my more complex scenarios, there are no errors in the
   Error log, only a couple of non-threatening warnings.

   Steps to reproduce:
   1) Create scala project "TestProject"
   2) Say yes when given the chance to switch to scala perspective.
   3) create new package in src called "test"
   4) create new class in test called "Foo"
   5) create new class in test called "Bar"
   6) Edit Foo.scala and change "class Foo {" to be "class Foo2 {"
   and save.
   Observation 1:
   a) The "Problems" view briefly changes its text to Italic,
   indicating that it is compiling something
   b) No problems show up.
   c) workspace/TestProject/bin/test contain files Bar.class,
   Foo.class, and Foo2.class
   Conclusion:
   Renaming a class in a scala file fails to delete the orignal
   .class file even though it creates a new one. (But see below)

   Further testing:
   1) If I "clean project", I correctly get "not found: type Foo"
   2) If I then rename Foo2 back to Foo, the error correctly goes away.
   3) Somewhat surprisingly, bin directory only has Bar.class and
   Foo.class this time. It does not still have Foo2.class
   Conclusion:
   Contrary to above, class files are sometimes deleted when the
   class is renamed but possibly not when anything depends on them.
   Confirming this last hypothesis hard to do.

   -Tupshin


   Sean McDirmid wrote:

       Could you send me a copy of your project? This sounds like it
       is easy to reproduce.
       On Tue, Feb 10, 2009 at 11:29 AM, Tupshin Harper
       <tupshin [at] tupshin [dot] com <mailto:tupshin [at] tupshin [dot] com>
       <mailto:tupshin [at] tupshin [dot] com <mailto:tupshin [at] tupshin [dot] com>>> wrote:

          After converting every last line to scala and getting rid
       of the
          last trace of Java in that same project, if
          a) there is any single compilation error in any file (even a
          standalone one that has no interdependencies on any other
       file in
          the project) and
          b) I deliberately introduce new compilation errors by
       renaming a
          class (not doing file rename, just changing the name of the
       class
          inside the file)
          then a cascading series of 44 errors show up in my project "not
          found: type X", "nof found: value y", etc etc etc etc etc.
          And when I rename the class back to what it should be and
       save it,
          all of those errors stay until I do a "clean project", at which
          point I am back to the one original (and correct) error.

          Then when I tried to verify this test case and delete the file
          with an error in it, I get:
          Can't delete file...TestListerner.scala [in com.foo.bar[in
       src [in
          Bar]]] does not exist
          Which I couldn't resolve except by quitting eclipse and
       restarting
          it (closing and reopening the project didn't work).

          I then tried to reproduce the same problem with no existing
       errors
          in the project, and this time, had an even worse (IMO) result,
          which was a failure to detect *any* error after renaming
       the class
          until I cleaned the project at which point it succesfully
       detected
          the missing class.

          I then went to see whether this problem could have to do
       with my
          project still having a Java nature in addition to the scala
       one,
          so I created a new scala project and copied my source tree from
          hybrid project to scala project.
          Problems:
          1) There are immediately two representation of every scala
       file,
          one with a "J" icon and one with a "S" icon.
          2) Most of the package icons have an error symbol in them
       despite
          there being no classes with errors in those packages. (this
          resolves itself after closing and reopening the project
          3) I then deliberately reintroduced one isolated
       compilation error
          in this pristine scala-only project, renamed one class to
          introduce a second error, and had it fail to detect the
          compilation problems that should occur when that class is
       missing
          until I cleaned the project.

          In the meantime, the error log shows a whole progression of fun
          things like:
          While loading class "lampion.presentation.Presentations$",
       thread
          "Thread[worker-Foo,6,main]" timed out waiting (5000ms) for
       thread
          "Thread[main,6,main]" to finish starting bundle
                "reference:file:plugins\ch.epfl.lamp.sdt.core_2.8.0.r17010-b20090131023503.jar
          [920]". To avoid deadlock, thread
       "Thread[worker-LBDone,6,main]"
          is proceeding but "lampion.presentation.Presentations$" may
       not be
          fully initialized.
          org.eclipse.core.internal.resources.ResourceException: Resource
          '/Bar' is not open. (After I deliberately closed the project)
          File not found:
          G:\Users\tupshin\AppData\Local\Temp\7zE7E36.tmp\Foo.jar
          "Paste" did not complete normally.  Please see the log for more
          information. An exception stack trace is not available. (when
          trying to copy a very short string into a scala file which
       worked
          fine when I typed it manuall)
          etc.
          etc.

          An exercise in frustration indeed, and none of this associated
          with mixed Scala/Java project.

          -Tupshin


          Sean McDirmid wrote:

              But we are not talking about users, we are talking
       about the
              developers of the project deciding not to report bugs!
       This is
              really serious and basically unworkable for a project.
              Mixed scala/java is a big question: it has nothing to
       do with
              the Scala-on-scala dependency tracker. Before, with
       separate
              Java/Scala projects we just reused the classfile
       invalidation
              events provided by the JDT, very very simple but we didn't
              support mixed Scala/Java projects then. I'm not sure
       what the
              deal is now, I did nothing with mixed Scala/Java
       projects, its
              definitely quite possible that it doesn't work. The
       plugin is
              always a moving target :(

              Why did we ever decide that it was a good idea to mix Scala
              and Java code in the same project? Is having two separate
              projects too much inconvenience?
              On Tue, Feb 10, 2009 at 10:40 AM, Josh Suereth
              <joshua [dot] suereth [at] gmail [dot] com
       <mailto:joshua [dot] suereth [at] gmail [dot] com>
       <mailto:joshua [dot] suereth [at] gmail [dot] com
       <mailto:joshua [dot] suereth [at] gmail [dot] com>>
              <mailto:joshua [dot] suereth [at] gmail [dot] com
       <mailto:joshua [dot] suereth [at] gmail [dot] com>
              <mailto:joshua [dot] suereth [at] gmail [dot] com
       <mailto:joshua [dot] suereth [at] gmail [dot] com>>>> wrote:

                 Sean, for future reference:
                 (09:29:37 PM) tupshin: I took a 20 class java
       project and
                 converted it to scala one class at a time using
       eclipse. Was an
                 exercise in frustration. I probably hit "clean
       project" a few
                 hundred times while doing the conversion.
                 (09:29:58 PM) tupshin: Just to see what the *real*
       remaining
                 compilation errors were.

                 I've seen the same myself, and If you want I'll make
       a bug
              report,
                 however it won't be very helpful.   By the time I
       diagnose what
                 the problem was, there's no context for me to figure out
              what's in
                 my .log *and* it's not very reproducable (that
       doesn't mean it
                 doesn't exist, just that we can't figure out how to
              reproduce it).

                 The easiest way for scalac to get freaked out is to
       use it with
                 java.  If your dependency analysis doesn't handle this
              case, then
                 I'd rather go with David's solution, as I live in
       the Java
              world.

                 Also note that a lot of people never report bugs
       against OS
                 projects (90% of my coworkers for instance).

                 -Josh

                 On Mon, Feb 9, 2009 at 9:37 PM, Miles Sabin
              <miles [at] milessabin [dot] com <mailto:miles [at] milessabin [dot] com>
       <mailto:miles [at] milessabin [dot] com <mailto:miles [at] milessabin [dot] com>>
                 <mailto:miles [at] milessabin [dot] com
       <mailto:miles [at] milessabin [dot] com>

              <mailto:miles [at] milessabin [dot] com
       <mailto:miles [at] milessabin [dot] com>>>> wrote:

                     Sean, you're useful to to project if you're still
              willing and
                     able to
                     make concrete contributions. Code would be nice
       (there are
                     plenty of
                     open bugs in areas where you're the best
       authority) and
                     documentation
                     would be wonderful (the scalac-IDE interface and
              incremental AST
                     maintenance, hint, hint).

                     But if you're not going to do that and instead
       just respond
                     negatively
                     to efforts like the one David is making then it
       might be
                     helpful for
                     you to take a little break, at least until
       you're able
              to adopt a
                     slightly more detached attitude to the project.

                     Cheers,


                     Miles

                     --
                     Miles Sabin
                     tel:    +44 (0)1273 720 779
                     mobile: +44 (0)7813 944 528
                     skype:  milessabin









Naftoli Gugenheim
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Tracking changed files for recompilation
One "solution" (for eclipse) would be to use the refactor command, (when it works -- does it yet?) instead of manually renaming, and put the logic of deleting the old class files in the refactor component, not the build component.

On Tue, Feb 10, 2009 at 2:48 AM, Tupshin Harper <tupshin [at] tupshin [dot] com> wrote:
OK. Good to know that it's an understood problem. I'll weigh in as completely neutral on *how* it gets fixed, but unless it does get fixed, then doing any significant amount of refactoring is an exceedingly painful process.

In the interests of correctness, it would seem essential to flush the symbols of a class before recompiling until and unless there existed a more sophisticated approach that did a first pass to see if symbols were reusable and a second pass to do a full compile. The current approach is proven incorrect and therefore must change.

-Tupshin

Sean McDirmid wrote:
Ah, ok. I understand, everything is working as expected. We aren't cleaning up stale classes in between recompiles. Its easy enough to delete the stale class files, but it is much more difficult to evict the stale class symbols from symbol loaders.
scalac doesn't generate class files on a bad build, so that explains why Foo2.class is never generated (in either case) while it doesn't delete class files either after a rebuild (which is why Foo.class hangs around).
In scalac non-resident, we can fix this by deleting all class files generated from the source file on its previous compile before it is recompiled. I'll let David and Martin decide if they want scalac to do that. For scalac resident, we have to flush out all symbols of a class before the source file is recompiled (rather than just see if we can reuse symbols as is the case now). Martin should decide if he wants to do that or not, if Martin does that for FSC we can piggy back on the change for the plugin.
FYI, in Java, the ejc does the right thing when rebuilding source files. I don't believe javac does (i.e., scalac is mimicking javac's behavior).

On Tue, Feb 10, 2009 at 1:34 PM, Tupshin Harper <tupshin [at] tupshin [dot] com <mailto:tupshin [at] tupshin [dot] com>> wrote:

   I could, but I really really don't need to as you can reproduce it
   yourself very quickly.

   Test scenario:
   1) I just updated to scala plugin 2.8.0.r17068-b20090210023355,
   but identical problem reproduced with a version from a couple of
   weeks ago
   2) Eclipse if version 3.4.1
   3) Created brand new workspace and did a fresh launch of eclipse.
   4) System is 64 bit Windows Vista. If you are unable to reproduce,
   I will try it on a Debian Linux system, but that will take some
   time to set up.void
   5) Unlike my more complex scenarios, there are no errors in the
   Error log, only a couple of non-threatening warnings.

   Steps to reproduce:
   1) Create scala project "TestProject"
   2) Say yes when given the chance to switch to scala perspective.
   3) create new package in src called "test"
   4) create new class in test called "Foo"
   5) create new class in test called "Bar"
   6) Edit Foo.scala and change "class Foo {" to be "class Foo2 {"
   and save.
   Observation 1:
   a) The "Problems" view briefly changes its text to Italic,
   indicating that it is compiling something
   b) No problems show up.
   c) workspace/TestProject/bin/test contain files Bar.class,
   Foo.class, and Foo2.class
   Conclusion:
   Renaming a class in a scala file fails to delete the orignal
   .class file even though it creates a new one. (But see below)

   Further testing:
   1) If I "clean project", I correctly get "not found: type Foo"
   2) If I then rename Foo2 back to Foo, the error correctly goes away.
   3) Somewhat surprisingly, bin directory only has Bar.class and
   Foo.class this time. It does not still have Foo2.class
   Conclusion:
   Contrary to above, class files are sometimes deleted when the
   class is renamed but possibly not when anything depends on them.
   Confirming this last hypothesis hard to do.

   -Tupshin


   Sean McDirmid wrote:

       Could you send me a copy of your project? This sounds like it
       is easy to reproduce.
       On Tue, Feb 10, 2009 at 11:29 AM, Tupshin Harper
       <tupshin [at] tupshin [dot] com <mailto:tupshin [at] tupshin [dot] com>
       <mailto:tupshin [at] tupshin [dot] com <mailto:tupshin [at] tupshin [dot] com>>> wrote:

          After converting every last line to scala and getting rid
       of the
          last trace of Java in that same project, if
          a) there is any single compilation error in any file (even a
          standalone one that has no interdependencies on any other
       file in
          the project) and
          b) I deliberately introduce new compilation errors by
       renaming a
          class (not doing file rename, just changing the name of the
       class
          inside the file)
          then a cascading series of 44 errors show up in my project "not
          found: type X", "nof found: value y", etc etc etc etc etc.
          And when I rename the class back to what it should be and
       save it,
          all of those errors stay until I do a "clean project", at which
          point I am back to the one original (and correct) error.

          Then when I tried to verify this test case and delete the file
          with an error in it, I get:
          Can't delete file...TestListerner.scala [in com.foo.bar[in
       src [in
          Bar]]] does not exist
          Which I couldn't resolve except by quitting eclipse and
       restarting
          it (closing and reopening the project didn't work).

          I then tried to reproduce the same problem with no existing
       errors
          in the project, and this time, had an even worse (IMO) result,
          which was a failure to detect *any* error after renaming
       the class
          until I cleaned the project at which point it succesfully
       detected
          the missing class.

          I then went to see whether this problem could have to do
       with my
          project still having a Java nature in addition to the scala
       one,
          so I created a new scala project and copied my source tree from
          hybrid project to scala project.
          Problems:
          1) There are immediately two representation of every scala
       file,
          one with a "J" icon and one with a "S" icon.
          2) Most of the package icons have an error symbol in them
       despite
          there being no classes with errors in those packages. (this
          resolves itself after closing and reopening the project
          3) I then deliberately reintroduced one isolated
       compilation error
          in this pristine scala-only project, renamed one class to
          introduce a second error, and had it fail to detect the
          compilation problems that should occur when that class is
       missing
          until I cleaned the project.

          In the meantime, the error log shows a whole progression of fun
          things like:
          While loading class "lampion.presentation.Presentations$",
       thread
          "Thread[worker-Foo,6,main]" timed out waiting (5000ms) for
       thread
          "Thread[main,6,main]" to finish starting bundle
                "reference:file:plugins\ch.epfl.lamp.sdt.core_2.8.0.r17010-b20090131023503.jar
          [920]". To avoid deadlock, thread
       "Thread[worker-LBDone,6,main]"
          is proceeding but "lampion.presentation.Presentations$" may
       not be
          fully initialized.
          org.eclipse.core.internal.resources.ResourceException: Resource
          '/Bar' is not open. (After I deliberately closed the project)
          File not found:
          G:\Users\tupshin\AppData\Local\Temp\7zE7E36.tmp\Foo.jar
          "Paste" did not complete normally.  Please see the log for more
          information. An exception stack trace is not available. (when
          trying to copy a very short string into a scala file which
       worked
          fine when I typed it manuall)
          etc.
          etc.

          An exercise in frustration indeed, and none of this associated
          with mixed Scala/Java project.

          -Tupshin


          Sean McDirmid wrote:

              But we are not talking about users, we are talking
       about the
              developers of the project deciding not to report bugs!
       This is
              really serious and basically unworkable for a project.
              Mixed scala/java is a big question: it has nothing to
       do with
              the Scala-on-scala dependency tracker. Before, with
       separate
              Java/Scala projects we just reused the classfile
       invalidation
              events provided by the JDT, very very simple but we didn't
              support mixed Scala/Java projects then. I'm not sure
       what the
              deal is now, I did nothing with mixed Scala/Java
       projects, its
              definitely quite possible that it doesn't work. The
       plugin is
              always a moving target :(

              Why did we ever decide that it was a good idea to mix Scala
              and Java code in the same project? Is having two separate
              projects too much inconvenience?
              On Tue, Feb 10, 2009 at 10:40 AM, Josh Suereth
              <joshua [dot] suereth [at] gmail [dot] com
       <mailto:joshua [dot] suereth [at] gmail [dot] com>
       <mailto:joshua [dot] suereth [at] gmail [dot] com
       <mailto:joshua [dot] suereth [at] gmail [dot] com>>
              <mailto:joshua [dot] suereth [at] gmail [dot] com
       <mailto:joshua [dot] suereth [at] gmail [dot] com>
              <mailto:joshua [dot] suereth [at] gmail [dot] com
       <mailto:joshua [dot] suereth [at] gmail [dot] com>>>> wrote:

                 Sean, for future reference:
                 (09:29:37 PM) tupshin: I took a 20 class java
       project and
                 converted it to scala one class at a time using
       eclipse. Was an
                 exercise in frustration. I probably hit "clean
       project" a few
                 hundred times while doing the conversion.
                 (09:29:58 PM) tupshin: Just to see what the *real*
       remaining
                 compilation errors were.

                 I've seen the same myself, and If you want I'll make
       a bug
              report,
                 however it won't be very helpful.   By the time I
       diagnose what
                 the problem was, there's no context for me to figure out
              what's in
                 my .log *and* it's not very reproducable (that
       doesn't mean it
                 doesn't exist, just that we can't figure out how to
              reproduce it).

                 The easiest way for scalac to get freaked out is to
       use it with
                 java.  If your dependency analysis doesn't handle this
              case, then
                 I'd rather go with David's solution, as I live in
       the Java
              world.

                 Also note that a lot of people never report bugs
       against OS
                 projects (90% of my coworkers for instance).

                 -Josh

                 On Mon, Feb 9, 2009 at 9:37 PM, Miles Sabin
              <miles [at] milessabin [dot] com <mailto:miles [at] milessabin [dot] com>
       <mailto:miles [at] milessabin [dot] com <mailto:miles [at] milessabin [dot] com>>
                 <mailto:miles [at] milessabin [dot] com
       <mailto:miles [at] milessabin [dot] com>

              <mailto:miles [at] milessabin [dot] com
       <mailto:miles [at] milessabin [dot] com>>>> wrote:

                     Sean, you're useful to to project if you're still
              willing and
                     able to
                     make concrete contributions. Code would be nice
       (there are
                     plenty of
                     open bugs in areas where you're the best
       authority) and
                     documentation
                     would be wonderful (the scalac-IDE interface and
              incremental AST
                     maintenance, hint, hint).

                     But if you're not going to do that and instead
       just respond
                     negatively
                     to efforts like the one David is making then it
       might be
                     helpful for
                     you to take a little break, at least until
       you're able
              to adopt a
                     slightly more detached attitude to the project.

                     Cheers,


                     Miles

                     --
                     Miles Sabin
                     tel:    +44 (0)1273 720 779
                     mobile: +44 (0)7813 944 528
                     skype:  milessabin









Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland