Visual studio plugin - propose VS2010 support only

On 05/19/10 09:04 AM, Jeremy Bell wrote:
> It is perhaps a bit early to talk about this, since visual studio support
> for scala isn't planned until 2011 (if my understanding is correct), but I
> thought I would propose that we not bother attempting to support Visual
> Studio 2008 or earlier. The language service API went through major
> revisions in visual studio 2010 (although there is some backwards
> compatibility), and implementing support for a new language is much
> easier/cleaner in 2010 than it was previously (using MEF instead of COM
> interop, for the most part). Also, deployment of custom MSBuild actions in
> visual studio 2010 is easier since you can now define them inline in the
> project file, making scala projects potentially more portable.
>

I think this is a good idea. The amount of work needed to support
VS2008 might push the time-line past 2011.

+1

Eric

  Jeremy,
The only news so far about the VS plugin is a catalog of resources [1], aiming at VS 2010 implementation.   Even with that modern API, developing that plugin will take months.   It would be interesting therefore to explore a shortcut (that might just work) sketched in Sec. 2 of [2]. If you can bring that idea to fruition, wow, you really deserve a Scala mug ;-)   Other than that, progress on Scala.NET is reported at this website (warning: for compiler hackers)
http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/  
Miguel     [1] http://www.sts.tu-harburg.de/people/mi.garcia/ScalaCompilerCorner/ResourcesVSExtensibility.pdf   [2] http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/CILMdAsScala.pdf    

"Jeremy Bell" <bell [dot] jeremy [at] gmail [dot] com> wrote in message AANLkTik_O4qscLLsLWasU_81N5KZM57IZ-dsqZjndM5h [at] mail [dot] gmail [dot] com" rel="nofollow">news:AANLkTik_O4qscLLsLWasU_81N5KZM57IZ-dsqZjndM5h [at] mail [dot] gmail [dot] com...It is perhaps a bit early to talk about this, since visual studio support for scala isn't planned until 2011 (if my understanding is correct), but I thought I would propose that we not bother attempting to support Visual Studio 2008 or earlier. The language service API went through major revisions in visual studio 2010 (although there is some backwards compatibility), and implementing support for a new language is much easier/cleaner in 2010 than it was previously (using MEF instead of COM interop, for the most part). Also, deployment of custom MSBuild actions in visual studio 2010 is easier since you can now define them inline in the project file, making scala projects potentially more portable.

It would be nice, but unfortunately I have no experience with eclipse programming (plugins, languages, and the like). My background is mostly on the .Net side of things and the Visual Studio SDK (2008 mostly, I'm still learning the new 2010 stuff). I've worked on simple domain specific integration packages, mostly, but I've never done a language service.
The furthest I got was to implement a simple MSBuild action that called the old scala .net compiler on specific scala sources in the project, but I couldn't figure out how to tell visual studio to merge the generated il files into the output assembly (plus the old compiler would flake out half the time on certain scala programs - I'm guessing this is being worked on). It seems like it should be straightforward though - I just have to keep working on it. 
As for the other functionality - we might be able to do it in stages (note: this is just my naive from-a-distance take on things). 

1. Once we get the custom MSBuild action working, that's a usable deliverable already. I can always just use the eclipse editor as if it were a java-based scala project, and add a build step to build the VS project using MSBuild on the command line until the other pieces are in place. It wouldn't be perfect, plus unfortunately this means you'd have to hand-edit the project files to add scala sources, but I could always just add an integration package that automates this to some degree.
2. Next would be syntax highlighting, which is relatively easy in VS 2010 for simple context free highlighting (keywords, comments, strings, etc...).  
3. After that - using the presentation compiler to "tag" errors and warnings in the editor (the squiggly lines) and add them to errors/warnings tab. This difficulty in this step is just matching up the parsed output of the presentation compiler to the text spans in the editor, and determining when to schedule a presentation compiler "refresh". Additionally, we can integrate the presentation compiler output with the syntax highlighter segment to provide more contextual syntax highlighting (methods vs ids vs literals, etc..).
4. Integrate more of the presentation compiler output (along with reflection) to the editor, to add intellisense support (popup menu when type the period after a local variable, and overlays when you mouse over types, ids, and so forth). 
5. Profit! (just kidding)

On Thu, May 20, 2010 at 2:35 PM, Miguel Garcia <miguel [dot] garcia [at] tuhh [dot] de> wrote:

  Jeremy,
The only news so far about the VS plugin is a catalog of resources [1], aiming at VS 2010 implementation.   Even with that modern API, developing that plugin will take months.   It would be interesting therefore to explore a shortcut (that might just work) sketched in Sec. 2 of [2]. If you can bring that idea to fruition, wow, you really deserve a Scala mug ;-)   Other than that, progress on Scala.NET is reported at this website (warning: for compiler hackers)
http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/  
Miguel     [1] http://www.sts.tu-harburg.de/people/mi.garcia/ScalaCompilerCorner/ResourcesVSExtensibility.pdf   [2] http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/CILMdAsScala.pdf    

"Jeremy Bell" <bell [dot] jeremy [at] gmail [dot] com> wrote in message AANLkTik_O4qscLLsLWasU_81N5KZM57IZ-dsqZjndM5h [at] mail [dot] gmail [dot] com" target="_blank" rel="nofollow">news:AANLkTik_O4qscLLsLWasU_81N5KZM57IZ-dsqZjndM5h [at] mail [dot] gmail [dot] com...It is perhaps a bit early to talk about this, since visual studio support for scala isn't planned until 2011 (if my understanding is correct), but I thought I would propose that we not bother attempting to support Visual Studio 2008 or earlier. The language service API went through major revisions in visual studio 2010 (although there is some backwards compatibility), and implementing support for a new language is much easier/cleaner in 2010 than it was previously (using MEF instead of COM interop, for the most part). Also, deployment of custom MSBuild actions in visual studio 2010 is easier since you can now define them inline in the project file, making scala projects potentially more portable.

Jeremy,   The step by step approach you mention is very reasonable, and my goal is to make life easier for all Scala.NET adopters. The suggestions I made in [1] ("other projects for the Scala.NET ecosystem") did not mention VS because that's a more ambitious goal, even more so if one plans to develop the VS plugin for Scala in Scala itself. About that I have a comment. Generics. That's an area where the compiler is being updated but it's not done yet. Not that the presentation compiler needs Generics (nor the compiler) but programming against VS 2010 APIs will require support for .NET generics from the compiler. Patience :-)   OK, I guess I have to ask a round of feedback from other Scala.NET fans, to come up with a picture of what pieces of the puzzle can be brought in place (in the meantime without Generics). And, small contributions also count, for example among the laundry-list [1] there are to-do's like "generate modern remoting code (i.e., WCF-based)". For that case, a useful contribution consists in distilling the recipe of the remoting pattern to generate. With that, other contributors (say, less versed in .NET) may contribute its implementation in the GenMSIL component.   Coming back to VS, I haven't spent too much time thinking how much of 'glue-code' vs. 'complex-AST-processing-code' a VS plugin is made of (for a language like Scala). If most of the latter services are offered by the Scala compiler (for which we have a .dll already), then I see no technical reason against developing the VS plugin in C#, because maintaining 'glue code' is no big deal irrespective of language [2].   Also here some feedback would be welcome (from those involved in IDE integrations) because there has been a conscious effort to make those "language services" IDE-agnostic (those services have been used in JVM-based IDEs so far but they are IDE-agnostic all the same). I'll search for other references, off the top of my head I can mention the work by Mirko Stocker on refactoring [3] [4] but I'm sure there are more.   Not sure if you can make a profit out of the VS plugin for Scala, but for sure you can be the first to enter that market :-)       Miguel  
[1] http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/CILMdAsScala.pdf   [2] please don't make me write complex algorithms in C#   [3] http://scala.ifs.hsr.ch/doc/scalarefactoring-term.pdf    [4] Appendix A in http://scala.ifs.hsr.ch/hudson/job/Scala-Refactoring/lastSuccessfulBuild/artifact/scala-refactoring/target/thesis.pdf    

out of curiosity, how does scala.net handle .net struct types? My main use case for scala on .net (xna/silverlight) isn't very feasible without struct support, at least the ability to create boxed structs and convert unboxed structs from a library to boxed structs in scala (I'm aware of a high amount of resistance to value types in the scala community). 

On Fri, May 21, 2010 at 1:13 PM, Miguel Garcia <miguel [dot] garcia [at] tuhh [dot] de> wrote:

Jeremy,   The step by step approach you mention is very reasonable, and my goal is to make life easier for all Scala.NET adopters. The suggestions I made in [1] ("other projects for the Scala.NET ecosystem") did not mention VS because that's a more ambitious goal, even more so if one plans to develop the VS plugin for Scala in Scala itself. About that I have a comment. Generics. That's an area where the compiler is being updated but it's not done yet. Not that the presentation compiler needs Generics (nor the compiler) but programming against VS 2010 APIs will require support for .NET generics from the compiler. Patience :-)   OK, I guess I have to ask a round of feedback from other Scala.NET fans, to come up with a picture of what pieces of the puzzle can be brought in place (in the meantime without Generics). And, small contributions also count, for example among the laundry-list [1] there are to-do's like "generate modern remoting code (i.e., WCF-based)". For that case, a useful contribution consists in distilling the recipe of the remoting pattern to generate. With that, other contributors (say, less versed in .NET) may contribute its implementation in the GenMSIL component.   Coming back to VS, I haven't spent too much time thinking how much of 'glue-code' vs. 'complex-AST-processing-code' a VS plugin is made of (for a language like Scala). If most of the latter services are offered by the Scala compiler (for which we have a .dll already), then I see no technical reason against developing the VS plugin in C#, because maintaining 'glue code' is no big deal irrespective of language [2].   Also here some feedback would be welcome (from those involved in IDE integrations) because there has been a conscious effort to make those "language services" IDE-agnostic (those services have been used in JVM-based IDEs so far but they are IDE-agnostic all the same). I'll search for other references, off the top of my head I can mention the work by Mirko Stocker on refactoring [3] [4] but I'm sure there are more.   Not sure if you can make a profit out of the VS plugin for Scala, but for sure you can be the first to enter that market :-)       Miguel  
[1] http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/CILMdAsScala.pdf   [2] please don't make me write complex algorithms in C#   [3] http://scala.ifs.hsr.ch/doc/scalarefactoring-term.pdf    [4] Appendix A in http://scala.ifs.hsr.ch/hudson/job/Scala-Refactoring/lastSuccessfulBuild/artifact/scala-refactoring/target/thesis.pdf    

Jeremy,   Knowing about interoperability requirements that Scala.NET developers expect is a good thing, so as to take them into account early in the game. If the comments below do not address all issues please feel free to open an enhancement bug ticket.   Regarding C# struct types, what the compiler sees is the metadata in compiled assemblies, where that kind of types inherit from System.ValueType. The contract of System.ValueType is in agreement with that of Scala case classes (i.e., the contract that System.ValueType specializes from Object.ReferenceEquals and Object.Equals).   That's about consuming struct types defined externally. At runtime, we won't just want to receive values of those types but in general pass them to other consumers (consumers not written in Scala.NET). I can't say whether all Scala case classes  will become .NET value types, but it appears a plausible alternative.   Similar interoperability considerations (i.e., contracts beyond those enforced by types) will be spelled out for:   System.Enumeration vs. scala.Enumeration
nullable types vs. scala.Option
LINQ
etc.   And hey, interoperability is already working in the other direction (.NET languages can consume assemblies produced by Scala.NET). This last scenario was already touched upon when discussing the VS plugin: a VS plugin for Scala written in, say, C# can use the presentation compiler and other language services as the "Model" in "Model-View-Controller" sense.   Speaking of which, I found no documentation on those language services, but their IDE-agnostic nature is being leveraged as of late by the Emacs-integration http://github.com/aemoncannon/ensime .       Miguel  
 

"Jeremy Bell" <bell [dot] jeremy [at] gmail [dot] com> wrote in message AANLkTimkNSLSASQLGRmUQ-iHEgHgKtpgPuU74ld7hYtp [at] mail [dot] gmail [dot] com" rel="nofollow">news:AANLkTimkNSLSASQLGRmUQ-iHEgHgKtpgPuU74ld7hYtp [at] mail [dot] gmail [dot] com...out of curiosity, how does scala.net handle .net struct types? My main use case for scala on .net (xna/silverlight) isn't very feasible without struct support, at least the ability to create boxed structs and convert unboxed structs from a library to boxed structs in scala (I'm aware of a high amount of resistance to value types in the scala community). 

On Fri, May 21, 2010 at 1:13 PM, Miguel Garcia <miguel [dot] garcia [at] tuhh [dot] de> wrote:

Jeremy,   The step by step approach you mention is very reasonable, and my goal is to make life easier for all Scala.NET adopters. The suggestions I made in [1] ("other projects for the Scala.NET ecosystem") did not mention VS because that's a more ambitious goal, even more so if one plans to develop the VS plugin for Scala in Scala itself. About that I have a comment. Generics. That's an area where the compiler is being updated but it's not done yet. Not that the presentation compiler needs Generics (nor the compiler) but programming against VS 2010 APIs will require support for .NET generics from the compiler. Patience :-)   OK, I guess I have to ask a round of feedback from other Scala.NET fans, to come up with a picture of what pieces of the puzzle can be brought in place (in the meantime without Generics). And, small contributions also count, for example among the laundry-list [1] there are to-do's like "generate modern remoting code (i.e., WCF-based)". For that case, a useful contribution consists in distilling the recipe of the remoting pattern to generate. With that, other contributors (say, less versed in .NET) may contribute its implementation in the GenMSIL component.   Coming back to VS, I haven't spent too much time thinking how much of 'glue-code' vs. 'complex-AST-processing-code' a VS plugin is made of (for a language like Scala). If most of the latter services are offered by the Scala compiler (for which we have a .dll already), then I see no technical reason against developing the VS plugin in C#, because maintaining 'glue code' is no big deal irrespective of language [2].   Also here some feedback would be welcome (from those involved in IDE integrations) because there has been a conscious effort to make those "language services" IDE-agnostic (those services have been used in JVM-based IDEs so far but they are IDE-agnostic all the same). I'll search for other references, off the top of my head I can mention the work by Mirko Stocker on refactoring [3] [4] but I'm sure there are more.   Not sure if you can make a profit out of the VS plugin for Scala, but for sure you can be the first to enter that market :-)       Miguel  
[1] http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/CILMdAsScala.pdf   [2] please don't make me write complex algorithms in C#   [3] http://scala.ifs.hsr.ch/doc/scalarefactoring-term.pdf    [4] Appendix A in http://scala.ifs.hsr.ch/hudson/job/Scala-Refactoring/lastSuccessfulBuild/artifact/scala-refactoring/target/thesis.pdf    

Does that mean that case classes would become non-allocating on the .net VM? If so, that would be sufficient for my needs.
For reference, here are my other requirements for being able to use scala on .net: I would use scala for .net in my silverlight/XNA projects, which would run on the PC, XBox360, and soon on windows phone 7. In general, if you can take arbitrary silverlight-based or xna-based code, be able to convert it to scala, and get a similar performance profile, then it meets my requirements. More specifically: 

• Support for generics of course. 
• Support for non-allocating value types - both consuming them from external libraries and producing them in scala code without causing an allocation. I spoke at length about this on scala-debate, but in summary:
• It is critically important that value types not be simulated with immutable reference types that are still garbage collected. 
• The non-allocating, non-garbage-collected nature of structs in .net is part of the contract.
• Software written in .net takes advantage of this and wouldn't work well otherwise. 
• For example, physics engines on the .net platform rely on it to ensure that zero allocations occur during each timestep, because allocations cause unpredictable step times (due to garbage collector pauses) which are visible to the user, even on the latest .net vm. In addition, I write software for XBox360 and soon for windows phone 7, where the .Net compact framework doesn't currently have a generational garbage collector (by design), so garbage collection pauses are even more noticeable. 
• There are far too many object instantiations in typical software of this type for the GC to handle efficiently, no matter how advanced it is. Object pooling/recycling can be used at a high level where there are few or no allocations (to hold geometries that don't often change from time-step to time-step, etc..), but this is prohibitively expensive at a low level (vector math, contact joints, corrective forces, etc..) where there may be over a thousand potentially escaping objects being allocated during each time-step (of which there are 30 - 60 per second).
• Preferably, value types in scala would be locally mutable. By that I mean: local struct variables or struct variables in primitive arrays would be mutable, but structs in generic containers (other than a built in array) are immutable. However, I would be willing to accept complete immutability as a requirement to any value type implementation for scala - I understand this might make it easier to "fit in" to the existing scala type system. 
• This would, however, introduce some overhead in certain use cases. For example, certain vector/matrix calculations can be more efficiently done "in-place".
• Immutable value types also lead to more verbose code. 
• Neither of these issues are show stoppers per say, but they would make me less inclined to write this kind of code in scala. It would be messier than what I would write in C#.
• Enumeration support. Clarification: Lightweight enumeration support. In other words, I would never use Java enumerations in performance critical code, which is a pity because enumerations make code more readable. .Net enumerations are more lightweight, integer based, non-allocating value types which are used extensively in XNA, silverlight, and various .net libraries.
• Also important to make sure [Flags] based enumeration bitfields are supported - they're used in a couple of places in XNA and silverlight, if I remember correctly.
• ref and out parameters. At least, the ability to call an external function which takes ref/out parameters. Personally, I don't use them in my own code (I consider them bad design), but unfortunately they're everywhere in the XNA library. Note this also means the ability to pass a "struct" type as a ref/out parameter without boxing it.
• Support for events and delegates. You can simulate this in scala, except that in C# you can do "eventName -= someObject.MethodName" to unsubscribe to an event. This is not possible directly in scala - you have to save off the generated closure to use to unsubscribe later. Here is a fleshed out Event class which I wrote based off of ideas from Max Bolingbroke's blog (note the += returns the passed in closure, so if you call += and give it "object.MethodName", you are actually creating an anonymous closure that calls object.MethodName, so you need to save off a reference to the closure that is created so you can pass it to -=):

// Warning: experimental, not tested

object EventArgs {

      val Empty: EventArgs =

            new EventArgs("EventArgs.Empty");

}

 

class EventArgs(val Message: String) {

      def this() = {

            this("");

      }

}

 

// http://blog.omega-prime.co.uk/?p=21

trait IEvent[T <: EventArgs] {

      protected var invocationList :

            List[(AnyRef, T) => Unit] = Nil

     

      def +=(invoker : (AnyRef, T) => Unit) (AnyRef, T) => Unit {

      invocationList = invoker :: invocationList

      invoker

      }

 

      def -=(invoker : (AnyRef, T) => Unit) {

            invocationList = invocationList filter

                        ((x : (AnyRef, T) => Unit) => (x != invoker))

      }

}

 

// http://blog.omega-prime.co.uk/?p=21

class Event[T <: EventArgs]

      extends Function1[AnyRef, T] with IEvent[T] {

 

  override def apply(sender: AnyRef, args : T) {

    for (val invoker <- invocationList) {

      invoker(sender, args)

    }

  }

}

 

trait IBasicEvent extends IEvent[EventArgs] {}

class BasicEvent extends Event[EventArgs] with IBasicEvent {}

 

example:

trait IDrawable

{

      protected val drawOrderChanged : BasicEvent = new BasicEvent();

      protected val visibleChanged : BasicEvent = new BasicEvent();

      def DrawOrderChanged : IBasicEvent = drawOrderChanged;

      def VisibleChanged : IBasicEvent = visibleChanged;

     

      protected var drawOrder : Int;

      def DrawOrder : Int = drawOrder;

      def DrawOrder_=(value : Int) = {

            if(drawOrder != value) {

                  drawOrder = value;

                  drawOrderChanged(this, EventArgs.Empty);

            }

      }

 

      protected var visible : Boolean;

      def Visible : Boolean = visible;

      def Visible_=(value: Boolean) = {

            if(visible != value) {

                  visible = value;

                  visibleChanged(this, EventArgs.Empty);

            }

      }

     

      def Draw(gameTime: GameTime);

}

On Sun, May 23, 2010 at 8:35 AM, Miguel Garcia <miguel [dot] garcia [at] tuhh [dot] de> wrote:

Jeremy,   Knowing about interoperability requirements that Scala.NET developers expect is a good thing, so as to take them into account early in the game. If the comments below do not address all issues please feel free to open an enhancement bug ticket.   Regarding C# struct types, what the compiler sees is the metadata in compiled assemblies, where that kind of types inherit from System.ValueType. The contract of System.ValueType is in agreement with that of Scala case classes (i.e., the contract that System.ValueType specializes from Object.ReferenceEquals and Object.Equals).   That's about consuming struct types defined externally. At runtime, we won't just want to receive values of those types but in general pass them to other consumers (consumers not written in Scala.NET). I can't say whether all Scala case classes  will become .NET value types, but it appears a plausible alternative.   Similar interoperability considerations (i.e., contracts beyond those enforced by types) will be spelled out for:   System.Enumeration vs. scala.Enumeration
nullable types vs. scala.Option
LINQ
etc.   And hey, interoperability is already working in the other direction (.NET languages can consume assemblies produced by Scala.NET). This last scenario was already touched upon when discussing the VS plugin: a VS plugin for Scala written in, say, C# can use the presentation compiler and other language services as the "Model" in "Model-View-Controller" sense.   Speaking of which, I found no documentation on those language services, but their IDE-agnostic nature is being leveraged as of late by the Emacs-integration http://github.com/aemoncannon/ensime .       Miguel  
 

"Jeremy Bell" <bell [dot] jeremy [at] gmail [dot] com> wrote in message AANLkTimkNSLSASQLGRmUQ-iHEgHgKtpgPuU74ld7hYtp [at] mail [dot] gmail [dot] com" target="_blank" rel="nofollow">news:AANLkTimkNSLSASQLGRmUQ-iHEgHgKtpgPuU74ld7hYtp [at] mail [dot] gmail [dot] com...out of curiosity, how does scala.net handle .net struct types? My main use case for scala on .net (xna/silverlight) isn't very feasible without struct support, at least the ability to create boxed structs and convert unboxed structs from a library to boxed structs in scala (I'm aware of a high amount of resistance to value types in the scala community). 

On Fri, May 21, 2010 at 1:13 PM, Miguel Garcia <miguel [dot] garcia [at] tuhh [dot] de> wrote:

Jeremy,   The step by step approach you mention is very reasonable, and my goal is to make life easier for all Scala.NET adopters. The suggestions I made in [1] ("other projects for the Scala.NET ecosystem") did not mention VS because that's a more ambitious goal, even more so if one plans to develop the VS plugin for Scala in Scala itself. About that I have a comment. Generics. That's an area where the compiler is being updated but it's not done yet. Not that the presentation compiler needs Generics (nor the compiler) but programming against VS 2010 APIs will require support for .NET generics from the compiler. Patience :-)   OK, I guess I have to ask a round of feedback from other Scala.NET fans, to come up with a picture of what pieces of the puzzle can be brought in place (in the meantime without Generics). And, small contributions also count, for example among the laundry-list [1] there are to-do's like "generate modern remoting code (i.e., WCF-based)". For that case, a useful contribution consists in distilling the recipe of the remoting pattern to generate. With that, other contributors (say, less versed in .NET) may contribute its implementation in the GenMSIL component.   Coming back to VS, I haven't spent too much time thinking how much of 'glue-code' vs. 'complex-AST-processing-code' a VS plugin is made of (for a language like Scala). If most of the latter services are offered by the Scala compiler (for which we have a .dll already), then I see no technical reason against developing the VS plugin in C#, because maintaining 'glue code' is no big deal irrespective of language [2].   Also here some feedback would be welcome (from those involved in IDE integrations) because there has been a conscious effort to make those "language services" IDE-agnostic (those services have been used in JVM-based IDEs so far but they are IDE-agnostic all the same). I'll search for other references, off the top of my head I can mention the work by Mirko Stocker on refactoring [3] [4] but I'm sure there are more.   Not sure if you can make a profit out of the VS plugin for Scala, but for sure you can be the first to enter that market :-)       Miguel  
[1] http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/CILMdAsScala.pdf   [2] please don't make me write complex algorithms in C#   [3] http://scala.ifs.hsr.ch/doc/scalarefactoring-term.pdf    [4] Appendix A in http://scala.ifs.hsr.ch/hudson/job/Scala-Refactoring/lastSuccessfulBuild/artifact/scala-refactoring/target/thesis.pdf    

Jeremy,   I see performance (e.g., avoiding wrappers) as common theme in the requirements for your planned use of Scala.NET. Although Scala.NET is no DSL for the CoreCLR VM (used in Silverlight) nor for XNA, the features you mention actually fall under plain old CLR, which is being targeted. No surprises there, although I have a comment about syntax, for example regarding event subscription. There's syntax for that (in widespread use)  in C#, syntax that may seem natural but which in fact resulted from adding just one more feature to C# (and so on so forth). Instead, Scala manages to capture programming intent with existing constructs. Whatever the syntax, what really makes an "event" in the .NET platform is its rendition in assembly metadata, that's why we're adding interoperability at that level. In that sense, your message contributes a more detailed discussion (and serves as a reminder) of issues we had already identified.  
Miguel  
 

"Jeremy Bell" <bell [dot] jeremy [at] gmail [dot] com> wrote in message AANLkTimgD9-LSfJiPO9raH0_dNiuAtlMThrGDp9AfFlO [at] mail [dot] gmail [dot] com" rel="nofollow">news:AANLkTimgD9-LSfJiPO9raH0_dNiuAtlMThrGDp9AfFlO [at] mail [dot] gmail [dot] com... Does that mean that case classes would become non-allocating on the .net VM? If so, that would be sufficient for my needs.
For reference, here are my other requirements for being able to use scala on .net: I would use scala for .net in my silverlight/XNA projects, which would run on the PC, XBox360, and soon on windows phone 7. In general, if you can take arbitrary silverlight-based or xna-based code, be able to convert it to scala, and get a similar performance profile, then it meets my requirements. More specifically: 

• Support for generics of course. 
• Support for non-allocating value types - both consuming them from external libraries and producing them in scala code without causing an allocation. I spoke at length about this on scala-debate, but in summary:
  • It is critically important that value types not be simulated with immutable reference types that are still garbage collected. 
  • The non-allocating, non-garbage-collected nature of structs in .net is part of the contract.
  • Software written in .net takes advantage of this and wouldn't work well otherwise. 
    • For example, physics engines on the .net platform rely on it to ensure that zero allocations occur during each timestep, because allocations cause unpredictable step times (due to garbage collector pauses) which are visible to the user, even on the latest .net vm. In addition, I write software for XBox360 and soon for windows phone 7, where the .Net compact framework doesn't currently have a generational garbage collector (by design), so garbage collection pauses are even more noticeable. 
    • There are far too many object instantiations in typical software of this type for the GC to handle efficiently, no matter how advanced it is. Object pooling/recycling can be used at a high level where there are few or no allocations (to hold geometries that don't often change from time-step to time-step, etc..), but this is prohibitively expensive at a low level (vector math, contact joints, corrective forces, etc..) where there may be over a thousand potentially escaping objects being allocated during each time-step (of which there are 30 - 60 per second).
  • Preferably, value types in scala would be locally mutable. By that I mean: local struct variables or struct variables in primitive arrays would be mutable, but structs in generic containers (other than a built in array) are immutable. However, I would be willing to accept complete immutability as a requirement to any value type implementation for scala - I understand this might make it easier to "fit in" to the existing scala type system. 
    • This would, however, introduce some overhead in certain use cases. For example, certain vector/matrix calculations can be more efficiently done "in-place".
    • Immutable value types also lead to more verbose code. 
    • Neither of these issues are show stoppers per say, but they would make me less inclined to write this kind of code in scala. It would be messier than what I would write in C#.
• Enumeration support. Clarification: Lightweight enumeration support. In other words, I would never use Java enumerations in performance critical code, which is a pity because enumerations make code more readable. .Net enumerations are more lightweight, integer based, non-allocating value types which are used extensively in XNA, silverlight, and various .net libraries.
  • Also important to make sure [Flags] based enumeration bitfields are supported - they're used in a couple of places in XNA and silverlight, if I remember correctly.
• ref and out parameters. At least, the ability to call an external function which takes ref/out parameters. Personally, I don't use them in my own code (I consider them bad design), but unfortunately they're everywhere in the XNA library. Note this also means the ability to pass a "struct" type as a ref/out parameter without boxing it.
• Support for events and delegates. You can simulate this in scala, except that in C# you can do "eventName -= someObject.MethodName" to unsubscribe to an event. This is not possible directly in scala - you have to save off the generated closure to use to unsubscribe later. Here is a fleshed out Event class which I wrote based off of ideas from Max Bolingbroke's blog (note the += returns the passed in closure, so if you call += and give it "object.MethodName", you are actually creating an anonymous closure that calls object.MethodName, so you need to save off a reference to the closure that is created so you can pass it to -=):

// Warning: experimental, not tested

object EventArgs {

      val Empty: EventArgs =

            new EventArgs("EventArgs.Empty");

}

 

class EventArgs(val Message: String) {

      def this() = {

            this("");

      }

}

 

// http://blog.omega-prime.co.uk/?p=21

trait IEvent[T <: EventArgs] {

      protected var invocationList :

            List[(AnyRef, T) => Unit] = Nil

     

      def +=(invoker : (AnyRef, T) => Unit) (AnyRef, T) => Unit {

      invocationList = invoker :: invocationList

      invoker

      }

 

      def -=(invoker : (AnyRef, T) => Unit) {

            invocationList = invocationList filter

                        ((x : (AnyRef, T) => Unit) => (x != invoker))

      }

}

 

// http://blog.omega-prime.co.uk/?p=21

class Event[T <: EventArgs]

      extends Function1[AnyRef, T] with IEvent[T] {

 

  override def apply(sender: AnyRef, args : T) {

    for (val invoker <- invocationList) {

      invoker(sender, args)

    }

  }

}

 

trait IBasicEvent extends IEvent[EventArgs] {}

class BasicEvent extends Event[EventArgs] with IBasicEvent {}

 

example:

trait IDrawable

{

      protected val drawOrderChanged : BasicEvent = new BasicEvent();

      protected val visibleChanged : BasicEvent = new BasicEvent();

      def DrawOrderChanged : IBasicEvent = drawOrderChanged;

      def VisibleChanged : IBasicEvent = visibleChanged;

     

      protected var drawOrder : Int;

      def DrawOrder : Int = drawOrder;

      def DrawOrder_=(value : Int) = {

            if(drawOrder != value) {

                  drawOrder = value;

                  drawOrderChanged(this, EventArgs.Empty);

            }

      }

 

      protected var visible : Boolean;

      def Visible : Boolean = visible;

      def Visible_=(value: Boolean) = {

            if(visible != value) {

                  visible = value;

                  visibleChanged(this, EventArgs.Empty);

            }

      }

     

      def Draw(gameTime: GameTime);

}

On Sun, May 23, 2010 at 8:35 AM, Miguel Garcia <miguel [dot] garcia [at] tuhh [dot] de> wrote:

Jeremy,   Knowing about interoperability requirements that Scala.NET developers expect is a good thing, so as to take them into account early in the game. If the comments below do not address all issues please feel free to open an enhancement bug ticket.   Regarding C# struct types, what the compiler sees is the metadata in compiled assemblies, where that kind of types inherit from System.ValueType. The contract of System.ValueType is in agreement with that of Scala case classes (i.e., the contract that System.ValueType specializes from Object.ReferenceEquals and Object.Equals).   That's about consuming struct types defined externally. At runtime, we won't just want to receive values of those types but in general pass them to other consumers (consumers not written in Scala.NET). I can't say whether all Scala case classes  will become .NET value types, but it appears a plausible alternative.   Similar interoperability considerations (i.e., contracts beyond those enforced by types) will be spelled out for:   System.Enumeration vs. scala.Enumeration
nullable types vs. scala.Option
LINQ
etc.   And hey, interoperability is already working in the other direction (.NET languages can consume assemblies produced by Scala.NET). This last scenario was already touched upon when discussing the VS plugin: a VS plugin for Scala written in, say, C# can use the presentation compiler and other language services as the "Model" in "Model-View-Controller" sense.   Speaking of which, I found no documentation on those language services, but their IDE-agnostic nature is being leveraged as of late by the Emacs-integration http://github.com/aemoncannon/ensime .       Miguel  
 

"Jeremy Bell" <bell [dot] jeremy [at] gmail [dot] com> wrote in message AANLkTimkNSLSASQLGRmUQ-iHEgHgKtpgPuU74ld7hYtp [at] mail [dot] gmail [dot] com" target="_blank" rel="nofollow">news:AANLkTimkNSLSASQLGRmUQ-iHEgHgKtpgPuU74ld7hYtp [at] mail [dot] gmail [dot] com... out of curiosity, how does scala.net handle .net struct types? My main use case for scala on .net (xna/silverlight) isn't very feasible without struct support, at least the ability to create boxed structs and convert unboxed structs from a library to boxed structs in scala (I'm aware of a high amount of resistance to value types in the scala community). 

On Fri, May 21, 2010 at 1:13 PM, Miguel Garcia <miguel [dot] garcia [at] tuhh [dot] de> wrote:

Jeremy,   The step by step approach you mention is very reasonable, and my goal is to make life easier for all Scala.NET adopters. The suggestions I made in [1] ("other projects for the Scala.NET ecosystem") did not mention VS because that's a more ambitious goal, even more so if one plans to develop the VS plugin for Scala in Scala itself. About that I have a comment. Generics. That's an area where the compiler is being updated but it's not done yet. Not that the presentation compiler needs Generics (nor the compiler) but programming against VS 2010 APIs will require support for .NET generics from the compiler. Patience :-)   OK, I guess I have to ask a round of feedback from other Scala.NET fans, to come up with a picture of what pieces of the puzzle can be brought in place (in the meantime without Generics). And, small contributions also count, for example among the laundry-list [1] there are to-do's like "generate modern remoting code (i.e., WCF-based)". For that case, a useful contribution consists in distilling the recipe of the remoting pattern to generate. With that, other contributors (say, less versed in .NET) may contribute its implementation in the GenMSIL component.   Coming back to VS, I haven't spent too much time thinking how much of 'glue-code' vs. 'complex-AST-processing-code' a VS plugin is made of (for a language like Scala). If most of the latter services are offered by the Scala compiler (for which we have a .dll already), then I see no technical reason against developing the VS plugin in C#, because maintaining 'glue code' is no big deal irrespective of language [2].   Also here some feedback would be welcome (from those involved in IDE integrations) because there has been a conscious effort to make those "language services" IDE-agnostic (those services have been used in JVM-based IDEs so far but they are IDE-agnostic all the same). I'll search for other references, off the top of my head I can mention the work by Mirko Stocker on refactoring [3] [4] but I'm sure there are more.   Not sure if you can make a profit out of the VS plugin for Scala, but for sure you can be the first to enter that market :-)       Miguel  
[1] http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/CILMdAsScala.pdf   [2] please don't make me write complex algorithms in C#   [3] http://scala.ifs.hsr.ch/doc/scalarefactoring-term.pdf    [4] Appendix A in http://scala.ifs.hsr.ch/hudson/job/Scala-Refactoring/lastSuccessfulBuild/artifact/scala-refactoring/target/thesis.pdf    

  Visual Studio - Lua Language Support
http://vslua.codeplex.com/   Moving code blocks among code regions using VS 2010 Extensions
http://www.codeproject.com/KB/macros/MoveToRegionVSX.aspx   CodeBlog: Writing a Blogging Extension for Visual Studio 2010
http://www.devx.com/VS_2010/Article/44073   Visual Studio SDK, Extending the Editor
http://msdn.microsoft.com/en-us/library/dd885242(v=VS.100).aspx     Miguel    

  Visual Studio SDK Language Services

The purpose of a language service in Visual Studio is to provide language-specific support for editing source code in the integrated development environment (IDE). You implement a language service as part of a VSPackage.

This section discusses the structure and implementation of a Visual Studio language service.

http://msdn.microsoft.com/en-us/library/bb165099(v=VS.100).aspx