This page is no longer maintained — Please continue to the home page at www.scala-lang.org

implementing macros for Scala, looking for feedback!

41 replies
Eugene Burmako
Joined: 2011-09-17,
User offline. Last seen 42 years 45 weeks ago.
Hi folks! My name is Eugene Burmako. I'm a first-year PhD student from Martin's lab at EPFL. During the first year of studies all doctoral students here at EPFL have to carry out a couple of semester projects, and, of course, my projects will involve something Scala-related.

The idea, which I like the most at the moment, is to implement a metaprogramming facility for Scala. By that I mean a full-fledged system with quasiquotations and hygienic compile-time macros. My current inspiration is the macro system of Nemerle (http://nemerle.org/metaprogramming.ps). Sure, it's a huge amount of work, so I don't expect finishing all that stuff soon, but it can be done gradually.

This macro project somewhat intersects with another research direction of our lab, namely, with lightweight modular staging (http://www.stanford.edu/class/cs442/lectures_unrestricted/cs442-lms.pdf). However, despite of being applicable to similar problems, both LMS and macros each have distinctive scenarios where they really shine. That's why we decided that it'd be practical to explore both of these approaches to metaprogramming.

Having said that, I'd like to find compelling use-cases for macros, so that the functionality implemented will be immediately useful. Of course, macros are very elegant in themselves, and, from the standpoint of a language designer, I can name at least a handful of cool features that can be enabled by macros. However, what I'd really like to know is what scenarios are the most important to you in your everyday jobs. I would be very glad to receive your feedback!

P.S. The usual disclaimer applies here. No promises/commitments so far, it's just a research project. However, if it deems useful, I think, it might end up in trunk in some shape or another.
roland.kuhn
Joined: 2011-02-21,
User offline. Last seen 35 weeks 3 days ago.
Re: implementing macros for Scala, looking for feedback!
Hi Eugene,
this will probably be interesting for a lot of people, glad you tackle it! One little thing I find myself writing very often is a sealed trait hierarchy aka “the better Enum”:
sealed trait Statecase object Idle extends Statecase object Active extends State
sealed trait Messagecase object Kill extends Messagecase class DoIt(whatever: String) extends Message
It would be really great if the signal to noise ratio could be reduced: in the above only 6 out of 28 words are actually interesting. If my understanding of macros matches your proposal, this could be reduced to something like
Enum(State, Idle, Active)Enum(Message, Kill, class DoIt(whatever: String))
(if I did not completely miss the scope of macros from a very cursory glance over the paper.)
Regards,
Roland
On Sep 17, 2011, at 13:13 , Eugene Burmako wrote:
Hi folks! My name is Eugene Burmako. I'm a first-year PhD student from Martin's lab at EPFL. During the first year of studies all doctoral students here at EPFL have to carry out a couple of semester projects, and, of course, my projects will involve something Scala-related.

The idea, which I like the most at the moment, is to implement a metaprogramming facility for Scala. By that I mean a full-fledged system with quasiquotations and hygienic compile-time macros. My current inspiration is the macro system of Nemerle (http://nemerle.org/metaprogramming.ps). Sure, it's a huge amount of work, so I don't expect finishing all that stuff soon, but it can be done gradually.

This macro project somewhat intersects with another research direction of our lab, namely, with lightweight modular staging (http://www.stanford.edu/class/cs442/lectures_unrestricted/cs442-lms.pdf). However, despite of being applicable to similar problems, both LMS and macros each have distinctive scenarios where they really shine. That's why we decided that it'd be practical to explore both of these approaches to metaprogramming.

Having said that, I'd like to find compelling use-cases for macros, so that the functionality implemented will be immediately useful. Of course, macros are very elegant in themselves, and, from the standpoint of a language designer, I can name at least a handful of cool features that can be enabled by macros. However, what I'd really like to know is what scenarios are the most important to you in your everyday jobs. I would be very glad to receive your feedback!

P.S. The usual disclaimer applies here. No promises/commitments so far, it's just a research project. However, if it deems useful, I think, it might end up in trunk in some shape or another.


Roland KuhnTypesafe – Enterprise-Grade Scala from the Expertstwitter: @rolandkuhn
debasish
Joined: 2011-06-19,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!
Hi Eugene -
Thanks for giving time towards macros in Scala. Despite having a rich type system, I still think there are valid use cases for compile time macros in Scala. Two such cases come straight to my mind :
1. code generation. No language is boilerplate free and there occurs situations where you feel the necessity to generate code. In one of my recent projects I was writing a Redis client in Scala. Besides the basic client I also implemented a Redis Cluster, which to the end user is also yet another client. The user invokes same operations on the cluster as on a single connection. Most operations are a simple delegate to the appropriate shard while some operations need additional logic. Here I felt the necessity of macros in Scala which would have generated lots of code that I needed to write manually.
2. Inlining code is a good optimization. But often even though the user wants to inline, the compiler would not. Or even the user is not aware of an inline optimization. In such cases if he implements using a macro, it will be more efficient as the inlining is guaranteed.
Will let u know more such use cases as they come up.
Thanks.

On Sat, Sep 17, 2011 at 4:43 PM, Eugene Burmako <eugene [dot] burmako [at] epfl [dot] ch> wrote:
Hi folks! My name is Eugene Burmako. I'm a first-year PhD student from Martin's lab at EPFL. During the first year of studies all doctoral students here at EPFL have to carry out a couple of semester projects, and, of course, my projects will involve something Scala-related.

The idea, which I like the most at the moment, is to implement a metaprogramming facility for Scala. By that I mean a full-fledged system with quasiquotations and hygienic compile-time macros. My current inspiration is the macro system of Nemerle (http://nemerle.org/metaprogramming.ps). Sure, it's a huge amount of work, so I don't expect finishing all that stuff soon, but it can be done gradually.

This macro project somewhat intersects with another research direction of our lab, namely, with lightweight modular staging (http://www.stanford.edu/class/cs442/lectures_unrestricted/cs442-lms.pdf). However, despite of being applicable to similar problems, both LMS and macros each have distinctive scenarios where they really shine. That's why we decided that it'd be practical to explore both of these approaches to metaprogramming.

Having said that, I'd like to find compelling use-cases for macros, so that the functionality implemented will be immediately useful. Of course, macros are very elegant in themselves, and, from the standpoint of a language designer, I can name at least a handful of cool features that can be enabled by macros. However, what I'd really like to know is what scenarios are the most important to you in your everyday jobs. I would be very glad to receive your feedback!

P.S. The usual disclaimer applies here. No promises/commitments so far, it's just a research project. However, if it deems useful, I think, it might end up in trunk in some shape or another.



--
Debasish Ghosh
http://manning.com/ghosh

Twttr: @debasishg
Blog: http://debasishg.blogspot.com
Code: http://github.com/debasishg
missingfaktor
Joined: 2010-04-13,
User offline. Last seen 1 year 3 days ago.
Re: implementing macros for Scala, looking for feedback!
data Message(header: String, contents: String) deriving (Equal, Show)

could expand to:
case class Message(header: String, contents: String)
object Message extends ((String, String) => Message) {   implicit def messageShow: Show[Message] = showA  implicit val messageEqual: Equal[Message] = equalA   def header = (_: Message).header  def contents = (_: Message).contents }
And maybe lenses too.
On Sat, Sep 17, 2011 at 7:08 PM, Debasish Ghosh <ghosh [dot] debasish [at] gmail [dot] com> wrote:
Hi Eugene -
Thanks for giving time towards macros in Scala. Despite having a rich type system, I still think there are valid use cases for compile time macros in Scala. Two such cases come straight to my mind :
1. code generation. No language is boilerplate free and there occurs situations where you feel the necessity to generate code. In one of my recent projects I was writing a Redis client in Scala. Besides the basic client I also implemented a Redis Cluster, which to the end user is also yet another client. The user invokes same operations on the cluster as on a single connection. Most operations are a simple delegate to the appropriate shard while some operations need additional logic. Here I felt the necessity of macros in Scala which would have generated lots of code that I needed to write manually.
2. Inlining code is a good optimization. But often even though the user wants to inline, the compiler would not. Or even the user is not aware of an inline optimization. In such cases if he implements using a macro, it will be more efficient as the inlining is guaranteed.
Will let u know more such use cases as they come up.
Thanks.

On Sat, Sep 17, 2011 at 4:43 PM, Eugene Burmako <eugene [dot] burmako [at] epfl [dot] ch> wrote:
Hi folks! My name is Eugene Burmako. I'm a first-year PhD student from Martin's lab at EPFL. During the first year of studies all doctoral students here at EPFL have to carry out a couple of semester projects, and, of course, my projects will involve something Scala-related.

The idea, which I like the most at the moment, is to implement a metaprogramming facility for Scala. By that I mean a full-fledged system with quasiquotations and hygienic compile-time macros. My current inspiration is the macro system of Nemerle (http://nemerle.org/metaprogramming.ps). Sure, it's a huge amount of work, so I don't expect finishing all that stuff soon, but it can be done gradually.

This macro project somewhat intersects with another research direction of our lab, namely, with lightweight modular staging (http://www.stanford.edu/class/cs442/lectures_unrestricted/cs442-lms.pdf). However, despite of being applicable to similar problems, both LMS and macros each have distinctive scenarios where they really shine. That's why we decided that it'd be practical to explore both of these approaches to metaprogramming.

Having said that, I'd like to find compelling use-cases for macros, so that the functionality implemented will be immediately useful. Of course, macros are very elegant in themselves, and, from the standpoint of a language designer, I can name at least a handful of cool features that can be enabled by macros. However, what I'd really like to know is what scenarios are the most important to you in your everyday jobs. I would be very glad to receive your feedback!

P.S. The usual disclaimer applies here. No promises/commitments so far, it's just a research project. However, if it deems useful, I think, it might end up in trunk in some shape or another.



--
Debasish Ghosh
http://manning.com/ghosh

Twttr: @debasishg
Blog: http://debasishg.blogspot.com
Code: http://github.com/debasishg



--
Cheers,missingfaktor.
When you stand for what you believe in, you can change the world.
dcsobral
Joined: 2009-04-23,
User offline. Last seen 38 weeks 5 days ago.
Re: implementing macros for Scala, looking for feedback!

+1 on both. I'd be mentioning them if no one had done so!

I'll put forward conditional compilation based on Scala version. This
is something that has direct impact on 3rd party libraries. Of course,
it wouldn't help them with code *pre*-macros, so it wouldn't be of
immediate use.

On Sat, Sep 17, 2011 at 10:49, missingfaktor
wrote:
> data Message(header: String, contents: String) deriving (Equal, Show)
>
> could expand to:
> case class Message(header: String, contents: String)
> object Message extends ((String, String) => Message) {
>   implicit def messageShow: Show[Message] = showA
>   implicit val messageEqual: Equal[Message] = equalA
>   def header = (_: Message).header
>   def contents = (_: Message).contents
> }
> And maybe lenses too.
> On Sat, Sep 17, 2011 at 7:08 PM, Debasish Ghosh
> wrote:
>>
>> Hi Eugene -
>> Thanks for giving time towards macros in Scala. Despite having a rich type
>> system, I still think there are valid use cases for compile time macros in
>> Scala. Two such cases come straight to my mind :
>> 1. code generation. No language is boilerplate free and there occurs
>> situations where you feel the necessity to generate code. In one of my
>> recent projects I was writing a Redis client in Scala. Besides the basic
>> client I also implemented a Redis Cluster, which to the end user is also yet
>> another client. The user invokes same operations on the cluster as on a
>> single connection. Most operations are a simple delegate to the appropriate
>> shard while some operations need additional logic. Here I felt the necessity
>> of macros in Scala which would have generated lots of code that I needed to
>> write manually.
>> 2. Inlining code is a good optimization. But often even though the user
>> wants to inline, the compiler would not. Or even the user is not aware of an
>> inline optimization. In such cases if he implements using a macro, it will
>> be more efficient as the inlining is guaranteed.
>> Will let u know more such use cases as they come up.
>> Thanks.
>>
>> On Sat, Sep 17, 2011 at 4:43 PM, Eugene Burmako
>> wrote:
>>>
>>> Hi folks! My name is Eugene Burmako. I'm a first-year PhD student from
>>> Martin's lab at EPFL. During the first year of studies all doctoral students
>>> here at EPFL have to carry out a couple of semester projects, and, of
>>> course, my projects will involve something Scala-related.
>>>
>>> The idea, which I like the most at the moment, is to implement a
>>> metaprogramming facility for Scala. By that I mean a full-fledged system
>>> with quasiquotations and hygienic compile-time macros. My current
>>> inspiration is the macro system of Nemerle
>>> (http://nemerle.org/metaprogramming.ps). Sure, it's a huge amount of work,
>>> so I don't expect finishing all that stuff soon, but it can be done
>>> gradually.
>>>
>>> This macro project somewhat intersects with another research direction of
>>> our lab, namely, with lightweight modular staging
>>> (http://www.stanford.edu/class/cs442/lectures_unrestricted/cs442-lms.pdf).
>>> However, despite of being applicable to similar problems, both LMS and
>>> macros each have distinctive scenarios where they really shine. That's why
>>> we decided that it'd be practical to explore both of these approaches to
>>> metaprogramming.
>>>
>>> Having said that, I'd like to find compelling use-cases for macros, so
>>> that the functionality implemented will be immediately useful. Of course,
>>> macros are very elegant in themselves, and, from the standpoint of a
>>> language designer, I can name at least a handful of cool features that can
>>> be enabled by macros. However, what I'd really like to know is what
>>> scenarios are the most important to you in your everyday jobs. I would be
>>> very glad to receive your feedback!
>>>
>>> P.S. The usual disclaimer applies here. No promises/commitments so far,
>>> it's just a research project. However, if it deems useful, I think, it might
>>> end up in trunk in some shape or another.
>>
>>
>>
>> --
>> Debasish Ghosh
>> http://manning.com/ghosh
>>
>> Twttr: @debasishg
>> Blog: http://debasishg.blogspot.com
>> Code: http://github.com/debasishg
>
>
>
> --
> Cheers,
> missingfaktor.
> When you stand for what you believe in, you can change the world.
>

Philippe Lhoste
Joined: 2010-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!

On 17/09/2011 13:13, Eugene Burmako wrote:
> The idea, which I like the most at the moment, is to implement a
> metaprogramming facility for Scala. By that I mean a full-fledged system
> with quasiquotations and hygienic compile-time macros. My current
> inspiration is the macro system of Nemerle
> (http://nemerle.org/metaprogramming.ps). Sure, it's a huge amount of
> work, so I don't expect finishing all that stuff soon, but it can be
> done gradually.

Excellent project, taking inspiration from an excellent (from what I
have read) source.

> Having said that, I'd like to find compelling use-cases for macros, so
> that the functionality implemented will be immediately useful. Of
> course, macros are very elegant in themselves, and, from the standpoint
> of a language designer, I can name at least a handful of cool features
> that can be enabled by macros. However, what I'd really like to know is
> what scenarios are the most important to you in your everyday jobs. I
> would be very glad to receive your feedback!

One obvious point it in Scala itself, to avoid generating
Function1.scala to Function22.scala, idem for Product.scala and
Tuple.scala

Enum is another good case, eg. to automatically generate an index number
from a set of enums. I made a workaround by putting the case object
names in a list, but I dislike duplicating these names.

I can see them useful to make DSLs, too.
There is a little intersection with partial functions, for example, but
still I think it can be useful there.

Ken McDonald
Joined: 2011-02-13,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!
Just as a side note, there should ideally be something in the syntax of macro invocations that makes them easily distinguishable from standard Scala calls. Scala is already so semantically rich that adding in another feature that _looks_ like Scala calls could really muddy the waters when it came to understanding code.
Ideally this should be done by syntax-aware editors, but they're still struggling to get up to Scala's current level, so I think it's safe to assume they wouldn't highlight macros for quite some time.
Cheers,Ken
dcsobral
Joined: 2009-04-23,
User offline. Last seen 38 weeks 5 days ago.
Re: Re: implementing macros for Scala, looking for feedback!

On Sat, Sep 17, 2011 at 16:05, Ken McDonald wrote:
> Just as a side note, there should ideally be something in the syntax of
> macro invocations that makes them easily distinguishable from standard Scala
> calls. Scala is already so semantically rich that adding in another feature
> that _looks_ like Scala calls could really muddy the waters when it came to
> understanding code.
> Ideally this should be done by syntax-aware editors, but they're still
> struggling to get up to Scala's current level, so I think it's safe to
> assume they wouldn't highlight macros for quite some time.

That assumption is not really supported. Eclipse's ScalaIDE 2.0 used
*the compiler* to figure out Scala code. It, therefore, knows as much
as the compiler does. Its problems lie elsewhere.

It would be true of IDEA, presumably, but we'll have to wait and see.
It's not like such a feature could hit trunk tomorrow.

Bill La Forge
Joined: 2011-07-13,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!
My code never quite seems to fit the language I'm programming into and I always end up with too much boilerplate, even with a language as rich as Scala. Macros would help a lot.

I'm always creating factory classes which contain meta data about another class, with possibly multiple factories (supporting different uses) per class. Add to this I do aggregates of classes and I often end up with 3 or more classes of boilerplate.

Bill

On Sat, Sep 17, 2011 at 4:43 PM, Eugene Burmako <eugene [dot] burmako [at] epfl [dot] ch> wrote:
Hi folks! My name is Eugene Burmako. I'm a first-year PhD student from Martin's lab at EPFL. During the first year of studies all doctoral students here at EPFL have to carry out a couple of semester projects, and, of course, my projects will involve something Scala-related.

The idea, which I like the most at the moment, is to implement a metaprogramming facility for Scala. By that I mean a full-fledged system with quasiquotations and hygienic compile-time macros. My current inspiration is the macro system of Nemerle (http://nemerle.org/metaprogramming.ps). Sure, it's a huge amount of work, so I don't expect finishing all that stuff soon, but it can be done gradually.

This macro project somewhat intersects with another research direction of our lab, namely, with lightweight modular staging (http://www.stanford.edu/class/cs442/lectures_unrestricted/cs442-lms.pdf). However, despite of being applicable to similar problems, both LMS and macros each have distinctive scenarios where they really shine. That's why we decided that it'd be practical to explore both of these approaches to metaprogramming.

Having said that, I'd like to find compelling use-cases for macros, so that the functionality implemented will be immediately useful. Of course, macros are very elegant in themselves, and, from the standpoint of a language designer, I can name at least a handful of cool features that can be enabled by macros. However, what I'd really like to know is what scenarios are the most important to you in your everyday jobs. I would be very glad to receive your feedback!

P.S. The usual disclaimer applies here. No promises/commitments so far, it's just a research project. However, if it deems useful, I think, it might end up in trunk in some shape or another.

kolotyluk
Joined: 2010-06-04,
User offline. Last seen 5 weeks 16 hours ago.
Re: Re: implementing macros for Scala, looking for feedback!

When I first learned IBM 360 Assembler macros were a critical function
of the assembler because you were so close to the machine. You could
even write complete programs in the macro language itself. Eventually
when I was an undergraduate in the late 70's the conventional wisdom
seemed to be that Algol 68 was not much of a success, and people were
hinting that even macros were not such a great idea. In the 80's one of
my colleagues had done his master's thesis on extensibility in
programming languages, and his conclusion was that it was a bad idea. In
early 90's when I went back to grad school the conventional wisdom still
seemed to be extensibility was not such a great idea, and proponents of
Java seemed to believe that macros were such a failure in C & C++ that
Java was much better off without them.

These days I am able to see language extensibility working in Scala, and
it actually seems to work, but not in the way I would have dreamed of
even 15 years ago.

Now we are talking about macros again, and I naturally wonder about such
things as how do you know when there is an error in the macro vs. the
program, or how does the debugger work properly. I'm not saying macros
are a bad idea, but what has fundamentally changed to make people so
positive about them now? Is it that computing science has advanced to
the point where where now know how to do them better?

Cheers, Eric

Philippe Lhoste
Joined: 2010-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!

On 18/09/2011 03:36, Eric Kolotyluk wrote:
> early 90's when I went back to grad school the conventional wisdom still
> seemed to be extensibility was not such a great idea, and proponents of
> Java seemed to believe that macros were such a failure in C & C++ that
> Java was much better off without them.

It is a failure in C because it is unsafe, disregards types, can lead to
incorrect code (eg. #define M x*x instead of (x*x)) and so on.
Not sure about macros in C++, as I never used them.

> I'm not saying macros
> are a bad idea, but what has fundamentally changed to make people so
> positive about them now? Is it that computing science has advanced to
> the point where where now know how to do them better?

You mentioned only a part of computer history, but to my knowledge,
macros in Lisp (and derivatives) and some other languages, so called
hygienic macros, are safe to use and are seen as powerful features. And
it goes back far in the said history.

There might be downsides to the use of macros, perhaps debugging,
readability, I don't really know (I have yet to use a PL with good macro
support, but I like the idea).
I believe that macro support would strongly reduce the need for compiler
plugins, which need to be installed in your Scala distribution and thus
are harder to use in a project.

Scala might need macros less than some other languages (eg. you can
define a repeat(n) partial function, while other languages would have
used a macro to make new structures like that) but I believe it can
still be useful. I might be interesting to take a look at the existing
compiler plugins and see if a macro system can really replace them (or
simplify them).
Macros are about manipulating the AST, to generate new code.

Philippe Lhoste
Joined: 2010-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!

On 17/09/2011 13:13, Eugene Burmako wrote:
> Having said that, I'd like to find compelling use-cases for macros, so
> that the functionality implemented will be immediately useful.

Another interesting feature I though of:
Parse a format string and check it at compile time, instead of run-time.
We then go a step ahead in type safeness.

Ie., define format as a macro, and verify that in:
format("%s value is %d", name, name.value)
name is really a string, and name.value is an int.
And generate directly the StringBuilder stuff, instead of relying in
runtime code walking the string in search of %

Idem for a PHP-like templating system:
"Foo $name bar ${x map (_.toUpperCase)}" or similar: compile time check,
optimized code.
Idem for regular expressions.

Along with Scala's multiline strings, it can be powerful and efficient.

Bill La Forge
Joined: 2011-07-13,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: implementing macros for Scala, looking for feedback!
Very fond I was of IBM 360 Assembler macros. You could effectively write your own language, it was so powerful.

Bill La Forge

On Sun, Sep 18, 2011 at 7:06 AM, Eric Kolotyluk <eric [dot] kolotyluk [at] gmail [dot] com> wrote:
When I first learned IBM 360 Assembler macros were a critical function of the assembler because you were so close to the machine. You could even write complete programs in the macro language itself. Eventually when I was an undergraduate in the late 70's the conventional wisdom seemed to be that Algol 68 was not much of a success, and people were hinting that even macros were not such a great idea. In the 80's one of my colleagues had done his master's thesis on extensibility in programming languages, and his conclusion was that it was a bad idea. In early 90's when I went back to grad school the conventional wisdom still seemed to be extensibility was not  such a great idea, and proponents of Java seemed to believe that macros were such a failure in C & C++ that Java was much better off without them.

These days I am able to see language extensibility working in Scala, and it actually seems to work, but not in the way I would have dreamed of even 15 years ago.

Now we are talking about macros again, and I naturally wonder about such things as how do you know when there is an error in the macro vs. the program, or how does the debugger work properly. I'm not saying macros are a bad idea, but what has fundamentally changed to make people so positive about them now? Is it that computing science has advanced to the point where where now know how to do them better?

Cheers, Eric

Alex Repain
Joined: 2010-07-27,
User offline. Last seen 1 year 31 weeks ago.
Re: Re: implementing macros for Scala, looking for feedback!


2011/9/18 Philippe Lhoste <PhiLho [at] gmx [dot] net>
On 17/09/2011 13:13, Eugene Burmako wrote:
Having said that, I'd like to find compelling use-cases for macros, so
that the functionality implemented will be immediately useful.

Another interesting feature I though of:
Parse a format string and check it at compile time, instead of run-time. We then go a step ahead in type safeness.

I would add we also go a step ahead in performances. Format strings are about 100 times slower than StringBuilders, even used implicitly in concatenated stuff (in 2.8.0, maybe not anymore)

Ie., define format as a macro, and verify that in:
format("%s value is %d", name, name.value)
name is really a string, and name.value is an int.
And generate directly the StringBuilder stuff, instead of relying in runtime code walking the string in search of %

Idem for a PHP-like templating system:
"Foo $name bar ${x map (_.toUpperCase)}" or similar: compile time check, optimized code.
Idem for regular expressions.

Along with Scala's multiline strings, it can be powerful and efficient.

xeno.by
Joined: 2011-07-18,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!

Thanks for all the feedback - please, keep it going!

@All. I'm not really sure whether it will be possible to implement
macros that change syntax (I have yet to dig into scalac parser to
find that out), though I'd really like to implement unrestricted code
generation (i.e. to support not only expressions, but also class/
method/implicit/whatever definitions).

@Daniel. Conditional compilation for different Scala versions is a
brilliant idea!

@Eric. Well, error reporting and debugging are not that bad. There are
ways to retain most of the debug info, and there are ways to keep
macros hygienic and type-safe. You can find more info on that in my
today's blog (see the link below).

@Ken. As I see that, there is a trade-off that involves transparency
of macro invocations. Either we have transparent but not first-class
macros (as it's done in Nemerle), or we have first-class macros that
require explicit triggers (as it's done in Template Haskell). Both
approaches have their merits, so it's not an easy choice.

@Philippe, Alex. Yep, string interpolation, along with Perl-like regex
matching, is near the top of my favorite macro applications in the
small. The top being syntactic sugar for computation expressions. I
always loved the concept of monads, but was scared away by the amount
of boilerplate required to use them in most mainstream languages.

This very semester I will, most likely, go for quasiquotations that
will bring introspection capabilities of Scala to a new level (kinda
what LINQ and expression trees did to .NET, but even better). In the
meanwhile, I'll be posting about the progress in my blog: http://blog-en.xeno.by.
Here's the today's installation: http://xeno-by.blogspot.com/2011/09/macros-state-of-art.html.

ichoran
Joined: 2009-08-14,
User offline. Last seen 2 years 3 weeks ago.
Re: implementing macros for Scala, looking for feedback!
(1) I waste a lot of time on library pimping to add a method:

class XPimpsY[A](a: A) {
  def doSomething[B](b: B) = { ... }
}
implicit def y_to_x[A](y: Y[A]) = new XPimpsY[A]

when all I really want to write is doSomething[A,B](y: Y[A], b: B) = {...}

If Scala gets some independent lightweight extension capability this would be a moot point, but otherwise being able to macro this would be my most important use case.

(2) Using the ordinary PML pattern for collections is often arduous because one needs to write in highly generic CanBuildFroms and pass in a pile of implicits.  If I just want to do something like

   xs.filter(p).map(f).filter(q)

over and over again, it would be nice to be able to specify that xs.fmf(p)(f)(q) expanded to the above, and let the compiler worry about whether that was possible given the types of xs and f, not me when I'm writing the implicit conversions and such.

(3) The type system doesn't make it easy to deal in semi-programmatic ways with primitives.  For example, if you're doing vector math, you might want
  longVector + longVector => longVector
  floatVector + longVector => doubleVector
  floatVector + floatVector => floatVector
and you might want your type system to know this.  Actually telling the type system about this is painful, because you have to duplicate code for the addition each time.  Given how the JVM works, the code _must_ be duplicated (it's different bytecode each time), but if a macroing system could do the work instead of me, it would save a lot of time and potential for error.

(4) For loops are still painfully slow for certain operations; defining equivalent methods only gives equivalent performance if multiple dispatch can be avoided (which it often can't e.g. when nested).  If macroing could, as in the Nemerle example, allow
  For(i = 0; i<N; i+=1) { a(i) = b(i+1)-b(i) }
to be converted into the appropriate while loop (with variable hiding)
  {
    var i = 0
    while (i < N) {
      a(i) = b(i+1) - b(i)
      i += 1
    }
  }
it would save quite a bit of typing and potential for error in high-performance code.  This is exactly the sort of thing one can make DSLs for, but when it comes to primitives, the speed usually leaves much to be desired.

  --Rex
 

On Sat, Sep 17, 2011 at 7:13 AM, Eugene Burmako <eugene [dot] burmako [at] epfl [dot] ch> wrote:
Hi folks! My name is Eugene Burmako. I'm a first-year PhD student from Martin's lab at EPFL. During the first year of studies all doctoral students here at EPFL have to carry out a couple of semester projects, and, of course, my projects will involve something Scala-related.

The idea, which I like the most at the moment, is to implement a metaprogramming facility for Scala. By that I mean a full-fledged system with quasiquotations and hygienic compile-time macros. My current inspiration is the macro system of Nemerle (http://nemerle.org/metaprogramming.ps). Sure, it's a huge amount of work, so I don't expect finishing all that stuff soon, but it can be done gradually.

This macro project somewhat intersects with another research direction of our lab, namely, with lightweight modular staging (http://www.stanford.edu/class/cs442/lectures_unrestricted/cs442-lms.pdf). However, despite of being applicable to similar problems, both LMS and macros each have distinctive scenarios where they really shine. That's why we decided that it'd be practical to explore both of these approaches to metaprogramming.

Having said that, I'd like to find compelling use-cases for macros, so that the functionality implemented will be immediately useful. Of course, macros are very elegant in themselves, and, from the standpoint of a language designer, I can name at least a handful of cool features that can be enabled by macros. However, what I'd really like to know is what scenarios are the most important to you in your everyday jobs. I would be very glad to receive your feedback!

P.S. The usual disclaimer applies here. No promises/commitments so far, it's just a research project. However, if it deems useful, I think, it might end up in trunk in some shape or another.

etorreborre 2
Joined: 2011-02-23,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!
> I'll put forward conditional compilation based on Scala version.

I think that would still be helpful in the future. For example to avoid deprecation warnings like error/sys.error:
  •  error was deprecated in 2.9.0 in favor of sys.error
  • if you want to do a cross-compilation with 2.8.1 and 2.9.0 you had to put up with all the warnings.
Eric.
dcsobral
Joined: 2009-04-23,
User offline. Last seen 38 weeks 5 days ago.
Re: implementing macros for Scala, looking for feedback!

On Sun, Sep 18, 2011 at 20:39, etorreborre wrote:
>> I'll put forward conditional compilation based on Scala version.
>
> I think that would still be helpful in the future. For example to avoid
> deprecation warnings like error/sys.error:
>
>  error was deprecated in 2.9.0 in favor of sys.error
> if you want to do a cross-compilation with 2.8.1 and 2.9.0 you had to put up
> with all the warnings.

Yes, but, to illustrate my point, say macros were introduced in 2.9.
You wouldn't be able to use them to avoid this problem, because 2.8
wouldn't accept the macro.

etorreborre 2
Joined: 2011-02-23,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!
Right, I got that but I'm thinking that this may still be useful in the future to absorb deprecation cycles. I'd be surprised if there weren't any :-).
E.
dcsobral
Joined: 2009-04-23,
User offline. Last seen 38 weeks 5 days ago.
Re: implementing macros for Scala, looking for feedback!

On Mon, Sep 19, 2011 at 00:23, etorreborre wrote:
> Right, I got that but I'm thinking that this may still be useful in the
> future to absorb deprecation cycles. I'd be surprised if there weren't any
> :-).

Oh, I *hope* it will be useful for that! I know Odersky wants to put
breaks on wild changes -- and that's good -- but there's still much to
be improved on Scala.

Adam Jorgensen
Joined: 2011-04-17,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!
+1 to this.
I think it would be nice and I imagine it's quite possible to avoid the pitfalls of the C macro system.
A totally random and possibly incorrect thought: Macros in C are fairly comparable to a Templating System in a modern Web App
For applications that want to perform some scaffolding at run-time and use the output this might also be useful?
nicolas.oury@gm...
Joined: 2011-02-13,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!
I think macros are a great idea. I used to code a bit in LISP and find miss them really much.
I used them for DSLs.  (My particular case was to read generalised chemical reactions and output 2 functions : one to compute the activity of the reaction and another to execute the reaction when it is triggered)  
Contrary to one of the preceding mail, I think it should be a similar syntax for macro call and function call. However, it can be useful to have some of its arguments parsed with another grammar than the scala grammar. (I don't know how hard it is) For example, with the classical example of a Prolog-like rules as macros, it would be easier to parse it as a list than as a scala AST, and then deconstruct the scala AST to its list meaning. Some DSL might not be even writable as syntictally correct scala.  So suggestion 1: At macro definition: annotate the arguments of the macro by how it is to be parsed. (I know it is probably difficult because of compiler phases, but one of the strength of macros in LISP comes from the fact than LISP syntax is very generic: everything that is well bracketed is a LISP term and can be an argument to a macro)
There is also some interesting relation between macros and typing. Depending of the position where the macro is called, you might have a type (or part of a type) to push into the macro. (Then again, compiler phase issue...). So suggestion 2: allowing some macros to request an access to type information: getting (partial) informations about the type they must return. (or even type of a subterm?)
Last, I think hygienic macros are certainly a good default, but with macros, it can be good to allow things that are really hackish and inelegant. So making sure there is a way of any term generating program is probably a good idea too.
I hope you will do his project and have a lot of fun.
Best regards,
Nicolas.
Alan Burlison
Joined: 2011-08-26,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!

On 09/19/11 06:01, Adam Jorgensen wrote:

> A totally random and possibly incorrect thought: Macros in C are fairly
> comparable to a Templating System in a modern Web App

Umm, no they aren't. C macros provide static compile-time textual
substitution, templating systems provide run-time dynamic output formatting.

I'm not at all convinced that macros are a good idea for Scala to be
honest, I certainly don't miss them in Java. However, that's probably
due to the scars they have inflicted on me in C/C++, so perhaps
something less lethal can be devised for Scala.

Philippe Lhoste
Joined: 2010-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!

On 19/09/2011 09:22, nicolas [dot] oury [at] gmail [dot] com wrote:
> Contrary to one of the preceding mail, I think it should be a similar syntax for macro
> call and function call. However, it can be useful to have some of its arguments parsed
> with another grammar than the scala grammar. (I don't know how hard it is)

I am not sure how to conflate the two sentences.
In a number of languages using (hygienic) macros I looked at (Nermele, Seed7, some others
I don't recall), macro usage doesn't look like a function call in general, but rather
allows to use them to make some kind of DSL, eg. to make new operators, new loop
structures, etc.

Now, perhaps one problem is to stumble upon such macro in some code and to be stumped
because it might precisely not look like the regular language, but offers no hints it uses
some macro defined somewhere...

Adam Jorgensen
Joined: 2011-04-17,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!
Thanks for the correction, although I guess what I really was trying to say both are basically textual substitution systems. It seems thatthe macro systems in LISP, Scheme, etc are a bit less braindead and hence might actually be a neat idea in Scala :-)

On 19 September 2011 10:35, Alan Burlison <alan [dot] burlison [at] gmail [dot] com> wrote:
On 09/19/11 06:01, Adam Jorgensen wrote:

A totally random and possibly incorrect thought: Macros in C are fairly
comparable to a Templating System in a modern Web App

Umm, no they aren't.  C macros provide static compile-time textual substitution, templating systems provide run-time dynamic output formatting.

I'm not at all convinced that macros are a good idea for Scala to be honest, I certainly don't miss them in Java.  However, that's probably due to the scars they have inflicted on me in C/C++, so perhaps something less lethal can be devised for Scala.

Tony Morris 2
Joined: 2009-03-20,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Please see Template Haskell. I would hate for us to repeat mistakes out
of ignorance of lessons already learned.

(not directed at any comment in particular)

On 09/19/2011 10:04 PM, Adam Jorgensen wrote:
> Thanks for the correction, although I guess what I really was trying to say
> both are basically textual substitution systems. It seems that
> the macro systems in LISP, Scheme, etc are a bit less braindead and hence
> might actually be a neat idea in Scala :-)
>
> On 19 September 2011 10:35, Alan Burlison wrote:
>
>> On 09/19/11 06:01, Adam Jorgensen wrote:
>>
>> A totally random and possibly incorrect thought: Macros in C are fairly
>>> comparable to a Templating System in a modern Web App
>>>
>>
>> Umm, no they aren't. C macros provide static compile-time textual
>> substitution, templating systems provide run-time dynamic output formatting.
>>
>> I'm not at all convinced that macros are a good idea for Scala to be
>> honest, I certainly don't miss them in Java. However, that's probably due
>> to the scars they have inflicted on me in C/C++, so perhaps something less
>> lethal can be devised for Scala.
>>
>> --
>> Alan Burlison
>> --
>>
>

nicolas.oury@gm...
Joined: 2011-02-13,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!

Please see Template Haskell. 


I +1 this.  If I remember correctly, there was some interesting relation to compiler phases, with some interleaving between templatingand typing, which might be of interest here.
nicolas.oury@gm...
Joined: 2011-02-13,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: implementing macros for Scala, looking for feedback!

I am not sure how to conflate the two sentences.
In a number of languages using (hygienic) macros I looked at (Nermele, Seed7, some others I don't recall), macro usage doesn't look like a function call in general, but rather allows to use them to make some kind of DSL, eg. to make new operators, new loop structures, etc.

Now, perhaps one problem is to stumble upon such macro in some code and to be stumped because it might precisely not look like the regular language, but offers no hints it uses some macro defined somewhere...


I am not sure about other languages, but from my macro in LISP experience, macro call looks like a function call, except that the argumentsdo not have to make sense in the language itself.
eg: In a Lisp, you can define a macro prolog, that takes a series of rule and compile them to some data structures representing the code or some program to solve the system.
This would look like: (prolog  (Foo (X) <-  Bar(x))             (Bar(foo) <-))
Here, the call looks like a function call but the argument does not look like an argument written in Lisp. The macro will parse the list and makes sense of it. It surely is easier to parse a list than scala AST, when the argument is not a scala program.
Hopes tat makes my point clearer,
Nicolas. 
dcsobral
Joined: 2009-04-23,
User offline. Last seen 38 weeks 5 days ago.
Re: Re: implementing macros for Scala, looking for feedback!

On Mon, Sep 19, 2011 at 07:42, Philippe Lhoste wrote:
> On 19/09/2011 09:22, nicolas [dot] oury [at] gmail [dot] com wrote:
>>
>> Contrary to one of the preceding mail, I think it should be a similar
>> syntax for macro
>> call and function call. However, it can be useful to have some of its
>> arguments parsed
>> with another grammar than the scala grammar. (I don't know how hard it is)
>
> I am not sure how to conflate the two sentences.
> In a number of languages using (hygienic) macros I looked at (Nermele,
> Seed7, some others I don't recall), macro usage doesn't look like a function
> call in general, but rather allows to use them to make some kind of DSL, eg.
> to make new operators, new loop structures, etc.

On Nemerle macros normally appear as either functional calls or
annotations -- which seems ideal for Scala. They can _also_ add
syntax, by defining how the parameters are separated by literal
tokens, but I doubt that can be achieved.

See http://nemerle.org/macros.html. One of the first examples show the
macro being used function-call like, and the first for-loop example
shows that as well. And, then, there's the serializable example, which
is interesting because it shows the (same) macro being inserted at two
points in the compiler phases, first to modify the type, and, then, to
add a method definition when all types are already settled.

>
> Now, perhaps one problem is to stumble upon such macro in some code and to
> be stumped because it might precisely not look like the regular language,
> but offers no hints it uses some macro defined somewhere...
>
> --
> Philippe Lhoste

Arno Haase
Joined: 2009-11-06,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Love to see you tackle this - I have been wanting this for a long time!

Usage examples:

* adding change notification support to classes

* 'cached' semantics for defs - store each result in some sort of cache
(that can be passed in as an additional implicit parameter) and evaluate
the def exactly once for every combination of parameters

* the @beanGetter / @beanSetter stuff from the compiler itself

* DSLs, especially for interop with typical Java frameworks, require
code to be generated rather than wrapping stuff at runtime

- - Arno

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk53XRoACgkQbmZsMyUPuXT+PQCg7t4zqqy1JKUCRGZ0pImInKOY
tIYAmwSGToKU9o32HiI/SnZ7iwUQu4vN
=wGXv
-----END PGP SIGNATURE-----

Skavookie
Joined: 2011-02-20,
User offline. Last seen 1 year 24 weeks ago.
Re: implementing macros for Scala, looking for feedback!

I have been occasionally than king that that maybe there is a
sufficientately generaic that can be macro system that can be applied
across programing languages. For example, for each new programing
language you can write plugins to make it work, I have in Lisp from
because that is particularly good for macros. So the plugins convert
X -> Lisp AST -> User Transforms -> Lisp ASP -> X. The Lisp (or want
ever) from is so that it can be as as flexible as prospiele and make
many macros against many languages. So they don't have to make one
macro accoros many languages. I'm sorry my communications skills are
so bad.

On Sep 17, 4:13 am, Eugene Burmako wrote:
> Hi folks! My name is Eugene Burmako. I'm a first-year PhD student from
> Martin's lab at EPFL. During the first year of studies all doctoral students
> here at EPFL have to carry out a couple of semester projects, and, of
> course, my projects will involve something Scala-related.
>
> The idea, which I like the most at the moment, is to implement a
> metaprogramming facility for Scala. By that I mean a full-fledged system
> with quasiquotations and hygienic compile-time macros. My current
> inspiration is the macro system of Nemerle (http://nemerle.org/metaprogramming.ps). Sure, it's a huge amount of work, so
> I don't expect finishing all that stuff soon, but it can be done gradually.
>
> This macro project somewhat intersects with another research direction of
> our lab, namely, with lightweight modular staging (http://www.stanford.edu/class/cs442/lectures_unrestricted/cs442-lms.pdf).
> However, despite of being applicable to similar problems, both LMS and
> macros each have distinctive scenarios where they really shine. That's why
> we decided that it'd be practical to explore both of these approaches to
> metaprogramming.
>
> Having said that, I'd like to find compelling use-cases for macros, so that
> the functionality implemented will be immediately useful. Of course, macros
> are very elegant in themselves, and, from the standpoint of a language
> designer, I can name at least a handful of cool features that can be enabled
> by macros. However, what I'd really like to know is what scenarios are the
> most important to you in your everyday jobs. I would be very glad to receive
> your feedback!
>
> P.S. The usual disclaimer applies here. No promises/commitments so far, it's
> just a research project. However, if it deems useful, I think, it might end
> up in trunk in some shape or another.

Ken McDonald
Joined: 2011-02-13,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!


On Monday, September 19, 2011 7:04:42 AM UTC-5, Adam Jorgensen wrote:
Thanks for the correction, although I guess what I really was trying to say both are basically textual substitution systems. It seems thatthe macro systems in LISP, Scheme, etc are a bit less braindead and hence might actually be a neat idea in Scala :-)

Well, I've used LISP macros a reasonable amount a _long_ time ago, and what I remember from the experience is that it always felt ugly and unreliable, compared to programming in pure Lisp. And needless to say, macros in C/C++ are a godawful mess that, depending on how extensively they are used, can basically require developers to program and debug in two languages at once, one of which is not very amenable to IDE support.
So out of the two macro systems I have some experience with, one has been less than ideal and one has been just awful. This isn't to say that working on a macro system for Scala isn't worthwhile, but I think it will need some new ideas as to the nature of what "macro" means, and very careful design. Recently, I had to generate a lot of boilerplate (many setter/getter type functions which only differed in the name and type of the property they operated on), and to do so, wrote a little Scala program than actually generated them. It would've been nice to have a macro system that could've done that.
Cheers,Ken 
Tony Morris 2
Joined: 2009-03-20,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/20/2011 07:40 AM, Ken McDonald wrote:
> (many setter/getter type
> functions which only differed in the name and type of the property they
> operated on), and to do so, wrote a little Scala program than actually
> generated them. It would've been nice to have a macro system that could've
> done that.

:(

Ken McDonald
Joined: 2011-02-13,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!


On Sunday, September 18, 2011 1:19:56 PM UTC-5, Eugene Burmako wrote:
Thanks for all the feedback - please, keep it going!

@All. I'm not really sure whether it will be possible to implement
macros that change syntax (I have yet to dig into scalac parser to
find that out), though I'd really like to implement unrestricted code
generation (i.e. to support not only expressions, but also class/
method/implicit/whatever definitions).


Unless there is a compelling use case for doing the above, please don't do it! Scala syntax is already highly flexible due to a number of features; interchangeability of method invocations and operators, use of implicits, and several other things. Unless others can give a real use case for needing further syntactic flexibility, I think that additional such flexibility would make Scala harder to learn without adding much real value.
Ken 
dcsobral
Joined: 2009-04-23,
User offline. Last seen 38 weeks 5 days ago.
Re: implementing macros for Scala, looking for feedback!

On Mon, Sep 19, 2011 at 18:40, Ken McDonald wrote:
>
> Well, I've used LISP macros a reasonable amount a _long_ time ago, and what
> I remember from the experience is that it always felt ugly and unreliable,
> compared to programming in pure Lisp. And needless to say, macros in C/C++
> are a godawful mess that, depending on how extensively they are used, can
> basically require developers to program and debug in two languages at once,
> one of which is not very amenable to IDE support.

One difference is that Scala is typed, and so would the macros. That,
plus hygiene, will go a long way towards increasing their safety.

Tiark Rompf
Joined: 2009-02-18,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!

Hey Tony,

> Please see Template Haskell. I would hate for us to repeat mistakes out
> of ignorance of lessons already learned.

what are (in your view) the lessons to be learned from Template Haskell?

- Tiark

> (not directed at any comment in particular)
>
> On 09/19/2011 10:04 PM, Adam Jorgensen wrote:
>> Thanks for the correction, although I guess what I really was trying to say
>> both are basically textual substitution systems. It seems that
>> the macro systems in LISP, Scheme, etc are a bit less braindead and hence
>> might actually be a neat idea in Scala :-)
>>
>> On 19 September 2011 10:35, Alan Burlison wrote:
>>
>>> On 09/19/11 06:01, Adam Jorgensen wrote:
>>>
>>> A totally random and possibly incorrect thought: Macros in C are fairly
>>>> comparable to a Templating System in a modern Web App
>>>>
>>>
>>> Umm, no they aren't. C macros provide static compile-time textual
>>> substitution, templating systems provide run-time dynamic output formatting.
>>>
>>> I'm not at all convinced that macros are a good idea for Scala to be
>>> honest, I certainly don't miss them in Java. However, that's probably due
>>> to the scars they have inflicted on me in C/C++, so perhaps something less
>>> lethal can be devised for Scala.
>>>
>>> --
>>> Alan Burlison
>>> --
>>>
>>
>
>
> - --
> Tony Morris
> http://tmorris.net/
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQEcBAEBAgAGBQJOdzBcAAoJEPxHMY3rBz0PIjAH/0YwIH3b+CHtjRyWvC5xg2FP
> cvL2rTwELj52tHDy5e7+4/1um2nmy1T6xTdaDF+T3UdofKRnoAYDK3tUKpX9Tv/j
> m5er0Te6rqR+bw3x0B2wdZEpCwXuMlaNWnfn74uONs7hpB1EMkx0h6BmHtKYJtdN
> iS3bP8hgrMcThchK+VBmWjoJkup3doulbttEZSwa2w4gNc+T6XwSIDQoJ+DOYdgA
> ESrJu27Vc5rrctVb4De2iuspucp/GlHLq+rgbCie7EfQWkXMzB1wD0vuN9MHB1mk
> gqPphx0HKtrsNPwDIk0yxj+q08v2yfPybdf0J3b2MKJBdgRu+7rAzzsAhIFCb0Y=
> =rkgK
> -----END PGP SIGNATURE-----

DaveScala
Joined: 2011-03-18,
User offline. Last seen 1 year 21 weeks ago.
Re: implementing macros for Scala, looking for feedback!

Hi Andreas,

thanks for the link.
yes I think it is certainly possible although my level is not there
yet.
I learned about datatype-generic programming (DGP), origami patterns
and higher kinded shapes of data a month or so ago via
these links:
http://www.cs.ox.ac.uk/jeremy.gibbons/publications/scalagp.pdf
http://vasilrem.com/blog/software-development/effective-scala-programmin...
http://www.slideshare.net/remeniuk/algebraic-data-types-and-origami-patt...
http://www.slideshare.net/remeniuk/effective-scala-programming-patterns

Although this approach is certainly better I think a pragmatic macro/
template way to solve a pattern would also be welcome.
because everybody understands the concept of a macro/template because
of Excel and Word, but a few understand what a bifunctor/bimap
etcetera is.

> Date: Tue, 20 Sep 2011 01:02:10 -0700
> Subject: Re: Fwd: Re: [scala-user] implementing macros for Scala, looking for feedback!
>
> Hi Dave!
> It actually possible to encode some OO patterns in a Library. So you
> wouldn't need macros but could use that lib. For further reference see
> this post:http://debasishg.blogspot.com/2011/07/datatype-generic-
> programming-in-scala.html
> And the corresponding paper - ORIGAMI Pattern
>
> Regards Andreas
>

> > I have a use case for a template macro that is for automatic creation
> > of plumbing code for work arounds.
> >
> > I am not that familiar with Nemerle templating. I only read some
> > documentation about it. Is it supposed after compiling to binary as a
> > plugin for the compiler and still inserts automatically code when
> > someone is extending some class?
> >
> > I have here some code:
> > This code has to abstract methods
> > def isV(v: Visitor):Boolean
> > def asV(v: Visitor):V
> >
> > that need to be implemented when down the inheritance tree
> > For instance in:
> >
> > class Lit(val value: Int) extends Node[AleVisitor] with PrintExp {
> > def print(print: Print) { Console.print(value) }
> > def accept(v: AleVisitor) { v.visitLit(this) }
> > def isV(v: Visitor):Boolean = v.isInstanceOf[AleVisitor]
> > def asV(v: Visitor) = v.asInstanceOf[AleVisitor]
> > }
> >
> > It is always the pattern:
> > def isV(v: Visitor):Boolean = v.isInstanceOf[V]
> > def asV(v: Visitor) = v.asInstanceOf[V]
> > that V as to be substituted with the concrete Visitor class in this
> > case AleVisitor
> >
> > It would be handy that this plumbing code would be automatically
> > generated by the macro even when the macro in the abstract class in
> > the framework package is already compiled so that it works like a
> > plugin for the compiler and that the one that extends a class can
> > focus on its own domain problem and does not have to bother about
> > extending plumbing code.
> >
> > expressionproblem.scala
> > =======================
> > package exprframework {
> >
> > trait Exp {
> > def handle(v: Visitor)
> > }
> >
> > trait Visitor {
> > def apply(e: Exp)
> > def default(e: Exp)
> > }
> >
> > abstract class Node[V <: Visitor] extends Exp {
> > final def handle(v: Visitor) {
> > if (isV(v)) accept(asV(v))
> > else v.default(this)
> > }
> > private[exprframework] def accept(v: V)
> > def isV(v: Visitor):Boolean
> > def asV(v: Visitor):V
> > }
> >
> > abstract class Op[E <: Exp] extends Visitor {
> > final def apply(e: Exp) {
> > if (isE(e)) call(asE(e))
> > else e.handle(this)
> > }
> > private[exprframework] def call(e: E)
> > def default(e: Exp) {
> > throw new IllegalArgumentException("Expression problem
> > occurred!")
> > }
> > def isE(e: Exp):Boolean
> > def asE(e: Exp):E
> > }}
> >
> > //implementation
> > package impl{
> > import exprframework._
> > trait AleVisitor extends Visitor {
> > def visitLit(lit: Lit)
> > def visitAdd(add: Add)
> > }
> >
> > trait PrintExp extends Exp {
> > def print(print: Print)
> > }
> >
> > class Print extends Op[PrintExp] with Visitor {
> > def call(e: PrintExp) { e.print(this) }
> > def isE(e: Exp):Boolean = e.isInstanceOf[PrintExp]
> > def asE(e: Exp) = e.asInstanceOf[PrintExp]
> > }
> >
> > class Lit(val value: Int) extends Node[AleVisitor] with PrintExp {
> > def print(print: Print) { Console.print(value) }
> > def accept(v: AleVisitor) { v.visitLit(this) }
> > def isV(v: Visitor):Boolean = v.isInstanceOf[AleVisitor]
> > def asV(v: Visitor) = v.asInstanceOf[AleVisitor]
> > }
> >
> > class Add(val left: Exp, val right: Exp) extends Node[AleVisitor]
> > with PrintExp {
> > def print(print: Print) {
> > print.apply(left); Console.print("+"); print.apply(right)
> > }
> > def accept(v: AleVisitor) { v.visitAdd(this) }
> > def isV(v: Visitor):Boolean = v.isInstanceOf[AleVisitor]
> > def asV(v: Visitor) = v.asInstanceOf[AleVisitor]
> >
> > }}
> >
> > //extension
> > package extension {
> > import exprframework._
> > import impl._
> >
> > trait EvalExp extends PrintExp {
> > def eval(eval: Eval): Int
> > }
> >
> > class Eval extends Op[EvalExp] with AleVisitor {
> > private [this] var result = 0
> > final def eval(e: Exp) = { apply(e); result }
> > def call(e: EvalExp) { result = e.eval(this) }
> > def visitLit(lit: Lit) { result = lit.value }
> > def visitAdd(add: Add) { result = eval(add.left) +
> > eval(add.right) }
> > def isE(e: Exp):Boolean = e.isInstanceOf[EvalExp]
> > def asE(e: Exp) = e.asInstanceOf[EvalExp]
> > }}
> >
> > //further extension
> > package impl {
> > import exprframework._
> > import extension._
> >
> > trait NaleVisitor extends AleVisitor {
> > def visitNeg(neg: Neg)
> > }
> >
> > class Neg(val exp: Exp) extends Node[NaleVisitor] with EvalExp {
> > def print(print: Print) {
> > Console.print("-("); print.apply(exp); Console.print(")")
> > }
> > def eval(eval: Eval) = { -eval.eval(exp) }
> > def accept(v: NaleVisitor) { v.visitNeg(this) }
> > def isV(v: Visitor):Boolean = v.isInstanceOf[NaleVisitor]
> > def asV(v: Visitor) = v.asInstanceOf[NaleVisitor]
> >
> > }}
> >
> > package main {
> > import exprframework._
> > import impl._
> > import extension._
> > object Main extends App {
> > val lit = new Lit(5)
> > val neg = new Neg(lit)
> > val eval = new Eval
> > println(eval.eval(neg))
> > }
> >
> >
> >
> > }

nicolas.oury@gm...
Joined: 2011-02-13,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: implementing macros for Scala, looking for feedback!


Unless there is a compelling use case for doing the above, please don't do it! Scaisla syntax is already highly flexible due to a number of features; interchangeability of method invocations and operators, use of implicits, and several other things. Unless others can give a real use case for needing further syntactic flexibility, I think that additional such flexibility would make Scala harder to learn without adding much real value.
Ken 

Some DSLs would not play well with scala syntax. In Graham's On Lisp, he builds a syntax for Prolog in Lisp using macros. ( http://lib.store.yahoo.net/lib/paulgraham/onlisp.pdf chapter 24) In practice, this allows to build macro-based DSLs that are directly usable by domain expert with little to no knowledge of lisp.
It is more difficult to do so, and sometimes impossible, without a flexible syntax. Scala syntax is flexible but still has numerous assumptions on the  language which make creating some DSLs quite difficult.
On top of that, there is no syntax to generate trait, type or classes at compile-time.Macro would be a great plus too.
On a different note, I have a question: how much of something like the continuation plugin cold be implemented as a set of Nemerle macro?It would be a good question to ask for different plugins.
Best,Nicolas.

Stephen Haberman 2
Joined: 2011-07-23,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!
Awesome to hear this!
I've been hobbling along doing "kinda sorta" metaprogramming in Java with annotation processors, but the limitations are painful. It would be great to finally have real compile-time metaprogramming.

If you're looking for more inspiration, I'm a fan of boo, another .NET language that has metaprogramming support:
http://boo.codehaus.org/
- Stephen
Ken McDonald
Joined: 2011-02-13,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: implementing macros for Scala, looking for feedback!


On Tuesday, September 20, 2011 9:44:04 AM UTC-5, Nicolas Oury wrote:


Unless there is a compelling use case for doing the above, please don't do it! Scaisla syntax is already highly flexible due to a number of features; interchangeability of method invocations and operators, use of implicits, and several other things. Unless others can give a real use case for needing further syntactic flexibility, I think that additional such flexibility would make Scala harder to learn without adding much real value.
Ken 

Some DSLs would not play well with scala syntax. In Graham's On Lisp, he builds a syntax for Prolog in Lisp using macros. ( http://lib.store.yahoo.net/lib/paulgraham/onlisp.pdf chapter 24) In practice, this allows to build macro-based DSLs that are directly usable by domain expert with little to no knowledge of lisp.
It is more difficult to do so, and sometimes impossible, without a flexible syntax. Scala syntax is flexible but still has numerous assumptions on the  language which make creating some DSLs quite difficult.
That's true of course, but that's the nature of a formal language; it is defined with syntactic and semantic rules, and those rules will naturally preclude doing some things. I wouldn't be surprised if building a "perfectly expressive" language is equivalent to solving the Halting Problem :-)
Please don't think that I'm objecting to macros--I just think they should be approached slowly. Implement macro processing with a few small features, see if everything works alright, and go from there. Perhaps it's just that I'm still traumatized by having to work with C macros--it does sound like they've been put on a better formal footing now.
And I certainly wouldn't mind the ability to automatically generate boilerplate at compile time.
Ken
Bakos Gábor
Joined: 2009-06-08,
User offline. Last seen 42 years 45 weeks ago.
Re: implementing macros for Scala, looking for feedback!

Hi Eugene,

I am not sure how important what will be in F#, but it might worth
looking at it[0]:
- I think type providers-like functionality were already asked for
- Auto-properties might be a nice addition
- Maybe some people get familiar with LINQ, so being able to use a
similar construct for various tasks would be welcome
(- I think the measure types can be implemented as a library, but the
precedence rules can be offensive.)

Personally I do not see these as must have features, but I guess
Microsoft were made researches on the most wanted functionalities.
Cheers, gabor

[0]: http://msdn.microsoft.com/en-us/library/hh370982(v=VS.110).aspx

On 2011.09.17. 13:13, Eugene Burmako wrote:
> Hi folks! My name is Eugene Burmako. I'm a first-year PhD student from
> Martin's lab at EPFL. During the first year of studies all doctoral
> students here at EPFL have to carry out a couple of semester projects,
> and, of course, my projects will involve something Scala-related.
>
> The idea, which I like the most at the moment, is to implement a
> metaprogramming facility for Scala. By that I mean a full-fledged system
> with quasiquotations and hygienic compile-time macros. My current
> inspiration is the macro system of Nemerle
> (http://nemerle.org/metaprogramming.ps). Sure, it's a huge amount of
> work, so I don't expect finishing all that stuff soon, but it can be
> done gradually.
>
> This macro project somewhat intersects with another research direction
> of our lab, namely, with lightweight modular staging
> (http://www.stanford.edu/class/cs442/lectures_unrestricted/cs442-lms.pdf).
> However, despite of being applicable to similar problems, both LMS and
> macros each have distinctive scenarios where they really shine. That's
> why we decided that it'd be practical to explore both of these
> approaches to metaprogramming.
>
> Having said that, I'd like to find compelling use-cases for macros, so
> that the functionality implemented will be immediately useful. Of
> course, macros are very elegant in themselves, and, from the standpoint
> of a language designer, I can name at least a handful of cool features
> that can be enabled by macros. However, what I'd really like to know is
> what scenarios are the most important to you in your everyday jobs. I
> would be very glad to receive your feedback!
>
> P.S. The usual disclaimer applies here. No promises/commitments so far,
> it's just a research project. However, if it deems useful, I think, it
> might end up in trunk in some shape or another.

H-star Development
Joined: 2010-04-14,
User offline. Last seen 2 years 26 weeks ago.
Re: Re: implementing macros for Scala, looking for feedback!

DYNAMIC MIXINS!!!!1111

aka

subclass, proxy, and add another trait

Am 23.09.2011 19:48, schrieb Bakos Gábor:
> Hi Eugene,
>
> I am not sure how important what will be in F#, but it might worth
> looking at it[0]:
> - I think type providers-like functionality were already asked for
> - Auto-properties might be a nice addition
> - Maybe some people get familiar with LINQ, so being able to use a
> similar construct for various tasks would be welcome
> (- I think the measure types can be implemented as a library, but the
> precedence rules can be offensive.)
>
> Personally I do not see these as must have features, but I guess
> Microsoft were made researches on the most wanted functionalities.
> Cheers, gabor
>
> [0]: http://msdn.microsoft.com/en-us/library/hh370982(v=VS.110).aspx
>
> On 2011.09.17. 13:13, Eugene Burmako wrote:
>> Hi folks! My name is Eugene Burmako. I'm a first-year PhD student from
>> Martin's lab at EPFL. During the first year of studies all doctoral
>> students here at EPFL have to carry out a couple of semester projects,
>> and, of course, my projects will involve something Scala-related.
>>
>> The idea, which I like the most at the moment, is to implement a
>> metaprogramming facility for Scala. By that I mean a full-fledged system
>> with quasiquotations and hygienic compile-time macros. My current
>> inspiration is the macro system of Nemerle
>> (http://nemerle.org/metaprogramming.ps). Sure, it's a huge amount of
>> work, so I don't expect finishing all that stuff soon, but it can be
>> done gradually.
>>
>> This macro project somewhat intersects with another research direction
>> of our lab, namely, with lightweight modular staging
>> (http://www.stanford.edu/class/cs442/lectures_unrestricted/cs442-lms.pdf).
>> However, despite of being applicable to similar problems, both LMS and
>> macros each have distinctive scenarios where they really shine. That's
>> why we decided that it'd be practical to explore both of these
>> approaches to metaprogramming.
>>
>> Having said that, I'd like to find compelling use-cases for macros, so
>> that the functionality implemented will be immediately useful. Of
>> course, macros are very elegant in themselves, and, from the standpoint
>> of a language designer, I can name at least a handful of cool features
>> that can be enabled by macros. However, what I'd really like to know is
>> what scenarios are the most important to you in your everyday jobs. I
>> would be very glad to receive your feedback!
>>
>> P.S. The usual disclaimer applies here. No promises/commitments so far,
>> it's just a research project. However, if it deems useful, I think, it
>> might end up in trunk in some shape or another.
>
>

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland