Macro support in Scala

On Sat, Dec 11, 2010 at 07:44:45PM +0100, Arno Haase wrote:
> I would love to have something like Lisp macros is Scala, i.e. the
> possibility to have code that transforms other code. And I plan to get
> working on that. Is anyone aware of a similar effort?

You will want to look at this.

https://github.com/TiarkRompf/scala-virtualized/

I always assumed the point of creating supple typeful languages was
specifically to avoid the need for macro support. Call-by-name. Closures.
Lazy values. Top-level objects. Monadic comprehensions. Sugar for
properties. Sugar for apply() and update(). Extractors. Traits and
mix-ins. Implicit conversions and arguments. Named and default parameters.
These all can be seen as satisfying some classic use-case for macros, and
doing so more clearly and safely than the macro-based solution would.

If there's still a solid use-case for macros in Scala, that strikes me as an
excellent place to start thinking about extending the language. The only
one that's springing to mind for me is automatic delegation and run-time
mix-in, along the lines of the auto-proxy plugin.

--Dave Griffith

On Sat, Dec 11, 2010 at 12:36 PM, Dave Griffith
wrote:
>
>
> I always assumed the point of creating supple typeful languages was
> specifically to avoid the need for macro support.  Call-by-name.  Closures.
> Lazy values.  Top-level objects.  Monadic comprehensions.  Sugar for
> properties.  Sugar for apply() and update().  Extractors.  Traits and
> mix-ins.  Implicit conversions and arguments.  Named and default parameters.
> These all can be seen as satisfying some classic use-case for macros, and
> doing so more clearly and safely than the macro-based solution would.

The thing that from my mind makes a clear case for the utility of
macros is that Scala itself provides several facilities that could
potentially be expressed in a sufficiently powerful macro library, a
few being for comprehensions, case classes, and extractors. Macros can
be extremely useful in the case that you need to abstract not merely
over types, which Scala allows, but also over names, which Scala does
not. Consider, for example, something like ScalaQuery or Lift's Mapper
framework - the classes that are used to describe tables and columns
in these frameworks are exactly the sorts of things that I would like
to see produced by macros rather than by hand.

Kris

> If there's still a solid use-case for macros in Scala, that strikes me as an
> excellent place to start thinking about extending the language.   The only
> one that's springing to mind for me is automatic delegation and run-time
> mix-in, along the lines of the auto-proxy plugin.
>
> --Dave Griffith
> --
> View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Macro-support-in...
> Sent from the Scala mailing list archive at Nabble.com.
>

Oh, this looks interesting. Is there any chance that some of this ends
up in trunk?

Regards,
Ruediger

2010/12/11 Paul Phillips :
> On Sat, Dec 11, 2010 at 07:44:45PM +0100, Arno Haase wrote:
>> I would love to have something like Lisp macros is Scala, i.e. the
>> possibility to have code that transforms other code. And I plan to get
>> working on that. Is anyone aware of a similar effort?
>
> You will want to look at this.
>
> https://github.com/TiarkRompf/scala-virtualized/
>
> --
> Paul Phillips      | Christ died for our sins.  Dare we make his martyrdom
> Analgesic          | meaningless by not committing them?
> Empiricist         |     -- Jules Feiffer
> up hill, pi pals!  |----------* http://www.improving.org/paulp/ *----------
>

On Sat, Dec 11, 2010 at 08:37:21PM +0100, Ruediger Keller wrote:
> Oh, this looks interesting. Is there any chance that some of this ends
> up in trunk?

Here is the relevant paper. Based on the identity of the second author,
it's hard to rule out the possibility.

http://infoscience.epfl.ch/record/150347/files/gpce63-rompf.pdf

Well, try writing some code that will compile under 2.7 and 2.8 unchanged. :-)

On Sat, Dec 11, 2010 at 17:36, Dave Griffith <dave [dot] l [dot] griffith [at] gmail [dot] com> wrote:

I always assumed the point of creating supple typeful languages was
specifically to avoid the need for macro support.  Call-by-name.  Closures.
Lazy values.  Top-level objects.  Monadic comprehensions.  Sugar for
properties.  Sugar for apply() and update().  Extractors.  Traits and
mix-ins.  Implicit conversions and arguments.  Named and default parameters.
These all can be seen as satisfying some classic use-case for macros, and
doing so more clearly and safely than the macro-based solution would.

If there's still a solid use-case for macros in Scala, that strikes me as an
excellent place to start thinking about extending the language.   The only
one that's springing to mind for me is automatic delegation and run-time
mix-in, along the lines of the auto-proxy plugin.

--Dave Griffith
--
View this message in context: http://scala-programming-language.1934581.n4.nabble.com/Macro-support-in-Scala-tp3083592p3083626.html
Sent from the Scala mailing list archive at Nabble.com.

--
Daniel C. Sobral

I travel to the future all the time.

On 2010, Dec 11, at 8:50 PM, Daniel Sobral wrote:
> Well, try writing some code that will compile under 2.7 and 2.8 unchanged. :-)

I can't really see any situation where I would want to do that unless I was trying to write a scala library and doing governance by macro, but I'll bite:

Are you suggesting scala needs macros because the macro language would become the language which would end the need for all further language evolution, or are you suggesting that the macros would be a least common denominator that would work in all languages whatever future enhancements Martin and the rest of the other language designers are able to come up with?

I think you need to choose. You probably can't have it both ways.

yours
Geir

Racket has the most advanced macro system AFAIK. You might look at
Ryan Culpepper's dissertation (syntax-parse) for inspiration on how to
present macros to the user. Also check out some of the earlier papers
on things like the module system. Now in Scheme/Racket the regularity
of the syntax makes the parsing aspects of macros easy. In infix
languages building an extensible parser is much harder. Typically
people punt and require macros to be marked in code with special
delimiters. This is the Template Haskell solution IIRC. I'm not sure
what the staged computation guys do. Some macro facility certainly
would be useful in Scala.

Cheers,
N.

On Sat, Dec 11, 2010 at 6:44 PM, Arno Haase
wrote:
> I would love to have something like Lisp macros is Scala, i.e. the
> possibility to have code that transforms other code. And I plan to get
> working on that. Is anyone aware of a similar effort?
...
> Any thoughts or ideas on this?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Wow, this looks amazing - thank you for pointing me to it!

I feel pretty humbled by the sophistication of the paper and the
comments to my post, and maybe the LMS approach also covers this - but
what I have in mind is to provide a way to add something like the
@BeanProperty annotation as part of a library rather than a compiler
plugin.

These kind of convenience annotations to reduce boilerplate are
sprouting all over the place, and I find them pretty useful myself - for
example I wrote a plugin for an @Cached annotation that provided very
specific, scoped caching that was very useful for an app I was working on.

Probably stuff like LINQ or even Rails could also be built on top of
what I have in mind.

So basically I would like to provide a simple way to write-your-own AST
transformations based on annotations - with an API that lives entirely
in the library path and is significantly simpler and less error prone to
use than actual plugin development but still covers many useful
scenarios (but of course not all).

Does that sound useful to others besides myself? Suggestions?

- - Arno

Am 11.12.2010 20:56, schrieb Paul Phillips:
> On Sat, Dec 11, 2010 at 08:37:21PM +0100, Ruediger Keller wrote:
>> Oh, this looks interesting. Is there any chance that some of this ends
>> up in trunk?
>
> Here is the relevant paper. Based on the identity of the second author,
> it's hard to rule out the possibility.
>
> http://infoscience.epfl.ch/record/150347/files/gpce63-rompf.pdf
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk0EDG0ACgkQbmZsMyUPuXS86wCg1yJ76WxbzcFdbuk0V0CchSdr
G5YAoOw/5tZ+D9gIoVgv06bs3Cvdrv+N
=vBau
-----END PGP SIGNATURE-----

On 11 Dec 2010 23:42, "Arno Haase" <arno [dot] haase [at] haase-consulting [dot] com> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Wow, this looks amazing - thank you for pointing me to it!
>
> I feel pretty humbled by the sophistication of the paper and the
> comments to my post, and maybe the LMS approach also covers this - but
> what I have in mind is to provide a way to add something like the
> @BeanProperty annotation as part of a library rather than a compiler
> plugin.
>
> These kind of convenience annotations to reduce boilerplate are
> sprouting all over the place, and I find them pretty useful myself - for
> example I wrote a plugin for an @Cached annotation that provided very
> specific, scoped caching that was very useful for an app I was working on.
>
> Probably stuff like LINQ or even Rails could also be built on top of
> what I have in mind.
>
> So basically I would like to provide a simple way to write-your-own AST
> transformations based on annotations - with an API that lives entirely
> in the library path and is significantly simpler and less error prone to
> use than actual plugin development but still covers many useful
> scenarios (but of course not all).
>
> Does that sound useful to others besides myself? Suggestions?
>
> - - Arno
>

It's a great idea, and one I'm also pursuing.  The problem is where you need to do any sort of method synthesis that requires type info.

Because after the typer phase, compilation has already failed any code that tried to use the as-yet ungenerated method...

> Am 11.12.2010 20:56, schrieb Paul Phillips:
> > On Sat, Dec 11, 2010 at 08:37:21PM +0100, Ruediger Keller wrote:
> >> Oh, this looks interesting. Is there any chance that some of this ends
> >> up in trunk?
> >
> > Here is the relevant paper.  Based on the identity of the second author,
> > it's hard to rule out the possibility.
> >
> >   http://infoscience.epfl.ch/record/150347/files/gpce63-rompf.pdf
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk0EDG0ACgkQbmZsMyUPuXS86wCg1yJ76WxbzcFdbuk0V0CchSdr
> G5YAoOw/5tZ+D9gIoVgv06bs3Cvdrv+N
> =vBau
> -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>> It's a great idea, and one I'm also pursuing. The problem is where you
>> need to do any sort of method synthesis that requires type info.
>
>> Because after the typer phase, compilation has already failed any code
>> that tried to use the as-yet ungenerated method...

Yes, I also think that's the big obstacle. Have you had any ideas for
(partial) solutions?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk0EEwsACgkQbmZsMyUPuXQzTgCfUeO1fFrKDQIzCQctftCxuKEF
EXcAoJ0rvFl7FdEG4Spcp4Sj/RgzOzW0
=w+Zk
-----END PGP SIGNATURE-----

On 12 Dec 2010 00:10, "Arno Haase" <arno [dot] haase [at] haase-consulting [dot] com> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> >> It's a great idea, and one I'm also pursuing.  The problem is where you
> >> need to do any sort of method synthesis that requires type info.
> >
> >> Because after the typer phase, compilation has already failed any code
> >> that tried to use the as-yet ungenerated method...
>
> Yes, I also think that's the big obstacle. Have you had any ideas for
> (partial) solutions?

Several, but none of them has yet worked well enough for me to be happy.

> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk0EEwsACgkQbmZsMyUPuXQzTgCfUeO1fFrKDQIzCQctftCxuKEF
> EXcAoJ0rvFl7FdEG4Spcp4Sj/RgzOzW0
> =w+Zk
> -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>> Yes, I also think that's the big obstacle. Have you had any ideas for
>> (partial) solutions?
>
> Several, but none of them has yet worked well enough for me to be happy.

Do you have any experience with creating the synthetic defs before
namer/typer based on a simple dummy expression that has the right type
and replacing them afterwards with the real implementation?

The bigger obstacle seems to be resolving the marker annotations before
namer/typer.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk0EkUEACgkQbmZsMyUPuXSfNQCg9LRoficqhq92dy9t18K5qhqr
EvcAoNh8u53LlSfckw4fAleWbl2SBr76
=+GwC
-----END PGP SIGNATURE-----

On Sat, Dec 11, 2010 at 8:43 PM, Kris Nuttycombe <kris [dot] nuttycombe [at] gmail [dot] com> wrote:

On Sat, Dec 11, 2010 at 12:36 PM, Dave Griffith
<dave [dot] l [dot] griffith [at] gmail [dot] com> wrote:
>
>
> I always assumed the point of creating supple typeful languages was
> specifically to avoid the need for macro support.  Call-by-name.  Closures.
> Lazy values.  Top-level objects.  Monadic comprehensions.  Sugar for
> properties.  Sugar for apply() and update().  Extractors.  Traits and
> mix-ins.  Implicit conversions and arguments.  Named and default parameters.
> These all can be seen as satisfying some classic use-case for macros, and
> doing so more clearly and safely than the macro-based solution would.

The thing that from my mind makes a clear case for the utility of
macros is that Scala itself provides several facilities that could
potentially be expressed in a sufficiently powerful macro library, a
few being for comprehensions, case classes, and extractors.

I agree that for comprehensions could be implemented in a (hygienic)
macro system. Case classes no, because their expansion is type dependent (for instance an equals method is generated only if no other user-defined equals method is defined or inherited). Extractors could possibly be shortened by a macro system, but they do not entail any code generated themselves.

Cheers

 -- Martin

On Sat, Dec 11, 2010 at 8:37 PM, Ruediger Keller <ruediger [at] rk42 [dot] de> wrote:

Oh, this looks interesting. Is there any chance that some of this ends
up in trunk?

It's well possible, but no commitments yet. We want to get more experience with this first. -- Martin
 

Regards,
Ruediger


2010/12/11 Paul Phillips <paulp [at] improving [dot] org>:
> On Sat, Dec 11, 2010 at 07:44:45PM +0100, Arno Haase wrote:
>> I would love to have something like Lisp macros is Scala, i.e. the
>> possibility to have code that transforms other code. And I plan to get
>> working on that. Is anyone aware of a similar effort?
>
> You will want to look at this.
>
> https://github.com/TiarkRompf/scala-virtualized/
>
> --
> Paul Phillips      | Christ died for our sins.  Dare we make his martyrdom
> Analgesic          | meaningless by not committing them?
> Empiricist         |     -- Jules Feiffer
> up hill, pi pals!  |----------* http://www.improving.org/paulp/ *----------
>