This page is no longer maintained — Please continue to the home page at www.scala-lang.org

A standard source code format - a very rough sketch

46 replies
bjohanns
Joined: 2009-10-23,
User offline. Last seen 1 year 37 weeks ago.

Hello list,

just while thinking about sip-12 it occurred to me that it might be
nice to have the possibility to store a given source code in a way
that is independent of its formatting.

Why? Because it would be nice to be able to reformat a given source
code to "my" taste without triggering all the diff gizmoz around. I
see that such a functionality will require special handling by the IDE
(firstplace).

So the roundtrip would be:

-> IDE + Compiler => => VCS => IDE + Compiler
+ reformatter =>

Whether this is simply a "defined" source code
format which took away any optional formattings (e.g. remove all
semicolons, brackets, whitespace etc. where possible) or if it would
be some kind of AST-XML which allows to reconstruct the source code
entirely (inclusive comments) is a free decision.

This is a feature I (among others) always miss. I've come across many
source files (in different languages) that would benefit from a
reformat. But this always triggers the discussion about the change
history [oh no, you can no longer diff them with one of the previous
versions - don't do that!].

Here scala could take the lead by enabling such a "standard repository
format for sourcecode" in eclipse.

[I post this here and not in tools because it seems to me that the
compiler is heavily involved in source code handling / interpretation
and might be the first place to enable such standardization
operations]

Just me... dreaming...
Bernd

Russ P.
Joined: 2009-01-31,
User offline. Last seen 1 year 26 weeks ago.
Re: A standard source code format - a very rough sketch
Seems like a good idea to me. It would allow you to take anyone's code and automatically reformat it to your own taste without causing superficial diff and VCS problems. Since I'm one of the few programmers who seems to know how to properly format code, I would love to have that capability! 8^) But it seems that you'd want a front end on your VCS to automatically convert to standard format, otherwise it would quickly become a hassle to do for every commit and dif. Is that what you have in mind?

--Russ P.


On Mon, Oct 17, 2011 at 11:03 PM, Bernd <bjohanns2002 [at] yahoo [dot] de> wrote:
Hello list,

just while thinking about sip-12 it occurred to me that it might be
nice to have the possibility to store a given source code in a way
that is independent of its formatting.

Why? Because it would be nice to be able to reformat a given source
code to "my" taste without triggering all the diff gizmoz around. I
see that such a functionality will require special handling by the IDE
(firstplace).

So the roundtrip would be:

<code> -> IDE + Compiler => <standard format> => VCS => IDE + Compiler
+ reformatter => <code>

Whether this <standard format> is simply a "defined" source code
format which took away any optional formattings (e.g. remove all
semicolons, brackets, whitespace etc. where possible) or if it would
be some kind of AST-XML which allows to reconstruct the source code
entirely (inclusive comments) is a free decision.

This is a feature I (among others) always miss. I've come across many
source files (in different languages) that would benefit from a
reformat. But this always triggers the discussion about the change
history [oh no, you can no longer diff them with one of the previous
versions - don't do that!].

Here scala could take the lead by enabling such a "standard repository
format for sourcecode" in eclipse.

[I post this here and not in tools because it seems to me that the
compiler is heavily involved in source code handling / interpretation
and might be the first place to enable such standardization
operations]

Just me... dreaming...
Bernd



--
http://RussP.us
Jason Zaugg
Joined: 2009-05-18,
User offline. Last seen 38 weeks 5 days ago.
Re: A standard source code format - a very rough sketch
On Tue, Oct 18, 2011 at 8:03 AM, Bernd <bjohanns2002 [at] yahoo [dot] de> wrote:
Hello list,

just while thinking about sip-12 it occurred to me that it might be
nice to have the possibility to store a given source code in a way
that is independent of its formatting.

Why? Because it would be nice to be able to reformat a given source
code to "my" taste without triggering all the diff gizmoz around. I
see that such a functionality will require special handling by the IDE
(firstplace).

So the roundtrip would be:

<code> -> IDE + Compiler => <standard format> => VCS => IDE + Compiler
+ reformatter => <code>

Whether this <standard format> is simply a "defined" source code
format which took away any optional formattings (e.g. remove all
semicolons, brackets, whitespace etc. where possible) or if it would
be some kind of AST-XML which allows to reconstruct the source code
entirely (inclusive comments) is a free decision.

This is a feature I (among others) always miss. I've come across many
source files (in different languages) that would benefit from a
reformat. But this always triggers the discussion about the change
history [oh no, you can no longer diff them with one of the previous
versions - don't do that!].

Here scala could take the lead by enabling such a "standard repository
format for sourcecode" in eclipse.

[I post this here and not in tools because it seems to me that the
compiler is heavily involved in source code handling / interpretation
and might be the first place to enable such standardization
operations]

You can already just pick a style for your project and format automatically with scalariform.
There is a SBT plugin [1] that can do this for you before each compile.
-jason
[1] https://github.com/typesafehub/sbt-scalariform
Jesper Nordenberg
Joined: 2008-12-27,
User offline. Last seen 42 years 45 weeks ago.
Re: A standard source code format - a very rough sketch

Bernd yahoo.de> writes:
> just while thinking about sip-12 it occurred to me that it might be
> nice to have the possibility to store a given source code in a way
> that is independent of its formatting.

It's a nice idea, but it won't work in practice I think. There are just too
many advantages in having the source code in a "standard" text format, you
can publish it on the web easily, diffs are easily readable etc. It's
too much work to create all the tools required to work with source code in
some other format. Having a standard format like the "Scala Style Guide" is
a reasonable compromise.

/Jesper Nordenberg

Philippe Lhoste
Joined: 2010-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: A standard source code format - a very rough sketch

On 18/10/2011 08:03, Bernd wrote:
> just while thinking about sip-12 it occurred to me that it might be
> nice to have the possibility to store a given source code in a way
> that is independent of its formatting.
>
> Why? Because it would be nice to be able to reformat a given source
> code to "my" taste without triggering all the diff gizmoz around. I
> see that such a functionality will require special handling by the IDE
> (firstplace).
>[...]
> This is a feature I (among others) always miss. I've come across many
> source files (in different languages) that would benefit from a
> reformat. But this always triggers the discussion about the change
> history [oh no, you can no longer diff them with one of the previous
> versions - don't do that!].

Interesting idea!
Where I work, sometime code reviewer is grumpy because I reformatted code to follow more
closely the corporate policy in terms of formatting.
Because it drowns the useful fix (perhaps two lines!) in a sea of meaningless diffs.
Meaningless, but useful! Sometime code is awful, because one forgot to set editor to ident
with spaces only (I prefer tabs, more flexible... but I follow policy, of course!), or
with the wrong amount, or because an automated merge messed up a bit the formatting.
Most of the case, ignore spaces setting in the diff program is enough to get rid of most
of it, but I also remove parentheses around return values (return (somevalue); makes my
eyes bleed!) or assert expression, etc. Or sometime I rename variables (you know, these
tempVal and similar), but then your proposal probably cannot do much.

Of course, then you need a special tool to do the diffs at the AST-XML (or whatever other
format) level.

Philippe Lhoste
Joined: 2010-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: A standard source code format - a very rough sketch

On 18/10/2011 09:41, Jason Zaugg wrote:
> You can already just pick a style for your project and format automatically with scalariform.

The point is that if you find a file not properly formatted by a peer, for example, and
reformat it, you want a zero diff result between the two files (before and after
reformatting).

Kevin Wright 2
Joined: 2010-05-30,
User offline. Last seen 26 weeks 4 days ago.
Re: Re: A standard source code format - a very rough sketch
It's an OLD idea, and one that never really gets off the ground (mostly for the reasons that Jesper listed).  Maybe its time will come, but we're still lacking a version control system that's designed to work in this way.
Try a query for "source code in database" or "seesoft" in your favourite web search engine.  You'll also find that a lot of the ideas are now being used in various static analysis tools.

On 18 October 2011 09:53, Philippe Lhoste <PhiLho [at] gmx [dot] net> wrote:
On 18/10/2011 08:03, Bernd wrote:
just while thinking about sip-12 it occurred to me that it might be
nice to have the possibility to store a given source code in a way
that is independent of its formatting.

Why? Because it would be nice to be able to reformat a given source
code to "my" taste without triggering all the diff gizmoz around. I
see that such a functionality will require special handling by the IDE
(firstplace).
[...]
This is a feature I (among others) always miss. I've come across many
source files (in different languages) that would benefit from a
reformat. But this always triggers the discussion about the change
history [oh no, you can no longer diff them with one of the previous
versions - don't do that!].

Interesting idea!
Where I work, sometime code reviewer is grumpy because I reformatted code to follow more closely the corporate policy in terms of formatting.
Because it drowns the useful fix (perhaps two lines!) in a sea of meaningless diffs.
Meaningless, but useful! Sometime code is awful, because one forgot to set editor to ident with spaces only (I prefer tabs, more flexible... but I follow policy, of course!), or with the wrong amount, or because an automated merge messed up a bit the formatting.
Most of the case, ignore spaces setting in the diff program is enough to get rid of most of it, but I also remove parentheses around return values (return (somevalue); makes my eyes bleed!) or assert expression, etc. Or sometime I rename variables (you know, these tempVal and similar), but then your proposal probably cannot do much.

Of course, then you need a special tool to do the diffs at the AST-XML (or whatever other format) level.

Nicolas 2
Joined: 2011-10-18,
User offline. Last seen 42 years 45 weeks ago.
Re: A standard source code format - a very rough sketch

Just my 2¢: how hard would it be to have scalariform run automatically
before any code commit?

This way, you don't loose the advantages of storing code in plain
text, you have nice looking code, and you avoid the diff problem
because not code gets in without being already properly formatted.

Is that feasible?

On Oct 18, 5:07 am, Kevin Wright wrote:
> It's an OLD idea, and one that never really gets off the ground (mostly for
> the reasons that Jesper listed).  Maybe its time will come, but we're still
> lacking a version control system that's designed to work in this way.
>
> Try a query for "source code in database" or "seesoft" in your favourite web
> search engine.  You'll also find that a lot of the ideas are now being used
> in various static analysis tools.
>
> On 18 October 2011 09:53, Philippe Lhoste wrote:
>
>
>
>
>
>
>
> > On 18/10/2011 08:03, Bernd wrote:
>
> >> just while thinking about sip-12 it occurred to me that it might be
> >> nice to have the possibility to store a given source code in a way
> >> that is independent of its formatting.
>
> >> Why? Because it would be nice to be able to reformat a given source
> >> code to "my" taste without triggering all the diff gizmoz around. I
> >> see that such a functionality will require special handling by the IDE
> >> (firstplace).
> >> [...]
>
> >> This is a feature I (among others) always miss. I've come across many
> >> source files (in different languages) that would benefit from a
> >> reformat. But this always triggers the discussion about the change
> >> history [oh no, you can no longer diff them with one of the previous
> >> versions - don't do that!].
>
> > Interesting idea!
> > Where I work, sometime code reviewer is grumpy because I reformatted code
> > to follow more closely the corporate policy in terms of formatting.
> > Because it drowns the useful fix (perhaps two lines!) in a sea of
> > meaningless diffs.
> > Meaningless, but useful! Sometime code is awful, because one forgot to set
> > editor to ident with spaces only (I prefer tabs, more flexible... but I
> > follow policy, of course!), or with the wrong amount, or because an
> > automated merge messed up a bit the formatting.
> > Most of the case, ignore spaces setting in the diff program is enough to
> > get rid of most of it, but I also remove parentheses around return values
> > (return (somevalue); makes my eyes bleed!) or assert expression, etc. Or
> > sometime I rename variables (you know, these tempVal and similar), but then
> > your proposal probably cannot do much.
>
> > Of course, then you need a special tool to do the diffs at the AST-XML (or
> > whatever other format) level.
>
> > --
> > Philippe Lhoste
> > --  (near) Paris -- France
> > --  http://Phi.Lho.free.fr
> > --  --  --  --  --  --  --  --  --  --  --  --  --  --

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Re: A standard source code format - a very rough sketch
What we need to do in the future is just deal directly with AST nodes in some visual fashion of our choice that doesn't have to be a textual representation....

- Josh

On Tue, Oct 18, 2011 at 1:20 PM, Nicolas <nicolaspayette [at] gmail [dot] com> wrote:
Just my 2¢: how hard would it be to have scalariform run automatically
before any code commit?

This way, you don't loose the advantages of storing code in plain
text, you have nice looking code, and you avoid the diff problem
because not code gets in without being already properly formatted.

Is that feasible?

On Oct 18, 5:07 am, Kevin Wright <kev [dot] lee [dot] wri [dot] [dot] [dot] [at] gmail [dot] com> wrote:
> It's an OLD idea, and one that never really gets off the ground (mostly for
> the reasons that Jesper listed).  Maybe its time will come, but we're still
> lacking a version control system that's designed to work in this way.
>
> Try a query for "source code in database" or "seesoft" in your favourite web
> search engine.  You'll also find that a lot of the ideas are now being used
> in various static analysis tools.
>
> On 18 October 2011 09:53, Philippe Lhoste <Phi [dot] [dot] [dot] [at] gmx [dot] net> wrote:
>
>
>
>
>
>
>
> > On 18/10/2011 08:03, Bernd wrote:
>
> >> just while thinking about sip-12 it occurred to me that it might be
> >> nice to have the possibility to store a given source code in a way
> >> that is independent of its formatting.
>
> >> Why? Because it would be nice to be able to reformat a given source
> >> code to "my" taste without triggering all the diff gizmoz around. I
> >> see that such a functionality will require special handling by the IDE
> >> (firstplace).
> >> [...]
>
> >> This is a feature I (among others) always miss. I've come across many
> >> source files (in different languages) that would benefit from a
> >> reformat. But this always triggers the discussion about the change
> >> history [oh no, you can no longer diff them with one of the previous
> >> versions - don't do that!].
>
> > Interesting idea!
> > Where I work, sometime code reviewer is grumpy because I reformatted code
> > to follow more closely the corporate policy in terms of formatting.
> > Because it drowns the useful fix (perhaps two lines!) in a sea of
> > meaningless diffs.
> > Meaningless, but useful! Sometime code is awful, because one forgot to set
> > editor to ident with spaces only (I prefer tabs, more flexible... but I
> > follow policy, of course!), or with the wrong amount, or because an
> > automated merge messed up a bit the formatting.
> > Most of the case, ignore spaces setting in the diff program is enough to
> > get rid of most of it, but I also remove parentheses around return values
> > (return (somevalue); makes my eyes bleed!) or assert expression, etc. Or
> > sometime I rename variables (you know, these tempVal and similar), but then
> > your proposal probably cannot do much.
>
> > Of course, then you need a special tool to do the diffs at the AST-XML (or
> > whatever other format) level.
>
> > --
> > Philippe Lhoste
> > --  (near) Paris -- France
> > --  http://Phi.Lho.free.fr
> > --  --  --  --  --  --  --  --  --  --  --  --  --  --

Kevin Wright 2
Joined: 2010-05-30,
User offline. Last seen 26 weeks 4 days ago.
Re: Re: A standard source code format - a very rough sketch
Only if you have never, *ever* manually formatted a tricky bit of code to make it more readable, then seen all that work ruined by some heavy-handed lout with their IDE's auto-formatter.
I work on a policy of writing all my code as though it's about to be published in a magazine, and know that auto-formatters can never get it quite right.  The only thing I'll use scalariform for is converting ascii arrows to their unicode equivalents.  Especially since moving from linux=> Mac and losing my beloved compose key.
Scalariform would also be fine as part of a diff tool, though I really wouldn't want to see those changes then committed, or overwriting the files that I originally saved.  FWIW, I feel that an AST-aware diff tool that makes use of the presentation compiler would be better still.


On 18 October 2011 18:20, Nicolas <nicolaspayette [at] gmail [dot] com> wrote:
Just my 2¢: how hard would it be to have scalariform run automatically
before any code commit?

This way, you don't loose the advantages of storing code in plain
text, you have nice looking code, and you avoid the diff problem
because not code gets in without being already properly formatted.

Is that feasible?

On Oct 18, 5:07 am, Kevin Wright <kev [dot] lee [dot] wri [dot] [dot] [dot] [at] gmail [dot] com> wrote:
> It's an OLD idea, and one that never really gets off the ground (mostly for
> the reasons that Jesper listed).  Maybe its time will come, but we're still
> lacking a version control system that's designed to work in this way.
>
> Try a query for "source code in database" or "seesoft" in your favourite web
> search engine.  You'll also find that a lot of the ideas are now being used
> in various static analysis tools.
>
> On 18 October 2011 09:53, Philippe Lhoste <Phi [dot] [dot] [dot] [at] gmx [dot] net> wrote:
>
>
>
>
>
>
>
> > On 18/10/2011 08:03, Bernd wrote:
>
> >> just while thinking about sip-12 it occurred to me that it might be
> >> nice to have the possibility to store a given source code in a way
> >> that is independent of its formatting.
>
> >> Why? Because it would be nice to be able to reformat a given source
> >> code to "my" taste without triggering all the diff gizmoz around. I
> >> see that such a functionality will require special handling by the IDE
> >> (firstplace).
> >> [...]
>
> >> This is a feature I (among others) always miss. I've come across many
> >> source files (in different languages) that would benefit from a
> >> reformat. But this always triggers the discussion about the change
> >> history [oh no, you can no longer diff them with one of the previous
> >> versions - don't do that!].
>
> > Interesting idea!
> > Where I work, sometime code reviewer is grumpy because I reformatted code
> > to follow more closely the corporate policy in terms of formatting.
> > Because it drowns the useful fix (perhaps two lines!) in a sea of
> > meaningless diffs.
> > Meaningless, but useful! Sometime code is awful, because one forgot to set
> > editor to ident with spaces only (I prefer tabs, more flexible... but I
> > follow policy, of course!), or with the wrong amount, or because an
> > automated merge messed up a bit the formatting.
> > Most of the case, ignore spaces setting in the diff program is enough to
> > get rid of most of it, but I also remove parentheses around return values
> > (return (somevalue); makes my eyes bleed!) or assert expression, etc. Or
> > sometime I rename variables (you know, these tempVal and similar), but then
> > your proposal probably cannot do much.
>
> > Of course, then you need a special tool to do the diffs at the AST-XML (or
> > whatever other format) level.
>
Razvan Cojocaru 3
Joined: 2010-07-28,
User offline. Last seen 42 years 45 weeks ago.
RE: Re: A standard source code format - a very rough sketch

And you guys are certain this tool could easily support git/cvs/svn/mercurial/unix diffs, merging etc, including line changes and so on?

 

It would be interesting, certainly… although if I use this for a DSL to which I have access only remotely with say a vi (it happens more often than you’d think in certain lines of business) I would obviously default back to “normal” conventions and reviews…

 

From: scala-debate [at] googlegroups [dot] com [mailto:scala-debate [at] googlegroups [dot] com] On Behalf Of Kevin Wright
Sent: October-18-11 1:41 PM
To: Nicolas
Cc: scala-debate
Subject: Re: [scala-debate] Re: A standard source code format - a very rough sketch

 

Only if you have never, *ever* manually formatted a tricky bit of code to make it more readable, then seen all that work ruined by some heavy-handed lout with their IDE's auto-formatter.


I work on a policy of writing all my code as though it's about to be published in a magazine, and know that auto-formatters can never get it quite right.  The only thing I'll use scalariform for is converting ascii arrows to their unicode equivalents.  Especially since moving from linux=> Mac and losing my beloved compose key.


Scalariform would also be fine as part of a diff tool, though I really wouldn't want to see those changes then committed, or overwriting the files that I originally saved.  FWIW, I feel that an AST-aware diff tool that makes use of the presentation compiler would be better still.

On 18 October 2011 18:20, Nicolas <nicolaspayette [at] gmail [dot] com> wrote:

Just my 2¢: how hard would it be to have scalariform run automatically
before any code commit?

This way, you don't loose the advantages of storing code in plain
text, you have nice looking code, and you avoid the diff problem
because not code gets in without being already properly formatted.

Is that feasible?


On Oct 18, 5:07 am, Kevin Wright <kev [dot] lee [dot] wri [dot] [dot] [dot] [at] gmail [dot] com> wrote:
> It's an OLD idea, and one that never really gets off the ground (mostly for
> the reasons that Jesper listed).  Maybe its time will come, but we're still
> lacking a version control system that's designed to work in this way.
>
> Try a query for "source code in database" or "seesoft" in your favourite web
> search engine.  You'll also find that a lot of the ideas are now being used
> in various static analysis tools.
>

> On 18 October 2011 09:53, Philippe Lhoste <Phi [dot] [dot] [dot] [at] gmx [dot] net> wrote:
>
>
>
>
>
>
>
> > On 18/10/2011 08:03, Bernd wrote:
>
> >> just while thinking about sip-12 it occurred to me that it might be
> >> nice to have the possibility to store a given source code in a way
> >> that is independent of its formatting.
>
> >> Why? Because it would be nice to be able to reformat a given source
> >> code to "my" taste without triggering all the diff gizmoz around. I
> >> see that such a functionality will require special handling by the IDE
> >> (firstplace).
> >> [...]
>
> >> This is a feature I (among others) always miss. I've come across many
> >> source files (in different languages) that would benefit from a
> >> reformat. But this always triggers the discussion about the change
> >> history [oh no, you can no longer diff them with one of the previous
> >> versions - don't do that!].
>
> > Interesting idea!
> > Where I work, sometime code reviewer is grumpy because I reformatted code
> > to follow more closely the corporate policy in terms of formatting.
> > Because it drowns the useful fix (perhaps two lines!) in a sea of
> > meaningless diffs.
> > Meaningless, but useful! Sometime code is awful, because one forgot to set
> > editor to ident with spaces only (I prefer tabs, more flexible... but I
> > follow policy, of course!), or with the wrong amount, or because an
> > automated merge messed up a bit the formatting.
> > Most of the case, ignore spaces setting in the diff program is enough to
> > get rid of most of it, but I also remove parentheses around return values
> > (return (somevalue); makes my eyes bleed!) or assert expression, etc. Or
> > sometime I rename variables (you know, these tempVal and similar), but then
> > your proposal probably cannot do much.
>
> > Of course, then you need a special tool to do the diffs at the AST-XML (or
> > whatever other format) level.
>

Bernd Johannes
Joined: 2011-01-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: A standard source code format - a very rough sketch

> Bernd yahoo.de> writes:
> > just while thinking about sip-12 it occurred to me that it might be
> > nice to have the possibility to store a given source code in a way
> > that is independent of its formatting.
>
> It's a nice idea, but it won't work in practice I think. There are just too
> many advantages in having the source code in a "standard" text format, you
> can publish it on the web easily, diffs are easily readable etc. It's
> too much work to create all the tools required to work with source code in
> some other format. Having a standard format like the "Scala Style Guide" is
> a reasonable compromise.
>
> /Jesper Nordenberg

The poor man's solution could be to have an option in eclipse preferences
"save standardized"

this would cause eclipse to do the following:

1. on store operations send the code to the compiler and ask it to perform a
standardization (yielding still perfectly readable ASCII source code but in a
completely defined format WITHOUT any option whatsoever to tweek things around
[which would defeat "standard"]).

=> so any tool "down the chain" can still cope with the content and present
reasonable results.

2. on load operations eclipse would take the standard formatted code and
execute its own formatter before presenting it to the user thus reverting the
standard format into the user preferred format

The immediate benefit is to be freed from the "reformat lock" which is
introduced by any VCS. And even more - everybody could at least work in that
formatting style that fits the bill best.

As scala has still the some avantgarde air to it I have the impression that
this feature could still be "right in time".

Greetings
Bernd

Bernd Johannes
Joined: 2011-01-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: A standard source code format - a very rough sketch

Am Dienstag, 18. Oktober 2011, 11:07:07 schrieb Kevin Wright:
> It's an OLD idea, and one that never really gets off the ground (mostly for
> the reasons that Jesper listed). Maybe its time will come, but we're still
> lacking a version control system that's designed to work in this way.
>
> Try a query for "source code in database" or "seesoft" in your favourite
> web search engine. You'll also find that a lot of the ideas are now being
> used in various static analysis tools.
>
I agree - while probably very useful something like an AST-XML or other fancy
formats won't be well met with current VCS infrastructures.

Unfortunatelly there is (still) no VCS concept around (at least I don't know
of any) that would permit the management of a "source" and "rendition of the
source" in a useable way.
In DMS context this is commonplace - we have all types of fancy input formats
- which get archived "as is" as it is the original, but the user most often
deals with PDF renditions of the source (ok, this solves another problem - but
its a simple example for source and rendition management).

Maybe when "the time is right" all will fall into place. But the option (of
the scala compiler) to import and export XML AST representations of a given
source could spark a flame to that candle.

Still dreaming... :-)
Greetings
Bernd

Bernd Johannes
Joined: 2011-01-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: A standard source code format - a very rough sketch

Am Dienstag, 18. Oktober 2011, 19:40:38 schrieb Kevin Wright:
> Only if you have never, *ever* manually formatted a tricky bit of code to
> make it more readable, then seen all that work ruined by some heavy-handed
> lout with their IDE's auto-formatter.

I agree that sometimes (or even "often") the autoformatting doesn't get even
close to be beautiful. But I am used to environments where even some crude
reformatting is actually better than the original sourcecode. So I think this
is a balanced decision.
In environments with "code owners" where somebody impersonates "the codebase"
or at least a part of it and all the various policies around it, you most
probably do not want to turn on the "standardize on save" feature.
But in environments where coder change quicker than the specs and most of the
code (here: java) is obviously copied and pasted together from g**gle without
any second thought to formatting - such a feature would be at least a small
step toward quality and maintainability. Here I would hit that switch for
sure.

> that I originally saved. FWIW, I feel that an AST-aware diff tool that
> makes use of the presentation compiler would be better still.

I see the potential of that - we could even alias name changes in some
circumstances - and moving some functions / procedures in the sourcecode
around woud do no harm. We could even recreate the source code according to
some "top down" or "bottom up" policies (most elemental functional bodies
first or last - or sort alphabetical - why not?). It could be amazing.

But currently I think that our tooling is simply not good enough to get along
such things. And I got the impression that Linus is not very inclined to
enhance GIT to adapt to some "high level code formatting" desires of people
that use languages that don't use * ;-)

Just my 6 cents
Greetings
Bernd

Bernd Johannes
Joined: 2011-01-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: A standard source code format - a very rough sketch

Am Dienstag, 18. Oktober 2011, 10:53:37 schrieb Philippe Lhoste:
[...]
> Most
> of the case, ignore spaces setting in the diff program is enough to get
> rid of most of it, but I also remove parentheses around return values
> (return (somevalue); makes my eyes bleed!) or assert expression, etc. Or
> sometime I rename variables (you know, these tempVal and similar), but
> then your proposal probably cannot do much.

When looking at the AST I think one could even identify and alias simple name
changes.

But a real standardization (speaking of real code representations in the VCS
again - no AST XMLs) should also mangle the source code very thouroughly to
get rid of any optional features which could be one way or the other - either
by deleting them or by adding them consistently (e.g. semicola, brackets,
whitespaces).

This would go clearly beyond simple whitespace reformatting. Adding or
removing brakets is a bit more than "simple reformatting". The list could go
further - e.g. automagically make explicit any infered return type of
functions (I here you scream ;-) or more fancy stuff.

But there is also a price to pay: anything that would get hit by the
standardization (when using simple unannotated ASCII source code) could only
be recreated/reformatted in a consistent way. E.g. either all your optional
semicolons are there or none of them (but you could choose - do I want them or
not). The same would apply for brackets around if (expressions) or return()
:types of functions or optional brackets on function() calls or optional
{function/if/else block parens} and so on.

This would be a good deal more than just reformatting...

>
> Of course, then you need a special tool to do the diffs at the AST-XML (or
> whatever other format) level.

...I don't see it in the near future - I think the scalarians have more
pressing business, but it would be NICE!!!

Just my 6 cents
Greetings
Bernd

Derek Williams 3
Joined: 2011-08-12,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: A standard source code format - a very rough sketch


On Oct 18, 2011 11:20 AM, "Nicolas" <nicolaspayette [at] gmail [dot] com> wrote:
>
> Just my 2¢: how hard would it be to have scalariform run automatically
> before any code commit?
>
> This way, you don't loose the advantages of storing code in plain
> text, you have nice looking code, and you avoid the diff problem
> because not code gets in without being already properly formatted.
>
> Is that feasible?

Akka automatically does this before compiling using the sbt scalariform plugin. On my phone at the moment so can't give you a link, but you can find it on github.

Nicolas 2
Joined: 2011-10-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: A standard source code format - a very rough sketch

> Akka automatically does this before compiling using the sbt scalariform
> plugin. On my phone at the moment so can't give you a link, but you can find
> it on github.

That's nice! But I was thinking more along the lines of having your
VCS centrally handle it, this way you don't have to rely on individual
developers having the right configuration. Maybe it's already
possible. I haven't done the research.

And regarding the issue mentionned by Kevin Wright of scalariform
doing a bad job on some tricky code, well, there is always the option
of excluding specific parts of code:
http://mdr.github.com/scalariform/#source-directives

dcsobral
Joined: 2009-04-23,
User offline. Last seen 38 weeks 5 days ago.
Re: Re: A standard source code format - a very rough sketch

On Tue, Oct 18, 2011 at 15:20, Nicolas wrote:
> Just my 2¢: how hard would it be to have scalariform run automatically
> before any code commit?

With git, it is trivial:

http://www.blog.project13.pl/index.php/fun/1076/git-hacking-pre-commit-h...
http://progit.org/book/ch7-3.html

>
> This way, you don't loose the advantages of storing code in plain
> text, you have nice looking code, and you avoid the diff problem
> because not code gets in without being already properly formatted.
>
> Is that feasible?
>
> On Oct 18, 5:07 am, Kevin Wright wrote:
>> It's an OLD idea, and one that never really gets off the ground (mostly for
>> the reasons that Jesper listed).  Maybe its time will come, but we're still
>> lacking a version control system that's designed to work in this way.
>>
>> Try a query for "source code in database" or "seesoft" in your favourite web
>> search engine.  You'll also find that a lot of the ideas are now being used
>> in various static analysis tools.
>>
>> On 18 October 2011 09:53, Philippe Lhoste wrote:
>>
>>
>>
>>
>>
>>
>>
>> > On 18/10/2011 08:03, Bernd wrote:
>>
>> >> just while thinking about sip-12 it occurred to me that it might be
>> >> nice to have the possibility to store a given source code in a way
>> >> that is independent of its formatting.
>>
>> >> Why? Because it would be nice to be able to reformat a given source
>> >> code to "my" taste without triggering all the diff gizmoz around. I
>> >> see that such a functionality will require special handling by the IDE
>> >> (firstplace).
>> >> [...]
>>
>> >> This is a feature I (among others) always miss. I've come across many
>> >> source files (in different languages) that would benefit from a
>> >> reformat. But this always triggers the discussion about the change
>> >> history [oh no, you can no longer diff them with one of the previous
>> >> versions - don't do that!].
>>
>> > Interesting idea!
>> > Where I work, sometime code reviewer is grumpy because I reformatted code
>> > to follow more closely the corporate policy in terms of formatting.
>> > Because it drowns the useful fix (perhaps two lines!) in a sea of
>> > meaningless diffs.
>> > Meaningless, but useful! Sometime code is awful, because one forgot to set
>> > editor to ident with spaces only (I prefer tabs, more flexible... but I
>> > follow policy, of course!), or with the wrong amount, or because an
>> > automated merge messed up a bit the formatting.
>> > Most of the case, ignore spaces setting in the diff program is enough to
>> > get rid of most of it, but I also remove parentheses around return values
>> > (return (somevalue); makes my eyes bleed!) or assert expression, etc. Or
>> > sometime I rename variables (you know, these tempVal and similar), but then
>> > your proposal probably cannot do much.
>>
>> > Of course, then you need a special tool to do the diffs at the AST-XML (or
>> > whatever other format) level.
>>
>> > --
>> > Philippe Lhoste
>> > --  (near) Paris -- France
>> > --  http://Phi.Lho.free.fr
>> > --  --  --  --  --  --  --  --  --  --  --  --  --  --

Randall R Schulz
Joined: 2008-12-16,
User offline. Last seen 1 year 29 weeks ago.
Re: A standard source code format - a very rough sketch

On Tuesday 18 October 2011, Jason Zaugg wrote:
> ...
>
> You can already just pick a style for your project and format
> automatically with scalariform.

The program that formats code the way I do does not exist.

> -jason
>
> [1] https://github.com/typesafehub/sbt-scalariform

Randall Schulz

Randall R Schulz
Joined: 2008-12-16,
User offline. Last seen 1 year 29 weeks ago.
Re: Re: A standard source code format - a very rough sketch

On Tuesday 18 October 2011, Nicolas wrote:
> Just my 2¢: how hard would it be to have scalariform run
> automatically before any code commit?
>
> This way, you don't loose the advantages of storing code in plain
> text, you have nice looking code, and you avoid the diff problem
> because not code gets in without being already properly formatted.
>
> Is that feasible?

Why are people so obsessed with eradicating individualism in source
code?

What does "style" mean?

Randall Schulz

Randall R Schulz
Joined: 2008-12-16,
User offline. Last seen 1 year 29 weeks ago.
Re: Re: A standard source code format - a very rough sketch

On Tuesday 18 October 2011, Kevin Wright wrote:
> Only if you have never, *ever* manually formatted a tricky bit of
> code to make it more readable, then seen all that work ruined by some
> heavy-handed lout with their IDE's auto-formatter.
>
> I work on a policy of writing all my code as though it's about to be
> published in a magazine, and know that auto-formatters can never get
> it quite right. The only thing I'll use scalariform for is
> converting ascii arrows to their unicode equivalents. Especially
> since moving from linux=> Mac and losing my beloved compose key.
>
> Scalariform would also be fine as part of a diff tool, though I
> really wouldn't want to see those changes then committed, or
> overwriting the files that I originally saved. FWIW, I feel that an
> AST-aware diff tool that makes use of the presentation compiler would
> be better still.
>
> ...

Oh, my god. THANK YOU!

I reject the notion that programmers should be allowed to treat every
single white-space choice as entirely arbitrary and random, relying on
some algorithm to put the result of the resulting mess into a readable
format.

Randall Schulz

Tony Morris 2
Joined: 2009-03-20,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: A standard source code format - a very rough sketch

On 10/19/2011 03:06 PM, Randall R Schulz wrote:
> On Tuesday 18 October 2011, Nicolas wrote:
>> Just my 2¢: how hard would it be to have scalariform run
>> automatically before any code commit?
>>
>> This way, you don't loose the advantages of storing code in plain
>> text, you have nice looking code, and you avoid the diff problem
>> because not code gets in without being already properly formatted.
>>
>> Is that feasible?
> Why are people so obsessed with eradicating individualism in source
> code?
>
> What does "style" mean?
>
>
> Randall Schulz

I think this is an awesome question, but one for which there is so
little data available.

If we could have field experts (to demonstrate that this compulsion
works against self-interest and group-interest) working alongside
experts in neuroscience to work out why this behaviour persists anyway,
maybe we can could get a reasonable answer.

I hope this happens in my life-time. Thanks for hearing my lament.

Philippe Lhoste
Joined: 2010-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: A standard source code format - a very rough sketch

On 18/10/2011 19:40, Kevin Wright wrote:
> Only if you have never, *ever* manually formatted a tricky bit of code to make it more
> readable, then seen all that work ruined by some heavy-handed lout with their IDE's
> auto-formatter.

I agree. I only used the Java formatter of Eclipse, but if it is good for quickly putting
a foreign code into rough shape following our policies, I often want to fix some
formatting details, and I don't want to do it on each save!

Philippe Lhoste
Joined: 2010-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: A standard source code format - a very rough sketch

On 19/10/2011 07:06, Randall R Schulz wrote:
> Why are people so obsessed with eradicating individualism in source
> code?
>
> What does "style" mean?

You never saw code produced by newbies (at programming in general) in the Processing
forum! At best, you have no indentation at all. At worst, the indentation is so random, it
makes the code quite unreadable. Inconsistent indentation can even trick in believing the
code is doing something else than it really does, if you are not attentive enough (often
with if/else without braces).
And, well, some regularity in the way to write code can help in reading. If you meet

if (foo) {
do();
}
if( bar ){
stuff ( ); }
if (doh )
{
raz();
}

in the same file, you might cringe (I do!).

Kevin Wright 2
Joined: 2010-05-30,
User offline. Last seen 26 weeks 4 days ago.
Re: Re: A standard source code format - a very rough sketch
Totally.  I have no issue running a formatter over something which is such a mess that I just can't figure it out.  Even then I tend to be specific, just fixing the indentation for example, and not a full format wich mucks about the line breaks between parameters, etc, etc.
I have *never* once seen a formatter produce code which doesn't then need some manual tweaking.  So the thought of this happening automatically, outside of my control, on every single commit just fills me with horror.


On 19 October 2011 09:09, Philippe Lhoste <PhiLho [at] gmx [dot] net> wrote:
On 19/10/2011 07:06, Randall R Schulz wrote:
Why are people so obsessed with eradicating individualism in source
code?

What does "style" mean?

You never saw code produced by newbies (at programming in general) in the Processing forum! At best, you have no indentation at all. At worst, the indentation is so random, it makes the code quite unreadable. Inconsistent indentation can even trick in believing the code is doing something else than it really does, if you are not attentive enough (often with if/else without braces).
And, well, some regularity in the way to write code can help in reading. If you meet

if (foo) {
 do();
}
if( bar ){
 stuff ( ); }
if  (doh   )
{
raz();
}

in the same file, you might cringe (I do!).

Philippe Lhoste
Joined: 2010-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: A standard source code format - a very rough sketch

On 19/10/2011 10:25, Kevin Wright wrote:
> Totally. I have no issue running a formatter over something which is such a mess that I
> just can't figure it out. Even then I tend to be specific, just fixing the indentation
> for example, and not a full format wich mucks about the line breaks between parameters,
> etc, etc.
>
> I have *never* once seen a formatter produce code which doesn't then need some manual
> tweaking. So the thought of this happening automatically, outside of my control, on every
> single commit just fills me with horror.

+1
In short, such tool is excellent when used at the start of the processing of a file (front
end?) but quite awful as a systematic step of a code commit process (back end?).

Stefan Wagner
Joined: 2011-04-08,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: A standard source code format - a very rough sketch

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 19.10.2011 07:06, schrieb Randall R Schulz:

> Why are people so obsessed with eradicating individualism in source
> code?

Most people start programming as individuals, not in teams, where
personal style isn't such a drawback. After being used to a personal
style, it is hard to change your habits, and learn a different one.

And since there is no single authority for style guides, there are
multiple more or less good habits per language.

Then, there are people switching to other languages, and importing their
guides or parts of it.

What would be needed is a measurement how fast you can read different
styles. What I'm missing often is the blank before open parenthesis,
like in

val hits = list.zipWithIndex.
map (line => {
keys.map (key => (
line._1.matches (".*\\b" + key + "\\b.*"), line._2, key
))
} ).flatten.filter (_._1 == true).
sortWith ((a, b) => (a._3 < b._3) || (a._3 == b._3) && a._2 < b._2).
map (e => (e._3, e._2))

In conventional text, the rule is, to have space outside the braces and
parenthesis (like you see [it] here), which helps eye and brain to
tokenize. (Exception: punctation characters, following the closing brace.

But most line-break decisions in above block would be hard to tell
algorithmically. Preferences might change with the width of your device
(no, not typing on my iPhone).

I guess some programmers even belive, that short code (less whitespace)
is faster.

Kevin Wright 2
Joined: 2010-05-30,
User offline. Last seen 26 weeks 4 days ago.
Re: Re: A standard source code format - a very rough sketch
Fortunately... rich pattern-matching, the ability to use braces in place of parenthesis, infix method notation, a good default ordering for tuples, and collection methods like collect and flatMap allow us to make that kind of code *much* more readable:
val hits = (list.zipWithIndex flatMap { (line,idx) =>
  keys collect {    case key if line matches (".*\\b" + key + "\\b.*") =>       (key, idx)   }
}).sorted
If you're instead looking for auto-formatting to make your original code more readable, then you're really focussing on the wrong problem and the wrong solution.


On 19 October 2011 11:35, Stefan Wagner <hirnstrom [at] arcor [dot] de> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 19.10.2011 07:06, schrieb Randall R Schulz:

> Why are people so obsessed with eradicating individualism in source
> code?

Most people start programming as individuals, not in teams, where
personal style isn't such a drawback. After being used to a personal
style, it is hard to change your habits, and learn a different one.

And since there is no single authority for style guides, there are
multiple more or less good habits per language.

Then, there are people switching to other languages, and importing their
guides or parts of it.

What would be needed is a measurement how fast you can read different
styles. What I'm missing often is the blank before open parenthesis,
like in

val hits = list.zipWithIndex.
 map (line => {
   keys.map (key => (
     line._1.matches (".*\\b" + key + "\\b.*"), line._2, key
     ))
   } ).flatten.filter (_._1 == true).
 sortWith ((a, b) => (a._3 < b._3) || (a._3 == b._3) && a._2 < b._2).
 map (e => (e._3, e._2))

In conventional text, the rule is, to have space outside the braces and
parenthesis (like you see [it] here), which helps eye and brain to
tokenize. (Exception: punctation characters, following the closing brace.

But most line-break decisions in above block would be hard to tell
algorithmically. Preferences might change with the width of your device
(no, not typing on my iPhone).

I guess some programmers even belive, that short code (less whitespace)
is faster.

Philippe Lhoste
Joined: 2010-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: A standard source code format - a very rough sketch

On 19/10/2011 13:21, Kevin Wright wrote:
> Fortunately... rich pattern-matching, the ability to use braces in place of parenthesis,
> infix method notation, a good default ordering for tuples, and collection methods like
> collect and flatMap allow us to make that kind of code *much* more readable:
>
> val hits = (list.zipWithIndex flatMap { (line,idx) =>
> keys collect {
> case key if line matches (".*\\b" + key + "\\b.*") =>
> (key, idx)
> }
> }).sorted

Hey, nice!
But I had to make two changes to make it compile, not sure if my changes are optimal, though.

val lsd =
"""
Picture yourself in a boat on a river with tangerine trees
And marmalade skies.
Somebody calls you, you answer quite slowly, a girl with
Kaleidoscope eyes.
"""

val list = io.Source.fromString(lsd).getLines
val keys = List("with", "girl", "eyes", "skies", "trees", "boat")
val hits = (list.zipWithIndex flatMap
{
case (line, idx) =>
keys collect
{
case key if line matches (".*\\b" + key + "\\b.*") =>
(key, idx)
}
}).toSeq.sorted
println("Found:")
println(hits.mkString("\n"))

(Yes, my formatting is unconventional in the Scala / Java world...)

H-star Development
Joined: 2010-04-14,
User offline. Last seen 2 years 26 weeks ago.
Re: Re: A standard source code format - a very rough sketch

you wasted some lines there...

(but i'Ve seen that in some java projects, they like doing that)

-------- Original-Nachricht --------
> Datum: Wed, 19 Oct 2011 15:24:08 +0200
> Von: Philippe Lhoste
> An: scala-debate [at] googlegroups [dot] com
> Betreff: [scala-debate] Re: A standard source code format - a very rough sketch

> On 19/10/2011 13:21, Kevin Wright wrote:
> > Fortunately... rich pattern-matching, the ability to use braces in place
> of parenthesis,
> > infix method notation, a good default ordering for tuples, and
> collection methods like
> > collect and flatMap allow us to make that kind of code *much* more
> readable:
> >
> > val hits = (list.zipWithIndex flatMap { (line,idx) =>
> > keys collect {
> > case key if line matches (".*\\b" + key + "\\b.*") =>
> > (key, idx)
> > }
> > }).sorted
>
> Hey, nice!
> But I had to make two changes to make it compile, not sure if my changes
> are optimal, though.
>
> val lsd =
> """
> Picture yourself in a boat on a river with tangerine trees
> And marmalade skies.
> Somebody calls you, you answer quite slowly, a girl with
> Kaleidoscope eyes.
> """
>
> val list = io.Source.fromString(lsd).getLines
> val keys = List("with", "girl", "eyes", "skies", "trees", "boat")
> val hits = (list.zipWithIndex flatMap
> {
> case (line, idx) =>
> keys collect
> {
> case key if line matches (".*\\b" + key + "\\b.*") =>
> (key, idx)
> }
> }).toSeq.sorted
> println("Found:")
> println(hits.mkString("\n"))
>
> (Yes, my formatting is unconventional in the Scala / Java world...)
>

Razvan Cojocaru 3
Joined: 2010-07-28,
User offline. Last seen 42 years 45 weeks ago.
RE: Re: A standard source code format - a very rough sketch

We used to have very strict rules, enforced by a commit-hook checker, down
to the indentation and naming convention (there was an approved dictionary).

I must admit that life was good :) I had to change my personal style, but
not much.

In time, the number of developers went up and we relaxed that more and more
until there's only a few rules enforced now, like line length at 110 and no
tabs... something like that :)

What happened was, many junior developers write code which is so crappy and
hard to follow, that the poor formatting of the said code is the least of
the issues. While the 'master coders' can maintain a reasonably clean style
without a checker - they got to know the value.

What I still find occasionally is that, because of the different formatting
preferences, some morons (including me at times) re-format large files and,
when chasing a bug, one can't find that needle in the hay-stack... but it
happens infrequently enough (and only for specific developers) not to make
me go back to a checker. This is especially painful when a Swing developer
leaves and I or someone has to make a change or fix a bug - OMG !

Some people don't understand that this goes both ways: don't you reformat my
code, if I allow you to have a different style :)

Anyways - been there done that :) on both ends of it...

Cheers,
Razie

-----Original Message-----
From: scala-debate [at] googlegroups [dot] com [mailto:scala-debate [at] googlegroups [dot] com]
On Behalf Of Tony Morris
Sent: October-19-11 1:09 AM
To: scala-debate [at] googlegroups [dot] com
Subject: Re: [scala-debate] Re: A standard source code format - a very rough
sketch

On 10/19/2011 03:06 PM, Randall R Schulz wrote:
> On Tuesday 18 October 2011, Nicolas wrote:
>> Just my 2¢: how hard would it be to have scalariform run
>> automatically before any code commit?
>>
>> This way, you don't loose the advantages of storing code in plain
>> text, you have nice looking code, and you avoid the diff problem
>> because not code gets in without being already properly formatted.
>>
>> Is that feasible?
> Why are people so obsessed with eradicating individualism in source
> code?
>
> What does "style" mean?
>
>
> Randall Schulz

I think this is an awesome question, but one for which there is so little
data available.

If we could have field experts (to demonstrate that this compulsion works
against self-interest and group-interest) working alongside experts in
neuroscience to work out why this behaviour persists anyway, maybe we can
could get a reasonable answer.

I hope this happens in my life-time. Thanks for hearing my lament.

--
Tony Morris
http://tmorris.net/

Kevin Wright 2
Joined: 2010-05-30,
User offline. Last seen 26 weeks 4 days ago.
Re: Re: A standard source code format - a very rough sketch
That's what I get for writing it off the cuff and not testing  :)
You still seem to be favouring a lot of vertical whitespace, that's almost 1/3 of your lines spent without helping to delimit structure.  With the changes in, my preference would still be this:

val lsd = """|Picture yourself in a boat on a river with tangerine trees
             |And marmalade skies.
             |Somebody calls you, you answer quite slowly, a girl with
             |Kaleidoscope eyes.             |""".stripMargin

val list = (Source fromString lsd).getLines
val keys = List("with", "girl", "eyes", "skies", "trees", "boat")
val hits = (list.zipWithIndex flatMap {
  case (line, idx) =>    keys collect {
      case key if line matches (".*\\b" + key + "\\b.*") =>
        (key, idx)
    }
}).toSeq.sorted
println("Found:")
println(hits mkString "\n")


On 19 October 2011 14:24, Philippe Lhoste <PhiLho [at] gmx [dot] net> wrote:
On 19/10/2011 13:21, Kevin Wright wrote:
Fortunately... rich pattern-matching, the ability to use braces in place of parenthesis,
infix method notation, a good default ordering for tuples, and collection methods like
collect and flatMap allow us to make that kind of code *much* more readable:

val hits = (list.zipWithIndex flatMap { (line,idx) =>
  keys collect {
    case key if line matches (".*\\b" + key + "\\b.*") =>
   (key, idx)
  }
}).sorted

Hey, nice!
But I had to make two changes to make it compile, not sure if my changes are optimal, though.

val lsd =
"""
Picture yourself in a boat on a river with tangerine trees
And marmalade skies.
Somebody calls you, you answer quite slowly, a girl with
Kaleidoscope eyes.
"""

val list = io.Source.fromString(lsd).getLines
val keys = List("with", "girl", "eyes", "skies", "trees", "boat")
val hits = (list.zipWithIndex flatMap
 {
   case (line, idx) =>
     keys collect
     {
       case key if line matches (".*\\b" + key + "\\b.*") =>
         (key, idx)
     }
 }).toSeq.sorted
println("Found:")
println(hits.mkString("\n"))

(Yes, my formatting is unconventional in the Scala / Java world...)


H-star Development
Joined: 2010-04-14,
User offline. Last seen 2 years 26 weeks ago.
Re: RE: Re: A standard source code format - a very rough sketch

simple solution: fix & commit xor reformat & commit, never both

-------- Original-Nachricht --------
> Datum: Wed, 19 Oct 2011 09:40:40 -0400
> Von: "Razvan Cojocaru"
> An: tmorris [at] tmorris [dot] net, scala-debate [at] googlegroups [dot] com
> Betreff: RE: [scala-debate] Re: A standard source code format - a very rough sketch

> We used to have very strict rules, enforced by a commit-hook checker, down
> to the indentation and naming convention (there was an approved
> dictionary).
>
> I must admit that life was good :) I had to change my personal style, but
> not much.
>
> In time, the number of developers went up and we relaxed that more and
> more
> until there's only a few rules enforced now, like line length at 110 and
> no
> tabs... something like that :)
>
> What happened was, many junior developers write code which is so crappy
> and
> hard to follow, that the poor formatting of the said code is the least of
> the issues. While the 'master coders' can maintain a reasonably clean
> style
> without a checker - they got to know the value.
>
> What I still find occasionally is that, because of the different
> formatting
> preferences, some morons (including me at times) re-format large files
> and,
> when chasing a bug, one can't find that needle in the hay-stack... but it
> happens infrequently enough (and only for specific developers) not to make
> me go back to a checker. This is especially painful when a Swing developer
> leaves and I or someone has to make a change or fix a bug - OMG !
>
> Some people don't understand that this goes both ways: don't you reformat
> my
> code, if I allow you to have a different style :)
>
> Anyways - been there done that :) on both ends of it...
>
> Cheers,
> Razie
>
> -----Original Message-----
> From: scala-debate [at] googlegroups [dot] com [mailto:scala-debate [at] googlegroups [dot] com]
> On Behalf Of Tony Morris
> Sent: October-19-11 1:09 AM
> To: scala-debate [at] googlegroups [dot] com
> Subject: Re: [scala-debate] Re: A standard source code format - a very
> rough
> sketch
>
> On 10/19/2011 03:06 PM, Randall R Schulz wrote:
> > On Tuesday 18 October 2011, Nicolas wrote:
> >> Just my 2¢: how hard would it be to have scalariform run
> >> automatically before any code commit?
> >>
> >> This way, you don't loose the advantages of storing code in plain
> >> text, you have nice looking code, and you avoid the diff problem
> >> because not code gets in without being already properly formatted.
> >>
> >> Is that feasible?
> > Why are people so obsessed with eradicating individualism in source
> > code?
> >
> > What does "style" mean?
> >
> >
> > Randall Schulz
>
> I think this is an awesome question, but one for which there is so little
> data available.
>
> If we could have field experts (to demonstrate that this compulsion works
> against self-interest and group-interest) working alongside experts in
> neuroscience to work out why this behaviour persists anyway, maybe we can
> could get a reasonable answer.
>
> I hope this happens in my life-time. Thanks for hearing my lament.
>
> --
> Tony Morris
> http://tmorris.net/
>
>
>

Jim Powers
Joined: 2011-01-24,
User offline. Last seen 36 weeks 2 days ago.
Re: Re: A standard source code format - a very rough sketch
On Tue, Oct 18, 2011 at 1:40 PM, Kevin Wright <kev [dot] lee [dot] wright [at] gmail [dot] com> wrote:
[...]  FWIW, I feel that an AST-aware diff tool that makes use of the presentation compiler would be better still.

Would such a tool be able to work against comments? 
-- Jim Powers
Razvan Cojocaru 3
Joined: 2010-07-28,
User offline. Last seen 42 years 45 weeks ago.
RE: RE: Re: A standard source code format - a very rough sketch

What's your point? That we're human and forget sometimes? Duh!

:)

-----Original Message-----
From: Dennis Haupt [mailto:h-star [at] gmx [dot] de]
Sent: October-19-11 10:31 AM
To: Razvan Cojocaru; scala-debate [at] googlegroups [dot] com; tmorris [at] tmorris [dot] net
Subject: Re: RE: [scala-debate] Re: A standard source code format - a very rough sketch

simple solution: fix & commit xor reformat & commit, never both

-------- Original-Nachricht --------
> Datum: Wed, 19 Oct 2011 09:40:40 -0400
> Von: "Razvan Cojocaru"
> An: tmorris [at] tmorris [dot] net, scala-debate [at] googlegroups [dot] com
> Betreff: RE: [scala-debate] Re: A standard source code format - a very
> rough sketch

> We used to have very strict rules, enforced by a commit-hook checker,
> down to the indentation and naming convention (there was an approved
> dictionary).
>
> I must admit that life was good :) I had to change my personal style,
> but not much.
>
> In time, the number of developers went up and we relaxed that more and
> more until there's only a few rules enforced now, like line length at
> 110 and no tabs... something like that :)
>
> What happened was, many junior developers write code which is so
> crappy and hard to follow, that the poor formatting of the said code
> is the least of the issues. While the 'master coders' can maintain a
> reasonably clean style without a checker - they got to know the value.
>
> What I still find occasionally is that, because of the different
> formatting preferences, some morons (including me at times) re-format
> large files and, when chasing a bug, one can't find that needle in the
> hay-stack... but it happens infrequently enough (and only for specific
> developers) not to make me go back to a checker. This is especially
> painful when a Swing developer leaves and I or someone has to make a
> change or fix a bug - OMG !
>
> Some people don't understand that this goes both ways: don't you
> reformat my code, if I allow you to have a different style :)
>
> Anyways - been there done that :) on both ends of it...
>
> Cheers,
> Razie
>
> -----Original Message-----
> From: scala-debate [at] googlegroups [dot] com
> [mailto:scala-debate [at] googlegroups [dot] com]
> On Behalf Of Tony Morris
> Sent: October-19-11 1:09 AM
> To: scala-debate [at] googlegroups [dot] com
> Subject: Re: [scala-debate] Re: A standard source code format - a very
> rough sketch
>
> On 10/19/2011 03:06 PM, Randall R Schulz wrote:
> > On Tuesday 18 October 2011, Nicolas wrote:
> >> Just my 2¢: how hard would it be to have scalariform run
> >> automatically before any code commit?
> >>
> >> This way, you don't loose the advantages of storing code in plain
> >> text, you have nice looking code, and you avoid the diff problem
> >> because not code gets in without being already properly formatted.
> >>
> >> Is that feasible?
> > Why are people so obsessed with eradicating individualism in source
> > code?
> >
> > What does "style" mean?
> >
> >
> > Randall Schulz
>
> I think this is an awesome question, but one for which there is so
> little data available.
>
> If we could have field experts (to demonstrate that this compulsion
> works against self-interest and group-interest) working alongside
> experts in neuroscience to work out why this behaviour persists
> anyway, maybe we can could get a reasonable answer.
>
> I hope this happens in my life-time. Thanks for hearing my lament.
>
> --
> Tony Morris
> http://tmorris.net/
>
>
>

Bernd Johannes
Joined: 2011-01-28,
User offline. Last seen 42 years 45 weeks ago.
Re: A standard source code format - a very rough sketch

Am Mittwoch, 19. Oktober 2011, 07:06:14 schrieb Randall R Schulz:
> On Tuesday 18 October 2011, Nicolas wrote:
> > Just my 2¢: how hard would it be to have scalariform run
> > automatically before any code commit?
> >
> > This way, you don't loose the advantages of storing code in plain
> > text, you have nice looking code, and you avoid the diff problem
> > because not code gets in without being already properly formatted.
> >
> > Is that feasible?
>
> Why are people so obsessed with eradicating individualism in source
> code?
>
Oh I hope nobody is obsessed - this would call for an exorcism ;-).

But joke aside - I hope for freedom to reformat code to my taste, just because
I like it better "that" way.
Somehow my internal compiler doesn't cope well with some formatting styles (or
non-styles). And I want to be able to "set things right" (in my very
subjective ways).

E.g. I'm having a hard time grasping the meaning of code parts where the
indentation is contradicting the actual flow. I can't ignore the misleading
information given by the unusual indentation. Something a simple reformat
could set right.

But most of the time this simple wish causes a lot of fuss. A standardized VCS
"save format" could remedy the situation to a certain extend. (And such a
feature would be optional anyway - so anybody who is lucky enough to work in a
context where "ugly" code is nonexistent can simply leave it "off").

> What does "style" mean?

a mental function which decides which optional characters (or optional paths)
are emitted in a given context?

Greetings
Bernd

Philippe Lhoste
Joined: 2010-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: A standard source code format - a very rough sketch

On 19/10/2011 15:31, Dennis Haupt wrote:
> you wasted some lines there...

Yes, lines are cheap these times... I don't pay per wasted line anyway.
^_^

On 19/10/2011 16:21, Kevin Wright wrote:
> You still seem to be favouring a lot of vertical whitespace, that's
> almost 1/3 of your lines spent without helping to delimit structure.

I disagree (of course!), it helps delimiting the structures, to my eyes
at least. I find alignment of braces visually nice and helpful. Funnily,
I work for a Java shop which precisely think like me (I use this style
in C, C++, JavaScript, CSS, etc.).
But I don't mind the K&R style either, I adapt to the style of the
project I am working on (using my own style on my own projects, of course).

Raoul Duke
Joined: 2009-01-05,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: A standard source code format - a very rough sketch

for the love of all things sane with respect to usability and having
team members get along and yet still have things work well for them
individual selves, why aren't we just storing everything as some sort
of canonical AST under the covers, and letting each and every single
developer present it however they want whenever they want? intentional
programming on the cheap, please. man, the software world sucks. :-}

dcsobral
Joined: 2009-04-23,
User offline. Last seen 38 weeks 5 days ago.
Re: Re: A standard source code format - a very rough sketch

On Wed, Oct 19, 2011 at 09:21, Kevin Wright wrote:
> Fortunately... rich pattern-matching, the ability to use braces in place of
> parenthesis, infix method notation, a good default ordering for tuples, and
> collection methods like collect and flatMap allow us to make that kind of
> code *much* more readable:
> val hits = (list.zipWithIndex flatMap { (line,idx) =>

case (line, idx)

>   keys collect {
>     case key if line matches (".*\\b" + key + "\\b.*") =>

".*\\b\{key}\\b.*" ;-)

>       (key, idx)
>   }
> }).sorted
> If you're instead looking for auto-formatting to make your original code
> more readable, then you're really focussing on the wrong problem and the
> wrong solution.
>
>
> On 19 October 2011 11:35, Stefan Wagner wrote:
>>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Am 19.10.2011 07:06, schrieb Randall R Schulz:
>>
>> > Why are people so obsessed with eradicating individualism in source
>> > code?
>>
>> Most people start programming as individuals, not in teams, where
>> personal style isn't such a drawback. After being used to a personal
>> style, it is hard to change your habits, and learn a different one.
>>
>> And since there is no single authority for style guides, there are
>> multiple more or less good habits per language.
>>
>> Then, there are people switching to other languages, and importing their
>> guides or parts of it.
>>
>> What would be needed is a measurement how fast you can read different
>> styles. What I'm missing often is the blank before open parenthesis,
>> like in
>>
>> val hits = list.zipWithIndex.
>>  map (line => {
>>    keys.map (key => (
>>      line._1.matches (".*\\b" + key + "\\b.*"), line._2, key
>>      ))
>>    } ).flatten.filter (_._1 == true).
>>  sortWith ((a, b) => (a._3 < b._3) || (a._3 == b._3) && a._2 < b._2).
>>  map (e => (e._3, e._2))
>>
>> In conventional text, the rule is, to have space outside the braces and
>> parenthesis (like you see [it] here), which helps eye and brain to
>> tokenize. (Exception: punctation characters, following the closing brace.
>>
>> But most line-break decisions in above block would be hard to tell
>> algorithmically. Preferences might change with the width of your device
>> (no, not typing on my iPhone).
>>
>> I guess some programmers even belive, that short code (less whitespace)
>> is faster.
>>
>

Kevin Wright 2
Joined: 2010-05-30,
User offline. Last seen 26 weeks 4 days ago.
Re: Re: A standard source code format - a very rough sketch

>   keys collect {
>     case key if line matches (".*\\b" + key + "\\b.*") =>

".*\\b\{key}\\b.*" ;-)


+1
It's scala-debate, so I guess trunk features are safe enough!
Stefan Langer
Joined: 2009-10-23,
User offline. Last seen 42 years 45 weeks ago.
Re: A standard source code format - a very rough sketch

I've seen efforts like this start out and then fail. I think the
biggest problem here is that you are trying to solve a social probelm
through technical means which will always fail in the end.
The social problem: Every decent programmer takes pride in what he
writes and has his own style. Code is like a painting it is personal
taste and style. This includes coding style and formatting. In order
to reduce the probelms that stem from that organizations tend to
proclaim style guides. If these are too strict then the programmer
gets pissed he gets frustrated and he will losse interest resulting in
worse code. So live with the style and try to guide them. Most
codesbases will stabilize within loose rulez and everything is fine.
Now you might say but this is exactly what my proposal solves... but
unless you can customize your tool to incorporate all styles (highly
unlikely) and do this reliable without introducing new dependencies,
like installing hooks on new repositories making the developer install
certain software, taking away the ability to use simple tools like
diff, patch, grep to get a job done, you will always run into the
problem that you upset developers which will lead to frustration a
long the path which will lead to loosing interest and bad code.

Just my 2 cts.

Stefan

2011/10/18 Bernd :
> Hello list,
>
> just while thinking about sip-12 it occurred to me that it might be
> nice to have the possibility to store a given source code in a way
> that is independent of its formatting.
>
> Why? Because it would be nice to be able to reformat a given source
> code to "my" taste without triggering all the diff gizmoz around. I
> see that such a functionality will require special handling by the IDE
> (firstplace).
>
> So the roundtrip would be:
>
> -> IDE + Compiler => => VCS => IDE + Compiler
> + reformatter =>
>
> Whether this is simply a "defined" source code
> format which took away any optional formattings (e.g. remove all
> semicolons, brackets, whitespace etc. where possible) or if it would
> be some kind of AST-XML which allows to reconstruct the source code
> entirely (inclusive comments) is a free decision.
>
> This is a feature I (among others) always miss. I've come across many
> source files (in different languages) that would benefit from a
> reformat. But this always triggers the discussion about the change
> history [oh no, you can no longer diff them with one of the previous
> versions - don't do that!].
>
> Here scala could take the lead by enabling such a "standard repository
> format for sourcecode" in eclipse.
>
> [I post this here and not in tools because it seems to me that the
> compiler is heavily involved in source code handling / interpretation
> and might be the first place to enable such standardization
> operations]
>
> Just me... dreaming...
> Bernd
>

Philippe Lhoste
Joined: 2010-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: A standard source code format - a very rough sketch

On 20/10/2011 01:56, Daniel Sobral wrote:
> ".*\\b\{key}\\b.*" ;-)

That's 2.10, yes?
In this case, I'd prefer """.*\b{key}\b.*"""
I dislike double backslashes... :-P (but not enough to put triple quotes around a
four-letter string!)

Mmm, I see now the backslash before the opening brace. Not sure if I must then write
""".*\b\{key}\b.*"""? I suppose string inferring works with triple quotes, otherwise it
looses some interest...

Stefan Wagner
Joined: 2011-04-08,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: A standard source code format - a very rough sketch

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 19.10.2011 23:58, schrieb Raoul Duke:
> for the love of all things sane with respect to usability and having
> team members get along and yet still have things work well for them
> individual selves, why aren't we just storing everything as some sort
> of canonical AST under the covers, and letting each and every single
> developer present it however they want whenever they want? intentional
> programming on the cheap, please. man, the software world sucks. :-}
>

Because, if you look at the 10 pages, eclipse uses to let you specify
how to indent your code - after filling this, there are still cases,
where I prefer to deviate from the corsett, to make the code more
readable, which is, by now, not handled by a rule.

Example - alignment of similar lines, to make the similarity more obvious:

case SOUTH: foo.man (SOUTH); takata (); fop.fop (1, 2); break;
case EAST: foo.man (EAST); takata (); fop.fop (2, 3); break;
case WEST: foo.man (WEST); takata (); fop.fop (3, 4); break;
case NORTH: foo.man (NORTH); takata (); fop.fop (4, 1); break;

It is easy to spot the similarities, and to find the difference.
Yes, I know, I could write
case SOUTH: fun (direction, 1, 2); break;
instead. It's just an example. From the old world too.

It is a great tool, to indent code for a first time, which comes from a
different culture, or from the opposite of culture, but you don't want
to repeat the fine tuning every time.

Stefan Wagner
Joined: 2011-04-08,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: A standard source code format - a very rough sketch

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 19.10.2011 15:24, schrieb Philippe Lhoste:
>
> val list = io.Source.fromString(lsd).getLines
> val keys = List("with", "girl", "eyes", "skies", "trees", "boat")
> val hits = (list.zipWithIndex flatMap
> {...
> println("Found:")
> println(hits.mkString("\n"))

Compare it to this:

val list = io.Source.fromString (lsd).getLines
val keys = List ("with", "girl", "eyes", "skies", "trees", "boat")
val hits = (list.zipWithIndex flatMap
{ ...
println ("Found:")
println (hits.mkString ("\n"))

for example, the `fromString` in the first line is much better readable,
if there is a space before the paren. Or the 2 `println` in the end.

dcsobral
Joined: 2009-04-23,
User offline. Last seen 38 weeks 5 days ago.
Re: Re: A standard source code format - a very rough sketch

On Thu, Oct 20, 2011 at 06:54, Philippe Lhoste wrote:
> On 20/10/2011 01:56, Daniel Sobral wrote:
>>
>> ".*\\b\{key}\\b.*" ;-)
>
> That's 2.10, yes?
> In this case, I'd prefer """.*\b{key}\b.*"""
> I dislike double backslashes... :-P (but not enough to put triple quotes
> around a four-letter string!)
>
> Mmm, I see now the backslash before the opening brace. Not sure if I must
> then write """.*\b\{key}\b.*"""? I suppose string inferring works with
> triple quotes, otherwise it looses some interest...

http://scala.github.com/sips//pending/string-interpolation.html

No, it doesn't work with triple quotes, as it depends on processing an
escape character.

John Nilsson
Joined: 2008-12-20,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: A standard source code format - a very rough sketch

Nah. Don't store the AST. With an eye towards event sourcing and operational transform: store a stream of refactorings instead.

This way there's no need to do tree diffs after the fact which should help with merging and other code history manipulations.

Plus, it would proably be an interesting reasearch area to determine the algebraic properies of the various refactorings. How can a rename be combined with a move for example.

Tools could also determine how and if refactoeings depend on eachother. Maybe this couls be used to organize topic branches automatically.

BR,
John

Den 19 okt 2011 23:59 skrev "Raoul Duke" <raould [at] gmail [dot] com>:
for the love of all things sane with respect to usability and having
team members get along and yet still have things work well for them
individual selves, why aren't we just storing everything as some sort
of canonical AST under the covers, and letting each and every single
developer present it however they want whenever they want? intentional
programming on the cheap, please. man, the software world sucks. :-}
Philippe Lhoste
Joined: 2010-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: A standard source code format - a very rough sketch

On 20/10/2011 19:16, Daniel Sobral wrote:
>> Mmm, I see now the backslash before the opening brace. Not sure if I must
>> then write """.*\b\{key}\b.*"""? I suppose string inferring works with
>> triple quotes, otherwise it looses some interest...
>
> http://scala.github.com/sips//pending/string-interpolation.html
>
> No, it doesn't work with triple quotes, as it depends on processing an
> escape character.

OK, thanks for the link.
As I wrote, without support of multi-line strings, the feature is much less interesting...
Well, one can argue that this feature shouldn't be used to make PHP-like code, mixing big
templates with code in an awful mess, so perhaps it is half an advantage, after all...
^_^'

As usual, powerful tools, potential for big holes in foot...

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland