This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Re: Re: Yammer moving away from Scala

43 replies
Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.


On Wed, Nov 30, 2011 at 5:11 PM, Aleksey Nikiforov <lexn82 [at] gmail [dot] com> wrote:
On Wed, Nov 30, 2011 at 3:43 PM, Jesper Nordenberg <megagurka [at] yahoo [dot] com> wrote:
Aleksey Nikiforov skrev 2011-11-30 21:27:
When I first dicovered that for loops were not optimized I was surpised.
When I discovered a resistance to optimizing them I was shocked. The
decision to do nothing about for loops baffles me to this day.

For comprehensions are just syntactic sugar. Optimizing just them is not very useful, instead effort should be put into generic optimization of HOF's (and other inlining).

/Jesper Nordenberg


This is exactly the thinking that baffles me. There are for-loops in Java. Not comprehensions, LOOPs. Anyone coming from Java will expect for LOOPs or simlar functionality.

It does not matter to me if it is a syntactic sugar or if they are comprehensions. What matters to me is to be able to write the code:

var sum = 0; for (i <- 0 until array.length) sum += array(i)

And have it work as fast as possible. Scala fails that expectation. It fails it so much that this is one of the most frequent questions on stack overflow. Also, it appears to be the number 1 on the list of problems with Scala as viewed by the employee at Yammer who wrote the email:
1. Don't ever use a for-loop.
So, they were looking for loops as well, not comprehensions. Why not send them an email and ask them if perhaps they were "talking about for comprehensions or just foreach"?! If you think I was overreacting, I would LOVE to see their response.


It is nice to have for-comprehensions optimized. But I really do not care about for-comprehensions. I want something that looks like for loops and works like for loops. 99% of my code uses emulated for-loops via while-loops. It's painful and error prone, so I take this issue very personally.

I'm pretty surprised by this.  I don't think while loops *always* beat for-expression for speed in all cases... 

Not providing proper for-LOOPs is akin to saying: Scala is not meant for the applications that require for-loops. Which sends the message that Scala is not meant for performance-sensitive applications. So guess what, the folks at Yammer got the message and went back to Java.

You can tell me that they are comprehensions, or a more general case should be optimized. But I do not care about those uses at all. Ranges and closures can never be optimized to the level of while loops. And I need the LOOPs. And so do many other people not interested in functional aspects of Scala.


Lex
Joined: 2010-02-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala
On Wed, Nov 30, 2011 at 4:19 PM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com> wrote:
I'm pretty surprised by this.  I don't think while loops *always* beat for-expression for speed in all cases... 

If you are not iterating over collections, then while-loops always beat the for-comprehensions. Also I have custom collections, which works best when you use index access isntead of iterators due to other JVM optimizations.


Jesper Nordenberg
Joined: 2008-12-27,
User offline. Last seen 42 years 45 weeks ago.
Re: Yammer moving away from Scala

Aleksey Nikiforov skrev 2011-11-30 23:24:
> If you are not iterating over collections, then while-loops always beat
> the for-comprehensions. Also I have custom collections, which works best
> when you use index access isntead of iterators due to other JVM
> optimizations.

Hmmm, what benefits do you see in using Scala over Java if you only use
plain old imperative loops?

/Jesper Nordenberg

d_m
Joined: 2010-11-11,
User offline. Last seen 35 weeks 2 days ago.
Re: Re: Yammer moving away from Scala

On Wed, Nov 30, 2011 at 11:39:56PM +0100, Jesper Nordenberg wrote:
> Hmmm, what benefits do you see in using Scala over Java if you only
> use plain old imperative loops?

Tons, of course. :)

At my job we do much of our work on Arrays (gasp) using while loops
(gasp) and yet we benefit from many awesome features in Scala:

* cleaner-looking code
* traits
* unified primitive/boxed types
* much better generics
* type classes via implicits
* pattern matching/extractors

These are only a subset of course.

I understand why a lot of us are defensive about this issue (no one
likes to be criticized, or have their favorite language criticized).
But there are legitimate reasons to want fast imperative-style loops
besides while loop, and legitimate Scala users who need or would like
to see this kind of use optimized.

Lex
Joined: 2010-02-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Yammer moving away from Scala
On Wed, Nov 30, 2011 at 4:39 PM, Jesper Nordenberg <megagurka [at] yahoo [dot] com> wrote:
Hmmm, what benefits do you see in using Scala over Java if you only use plain old imperative loops?

/Jesper Nordenberg


There is nothing wrong with imperative programming. I very much like using Scala in imperative style and it works great, with the exception of loops.

Lex
Joined: 2010-02-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Yammer moving away from Scala
On Wed, Nov 30, 2011 at 5:00 PM, Jesper Nordenberg <megagurka [at] yahoo [dot] com> wrote:
Aleksey Nikiforov skrev 2011-11-30 23:44:
There is nothing wrong with imperative programming.

Actually there is, but let's not get into that discussion. Personally I, and I think many others, find it much more important that code written in a functional style is optimized to get the same performance as corresponding imperative code.

/Jesper Nordenberg

Is it important enough to scare all the other users away from the language?
Raoul Duke
Joined: 2009-01-05,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala

On Wed, Nov 30, 2011 at 3:01 PM, Aleksey Nikiforov wrote:
> On Wed, Nov 30, 2011 at 5:00 PM, Jesper Nordenberg
>>> There is nothing wrong with imperative programming.
>> Actually there is, but let's not get into that discussion. Personally I,

see what i mean?

scala might have cast the net too wide...

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Yammer moving away from Scala


On Wed, Nov 30, 2011 at 6:01 PM, Aleksey Nikiforov <lexn82 [at] gmail [dot] com> wrote:
On Wed, Nov 30, 2011 at 5:00 PM, Jesper Nordenberg <megagurka [at] yahoo [dot] com> wrote:
Aleksey Nikiforov skrev 2011-11-30 23:44:
There is nothing wrong with imperative programming.

Actually there is, but let's not get into that discussion. Personally I, and I think many others, find it much more important that code written in a functional style is optimized to get the same performance as corresponding imperative code.

/Jesper Nordenberg

Is it important enough to scare all the other users away from the language?

I don't think Scala does this.  For expressions look like loops to encourage imperative developers to use them and have them look familiar.
You are correct that for-expressions on Ranges (1 to 10) and Arrays and such could probably be optimised the compiler.  If they're not by -optimise, I know the ScalaCL plugin *will* optimise them for you.   This should be a flagable option on the compiler, but right now there are still options to have code that runs like a loop but is a for-expression.  There's no need to drop them.
Scala tries to support many different styles.  There's nothing *wrong* with imperative programming.  There are pros and cons to any approach.  A lot of us use the blended OO-functional approach in Scala because a lot of the pros outweight the cons.  I wrote a blog article about this.  
Imperative code has its place.  If we can have functional code compile down to imperative style (the language of the machine) in an efficient manner, I think we're all winning.   IF the compiler isn't doing it for you, sometimes you have to write it directly.   It's all about your needs.
Jesper Nordenberg
Joined: 2008-12-27,
User offline. Last seen 42 years 45 weeks ago.
Re: Yammer moving away from Scala

Aleksey Nikiforov skrev 2011-11-30 23:44:
> There is nothing wrong with imperative programming.

Actually there is, but let's not get into that discussion. Personally I,
and I think many others, find it much more important that code written
in a functional style is optimized to get the same performance as
corresponding imperative code.

/Jesper Nordenberg

Viktor Klang
Joined: 2008-12-17,
User offline. Last seen 1 year 27 weeks ago.
Re: Re: Yammer moving away from Scala


On Thu, Dec 1, 2011 at 12:06 AM, martin odersky <martin [dot] odersky [at] epfl [dot] ch> wrote:


On Thu, Dec 1, 2011 at 12:00 AM, Jesper Nordenberg <megagurka [at] yahoo [dot] com> wrote:
Aleksey Nikiforov skrev 2011-11-30 23:44:
There is nothing wrong with imperative programming.

Actually there is, but let's not get into that discussion. Personally I, and I think many others, find it much more important that code written in a functional style is optimized to get the same performance as corresponding imperative code.

/Jesper Nordenberg


That's a nice dream. But I challenge you to find a language implementation where this holds in the real world, not just in some select benchmarks.

Totally agree,
real world beats benchmarks any day of the week.
 

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Re: Yammer moving away from Scala


On Thu, Dec 1, 2011 at 12:00 AM, Jesper Nordenberg <megagurka [at] yahoo [dot] com> wrote:
Aleksey Nikiforov skrev 2011-11-30 23:44:
There is nothing wrong with imperative programming.

Actually there is, but let's not get into that discussion. Personally I, and I think many others, find it much more important that code written in a functional style is optimized to get the same performance as corresponding imperative code.

/Jesper Nordenberg


That's a nice dream. But I challenge you to find a language implementation where this holds in the real world, not just in some select benchmarks.
 -- Martin
odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Yammer moving away from Scala


On Thu, Dec 1, 2011 at 12:19 AM, Jesper Nordenberg <megagurka [at] yahoo [dot] com> wrote:
martin odersky skrev 2011-12-01 00:06:
That's a nice dream. But I challenge you to find a language
implementation where this holds in the real world, not just in some
select benchmarks.

Not sure what you're getting at. With proper inlining, unboxed types, type inference etc. you can match imperative performance for functional code. This can quite easily be done in C++11 for example (GHC also has language extensions which allow these types of optimizations, but I don't have much experience using those).

I'd really like to see case studies of large systems where this is the case. I am surprised to see you mention C++ because without GC you cannot even start implementing true functional programming. 
 -- Martin

Jesper Nordenberg
Joined: 2008-12-27,
User offline. Last seen 42 years 45 weeks ago.
Re: Yammer moving away from Scala

martin odersky skrev 2011-12-01 00:06:
> That's a nice dream. But I challenge you to find a language
> implementation where this holds in the real world, not just in some
> select benchmarks.

Not sure what you're getting at. With proper inlining, unboxed types,
type inference etc. you can match imperative performance for functional
code. This can quite easily be done in C++11 for example (GHC also has
language extensions which allow these types of optimizations, but I
don't have much experience using those).

/Jesper Nordenberg

Bradley Buchsbaum 2
Joined: 2010-08-20,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala
Musing on the the statement "imperative programming has its place". I understand what is meant, but it occurred to me that the technological universe for the last 50 years is more or less made up of it. Its place is exceedingly large.


On Wed, Nov 30, 2011 at 6:10 PM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com> wrote:


On Wed, Nov 30, 2011 at 6:01 PM, Aleksey Nikiforov <lexn82 [at] gmail [dot] com> wrote:
On Wed, Nov 30, 2011 at 5:00 PM, Jesper Nordenberg <megagurka [at] yahoo [dot] com> wrote:
Aleksey Nikiforov skrev 2011-11-30 23:44:
There is nothing wrong with imperative programming.

Actually there is, but let's not get into that discussion. Personally I, and I think many others, find it much more important that code written in a functional style is optimized to get the same performance as corresponding imperative code.

/Jesper Nordenberg

Is it important enough to scare all the other users away from the language?

I don't think Scala does this.  For expressions look like loops to encourage imperative developers to use them and have them look familiar.
You are correct that for-expressions on Ranges (1 to 10) and Arrays and such could probably be optimised the compiler.  If they're not by -optimise, I know the ScalaCL plugin *will* optimise them for you.   This should be a flagable option on the compiler, but right now there are still options to have code that runs like a loop but is a for-expression.  There's no need to drop them.
Scala tries to support many different styles.  There's nothing *wrong* with imperative programming.  There are pros and cons to any approach.  A lot of us use the blended OO-functional approach in Scala because a lot of the pros outweight the cons.  I wrote a blog article about this.  
Imperative code has its place.  If we can have functional code compile down to imperative style (the language of the machine) in an efficient manner, I think we're all winning.   IF the compiler isn't doing it for you, sometimes you have to write it directly.   It's all about your needs.



--
Bradley R. Buchsbaum
Rotman Research Institute
3560 Bathurst St.
Toronto, ON Canada M6A 2E1
email: bbuchsbaum [at] rotman-baycrest [dot] on [dot] ca
Stefan Wagner
Joined: 2011-04-08,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 30.11.2011 23:24, schrieb Aleksey Nikiforov:

> If you are not iterating over collections, then while-loops always beat the
> for-comprehensions. Also I have custom collections, which works best when
> you use index access isntead of iterators due to other JVM optimizations.

Not for me:

scala ForWhileBench 400000000

0 while for
1 0.574 0.147
2 1.199 0.259
3 1.309 0.44
4 1.727 0.55
5 2.151 0.671
6 2.585 0.844
7 3.025 1.004
8 3.467 1.237
9 3.963 1.379
10 4.888 1.758

The code is only marginally modified from the yammer-code, except:
a) Linebreaks
b) using Longs, to avoid Int-overrun
c) returning the result.

type I=Int
type O=Long

def fordef (max: I): O = {
var total = 0L
for (i <- 0 until max) {
total += i }
total
}

def whiledef (max: I): O = {
var total = 0L
var i = 0
while (i < max) {
i += 1
total += i
}
total
}

Ah yes - and I compiled it.

ARKBAN
Joined: 2011-08-11,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala

I'd suggest that making the standard for-loop faster would be a good way
keep potential Scala converts. (Conversely failing to optimize them
might alienate potential converts.) When we first started with Scala we
all likely used various Java idioms such as for loops because they were
familiar and we could only embrace so much Scala at a time. And yet
that's as afar as many people will get before making up their minds
about adopting Scala. It seems so obvious to use because we've already
fallen down the rabbit hole.

ARKBAN

On 11/30/2011 05:55 PM, Erik Osheim wrote:
> On Wed, Nov 30, 2011 at 11:39:56PM +0100, Jesper Nordenberg wrote:
>> Hmmm, what benefits do you see in using Scala over Java if you only
>> use plain old imperative loops?
> Tons, of course. :)
>
> At my job we do much of our work on Arrays (gasp) using while loops
> (gasp) and yet we benefit from many awesome features in Scala:
>
> * cleaner-looking code
> * traits
> * unified primitive/boxed types
> * much better generics
> * type classes via implicits
> * pattern matching/extractors
>
> These are only a subset of course.
>
> I understand why a lot of us are defensive about this issue (no one
> likes to be criticized, or have their favorite language criticized).
> But there are legitimate reasons to want fast imperative-style loops
> besides while loop, and legitimate Scala users who need or would like
> to see this kind of use optimized.
>

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Re: Yammer moving away from Scala
I think you should check out (if it's available) susseman's (sp?) talk at Strange Loop 2011.  
Imperative programming is natural because it's how we've designed computers to think.   However, there are probably ways to think of the problem that can "scale" much better.   For example, the Human body is made up of millions of cells and actively fighting adaptive viruses from taking down the system.   We're nowhere near this level of sophistication.   
FP has its place.  Imperative programming has its place.   I'm not going to say how big either of these places are, just that there problems that each solve and each should be used when appropriate, or the best tool for a job.  I started off an imperative programmer, and by golly I still program imperatively today (I've been doing *lots* of build work lately, and believe you me, it's got lots of imperative bits, *good* imperative bits).  But if we step back and look at computing, we may still be in infant stages as a science.
- Josh

On Wed, Nov 30, 2011 at 6:32 PM, Bradley Buchsbaum <bbuchsbaum [at] rotman-baycrest [dot] on [dot] ca> wrote:
Musing on the the statement "imperative programming has its place". I understand what is meant, but it occurred to me that the technological universe for the last 50 years is more or less made up of it. Its place is exceedingly large.


On Wed, Nov 30, 2011 at 6:10 PM, Josh Suereth <joshua [dot] suereth [at] gmail [dot] com> wrote:


On Wed, Nov 30, 2011 at 6:01 PM, Aleksey Nikiforov <lexn82 [at] gmail [dot] com> wrote:
On Wed, Nov 30, 2011 at 5:00 PM, Jesper Nordenberg <megagurka [at] yahoo [dot] com> wrote:
Aleksey Nikiforov skrev 2011-11-30 23:44:
There is nothing wrong with imperative programming.

Actually there is, but let's not get into that discussion. Personally I, and I think many others, find it much more important that code written in a functional style is optimized to get the same performance as corresponding imperative code.

/Jesper Nordenberg

Is it important enough to scare all the other users away from the language?

I don't think Scala does this.  For expressions look like loops to encourage imperative developers to use them and have them look familiar.
You are correct that for-expressions on Ranges (1 to 10) and Arrays and such could probably be optimised the compiler.  If they're not by -optimise, I know the ScalaCL plugin *will* optimise them for you.   This should be a flagable option on the compiler, but right now there are still options to have code that runs like a loop but is a for-expression.  There's no need to drop them.
Scala tries to support many different styles.  There's nothing *wrong* with imperative programming.  There are pros and cons to any approach.  A lot of us use the blended OO-functional approach in Scala because a lot of the pros outweight the cons.  I wrote a blog article about this.  
Imperative code has its place.  If we can have functional code compile down to imperative style (the language of the machine) in an efficient manner, I think we're all winning.   IF the compiler isn't doing it for you, sometimes you have to write it directly.   It's all about your needs.



--
Bradley R. Buchsbaum
Rotman Research Institute
3560 Bathurst St.
Toronto, ON Canada M6A 2E1
email: bbuchsbaum [at] rotman-baycrest [dot] on [dot] ca

ejc
Joined: 2010-09-21,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala

On Wed, Nov 30, 2011 at 8:39 PM, Josh Suereth wrote:
> I think you should check out (if it's available) susseman's (sp?) talk at
> Strange Loop 2011.
>

A very good talk indeed!

http://www.infoq.com/presentations/We-Really-Dont-Know-How-To-Compute

Thanks,
Eric

> Imperative programming is natural because it's how we've designed computers
> to think.   However, there are probably ways to think of the problem that
> can "scale" much better.   For example, the Human body is made up of
> millions of cells and actively fighting adaptive viruses from taking down
> the system.   We're nowhere near this level of sophistication.
>
> FP has its place.  Imperative programming has its place.   I'm not going to
> say how big either of these places are, just that there problems that each
> solve and each should be used when appropriate, or the best tool for a job.
>  I started off an imperative programmer, and by golly I still program
> imperatively today (I've been doing *lots* of build work lately, and believe
> you me, it's got lots of imperative bits, *good* imperative bits).  But if
> we step back and look at computing, we may still be in infant stages as a
> science.
>
> - Josh
>
>
> On Wed, Nov 30, 2011 at 6:32 PM, Bradley Buchsbaum
> wrote:
>>
>> Musing on the the statement "imperative programming has its place". I
>> understand what is meant, but it occurred to me that the technological
>> universe for the last 50 years is more or less made up of it. Its place is
>> exceedingly large.
>>
>>
>>
>> On Wed, Nov 30, 2011 at 6:10 PM, Josh Suereth
>> wrote:
>>>
>>>
>>>
>>> On Wed, Nov 30, 2011 at 6:01 PM, Aleksey Nikiforov
>>> wrote:
>>>>
>>>> On Wed, Nov 30, 2011 at 5:00 PM, Jesper Nordenberg
>>>> wrote:
>>>>>
>>>>> Aleksey Nikiforov skrev 2011-11-30 23:44:
>>>>>
>>>>>> There is nothing wrong with imperative programming.
>>>>>
>>>>>
>>>>> Actually there is, but let's not get into that discussion. Personally
>>>>> I, and I think many others, find it much more important that code written in
>>>>> a functional style is optimized to get the same performance as corresponding
>>>>> imperative code.
>>>>>
>>>>> /Jesper Nordenberg
>>>>
>>>>
>>>> Is it important enough to scare all the other users away from the
>>>> language?
>>>
>>>
>>> I don't think Scala does this.  For expressions look like loops to
>>> encourage imperative developers to use them and have them look familiar.
>>>
>>> You are correct that for-expressions on Ranges (1 to 10) and Arrays and
>>> such could probably be optimised the compiler.  If they're not by -optimise,
>>> I know the ScalaCL plugin *will* optimise them for you.   This should be a
>>> flagable option on the compiler, but right now there are still options to
>>> have code that runs like a loop but is a for-expression.  There's no need to
>>> drop them.
>>>
>>> Scala tries to support many different styles.  There's nothing *wrong*
>>> with imperative programming.  There are pros and cons to any approach.  A
>>> lot of us use the blended OO-functional approach in Scala because a lot of
>>> the pros outweight the cons.  I wrote a blog article about this.
>>>
>>> Imperative code has its place.  If we can have functional code compile
>>> down to imperative style (the language of the machine) in an efficient
>>> manner, I think we're all winning.   IF the compiler isn't doing it for you,
>>> sometimes you have to write it directly.   It's all about your needs.
>>
>>
>>
>>
>> --
>> Bradley R. Buchsbaum
>> Rotman Research Institute
>> 3560 Bathurst St.
>> Toronto, ON Canada M6A 2E1
>> email: bbuchsbaum [at] rotman-baycrest [dot] on [dot] ca
>
>

Lex
Joined: 2010-02-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala
Clearly you are doing something wrong.


On Wed, Nov 30, 2011 at 7:08 PM, Stefan Wagner <hirnstrom [at] arcor [dot] de> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 30.11.2011 23:24, schrieb Aleksey Nikiforov:

> If you are not iterating over collections, then while-loops always beat the
> for-comprehensions. Also I have custom collections, which works best when
> you use index access isntead of iterators due to other JVM optimizations.

Not for me:

scala ForWhileBench 400000000

0       while   for
1       0.574   0.147
2       1.199   0.259
3       1.309   0.44
4       1.727   0.55
5       2.151   0.671
6       2.585   0.844
7       3.025   1.004
8       3.467   1.237
9       3.963   1.379
10      4.888   1.758

The code is only marginally modified from the yammer-code, except:
a) Linebreaks
b) using Longs, to avoid Int-overrun
c) returning the result.

 type I=Int
 type O=Long

 def fordef (max: I): O = {
   var total = 0L
   for (i <- 0 until max) {
     total += i }
   total
 }

 def whiledef (max: I): O = {
   var total = 0L
   var i = 0
   while (i < max) {
     i += 1
     total += i
   }
   total
 }

Ah yes - and I compiled it.

- --

Tschööö--->...Stefan
- ---------------------------
Don't visit my homepage at:
http://home.arcor-online.net/hirnstrom
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7W03YACgkQQeATqGpDnRp1/wCfZRcKN/0XD0YDK20OGnBa3RoO
hb4An3I1mY5JdOhEQ4t8+CPspqD7JHJd
=hZsK
-----END PGP SIGNATURE-----

Lex
Joined: 2010-02-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala
These are results I am getting:

for:   554.
while: 270.

for:   481.
while: 261.

for:   481.
while: 260.

for:   481.
while: 261.

for:   481.
while: 260.


Here is the code:

object Test {
 
  def main(args: Array[String]) {
    test()
    test()
    test()
    test()
    test()
  }
 
  def test() {
    val max = 400000000
   
    var start = 0L

    start = System.currentTimeMillis
    fordef(max)
    val timeFor = System.currentTimeMillis - start

    start = System.currentTimeMillis
    whiledef(max)
    val timeWhile = System.currentTimeMillis - start

    println("for:   " + timeFor + ".")
    println("while: " + timeWhile + ".")
  }
 
  def fordef(max: Int) {
   var total = 0L
   for (i <- 0 until max) {
     total += i
   }
   println(total)
 }

 def whiledef(max: Int) {
   var total = 0L
   var i = 0; while (i < max) {
     total += i
     i += 1
   }
   println(total)
 }
 
}

Lex
Joined: 2010-02-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala
In your code increment first, then increase total, which is not what the fordef does.

while (i < max) {
     i += 1
     total += i
   }

This should be:

while (i < max) {
      total += i
      i += 1
   }

Surprisingly, this makes a big difference:

for:   556.
while: 304.

for:   481.
while: 300.

for:   481.
while: 301.

for:   481.
while: 300.

for:   480.
while: 301.


A whole 40ms or about 15% slower. Still, the while loop significantly outperforms the for counterpart.


On Wed, Nov 30, 2011 at 11:01 PM, Aleksey Nikiforov <lexn82 [at] gmail [dot] com> wrote:
These are results I am getting:

for:   554.
while: 270.

for:   481.
while: 261.

for:   481.
while: 260.

for:   481.
while: 261.

for:   481.
while: 260.


Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Re: Yammer moving away from Scala
You're not warming up the JVM in those tests.   Make sure you run the code a few thousand times to see what hotspot will do first.
- Josh

On Thu, Dec 1, 2011 at 12:08 AM, Aleksey Nikiforov <lexn82 [at] gmail [dot] com> wrote:
In your code increment first, then increase total, which is not what the fordef does.

while (i < max) {
     i += 1
     total += i
   }

This should be:

while (i < max) {
      total += i
      i += 1
   }

Surprisingly, this makes a big difference:

for:   556.
while: 304.

for:   481.
while: 300.

for:   481.
while: 301.

for:   481.
while: 300.

for:   480.
while: 301.


A whole 40ms or about 15% slower. Still, the while loop significantly outperforms the for counterpart.


On Wed, Nov 30, 2011 at 11:01 PM, Aleksey Nikiforov <lexn82 [at] gmail [dot] com> wrote:
These are results I am getting:

for:   554.
while: 270.

for:   481.
while: 261.

for:   481.
while: 260.

for:   481.
while: 261.

for:   481.
while: 260.



Jesper Nordenberg
Joined: 2008-12-27,
User offline. Last seen 42 years 45 weeks ago.
Re: Yammer moving away from Scala

martin odersky skrev 2011-12-01 00:26:
> I'd really like to see case studies of large systems where this is the
> case. I am surprised to see you mention C++ because without GC you
> cannot even start implementing true functional programming.

What is true functional programming, and why would you need a GC to
implement it? C++11 supports lambda expressions with zero runtime
overhead, local type inference, type parameters, unboxed immutable data
types, scoped objects and ref counting (for automatic and exact
deallocation). I honestly don't see how it's less of a FPL than Scala is.

/Jesper Nordenberg

Russ P.
Joined: 2009-01-31,
User offline. Last seen 1 year 26 weeks ago.
Re: Re: Yammer moving away from Scala
As you know, a key aspect of FP is immutability. How do you effectively "change" an immutable object? You discard it and create a new, modified version of it (e.g., the case class copy method). If you had to manually clean up all the discarded objects, wouldn't you waste half your time doing that and still be left will memory leaks all over the place? Or am I missing something?

--Russ P.


On Wed, Nov 30, 2011 at 11:34 PM, Jesper Nordenberg <megagurka [at] yahoo [dot] com> wrote:
martin odersky skrev 2011-12-01 00:26:
I'd really like to see case studies of large systems where this is the
case. I am surprised to see you mention C++ because without GC you
cannot even start implementing true functional programming.

What is true functional programming, and why would you need a GC to implement it? C++11 supports lambda expressions with zero runtime overhead, local type inference, type parameters, unboxed immutable data types, scoped objects and ref counting (for automatic and exact deallocation). I honestly don't see how it's less of a FPL than Scala is.

/Jesper Nordenberg




--
http://RussP.us
odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Yammer moving away from Scala


On Thu, Dec 1, 2011 at 8:34 AM, Jesper Nordenberg <megagurka [at] yahoo [dot] com> wrote:
martin odersky skrev 2011-12-01 00:26:
I'd really like to see case studies of large systems where this is the
case. I am surprised to see you mention C++ because without GC you
cannot even start implementing true functional programming.

What is true functional programming, and why would you need a GC to implement it? C++11 supports lambda expressions with zero runtime overhead, local type inference, type parameters, unboxed immutable data types, scoped objects and ref counting (for automatic and exact deallocation). I honestly don't see how it's less of a FPL than Scala is.

The equivalent of
  { x =>      val arr = Array.ofDim(10)     y => arr(y) += 1; arr(y)  }
Here you have a true closure, i.e. a lambda abstraction that closes over its environment. Scoped objects don't help you here. Ref-counting could help if it's done everywhere. But ref-counting tends to have terrible performance in a multi-threaded environment. Also ref-counting does not pick up cyclic structures. You really need GC to to functional programming.  
Cheers
 -- Martin    
 

/Jesper Nordenberg



--
Martin Odersky
Prof., EPFL and Chairman, Typesafe
PSED, 1015 Lausanne, Switzerland
Tel. EPFL: +41 21 693 6863
Tel. Typesafe: +41 21 691 4967

Jesper Nordenberg
Joined: 2008-12-27,
User offline. Last seen 42 years 45 weeks ago.
Re: Yammer moving away from Scala

Russ Paielli gmail.com> writes:
> As you know, a key aspect of FP is immutability. How do you effectively
> "change" an immutable object? You discard it and create a new, modified
> version of it (e.g., the case class copy method). If you had to manually
> clean up all the discarded objects, wouldn't you waste half your time
> doing that and still be left will memory leaks all over the place? Or
> am I missing something?

In C++ you basically have three options:

- Use value types, which means there won't be any garbage produced. This
works well for small, non-recursive types (tuples, points, options,
records etc.)

- Use reference counting. Works well for non-cyclic data (which includes
basically all functional data types), but there is some runtime overhead
especially for objects which can be passed between threads

- Use a conservative GC like Boehm

/Jesper Nordenberg

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Re: Yammer moving away from Scala


On Thu, Dec 1, 2011 at 9:52 AM, Jesper Nordenberg <megagurka [at] yahoo [dot] com> wrote:
Russ Paielli <russ.paielli <at> gmail.com> writes:
> As you know, a key aspect of FP is immutability. How do you effectively
> "change" an immutable object? You discard it and create a new, modified
> version of it (e.g., the case class copy method). If you had to manually
> clean up all the discarded objects, wouldn't you waste half your time
> doing that and still be left will memory leaks all over the place? Or
> am I missing something?

In C++ you basically have three options:

- Use value types, which means there won't be any garbage produced. This
works well for small, non-recursive types (tuples, points, options,
records etc.)

- Use reference counting. Works well for non-cyclic data (which includes
basically all functional data types), but there is some runtime overhead
especially for objects which can be passed between threads
 
- Use a conservative GC like Boehm


Out of curiosity: I wonder how common that is?  Are there well known large C++ systems that use conservative GC?

Cheers

 -- Martin

Paul Brauner
Joined: 2010-10-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala

I know inkscape does
(http://wiki.inkscape.org/wiki/index.php/Compiling_Inkscape#Boehm-GC).

Paul

On Thu, Dec 1, 2011 at 10:25, martin odersky wrote:
>
>
> On Thu, Dec 1, 2011 at 9:52 AM, Jesper Nordenberg
> wrote:
>>
>> Russ Paielli gmail.com> writes:
>> > As you know, a key aspect of FP is immutability. How do you effectively
>> > "change" an immutable object? You discard it and create a new, modified
>> > version of it (e.g., the case class copy method). If you had to manually
>> > clean up all the discarded objects, wouldn't you waste half your time
>> > doing that and still be left will memory leaks all over the place? Or
>> > am I missing something?
>>
>> In C++ you basically have three options:
>>
>> - Use value types, which means there won't be any garbage produced. This
>> works well for small, non-recursive types (tuples, points, options,
>> records etc.)
>>
>> - Use reference counting. Works well for non-cyclic data (which includes
>> basically all functional data types), but there is some runtime overhead
>> especially for objects which can be passed between threads
>>
>>
>> - Use a conservative GC like Boehm
>>
>
> Out of curiosity: I wonder how common that is?  Are there well known large
> C++ systems that use conservative GC?
>
> Cheers
>
>  -- Martin
>

Stefan Wagner
Joined: 2011-04-08,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 01.12.2011 05:53, schrieb Aleksey Nikiforov:
> Clearly you are doing something wrong.

You're right, I'am sorry. My code labelled the cases in the wrong way.
For 200 million to 2 billion iterations, I now get:

0 while for rekdef
1 0.681 2.629 0.765
2 1.31 6.221 1.499
3 1.979 7.652 2.413
4 2.649 9.109 3.216
5 3.308 12.299 4.01
6 3.947 13.593 5.209
7 4.845 16.433 5.486
8 6.255 18.343 5.959
9 5.947 20.686 6.697
10 6.607 24.32 7.429

Rekdef is a recursive implementation:

@tailrec
def rekdef (max: I, sofar: O): O =
if (max == 0) sofar
else rekdef (max-1, sofar+max)

def rekdef (max: I): O = rekdef (max, 0L)

Jesper Nordenberg
Joined: 2008-12-27,
User offline. Last seen 42 years 45 weeks ago.
Re: Yammer moving away from Scala

martin odersky epfl.ch> writes:
> The equivalent of
>
>   { x => 
>      val arr = Array.ofDim(10)
>      y => arr(y) += 1; arr(y)
>   }
>
> Here you have a true closure, i.e. a lambda abstraction that closes over
> its environment. Scoped objects don't help you here. Ref-counting could
> help if it's done everywhere. But ref-counting tends to have terrible
> performance in a multi-threaded environment. Also ref-counting does not
> pick up cyclic structures. 

Interesting that you picked an example containing mutation and arrays when
we are discussing functional programming. In these cases it's certainly
useful to have a GC (for example ref counting), but it doesn't have anything
to do with if the code is functional or imperative.

Regarding cycles, lots of functional data types don't contain cycles (basically
all Scala immutable collection types for example). And there are lots of
other functional data types that can be value types.

Regarding multi threaded performance of ref counting, you can restrict
object sharing between threads and for objects that is shared you can mitigate
the performance hit by keeping thread specific ref counts.

Another solution is to use a conservative GC like Boehm with C++.

> You really need GC to to functional programming.  

I don't agree (even if you consider ref counting as a form of GC). You
can certainly do some forms of functional programming without a GC, but
for other forms it's certainly an advantage to have one.

/Jesper Nordenberg

Sebastien Bocq
Joined: 2008-12-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala
2011/12/1 Jesper Nordenberg <megagurka [at] yahoo [dot] com>
martin odersky <martin.odersky <at> epfl.ch> writes:
> The equivalent of
>
>   { x => 
>      val arr = Array.ofDim(10)
>      y => arr(y) += 1; arr(y)
>   }
>
> Here you have a true closure, i.e. a lambda abstraction that closes over
> its environment. Scoped objects don't help you here. Ref-counting could
> help if it's done everywhere. But ref-counting tends to have terrible
> performance in a multi-threaded environment. Also ref-counting does not
> pick up cyclic structures. 

Interesting that you picked an example containing mutation and arrays when
we are discussing functional programming. In these cases it's certainly
useful to have a GC (for example ref counting), but it doesn't have anything
to do with if the code is functional or imperative.

Regarding cycles, lots of functional data types don't contain cycles (basically
all Scala immutable collection types for example). And there are lots of
other functional data types that can be value types.

Regarding multi threaded performance of ref counting, you can restrict
object sharing between threads and for objects that is shared you can mitigate
the performance hit by keeping thread specific ref counts.

Another solution is to use a conservative GC like Boehm with C++.

> You really need GC to to functional programming.  

I don't agree (even if you consider ref counting as a form of GC). You
can certainly do some forms of functional programming without a GC, but
for other forms it's certainly an advantage to have one.

/Jesper Nordenberg



This question has been nagging at me for a while. Wouldn't it be possible for a compiler to allocate an efficient and recyclable stack-like structure for a restricted subset of a functional programming language without relying heavily on the runtime? For example, for a pure and total subset with single-threaded (linearly owned?) data access, ...? Btw, does anybody knows a mailing-list where one can ask such general question on functional programming?

Cheers,
Sébastien
odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Re: Yammer moving away from Scala


2011/12/1 √iktor Ҡlang <viktor [dot] klang [at] gmail [dot] com>


On Thu, Dec 1, 2011 at 2:14 PM, Sébastien Bocq <sebastien [dot] bocq [at] gmail [dot] com> wrote:
2011/12/1 Jesper Nordenberg <megagurka [at] yahoo [dot] com>
martin odersky <martin.odersky <at> epfl.ch> writes:
> The equivalent of
>
>   { x => 
>      val arr = Array.ofDim(10)
>      y => arr(y) += 1; arr(y)
>   }
>
> Here you have a true closure, i.e. a lambda abstraction that closes over
> its environment. Scoped objects don't help you here. Ref-counting could
> help if it's done everywhere. But ref-counting tends to have terrible
> performance in a multi-threaded environment. Also ref-counting does not
> pick up cyclic structures. 

Interesting that you picked an example containing mutation and arrays when
we are discussing functional programming. In these cases it's certainly
useful to have a GC (for example ref counting), but it doesn't have anything
to do with if the code is functional or imperative.

Regarding cycles, lots of functional data types don't contain cycles (basically
all Scala immutable collection types for example). And there are lots of
other functional data types that can be value types.

Regarding multi threaded performance of ref counting, you can restrict
object sharing between threads and for objects that is shared you can mitigate
the performance hit by keeping thread specific ref counts.

Another solution is to use a conservative GC like Boehm with C++.

> You really need GC to to functional programming.  

I don't agree (even if you consider ref counting as a form of GC). You
can certainly do some forms of functional programming without a GC, but
for other forms it's certainly an advantage to have one.

I read some interesting papers a couple of years ago about a language with compile-time garbage collection.
Unfortunately don't have a link, but I believe it was on LtU.

There's the work on region analysis (Tofte and Talpin) which had led to an SML compiler without a garbage collector (MLKit). That was in the 90s. Seemed to work reasonably well but I do not think it caught on in a big way in the ML community.
Chers
 -- Martin
Viktor Klang
Joined: 2008-12-17,
User offline. Last seen 1 year 27 weeks ago.
Re: Re: Yammer moving away from Scala


On Thu, Dec 1, 2011 at 2:14 PM, Sébastien Bocq <sebastien [dot] bocq [at] gmail [dot] com> wrote:
2011/12/1 Jesper Nordenberg <megagurka [at] yahoo [dot] com>
martin odersky <martin.odersky <at> epfl.ch> writes:
> The equivalent of
>
>   { x => 
>      val arr = Array.ofDim(10)
>      y => arr(y) += 1; arr(y)
>   }
>
> Here you have a true closure, i.e. a lambda abstraction that closes over
> its environment. Scoped objects don't help you here. Ref-counting could
> help if it's done everywhere. But ref-counting tends to have terrible
> performance in a multi-threaded environment. Also ref-counting does not
> pick up cyclic structures. 

Interesting that you picked an example containing mutation and arrays when
we are discussing functional programming. In these cases it's certainly
useful to have a GC (for example ref counting), but it doesn't have anything
to do with if the code is functional or imperative.

Regarding cycles, lots of functional data types don't contain cycles (basically
all Scala immutable collection types for example). And there are lots of
other functional data types that can be value types.

Regarding multi threaded performance of ref counting, you can restrict
object sharing between threads and for objects that is shared you can mitigate
the performance hit by keeping thread specific ref counts.

Another solution is to use a conservative GC like Boehm with C++.

> You really need GC to to functional programming.  

I don't agree (even if you consider ref counting as a form of GC). You
can certainly do some forms of functional programming without a GC, but
for other forms it's certainly an advantage to have one.

I read some interesting papers a couple of years ago about a language with compile-time garbage collection.
Unfortunately don't have a link, but I believe it was on LtU.

Cheers,

 

/Jesper Nordenberg



This question has been nagging at me for a while. Wouldn't it be possible for a compiler to allocate an efficient and recyclable stack-like structure for a restricted subset of a functional programming language without relying heavily on the runtime? For example, for a pure and total subset with single-threaded (linearly owned?) data access, ...? Btw, does anybody knows a mailing-list where one can ask such general question on functional programming?

Cheers,
Sébastien



--
Viktor Klang

Akka Tech LeadTypesafe - Enterprise-Grade Scala from the Experts

Twitter: @viktorklang
Viktor Klang
Joined: 2008-12-17,
User offline. Last seen 1 year 27 weeks ago.
Re: Re: Yammer moving away from Scala


2011/12/1 martin odersky <martin [dot] odersky [at] epfl [dot] ch>


2011/12/1 √iktor Ҡlang <viktor [dot] klang [at] gmail [dot] com>


On Thu, Dec 1, 2011 at 2:14 PM, Sébastien Bocq <sebastien [dot] bocq [at] gmail [dot] com> wrote:
2011/12/1 Jesper Nordenberg <megagurka [at] yahoo [dot] com>
martin odersky <martin.odersky <at> epfl.ch> writes:
> The equivalent of
>
>   { x => 
>      val arr = Array.ofDim(10)
>      y => arr(y) += 1; arr(y)
>   }
>
> Here you have a true closure, i.e. a lambda abstraction that closes over
> its environment. Scoped objects don't help you here. Ref-counting could
> help if it's done everywhere. But ref-counting tends to have terrible
> performance in a multi-threaded environment. Also ref-counting does not
> pick up cyclic structures. 

Interesting that you picked an example containing mutation and arrays when
we are discussing functional programming. In these cases it's certainly
useful to have a GC (for example ref counting), but it doesn't have anything
to do with if the code is functional or imperative.

Regarding cycles, lots of functional data types don't contain cycles (basically
all Scala immutable collection types for example). And there are lots of
other functional data types that can be value types.

Regarding multi threaded performance of ref counting, you can restrict
object sharing between threads and for objects that is shared you can mitigate
the performance hit by keeping thread specific ref counts.

Another solution is to use a conservative GC like Boehm with C++.

> You really need GC to to functional programming.  

I don't agree (even if you consider ref counting as a form of GC). You
can certainly do some forms of functional programming without a GC, but
for other forms it's certainly an advantage to have one.

I read some interesting papers a couple of years ago about a language with compile-time garbage collection.
Unfortunately don't have a link, but I believe it was on LtU.

There's the work on region analysis (Tofte and Talpin) which had led to an SML compiler without a garbage collector (MLKit). That was in the 90s. Seemed to work reasonably well but I do not think it caught on in a big way in the ML community.

Thanks for the pointer, I think it was this paper I read: http://lambda-the-ultimate.org/node/2047
 
Cheers,


Chers
 -- Martin



--
Viktor Klang

Akka Tech LeadTypesafe - Enterprise-Grade Scala from the Experts

Twitter: @viktorklang
Sebastien Bocq
Joined: 2008-12-18,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala


2011/12/1 √iktor Ҡlang <viktor [dot] klang [at] gmail [dot] com>


2011/12/1 martin odersky <martin [dot] odersky [at] epfl [dot] ch>


2011/12/1 √iktor Ҡlang <viktor [dot] klang [at] gmail [dot] com>


On Thu, Dec 1, 2011 at 2:14 PM, Sébastien Bocq <sebastien [dot] bocq [at] gmail [dot] com> wrote:
2011/12/1 Jesper Nordenberg <megagurka [at] yahoo [dot] com>
martin odersky <martin.odersky <at> epfl.ch> writes:
> The equivalent of
>
>   { x => 
>      val arr = Array.ofDim(10)
>      y => arr(y) += 1; arr(y)
>   }
>
> Here you have a true closure, i.e. a lambda abstraction that closes over
> its environment. Scoped objects don't help you here. Ref-counting could
> help if it's done everywhere. But ref-counting tends to have terrible
> performance in a multi-threaded environment. Also ref-counting does not
> pick up cyclic structures. 

Interesting that you picked an example containing mutation and arrays when
we are discussing functional programming. In these cases it's certainly
useful to have a GC (for example ref counting), but it doesn't have anything
to do with if the code is functional or imperative.

Regarding cycles, lots of functional data types don't contain cycles (basically
all Scala immutable collection types for example). And there are lots of
other functional data types that can be value types.

Regarding multi threaded performance of ref counting, you can restrict
object sharing between threads and for objects that is shared you can mitigate
the performance hit by keeping thread specific ref counts.

Another solution is to use a conservative GC like Boehm with C++.

> You really need GC to to functional programming.  

I don't agree (even if you consider ref counting as a form of GC). You
can certainly do some forms of functional programming without a GC, but
for other forms it's certainly an advantage to have one.

I read some interesting papers a couple of years ago about a language with compile-time garbage collection.
Unfortunately don't have a link, but I believe it was on LtU.

There's the work on region analysis (Tofte and Talpin) which had led to an SML compiler without a garbage collector (MLKit). That was in the 90s. Seemed to work reasonably well but I do not think it caught on in a big way in the ML community.

Thanks for the pointer, I think it was this paper I read: http://lambda-the-ultimate.org/node/2047
 
Cheers,


Chers
 -- Martin



--
Viktor Klang

Akka Tech LeadTypesafe - Enterprise-Grade Scala from the Experts

Twitter: @viktorklang

Thanks for the pointers too. Region analysis looked promising, it is strange that this research was not pursued further. This reminds me of this famous quote:

"LISP programmers know the value of everything, but the cost of nothing" - Alan Perlis

Maybe the demand was not there at the time but maybe now would be a good time to get it out the fridge since functional programming seems to attract more performance minded people.

Cheers,
Sébastien
Paul Brauner
Joined: 2010-10-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala

Well, region analysis is still an active research subject AFAIK, but
it's not always applied to garbage collection. This talk from François
Pottier (http://gallium.inria.fr/~fpottier/slides/fpottier-2007-05-linear-bestiar...)
does a good job at summarizing this research type system wise.

Paul

2011/12/1 Sébastien Bocq :
>
>
> 2011/12/1 √iktor Ҡlang
>>
>>
>>
>> 2011/12/1 martin odersky
>>>
>>>
>>>
>>> 2011/12/1 √iktor Ҡlang
>>>>
>>>>
>>>>
>>>> On Thu, Dec 1, 2011 at 2:14 PM, Sébastien Bocq
>>>> wrote:
>>>>>
>>>>> 2011/12/1 Jesper Nordenberg
>>>>>>
>>>>>> martin odersky epfl.ch> writes:
>>>>>> > The equivalent of
>>>>>> >
>>>>>> >   { x =>
>>>>>> >      val arr = Array.ofDim(10)
>>>>>> >      y => arr(y) += 1; arr(y)
>>>>>> >   }
>>>>>> >
>>>>>> > Here you have a true closure, i.e. a lambda abstraction that closes
>>>>>> > over
>>>>>> > its environment. Scoped objects don't help you here. Ref-counting
>>>>>> > could
>>>>>> > help if it's done everywhere. But ref-counting tends to have
>>>>>> > terrible
>>>>>> > performance in a multi-threaded environment. Also ref-counting does
>>>>>> > not
>>>>>> > pick up cyclic structures.
>>>>>>
>>>>>> Interesting that you picked an example containing mutation and arrays
>>>>>> when
>>>>>> we are discussing functional programming. In these cases it's
>>>>>> certainly
>>>>>> useful to have a GC (for example ref counting), but it doesn't have
>>>>>> anything
>>>>>> to do with if the code is functional or imperative.
>>>>>>
>>>>>> Regarding cycles, lots of functional data types don't contain cycles
>>>>>> (basically
>>>>>> all Scala immutable collection types for example). And there are lots
>>>>>> of
>>>>>> other functional data types that can be value types.
>>>>>>
>>>>>> Regarding multi threaded performance of ref counting, you can restrict
>>>>>> object sharing between threads and for objects that is shared you can
>>>>>> mitigate
>>>>>> the performance hit by keeping thread specific ref counts.
>>>>>>
>>>>>> Another solution is to use a conservative GC like Boehm with C++.
>>>>>>
>>>>>> > You really need GC to to functional programming.
>>>>>>
>>>>>> I don't agree (even if you consider ref counting as a form of GC). You
>>>>>> can certainly do some forms of functional programming without a GC,
>>>>>> but
>>>>>> for other forms it's certainly an advantage to have one.
>>>>
>>>>
>>>> I read some interesting papers a couple of years ago about a language
>>>> with compile-time garbage collection.
>>>> Unfortunately don't have a link, but I believe it was on LtU.
>>>
>>>
>>> There's the work on region analysis (Tofte and Talpin) which had led to
>>> an SML compiler without a garbage collector (MLKit). That was in the 90s.
>>> Seemed to work reasonably well but I do not think it caught on in a big way
>>> in the ML community.
>>>
>>
>> Thanks for the pointer, I think it was this paper I read:
>> http://lambda-the-ultimate.org/node/2047
>>
>> Cheers,
>> √
>>
>>> Chers
>>>
>>>  -- Martin
>>>
>>
>>
>>
>> --
>> Viktor Klang
>>
>> Akka Tech Lead
>> Typesafe - Enterprise-Grade Scala from the Experts
>>
>> Twitter: @viktorklang
>>
>
> Thanks for the pointers too. Region analysis looked promising, it is strange
> that this research was not pursued further. This reminds me of this famous
> quote:
>
> "LISP programmers know the value of everything, but the cost of nothing" -
> Alan Perlis
>
> Maybe the demand was not there at the time but maybe now would be a good
> time to get it out the fridge since functional programming seems to attract
> more performance minded people.
>
> Cheers,
> Sébastien

Stefan Wagner
Joined: 2011-04-08,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 01.12.2011 05:53, schrieb Aleksey Nikiforov:
> Clearly you are doing something wrong.
>

I made a new test - now I just compiled with -optimise, and here is the
result:

scala ForWhileBench 2000000000

0 while for rekdef
1 0.642 1.0 0.737
2 1.289 1.914 1.448
3 1.888 2.504 2.167
4 2.882 3.371 2.902
5 3.13 4.225 3.63
6 3.745 5.106 4.329
7 4.423 5.92 5.048
8 5.031 6.742 5.813
9 5.667 7.553 6.569
10 7.13 8.427 7.239

It's only a 20% difference between while and for, and nearly no
differnce to the tailcall recursive solution.

Jori Jovanovich
Joined: 2011-04-24,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala

On Wed, Nov 30, 2011 at 4:55 PM, Erik Osheim <erik [at] plastic-idolatry [dot] com> wrote:
On Wed, Nov 30, 2011 at 11:39:56PM +0100, Jesper Nordenberg wrote:
> Hmmm, what benefits do you see in using Scala over Java if you only
> use plain old imperative loops?

Tons, of course. :)

At my job we do much of our work on Arrays (gasp) using while loops
(gasp) and yet we benefit from many awesome features in Scala:

 * cleaner-looking code
 * traits
 * unified primitive/boxed types
 * much better generics
 * type classes via implicits
 * pattern matching/extractors

These are only a subset of course.

I understand why a lot of us are defensive about this issue (no one
likes to be criticized, or have their favorite language criticized).
But there are legitimate reasons to want fast imperative-style loops
besides while loop, and legitimate Scala users who need or would like
to see this kind of use optimized.
 I agree 100% with this.  I like Odersky's statement that you should "prefer immutable but not demonize mutable".  I'd like the same stance toward imperative:  "prefer functional but don't demonize imperative".  I write mostly functional code for but for hot spots I tend to have to use imperative style.  I'd also like break/continue so that Scala wouldn't be a "second class" imperative language.  One of the biggest attraction of Scala to me is its support of diverse programming styles.

Viktor Klang
Joined: 2008-12-17,
User offline. Last seen 1 year 27 weeks ago.
Re: Re: Yammer moving away from Scala

a big, fat +1 on that!

On Dec 1, 2011 9:28 PM, "Jori Jovanovich" <jori [at] dimensiology [dot] com> wrote:

On Wed, Nov 30, 2011 at 4:55 PM, Erik Osheim <erik [at] plastic-idolatry [dot] com> wrote:
On Wed, Nov 30, 2011 at 11:39:56PM +0100, Jesper Nordenberg wrote:
> Hmmm, what benefits do you see in using Scala over Java if you only
> use plain old imperative loops?

Tons, of course. :)

At my job we do much of our work on Arrays (gasp) using while loops
(gasp) and yet we benefit from many awesome features in Scala:

 * cleaner-looking code
 * traits
 * unified primitive/boxed types
 * much better generics
 * type classes via implicits
 * pattern matching/extractors

These are only a subset of course.

I understand why a lot of us are defensive about this issue (no one
likes to be criticized, or have their favorite language criticized).
But there are legitimate reasons to want fast imperative-style loops
besides while loop, and legitimate Scala users who need or would like
to see this kind of use optimized.
 I agree 100% with this.  I like Odersky's statement that you should "prefer immutable but not demonize mutable".  I'd like the same stance toward imperative:  "prefer functional but don't demonize imperative".  I write mostly functional code for but for hot spots I tend to have to use imperative style.  I'd also like break/continue so that Scala wouldn't be a "second class" imperative language.  One of the biggest attraction of Scala to me is its support of diverse programming styles.

Ken McDonald
Joined: 2011-02-13,
User offline. Last seen 42 years 45 weeks ago.
Re: Yammer moving away from Scala


On Wednesday, November 30, 2011 5:00:30 PM UTC-6, Jesper Nordenberg wrote:
Aleksey Nikiforov skrev 2011-11-30 23:44:
> There is nothing wrong with imperative programming.

Actually there is, but let's not get into that discussion.

No, there isn't. I spent three years in a Ph.D. program in Functional Programming, and one of the things that caused me to quit was the amazing ability of FP researchers to declare that FP was better, _even when_ they couldn't cover 1/10th of the solution space covered by _IP_.
Saying, or even implying, "you wouldn't have that problem if you programmed functionally" isn't an answer, it's an evasion.
Ken
Tony Morris 2
Joined: 2009-03-20,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Yammer moving away from Scala

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/04/2011 05:09 AM, Kenneth McDonald wrote:
>
>
> On Wednesday, November 30, 2011 5:00:30 PM UTC-6, Jesper Nordenberg wrote:
>>
>> Aleksey Nikiforov skrev 2011-11-30 23:44:
>>> There is nothing wrong with imperative programming.
>>
>> Actually there is, but let's not get into that discussion.
>>
> No, there isn't. I spent three years in a Ph.D. program in Functional
> Programming, and one of the things that caused me to quit was the amazing
> ability of FP researchers to declare that FP was better, _even when_ they
> couldn't cover 1/10th of the solution space covered by _IP_.
>
> Saying, or even implying, "you wouldn't have that problem if you programmed
> functionally" isn't an answer, it's an evasion.
>
> Ken
>

I think you missed the bit where "[FP] couldn't cover 1/10th of the
solution space covered by _IP_" is a blatant false statement and
elementary mistake.

Aleksey is correct and I could demonstrate this to you in less than 3 years.

The illusion that learning/teaching even happens at universities is the
reason I quit lecturing. You didn't spend 3 years *learning* about
functional programming. Sorry to break the news to you.

DaveScala
Joined: 2011-03-18,
User offline. Last seen 1 year 21 weeks ago.
Re: Yammer moving away from Scala

> At my job we do much of our work on Arrays (gasp) using while loops
> (gasp) and yet we benefit from many awesome features in Scala:
I am interested why imperative style for/while loops are still
wanted.
Why isn't it possible to solve the problem with for-comprehensions and
in parallel the scala way for speedup?
PS I don't know what gasp is.

Erik Engbrecht
Joined: 2008-12-19,
User offline. Last seen 3 years 18 weeks ago.
Re: Yammer moving away from Scala
Gasp: http://www.urbandictionary.com/define.php?term=gasp
As for for-comprehensions, they are in some cases significantly slower than while loops.  Also, parallelization adds a non-trivial amount of overhead, so in order for it to result in a speedup the cost of the computations being performed needs to exceed that overhead by a sufficient enough margin that parallelization will result in a real speedup.  If you are performing a light computations (e.g. summing primitive numbers) and the collection you're doing it on is relatively short than trying to parallelize it will make the computation take longer.
DaveScala
Joined: 2011-03-18,
User offline. Last seen 1 year 21 weeks ago.
Re: Yammer moving away from Scala

On 4 dec, 19:09, Erik Engbrecht wrote:
> Gasp:http://www.urbandictionary.com/define.php?term=gasp
>
:D (I was looking for some high performance array library)

> As for for-comprehensions, they are in some cases significantly slower than
> while loops.  Also, parallelization adds a non-trivial amount of overhead,
> so in order for it to result in a speedup the cost of the computations
> being performed needs to exceed that overhead by a sufficient enough margin
> that parallelization will result in a real speedup.  If you are performing
> a light computations (e.g. summing primitive numbers) and the collection
> you're doing it on is relatively short than trying to parallelize it will
> make the computation take longer.
Yes, I think for some problems that imperative style for-loops might
be better than for-comprehensions.
After learning about Amdahl's Law and Gustafson's Law I think some
problems are better to be solved in parallel and some problems are
better to be solved sequentially.

I think it depends on (cannot prove it):
-the number of elements
-the number of cores available
-the problem itself and how you formulate the solution

And for sequential processing for-loops (optimized for bounded number
of elements) are faster
than while loops (optimized for unbounded number of elements)
which are faster than for-comprehensions (optimized for lists and it
takes time to iterate and bind an element).
So I think it is good to include plain imperative style for-loops as
well to Scala.

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland