This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Human perception of complexity

36 replies
Bernd Johannes
Joined: 2011-01-28,
User offline. Last seen 42 years 45 weeks ago.

Hello Scalaist,

we've seen discussions about scala complexity, complex problems, fear of
complexity and related matters around scala (but not exclusively) for years
now.

This topic deals (among others) with something very fundamental: computer -
human - interaction. A programming language along with its environment is a
digital tool that interacts with humans (developers).

As I got the impression over time that all those discussions tend to run in
circles maybe something can be done to push things forward. Is there a
possiblity to use the often taunted "academic" roots of scala as it strength
by inducing some real research?

There have been some language metric papers and mesurement approaches around
for different computer languages - most of them dealing with procedural and OO
metrics. So I think they are not simply applicable to scala and its specific
constructs (ok - the very basic ones like "LOC" or "LLOC" can be used as is,
but their value is very limited).

Can't somebody with an appropriate academic network try to inspire some
research in that direction? One of the basic questions would be "how to
measure scala code complexity" (and this simple question in itself will unfold
into a considerable avalanche of more detailed questions and some real
problems to solve. I don't expect this to be a copy and paste work...).

In the end scala would gain some "hard facts" about complexity and how to
measure it. And this in turn will lead to appropriate tools to do those
measurements - which will help to dispell complexity fears ("look - the metric
says: It's good" ... it's all about psychology ;-) )

Just some thoughts...

Greetings
Bernd

kolotyluk
Joined: 2010-06-04,
User offline. Last seen 5 weeks 15 hours ago.
Re: Human perception of complexity

Great thinking Bernd. While I would not have thought of this as a
computer-human-interaction issue at first, I can now see that it is.
Thanks for framing it.

Cheers, Eric

On 2011-11-20 10:50 PM, Bernd Johannes wrote:
> Hello Scalaist,
>
> we've seen discussions about scala complexity, complex problems, fear of
> complexity and related matters around scala (but not exclusively) for years
> now.
>
> This topic deals (among others) with something very fundamental: computer -
> human - interaction. A programming language along with its environment is a
> digital tool that interacts with humans (developers).
>
> As I got the impression over time that all those discussions tend to run in
> circles maybe something can be done to push things forward. Is there a
> possiblity to use the often taunted "academic" roots of scala as it strength
> by inducing some real research?
>
> There have been some language metric papers and mesurement approaches around
> for different computer languages - most of them dealing with procedural and OO
> metrics. So I think they are not simply applicable to scala and its specific
> constructs (ok - the very basic ones like "LOC" or "LLOC" can be used as is,
> but their value is very limited).
>
> Can't somebody with an appropriate academic network try to inspire some
> research in that direction? One of the basic questions would be "how to
> measure scala code complexity" (and this simple question in itself will unfold
> into a considerable avalanche of more detailed questions and some real
> problems to solve. I don't expect this to be a copy and paste work...).
>
> In the end scala would gain some "hard facts" about complexity and how to
> measure it. And this in turn will lead to appropriate tools to do those
> measurements - which will help to dispell complexity fears ("look - the metric
> says: It's good" ... it's all about psychology ;-) )
>
> Just some thoughts...
>
> Greetings
> Bernd
>

H-star Development
Joined: 2010-04-14,
User offline. Last seen 2 years 26 weeks ago.
Re: Human perception of complexity

i thought about this.

i think something's "descriptive complexity" is the smallest number of language elements necessary to describe it. it completely depends on the language you are using to describe it.

if something seems to be complex, you are most likely on a low abstraction level instead of a higher one - or that which you are using is simply crap, cannot be abstracted away and should not be used (like css based layouts ;)

the important question is: how much low level complexity does leak through and cannot be abstracted away?

-------- Original-Nachricht --------
> Datum: Mon, 21 Nov 2011 08:29:00 -0800
> Von: Eric Kolotyluk
> An: scala-debate [at] googlegroups [dot] com
> Betreff: Re: [scala-debate] Human perception of complexity

> Great thinking Bernd. While I would not have thought of this as a
> computer-human-interaction issue at first, I can now see that it is.
> Thanks for framing it.
>
> Cheers, Eric
>
> On 2011-11-20 10:50 PM, Bernd Johannes wrote:
> > Hello Scalaist,
> >
> > we've seen discussions about scala complexity, complex problems, fear of
> > complexity and related matters around scala (but not exclusively) for
> years
> > now.
> >
> > This topic deals (among others) with something very fundamental:
> computer -
> > human - interaction. A programming language along with its environment
> is a
> > digital tool that interacts with humans (developers).
> >
> > As I got the impression over time that all those discussions tend to run
> in
> > circles maybe something can be done to push things forward. Is there a
> > possiblity to use the often taunted "academic" roots of scala as it
> strength
> > by inducing some real research?
> >
> > There have been some language metric papers and mesurement approaches
> around
> > for different computer languages - most of them dealing with procedural
> and OO
> > metrics. So I think they are not simply applicable to scala and its
> specific
> > constructs (ok - the very basic ones like "LOC" or "LLOC" can be used as
> is,
> > but their value is very limited).
> >
> > Can't somebody with an appropriate academic network try to inspire some
> > research in that direction? One of the basic questions would be "how to
> > measure scala code complexity" (and this simple question in itself will
> unfold
> > into a considerable avalanche of more detailed questions and some real
> > problems to solve. I don't expect this to be a copy and paste work...).
> >
> > In the end scala would gain some "hard facts" about complexity and how
> to
> > measure it. And this in turn will lead to appropriate tools to do those
> > measurements - which will help to dispell complexity fears ("look - the
> metric
> > says: It's good" ... it's all about psychology ;-) )
> >
> > Just some thoughts...
> >
> > Greetings
> > Bernd
> >

Erik Post
Joined: 2011-06-10,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity
Hi Bernd,
Not saying this wouldn't be great to have, but I suspect that a lot of the folks calling Scala too academic won't necessarily be swayed by an 'academic' proof of how un-complex Scala is. I'd propose to just stick a few pages of side by side Scala vs. Java comparisons on docs.scala-lang.org comparing things like Hello World, singletons, getters/setters, functions vs. anonymous inner classes, etc. It think this might be a bit more fruitful with some of the 'too complex' crowd. I think it would be totally sweet to have an url to send people propagating the complexity nonsense.
Cheers,Erik

On Monday, 21 November 2011 07:50:14 UTC+1, Bernd wrote:
Hello Scalaist,

we've seen discussions about scala complexity, complex problems, fear of
complexity and related matters around scala (but not exclusively) for years
now.

This topic deals (among others) with something very fundamental: computer -
human - interaction.  A programming language along with its environment is a
digital tool that interacts with humans (developers).

As I got the impression over time that all those discussions tend to run in
circles maybe something can be done to push things forward. Is there a
possiblity to use the often taunted "academic" roots of scala as it strength
by inducing some real research?

There have been some language metric papers and mesurement approaches around
for different computer languages - most of them dealing with procedural and OO
metrics. So I think they are not simply applicable to scala and its specific
constructs (ok - the very basic ones like "LOC" or "LLOC" can be used as is,
but their value is very limited).

Can't somebody with an appropriate academic network try to inspire some
research in that direction? One of the basic questions would be "how to
measure scala code complexity" (and this simple question in itself will unfold
into a considerable avalanche of more detailed questions and some real
problems to solve. I don't expect this to be a copy and paste work...).

In the end scala would gain some "hard facts" about complexity and how to
measure it. And this in turn will lead to appropriate tools to do those
measurements - which will help to dispell complexity fears ("look - the metric
says: It's good" ... it's all about psychology ;-) )

Just some thoughts...

Greetings
Bernd

Raoul Duke
Joined: 2009-01-05,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Human perception of complexity

On Mon, Nov 21, 2011 at 9:28 AM, Erik Post wrote:
> 'academic' proof of how un-complex Scala is. I'd propose to just stick a few
> pages of side by side Scala vs. Java comparisons on docs.scala-lang.org
> comparing things like Hello World, singletons, getters/setters, functions
> vs. anonymous inner classes, etc. It think this might be a bit more fruitful
> with some of the 'too complex' crowd. I think it would be totally sweet to
> have an url to send people propagating the complexity nonsense.

but then out of fairness you might also have to show the examples of
"advanced" type situations where the code looks like utter jibberish
to a Regular Joe.

Erik Post
Joined: 2011-06-10,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Human perception of complexity

On Monday, 21 November 2011 19:11:16 UTC+1, raould wrote:
On Mon, Nov 21, 2011 at 9:28 AM, Erik Post <eriks [dot] [dot] [dot] [at] gmail [dot] com> wrote:
> 'academic' proof of how un-complex Scala is. I'd propose to just stick a few
> pages of side by side Scala vs. Java comparisons on docs.scala-lang.org
> comparing things like Hello World, singletons, getters/setters, functions
> vs. anonymous inner classes, etc. It think this might be a bit more fruitful
> with some of the 'too complex' crowd. I think it would be totally sweet to
> have an url to send people propagating the complexity nonsense.

but then out of fairness you might also have to show the examples of
"advanced" type situations where the code looks like utter jibberish
to a Regular Joe.

Are you suggesting that Java would compare favourably in such situations?
Bernd Johannes
Joined: 2011-01-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Human perception of complexity

Am Montag, 21. November 2011, 18:28:10 schrieb Erik Post:

Hi Erik,

> Not saying this wouldn't be great to have, but I suspect that a lot of the
> folks calling Scala too academic won't necessarily be swayed by an
> 'academic' proof of how un-complex Scala is. I'd propose to just stick a

I didn't think along the lines of persuading somebody of something. It's just
that it occurs to me that most of the discussion is centered around often
strong "feelings" of some sort. That's not bad in itself - and it's relevant
for sure. But it's very subjective (of course) and hard to correlate.
Drawing conclusions out of this is akin to alchemy.

I think we can do better than that. If some parts of scala are hard to grasp
(either by concept or by source deciphering) we should be able to identify
this in a more reproducible way.

This knowledge can then be used to define some relevant metrics to predict if
a given code is perceived as rather convoluted or clear.

And "well known" metrics are appealing to people who are in charge of quality.
At least in the context where I was working some time ago. Because they didn't
have the knowledge or the time to judge for themselves.

Apart from all this - I don't believe that the average scala code is more
complex than comparable solutions in other languages. It's rather that the
code has more "constrasts". There are long runs where successful abstraction
makes the code very clear and well... almost boring. And then there are the
spots where "things happen". Of course one can always pick such an example to
show how unwieldy scala is. But that's an indication for the power of the
language.

Greetings
Bernd

Raoul Duke
Joined: 2009-01-05,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Human perception of complexity

On Mon, Nov 21, 2011 at 12:35 PM, Erik Post wrote:
> Are you suggesting that Java would compare favourably in such situations?

presumably it depends on what the cases are. i'm guessing there are
examples where a single cast in java would do the trick. :-)

Erik Post
Joined: 2011-06-10,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Human perception of complexity
On Tuesday, 22 November 2011 20:21:45 UTC+1, raould wrote:
On Mon, Nov 21, 2011 at 12:35 PM, Erik Post <eriks [dot] [dot] [dot] [at] gmail [dot] com> wrote:
> Are you suggesting that Java would compare favourably in such situations?

presumably it depends on what the cases are. i'm guessing there are
examples where a single cast in java would do the trick. :-)

The same poisonous pixie dust is available in Scala, as you are no doubt aware. The fact that it takes a few more (greppable) characters there only adds to the awesomeness of the Scala side of our hypothetical comparison, don't you think?
Linas
Joined: 2009-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity

I would like to offer another perspective of a similar kind. The usefulness and perceived complexity of a software tool are directly related to how well this tool manages to compensate for mental and physical limitations of a user or at least to not push the user to the bounds of his/her mental and physical ability.

The bounds of a human mental ability are defined by the limitations of memory and limitations of knowledge. Limitations of memory are induced by: limited short term memory (up to 9 items max on average, new items push out old ones) and associative recall (hints are necessary to recall long term memories efficiently). Limitations of knowledge are induced by: the finitness of knowledge possesed by individuals and cognitive bias (making conclusions from incomplete information). 

The bounds of human physical ability are defined by the properties of physical body (two hands, two lengs, two eyes etc...) and the properties of the sensory and cognitive systems, eyes and visual perception being most relevant in this case.

Now, how does this all relate to Scala? Take for example limited short term memory and implicits - tracking the scope and effects of implicits in a Scala code of a reasonable complexity will exhaust short term memory of most normal humans. Another name for this problem in software is "invisibility". Take associative recall and Scalas propensity to use cryptic symbols for method names in libraries, like ":+, ::, etc" - how many average programmers have associative memories to the meaning of those symbols used in Scala libraries? Not that many. Take finitness of knowledge and the extensiveness and quality of documentation of Scala library - can your average programmer gain deep understanding of Scala concepts just by browsing the scaladocs? No? Then here enters cognitive bias, and the average programmer decides that the thing before him/her is ,well, incomprehensible.

Let's also not forget the properties of human visual sensory system. The human visual sensory system at its most basic level works as a sort of visual "tokenizer and parser". This visual tokenizer and parser has ability to automatically recognize certain visual features and present them to a concious of a user. This kind of visual recognition is then perceived as effortless.  What the human visual sensory system reacts to is: change in contrast, distinctness of shape, and motion (not relevant here).  Now take the average chunk of a professional Scala code, for example one in a Scala library, what does one find there visually: no dots between member access, no parentheses around call arguments - meaning call chains are hard to parse visually (and load short term memory since meaning of chain members has to be tracked along the chain); multiple nested parentheses in huge type definitions - a visual blur and short term memory load; a blur of :;=>$_ in chained operator calls, and chained closure invocations - makes it hard to understand what is where and what the limits and scopes are just by visual inspection.

The list of issues presented here is just of the top of my mind, thus by no way an extensive one. And I cannot point to exact papers of relevant research. However if someone wants to, they can find the relevant papers by searching with the terms mentioned.

Linas

On Nov 21, 2011 8:50 AM, "Bernd Johannes" <bjohanns [at] bacon [dot] de> wrote:
Hello Scalaist,

we've seen discussions about scala complexity, complex problems, fear of
complexity and related matters around scala (but not exclusively) for years
now.

This topic deals (among others) with something very fundamental: computer -
human - interaction.  A programming language along with its environment is a
digital tool that interacts with humans (developers).

As I got the impression over time that all those discussions tend to run in
circles maybe something can be done to push things forward. Is there a
possiblity to use the often taunted "academic" roots of scala as it strength
by inducing some real research?

There have been some language metric papers and mesurement approaches around
for different computer languages - most of them dealing with procedural and OO
metrics. So I think they are not simply applicable to scala and its specific
constructs (ok - the very basic ones like "LOC" or "LLOC" can be used as is,
but their value is very limited).

Can't somebody with an appropriate academic network try to inspire some
research in that direction? One of the basic questions would be "how to
measure scala code complexity" (and this simple question in itself will unfold
into a considerable avalanche of more detailed questions and some real
problems to solve. I don't expect this to be a copy and paste work...).

In the end scala would gain some "hard facts" about complexity and how to
measure it. And this in turn will lead to appropriate tools to do those
measurements - which will help to dispell complexity fears ("look - the metric
says: It's good" ... it's all about psychology ;-) )

Just some thoughts...

Greetings
Bernd

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Human perception of complexity

On Sat, Nov 26, 2011 at 5:13 PM, Linas wrote:
> Now take the average chunk of a professional Scala code, for example one in
> a Scala library, what does one find there visually: no dots between member
> access, no parentheses around call arguments - meaning call chains are hard
> to parse visually (and load short term memory since meaning of chain members
> has to be tracked along the chain);

When people say this I can only think they are having too hard a time
for no good reason. You can't say that unless you don't the parsing
rules, but it's not complicated. I can't even tell you how noisy
those unnecessary dots and parentheses are to me now. Here are a
couple real code samples from the compiler/library. I have difficulty
understanding how more dots, more parentheses, and presumably fewer
newlines would make these clearer or less taxing on your short-term
memory.

def apply(name: String): Symbol = {
val tpname = newTypeName(name)
(
findName(tpname)
orElse findName(tpname.companionName)
orElse optCompilerClass(name)
orElse optCompilerModule(name)
getOrElse NoSymbol
)
}

override lazy val iter = (
Iterator
continually (codec wrap charReader.read())
takeWhile (_ != -1)
map (_.toChar)
)

val trees = (
allValDefDerived(vd)
map (acc => atPos(vd.pos.focus)(acc derive annotations))
filterNot (_ eq EmptyTree)
)

Most of the function arguments in these examples have parens because
they're calling instance methods, but they could as easily be local
methods with no parens. It's not some random roll of the dice how the
tokens land. It only happens one way. Do the missing semicolons also
tax our short term memory? Do you see why it might seem to some of us
like the working definition of "overly complex" is awfully close to
"not what I'm used to" ?

Erik Engbrecht
Joined: 2008-12-19,
User offline. Last seen 3 years 18 weeks ago.
Re: Human perception of complexity
Paul,How much time had you spent with Scala, meaning hours hacking Scala code, before the absence of punctuation started feeling natural and excessive punctuation became distracting?  I think people coming from punctuation heavy languages are accustomed to seeing a lot of punctuation, and thus the wires in their brains deal with it quickly and get tripped up when it isn't there.  Likewise, someone who is used to punctuation light languages will probably see heavy punctuation as redundant clutter.  Strictly speaking it is redundant, so the later point-of-view is correct.  But it takes a while to rewire those neurons.  And I suspect a large portion of people new to Scala spend substantially more hours per day in a punctuation heavy language like Java than they do in Scala.
-Erik
extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Human perception of complexity

On Sat, Nov 26, 2011 at 6:21 PM, Erik Engbrecht
wrote:
> Strictly speaking it
> is redundant, so the later point-of-view is correct.  But it takes a while
> to rewire those neurons.

Of course it does. But the message to which I was responding was
arguing that it is somehow inherently more taxing on the short-term
memory not to have a dot or parenthesis every few characters, in an
appeal to fundamental properties of "the human visual sensory system".
I am suggesting the argument was mostly "it's not what I'm used to"
but in suit and tie.

H-star Development
Joined: 2010-04-14,
User offline. Last seen 2 years 26 weeks ago.
Re: Human perception of complexity

Am 27.11.2011 03:05, schrieb Paul Phillips:
> On Sat, Nov 26, 2011 at 5:13 PM, Linas wrote:
>> Now take the average chunk of a professional Scala code, for example one in
>> a Scala library, what does one find there visually: no dots between member
>> access, no parentheses around call arguments - meaning call chains are hard
>> to parse visually (and load short term memory since meaning of chain members
>> has to be tracked along the chain);
> When people say this I can only think they are having too hard a time
> for no good reason. You can't say that unless you don't the parsing
> rules, but it's not complicated. I can't even tell you how noisy
> those unnecessary dots and parentheses are to me now. Here are a
> couple real code samples from the compiler/library. I have difficulty
> understanding how more dots, more parentheses, and presumably fewer
> newlines would make these clearer or less taxing on your short-term
> memory.
>
> def apply(name: String): Symbol = {
> val tpname = newTypeName(name)
> (
> findName(tpname)
> orElse findName(tpname.companionName)
> orElse optCompilerClass(name)
> orElse optCompilerModule(name)
> getOrElse NoSymbol
> )
> }
>
> override lazy val iter = (
> Iterator
> continually (codec wrap charReader.read())
> takeWhile (_ != -1)
> map (_.toChar)
> )
>
> val trees = (
> allValDefDerived(vd)
> map (acc => atPos(vd.pos.focus)(acc derive annotations))
> filterNot (_ eq EmptyTree)
> )
>
> Most of the function arguments in these examples have parens because
> they're calling instance methods, but they could as easily be local
> methods with no parens. It's not some random roll of the dice how the
> tokens land. It only happens one way. Do the missing semicolons also
> tax our short term memory? Do you see why it might seem to some of us
> like the working definition of "overly complex" is awfully close to
> "not what I'm used to" ?
>
i agree. if you claim that ". between to symbols" is significantly
easier to understand than "whitespace between two symbols" and this is a
reason to call one language more complex than the other, then take a
look at human languages. they are full of random and inconsistent rules.
in german, everything has a random gender. doors are female, elevators
are male. in french, there isn't even an "it". in english, "wra" is
pronounced depending on the letters that follow: "p" or "ith" or "th".
all three make the "wra" sound completely different. the amount of rules
is bigger by far, too. if a random human language has a complexity of
100, i'd place all programming languages in a range of 5-10. dots and
inferred semicolons don't make a difference in that scale.

learning a new programming language is so much easier than learning a
new spoken language that the whole complexity discussion is a
distinction without difference. use your brain to learn, that's all. it
took me a few weeks, 1-2 hours per day, to get used to the lisp syntax.
LISP (in fact it was clojure, but that's really not a big difference).
the one where the only measure of grouping stuff together are
parentheses. no dots, no semicolons, no nothing. just (that).

Russ P.
Joined: 2009-01-31,
User offline. Last seen 1 year 26 weeks ago.
Re: Human perception of complexity

Someone should probably do the language complexity research you're
talking about, but I don't envy whoever does it. It's a very difficult
problem, and it is obviously very subjective too.

I know research psychologists who have attempted to quantify the
complexity of air traffic scenarios as perceived by air traffic
controllers. The main "quantitative" measurements they get are
subjective answers to the question of how difficult the scenario is
for them. That is hardly what one would call a precise, repeatable
measurement. It could depend on coffee intake, among other things.
Another more objective measure might be how many mistakes they make
(in a simulation), but they don't make many mistakes unless you
overwhelm them with more traffic than they can handle, and if you do
that, you are not testing them in a realistic environment.

As for language complexity, you would need to specify the experience
level of the programmer and the languages in which the programmer has
experience. Obviously, what seems complex for a beginner may not seem
complex for a senior developer, and Scala will seem more complex for
Lisp programmers than for Java programmers. And for Haskell
programmers, nothing will seem complex, but everything other than pure
FP will just seem stupid.

--Russ P.

On Nov 20, 10:50 pm, Bernd Johannes wrote:
> Hello Scalaist,
>
> we've seen discussions about scala complexity, complex problems, fear of
> complexity and related matters around scala (but not exclusively) for years
> now.
>
> This topic deals (among others) with something very fundamental: computer -
> human - interaction.  A programming language along with its environment is a
> digital tool that interacts with humans (developers).
>
> As I got the impression over time that all those discussions tend to run in
> circles maybe something can be done to push things forward. Is there a
> possiblity to use the often taunted "academic" roots of scala as it strength
> by inducing some real research?
>
> There have been some language metric papers and mesurement approaches around
> for different computer languages - most of them dealing with procedural and OO
> metrics. So I think they are not simply applicable to scala and its specific
> constructs (ok - the very basic ones like "LOC" or "LLOC" can be used as is,
> but their value is very limited).
>
> Can't somebody with an appropriate academic network try to inspire some
> research in that direction? One of the basic questions would be "how to
> measure scala code complexity" (and this simple question in itself will unfold
> into a considerable avalanche of more detailed questions and some real
> problems to solve. I don't expect this to be a copy and paste work...).
>
> In the end scala would gain some "hard facts" about complexity and how to
> measure it. And this in turn will lead to appropriate tools to do those
> measurements - which will help to dispell complexity fears ("look - the metric
> says: It's good" ... it's all about psychology ;-) )
>
> Just some thoughts...
>
> Greetings
> Bernd

Ravindra Jaju
Joined: 2011-11-27,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity
You've made very valid points, IMHO. Additionally, I can see that although you've put forth examples to drive home your points, the larger theme gets lost when the responses tend to gather around the examples themselves.

Allow me to substitute one of your words with another: average with typical. Using average just seems like a(n) (subconscious) act to tone down potentially aggressive responses to me.

With that substitution, I can think of the following situations:
1] A typical programmer does not worry too much about languages - but probably, more about their domains and immediate problems at hand to solve. :\%$^#/: look like noise to them. And =/= seems like the "not-equals-to" notation bastardized because the keyboard can not let them strike out the equals sign.
2] A typical library writer does not care about domains. :/ looks like a clever use of a feature scala gives them - they feel more expressive and accurate. The "information theory" junkies who like to encode more with less number of bits.

The problem? Humans can not work without context. There's a lot of context going on behind the scenes. Programming language syntax and natural language have a lot of things NOT in common here. I could catch only two words in a sentence (or misspell most of my words) and (within some limits) make absolutely usable sense of it. Humans are *biased* - as long as the bias is in line with the happenings, they are in tune.

When one does *not* have context, s/he looks for help. Google is the impulsive go-to friend. And google can not handle :/ and =/=. How does one get the new context then? It's (currently) a not-so-easy problem to solve.

Scala as a language is a swiss-army knife. It lends a lot more of its internal handles to the programmers. But people need to be aware of what they should use and when. Martin Odersky has a very appropriate write-up on various "levels" of Scala programmers, and IMO, it needs more emphasis than it gets. IMHO (as an example) - interactions between A1 and L3 level people has the potential to cause WW3. Atleast in the cyber-world. And the intensity will decrease between any other level combinations. (Alright - please note the jest - which I can not encode in typed characters!)

What Scala needs is probably a balance between who's allowed to interact with whom. Not in the language itself - but in terms of good practices and idioms.

Assembly is hard for many people. But C still exposes it and doesn't get butchered for being *hard*! Scala is similar - but it seems to have many more vocal and prominent L-level people than the A-level.

My 0.02 of whatever-your-favorite-currency-is!

--
jaju

On Sat, Nov 26, 2011 at 8:13 PM, Linas <vejobrolis [at] gmail [dot] com> wrote:

I would like to offer another perspective of a similar kind. The usefulness and perceived complexity of a software tool are directly related to how well this tool manages to compensate for mental and physical limitations of a user or at least to not push the user to the bounds of his/her mental and physical ability.

The bounds of a human mental ability are defined by the limitations of memory and limitations of knowledge. Limitations of memory are induced by: limited short term memory (up to 9 items max on average, new items push out old ones) and associative recall (hints are necessary to recall long term memories efficiently). Limitations of knowledge are induced by: the finitness of knowledge possesed by individuals and cognitive bias (making conclusions from incomplete information). 

The bounds of human physical ability are defined by the properties of physical body (two hands, two lengs, two eyes etc...) and the properties of the sensory and cognitive systems, eyes and visual perception being most relevant in this case.

Now, how does this all relate to Scala? Take for example limited short term memory and implicits - tracking the scope and effects of implicits in a Scala code of a reasonable complexity will exhaust short term memory of most normal humans. Another name for this problem in software is "invisibility". Take associative recall and Scalas propensity to use cryptic symbols for method names in libraries, like ":+, ::, etc" - how many average programmers have associative memories to the meaning of those symbols used in Scala libraries? Not that many. Take finitness of knowledge and the extensiveness and quality of documentation of Scala library - can your average programmer gain deep understanding of Scala concepts just by browsing the scaladocs? No? Then here enters cognitive bias, and the average programmer decides that the thing before him/her is ,well, incomprehensible.

Let's also not forget the properties of human visual sensory system. The human visual sensory system at its most basic level works as a sort of visual "tokenizer and parser". This visual tokenizer and parser has ability to automatically recognize certain visual features and present them to a concious of a user. This kind of visual recognition is then perceived as effortless.  What the human visual sensory system reacts to is: change in contrast, distinctness of shape, and motion (not relevant here).  Now take the average chunk of a professional Scala code, for example one in a Scala library, what does one find there visually: no dots between member access, no parentheses around call arguments - meaning call chains are hard to parse visually (and load short term memory since meaning of chain members has to be tracked along the chain); multiple nested parentheses in huge type definitions - a visual blur and short term memory load; a blur of :;=>$_ in chained operator calls, and chained closure invocations - makes it hard to understand what is where and what the limits and scopes are just by visual inspection.

The list of issues presented here is just of the top of my mind, thus by no way an extensive one. And I cannot point to exact papers of relevant research. However if someone wants to, they can find the relevant papers by searching with the terms mentioned.

Linas

On Nov 21, 2011 8:50 AM, "Bernd Johannes" <bjohanns [at] bacon [dot] de> wrote:
Hello Scalaist,

we've seen discussions about scala complexity, complex problems, fear of
complexity and related matters around scala (but not exclusively) for years
now.

This topic deals (among others) with something very fundamental: computer -
human - interaction.  A programming language along with its environment is a
digital tool that interacts with humans (developers).

As I got the impression over time that all those discussions tend to run in
circles maybe something can be done to push things forward. Is there a
possiblity to use the often taunted "academic" roots of scala as it strength
by inducing some real research?

There have been some language metric papers and mesurement approaches around
for different computer languages - most of them dealing with procedural and OO
metrics. So I think they are not simply applicable to scala and its specific
constructs (ok - the very basic ones like "LOC" or "LLOC" can be used as is,
but their value is very limited).

Can't somebody with an appropriate academic network try to inspire some
research in that direction? One of the basic questions would be "how to
measure scala code complexity" (and this simple question in itself will unfold
into a considerable avalanche of more detailed questions and some real
problems to solve. I don't expect this to be a copy and paste work...).

In the end scala would gain some "hard facts" about complexity and how to
measure it. And this in turn will lead to appropriate tools to do those
measurements - which will help to dispell complexity fears ("look - the metric
says: It's good" ... it's all about psychology ;-) )

Just some thoughts...

Greetings
Bernd


ichoran
Joined: 2009-08-14,
User offline. Last seen 2 years 3 weeks ago.
Re: Human perception of complexity
On Mon, Nov 21, 2011 at 12:06 PM, Dennis Haupt <h-star [at] gmx [dot] de> wrote:
i thought about this.

i think something's "descriptive complexity" is the smallest number of language elements necessary to describe it. it completely depends on the language you are using to describe it.

Alas, this is not well-aligned with subjective measures of complexity.

  "The girl the boy the dog bit hit cried."

Generated via very simple rules and using very few language elements, but I think once you figure out what this is saying that you will find that there is higher descriptive complexity version that is much easier to understand.

This is one reason why Bernd's proposal is a research proposal (and would take serious effort).

  --Rex

H-star Development
Joined: 2010-04-14,
User offline. Last seen 2 years 26 weeks ago.
Re: Human perception of complexity
Am 27.11.2011 10:11, schrieb Rex Kerr:
CAP_xLa0QnmM5LKM-ANjOXgavBPk96izHazr-399dVPjG-GE1eg [at] mail [dot] gmail [dot] com" type="cite"> On Mon, Nov 21, 2011 at 12:06 PM, Dennis Haupt <h-star [at] gmx [dot] de" rel="nofollow">h-star [at] gmx [dot] de> wrote:
i thought about this.

i think something's "descriptive complexity" is the smallest number of language elements necessary to describe it. it completely depends on the language you are using to describe it.

Alas, this is not well-aligned with subjective measures of complexity.

  "The girl the boy the dog bit hit cried."

Generated via very simple rules and using very few language elements, but I think once you figure out what this is saying that you will find that there is higher descriptive complexity version that is much easier to understand.

This is one reason why Bernd's proposal is a research proposal (and would take serious effort).

  --Rex

is that even valid english? i've never seen or heard something nested like this. anyway, if it is, this should also count:

dog bites boy. boy hits girls. girl cries.

it's one word shorter than your version and easier to understand. it is also less cuddled and can be read in one direction instead of forcing the reading to jump back and forth.

Ivan Todoroski
Joined: 2009-05-12,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity

On 27.11.2011 02:13, Linas wrote:
> Let's also not forget the properties of human visual sensory system. The
> human visual sensory system at its most basic level works as a sort of
> visual "tokenizer and parser".
> [...snip...]
> Now take the average chunk of a professional Scala code, for
> example one in a Scala library, what does one find there visually: no
> dots between member access, no parentheses around call arguments -
> meaning call chains are hard to parse visually

I wholeheartedly agree with everything you said in your message, EXCEPT
this part.

My experience is completely opposite to this. At least to my "visual
tokenizer and parser" the dots and other punctuation are a hindrance the
effortless visual scanning of the code.

The spaces make it much easier for me to see the distinct tokens. I
mean, it's not a coincidence that we use spaces to separate words in
sentences, rather.than;writing(like).this.

The frequent dots, semicolons and parentheses you find in typical code
are just noise that to my eyes blurs the different tokens together and
requires conscious effort to tease them apart.

I highly prefer the spaces when chaining method calls together and use
it whenever it makes sense. The spaces simply make it easier for my eyes
to parse the code. Of course, I also use the dot where it's warranted
and when doing otherwise would make the code awkward looking.

You can't dismiss me as being too familiar with Scala and therefore
getting used to the lack of punctuation, because I am actually a
complete Scala noob, merely playing around with it occasionally as my
very limited time allows.

For what its worth, I program in Ruby from time to time, and there too I
also prefer the space to separate method parameters whenever possible,
rather then using parentheses.

In other words, even in Ruby I prefer this:

puts "Hello"

to this:

puts("Hello");

ARKBAN
Joined: 2011-08-11,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity
Apologies for jumping in late, but I personally dislike the use of spaces instead of dots and loss of parenthesis. I work in a mixed Scala/Java/PHP/Javascript/Python environment and its far easier on my limited brain to make the Scala code look similar to the other languages. And that goes double for other members of my team that spend less time in Scala code.

I apologize for the tangent, but I am personally very concerned with the tendency I am seeing in Scala to "simplify" the language by removing (or make optional) what are considered by some as unnecessary syntax. It makes the language more alien to those who are coming from other languages. I've had to explain the "x => print(x)" vs "print(x)" vs "print" lambda shortcuts enough that I consider it a larger barrier to Scala adoption than any possible benefit a few keystrokes could save. I'd really hate to see Scala become -- LISP avoided (at least in part) because the syntax is so alien.

ARKBAN

On 11/26/2011 08:13 PM, Linas wrote:
CANHkwEqPK+j_GJZhCOQaRZBDCEmAtzVEz5aEqO9A4yU8WVKfhg [at] mail [dot] gmail [dot] com" type="cite">

I would like to offer another perspective of a similar kind. The usefulness and perceived complexity of a software tool are directly related to how well this tool manages to compensate for mental and physical limitations of a user or at least to not push the user to the bounds of his/her mental and physical ability.

The bounds of a human mental ability are defined by the limitations of memory and limitations of knowledge. Limitations of memory are induced by: limited short term memory (up to 9 items max on average, new items push out old ones) and associative recall (hints are necessary to recall long term memories efficiently). Limitations of knowledge are induced by: the finitness of knowledge possesed by individuals and cognitive bias (making conclusions from incomplete information). 

The bounds of human physical ability are defined by the properties of physical body (two hands, two lengs, two eyes etc...) and the properties of the sensory and cognitive systems, eyes and visual perception being most relevant in this case.

Now, how does this all relate to Scala? Take for example limited short term memory and implicits - tracking the scope and effects of implicits in a Scala code of a reasonable complexity will exhaust short term memory of most normal humans. Another name for this problem in software is "invisibility". Take associative recall and Scalas propensity to use cryptic symbols for method names in libraries, like ":+, ::, etc" - how many average programmers have associative memories to the meaning of those symbols used in Scala libraries? Not that many. Take finitness of knowledge and the extensiveness and quality of documentation of Scala library - can your average programmer gain deep understanding of Scala concepts just by browsing the scaladocs? No? Then here enters cognitive bias, and the average programmer decides that the thing before him/her is ,well, incomprehensible.

Let's also not forget the properties of human visual sensory system. The human visual sensory system at its most basic level works as a sort of visual "tokenizer and parser". This visual tokenizer and parser has ability to automatically recognize certain visual features and present them to a concious of a user. This kind of visual recognition is then perceived as effortless.  What the human visual sensory system reacts to is: change in contrast, distinctness of shape, and motion (not relevant here).  Now take the average chunk of a professional Scala code, for example one in a Scala library, what does one find there visually: no dots between member access, no parentheses around call arguments - meaning call chains are hard to parse visually (and load short term memory since meaning of chain members has to be tracked along the chain); multiple nested parentheses in huge type definitions - a visual blur and short term memory load; a blur of :;=>$_ in chained operator calls, and chained closure invocations - makes it hard to understand what is where and what the limits and scopes are just by visual inspection.

The list of issues presented here is just of the top of my mind, thus by no way an extensive one. And I cannot point to exact papers of relevant research. However if someone wants to, they can find the relevant papers by searching with the terms mentioned.

Linas

On Nov 21, 2011 8:50 AM, "Bernd Johannes" <bjohanns [at] bacon [dot] de" rel="nofollow">bjohanns [at] bacon [dot] de> wrote:
Hello Scalaist,

we've seen discussions about scala complexity, complex problems, fear of
complexity and related matters around scala (but not exclusively) for years
now.

This topic deals (among others) with something very fundamental: computer -
human - interaction.  A programming language along with its environment is a
digital tool that interacts with humans (developers).

As I got the impression over time that all those discussions tend to run in
circles maybe something can be done to push things forward. Is there a
possiblity to use the often taunted "academic" roots of scala as it strength
by inducing some real research?

There have been some language metric papers and mesurement approaches around
for different computer languages - most of them dealing with procedural and OO
metrics. So I think they are not simply applicable to scala and its specific
constructs (ok - the very basic ones like "LOC" or "LLOC" can be used as is,
but their value is very limited).

Can't somebody with an appropriate academic network try to inspire some
research in that direction? One of the basic questions would be "how to
measure scala code complexity" (and this simple question in itself will unfold
into a considerable avalanche of more detailed questions and some real
problems to solve. I don't expect this to be a copy and paste work...).

In the end scala would gain some "hard facts" about complexity and how to
measure it. And this in turn will lead to appropriate tools to do those
measurements - which will help to dispell complexity fears ("look - the metric
says: It's good" ... it's all about psychology ;-) )

Just some thoughts...

Greetings
Bernd

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Human perception of complexity

On Sun, Nov 27, 2011 at 8:44 AM, ARKBAN wrote:
> any possible benefit a few keystrokes could save.

This is always a strawman, because it's never about keystrokes. It's
desirable because it's *easier to read*. But yes, "it's not what I'm
used to."

Stefan Wagner
Joined: 2011-04-08,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 27.11.2011 12:28, schrieb Ivan Todoroski:

> In other words, even in Ruby I prefer this:
>
> puts "Hello"
>
> to this:
>
> puts("Hello");

You are free to use:

puts ("Hello")

which is what I would suggest for better reading experiences.

ARKBAN
Joined: 2011-08-11,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity
I agree its easier to read for you, but that does not make it universally true (or even true for the majority). I did not intend it to be a straw man argument, I'm simply saying that while its true for many of the readers of this list, we unfortunately are not a good sample of the entire collection of developers. Especially when we start including the non-Scala developers we are courting.

ARKBAN

On 11/27/2011 12:26 PM, Paul Phillips wrote:
CAEaELhQdBDGkS4F6ZrpYXFok04tdcRxqQW3CG+1d9HF3XK0wvQ [at] mail [dot] gmail [dot] com" type="cite">
On Sun, Nov 27, 2011 at 8:44 AM, ARKBAN arkban [at] arkban [dot] net (<arkban [at] arkban [dot] net>) wrote:
any possible benefit a few keystrokes could save.
This is always a strawman, because it's never about keystrokes.  It's
desirable because it's *easier to read*.  But yes, "it's not what I'm
used to."
Raoul Duke
Joined: 2009-01-05,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity

On Sun, Nov 27, 2011 at 5:23 PM, ARKBAN wrote:
> I agree its easier to read for you, but that does not make it universally
> true (or even true for the majority). I did not intend it to be a straw man
> argument, I'm simply saying that while its true for many of the readers of
> this list, we unfortunately are not a good sample of the entire collection
> of developers. Especially when we start including the non-Scala developers
> we are courting.

i was reading about msft's spec# and stuff once, and those tools
really spelled out in more englishy terms some of the usual programmer
notation conventions. i do not recall nor wish to spend the time to
look up specifics, but it was simple things like instead of having
"public int x" something like "x is int". seemingly a small thing. but
it actually felt a lot nicer to read through as e.g. a maintenance
programmer who just inherited Other People's Code.

it also depends on how many things i'm juggling in my head at the
time. what other things i'm trying to see and learn at the time.

if we have syntactic sugar or shorthands, it would be interesting if
our ides had a slider that let us go from e.g. super concise haskell
style over to evil verbose java style, and anywhere in between.

sincerely.

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Human perception of complexity

On Sun, Nov 27, 2011 at 5:23 PM, ARKBAN wrote:
> I agree its easier to read for you, but that does not make it universally
> true (or even true for the majority).

Yes, as I said in every single email so far, "it's not what I'm used
to." Let's not move the goalposts too far from the field.

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Human perception of complexity

On Sun, Nov 27, 2011 at 5:23 PM, ARKBAN wrote:
> I agree its easier to read for you, but that does not make it universally
> true (or even true for the majority).

Just as an experiment,

I.agree(it.is_easier_to(read).for(you)->but(!that.does(make_it)(Constant(true))
|| (true.for(the_majority))).

That's not supposed to be a strawman, but I admit I'm not very good
with the punctuation so it's probably clumsy. Shouldn't it be at
least a little appealing to write like that, if there is indeed some
reasonable argument against eliminating redundant punctuation beyond
"it's not what I'm used to"? Is it? Why not? What's the difference?
It's not like english is easier to parse; quite the opposite.

ARKBAN
Joined: 2011-08-11,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity

I am completely willing to admit the dot notation may be easier to read
if you are willing to relearn or are learning from scratch. I can't say
the same for older programmers (not without more empirical evidence).
I'm actively trying to win Scala converts, and in my experince these
sorts of syntax deviations make that more difficult.

ARKBAN

On 11/27/2011 08:38 PM, Raoul Duke wrote:
> On Sun, Nov 27, 2011 at 5:23 PM, ARKBAN wrote:
>> I agree its easier to read for you, but that does not make it universally
>> true (or even true for the majority). I did not intend it to be a straw man
>> argument, I'm simply saying that while its true for many of the readers of
>> this list, we unfortunately are not a good sample of the entire collection
>> of developers. Especially when we start including the non-Scala developers
>> we are courting.
> i was reading about msft's spec# and stuff once, and those tools
> really spelled out in more englishy terms some of the usual programmer
> notation conventions. i do not recall nor wish to spend the time to
> look up specifics, but it was simple things like instead of having
> "public int x" something like "x is int". seemingly a small thing. but
> it actually felt a lot nicer to read through as e.g. a maintenance
> programmer who just inherited Other People's Code.
>
> it also depends on how many things i'm juggling in my head at the
> time. what other things i'm trying to see and learn at the time.
>
> if we have syntactic sugar or shorthands, it would be interesting if
> our ides had a slider that let us go from e.g. super concise haskell
> style over to evil verbose java style, and anywhere in between.
>
> sincerely.

ARKBAN
Joined: 2011-08-11,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity

I appreciate the thought experiment, but I don't believe it is a valid
one. You are adding extra syntax to an existing language, that's kind of
the opposite of what Scala is offering.

I think I did not clearly explain my point, or it somehow got lost in
the shuffle. (Or perhaps I used too many superfluous characters ; ) My
concern about Scala's syntactic noise reduction comes from when viewed
with the goal of trying to win Scala converts. I want more Scala
developers, and for that I think we ought to be very careful about
keeping the syntax familiar. Scala's familiar dot-notation syntax is
actually why I decide to learn it (over Clojure). I deal with questions
of Scala's syntax nearly daily at work by those less familiar. I want to
keep using Scala at work but I'm in the minority, and alien syntax makes
that all the more difficult.

ARKBAN

On 11/27/2011 08:53 PM, Paul Phillips wrote:
> On Sun, Nov 27, 2011 at 5:23 PM, ARKBAN wrote:
>> I agree its easier to read for you, but that does not make it universally
>> true (or even true for the majority).
> Just as an experiment,
>
> I.agree(it.is_easier_to(read).for(you)->but(!that.does(make_it)(Constant(true))
> || (true.for(the_majority))).
>
> That's not supposed to be a strawman, but I admit I'm not very good
> with the punctuation so it's probably clumsy. Shouldn't it be at
> least a little appealing to write like that, if there is indeed some
> reasonable argument against eliminating redundant punctuation beyond
> "it's not what I'm used to"? Is it? Why not? What's the difference?
> It's not like english is easier to parse; quite the opposite.

Bernd Johannes
Joined: 2011-01-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Human perception of complexity
p, li { white-space: pre-wrap; }

Hiho,


Am Sonntag, 27. November 2011, 10:00:44 schrieb Russ P.:

> Someone should probably do the language complexity research you're

> talking about, but I don't envy whoever does it. It's a very difficult

> problem, and it is obviously very subjective too.


Of course it will be subjective - such research has to deal with humans :-). It's about finding correllations between groups of people and groups of features and some f(peopledescriptor, featuredescriptor) which predict with some reliability key features of recognition.

e.g.:

- how hard is it to comprehend (mental effort)

- how likely is it to get it wrong (error rate)

- how hard is it to learn (reduction of mental effort over repetitions and time)


I know - it won't be that easy. But I think the results would be quite valuable (not only for scala - but scala has its merrits as a playfield for such a research as it allows for many concepts/features to be explored).


And I guess there won't be any big surprises in the results. Somehow we all "feel" all this in our subjective context. But we are not fully aware of the components.


Unless we have such data we will be restricted to those gut feeling debates. This doesn't have to be bad (and allows for some sort of statistics already ;-) ). But the disadvantage is that it tends to alienate people which fight over beliefs without some reliable facts to cool things down a bit.


> I know research psychologists who have attempted to quantify the

> complexity of air traffic scenarios as perceived by air traffic

> controllers. The main "quantitative" measurements they get are

> subjective answers to the question of how difficult the scenario is

> for them. That is hardly what one would call a precise, repeatable

> measurement. It could depend on coffee intake, among other things.


Comming from biology I am familiar with the drawbacks of a science which is based on "rules" instead of "laws" - there are always exceptions and uncertainties. The only thing we have to our rescue is statistics. We can only make predictions decorated with error quantifiers. But we have done so quite successfully.


> As for language complexity, you would need to specify the experience

> level of the programmer and the languages in which the programmer has

> experience. Obviously, what seems complex for a beginner may not seem

> complex for a senior developer, and Scala will seem more complex for

> Lisp programmers than for Java programmers. And for Haskell

> programmers, nothing will seem complex, but everything other than pure

> FP will just seem stupid.


For sure - and it would be the hardest part of this research to get enough "subjects" to test for each group. Because to be able to draw conclustions one would have to test a good amount of individuals for each group.


On the other hand: the "laboratory part" would be quite cheap. Some good designed tests - some good designed questionaires and you're done.

Maybe it will happen some day.


Greetings

Bernd



Ivan Todoroski
Joined: 2009-05-12,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity

On 28.11.2011 01:48, Stefan Wagner wrote:
> Am 27.11.2011 12:28, schrieb Ivan Todoroski:
>
>> In other words, even in Ruby I prefer this:
>>
>> puts "Hello"
>>
>> to this:
>>
>> puts("Hello");
>
> You are free to use:
>
> puts ("Hello")
>
> which is what I would suggest for better reading experiences.

Of course I am free to do that, but I *prefer* less punctuation. My only
point was that I am a counter-example to the original claim by Linas
that humans generally prefer punctuation for visual parsing. I don't,
and judging by typical Scala code many others don't as well.

ARKBAN
Joined: 2011-08-11,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity

On 11/28/11 8:24 AM, Ivan Todoroski wrote:
> On 28.11.2011 01:48, Stefan Wagner wrote:
>> Am 27.11.2011 12:28, schrieb Ivan Todoroski:
>>
>>> In other words, even in Ruby I prefer this:
>>>
>>> puts "Hello"
>>>
>>> to this:
>>>
>>> puts("Hello");
>>
>> You are free to use:
>>
>> puts ("Hello")
>>
>> which is what I would suggest for better reading experiences.
>
> Of course I am free to do that, but I *prefer* less punctuation. My
> only point was that I am a counter-example to the original claim by
> Linas that humans generally prefer punctuation for visual parsing. I
> don't, and judging by typical Scala code many others don't as well.
>

I'd prefer either:

puts "Hello"

or:

puts( "Hello" )

Our brains are good at figuring out the entire word based on the prefix.
(Try reading a only the first three letters of every word in a sentence,
you'll probably be pretty close, as opposed to looking at the last three
letters.) Smushing the open parenthesis to be visual a part of the
"puts" method allows us some of the benefit of the parenthesis-less
version while not loosing the parenthesis entirely (useful when you
might need them for syntax reasons.

ARKBAN

Stefan Wagner
Joined: 2011-04-08,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 28.11.2011 14:39, schrieb ARKBAN:
> or:
>
> puts( "Hello" )
>
> Our brains are good at figuring out the entire word based on the prefix.
> (Try reading a only the first three letters of every word in a sentence,
> you'll probably be pretty close, as opposed to looking at the last three
> letters.) Smushing the open parenthesis to be visual a part of the
> "puts" method allows us some of the benefit of the parenthesis-less
> version while not loosing the parenthesis entirely (useful when you
> might need them for syntax reasons.

I don't see a reason to prefer

> puts( "Hello" )
over
> puts ("Hello")

Do you see, that you're using a different rule in your own text above
"... on the prefix. (Try reading ... letters.) Smushing the ..."?

Parenthesis and braces have space to the outside, not to the inside.
(Exceptions: Punctuation and additional Parenthesis (like in this
example)). Foobar.

val s = sum (foo - (diff (a, b)))

ARKBAN
Joined: 2011-08-11,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity

I readily admit I use very different rules for English -- that's because
it is English and not code. The interpreter/compiler for English is far
more advanced and tolerant than the interpreter for any computer language.

ARKBAN

On 11/28/11 11:09 AM, Stefan Wagner wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Am 28.11.2011 14:39, schrieb ARKBAN:
>> or:
>>
>> puts( "Hello" )
>>
>> Our brains are good at figuring out the entire word based on the prefix.
>> (Try reading a only the first three letters of every word in a sentence,
>> you'll probably be pretty close, as opposed to looking at the last three
>> letters.) Smushing the open parenthesis to be visual a part of the
>> "puts" method allows us some of the benefit of the parenthesis-less
>> version while not loosing the parenthesis entirely (useful when you
>> might need them for syntax reasons.
> I don't see a reason to prefer
>
>> puts( "Hello" )
> over
>> puts ("Hello")
> Do you see, that you're using a different rule in your own text above
> "... on the prefix. (Try reading ... letters.) Smushing the ..."?
>
> Parenthesis and braces have space to the outside, not to the inside.
> (Exceptions: Punctuation and additional Parenthesis (like in this
> example)). Foobar.
>
> val s = sum (foo - (diff (a, b)))
>
> - --
>
> Tschööö--->...Stefan
> - ---------------------------
> Don't visit my homepage at:
> http://home.arcor-online.net/hirnstrom
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk7TskcACgkQQeATqGpDnRqF4ACgj3V2dXGf85JZaHEE6rfsYRd6
> LDsAnidvBeiXbIdej5oytcKEAJHFShdo
> =yhG7
> -----END PGP SIGNATURE-----

ichoran
Joined: 2009-08-14,
User offline. Last seen 2 years 3 weeks ago.
Re: Human perception of complexity


On Sun, Nov 27, 2011 at 5:08 AM, HamsterofDeath <h-star [at] gmx [dot] de> wrote:
Am 27.11.2011 10:11, schrieb Rex Kerr:
On Mon, Nov 21, 2011 at 12:06 PM, Dennis Haupt <h-star [at] gmx [dot] de> wrote:
i thought about this.

i think something's "descriptive complexity" is the smallest number of language elements necessary to describe it. it completely depends on the language you are using to describe it.

Alas, this is not well-aligned with subjective measures of complexity.

  "The girl the boy the dog bit hit cried."

Generated via very simple rules and using very few language elements, but I think once you figure out what this is saying that you will find that there is higher descriptive complexity version that is much easier to understand.

This is one reason why Bernd's proposal is a research proposal (and would take serious effort).

  --Rex

is that even valid english? i've never seen or heard something nested like this. anyway, if it is, this should also count:

dog bites boy. boy hits girls. girl cries.

it's one word shorter than your version and easier to understand. it is also less cuddled and can be read in one direction instead of forcing the reading to jump back and forth.

It's valid English with perfect grammar, but it uses a form that so difficult to parse that you didn't quite manage to do it properly.  The key building block is that
  noun-verb
can be used as a modifier (with a transitive verb) as well as an independent clause (in the case of a reflexive verb), as in
  "Throw away the cookie the dog bit."
Now we have a nesting rule:
  noun-verb -> (noun-(noun-verb))-verb
which is perfectly valid but really hard to parse.

So, what does
  The girl the boy the dog bit hit cried
mean?  Why,
  The girl cried when hit by the boy who was bitten by the dog.

The difference between this and your version, is that yours tells a story via the linking of events; mine (both the original and the expansion) uses the events only as modifiers to distinguish which girl and boy.  (Even that the girl cried _because_ the boy hit her is only implied; if she cried because she didn't get what she wanted for her birthday, but everyone knew that she was "the girl who was hit by a boy", the sentence would work:
  You know that girl who was hit by the boy who was bitten by a dog?  Well, she cried.
)

Anyway, point is--very simple recursive rules can rapidly lead to something which is infuriatingly incomprehensible.  Programmers are better than most at unwrapping these things, I'd expect, but it still points out that descriptive complexity is a sorely inadequate measure of perceived complexity.

  --Rex

Bradley Buchsbaum
Joined: 2009-04-15,
User offline. Last seen 42 years 45 weeks ago.
Re: Human perception of complexity

I'll just comment briefly on one point raised in this message:

"Limitations of memory are induced by: limited short term memory (up
to 9 items max on average, new items push out old ones)"

Actually, the number of distinct "items" that a person can maintain in
working memory is more like 3 or 4. There are many caveats and
complexities associated with this (i.e can information be "chunked"?
etc.). The definitive modern discussion of this topic maybe be found
here*, if you're curious.

*http://web.missouri.edu/~cowann/docs/articles/2001/Cowan%20BBS
%202001.pdf

Learning involves the formation of long-term memory structures that
shift the burden from limited capacity working memory. But when we
first learn something complex, before these memory schema have been
formed, the information must be held in working memory. I can imagine
that things like multiple type parameters, implicit parameters, etc.
would "take up slots" in working memory before adequate learning has
occurred. And "taking up slots" is roughly same as saying: your brain,
especially the prefrontal cortex, is working hard.

Brad Buchsbaum

On Nov 26, 8:13 pm, Linas wrote:
> I would like to offer another perspective of a similar kind. The usefulness
> and perceived complexity of a software tool are directly related to how
> well this tool manages to compensate for mental and physical limitations of
> a user or at least to not push the user to the bounds of his/her mental and
> physical ability.
>
> The bounds of a human mental ability are defined by the limitations of
> memory and limitations of knowledge. Limitations of memory are induced by:
> limited short term memory (up to 9 items max on average, new items push out
> old ones) and associative recall (hints are necessary to recall long term
> memories efficiently). Limitations of knowledge are induced by: the
> finitness of knowledge possesed by individuals and cognitive bias (making
> conclusions from incomplete information).
>
> The bounds of human physical ability are defined by the properties of
> physical body (two hands, two lengs, two eyes etc...) and the properties of
> the sensory and cognitive systems, eyes and visual perception being most
> relevant in this case.
>
> Now, how does this all relate to Scala? Take for example limited short term
> memory and implicits - tracking the scope and effects of implicits in a
> Scala code of a reasonable complexity will exhaust short term memory of
> most normal humans. Another name for this problem in software is
> "invisibility". Take associative recall and Scalas propensity to use
> cryptic symbols for method names in libraries, like ":+, ::, etc" - how
> many average programmers have associative memories to the meaning of those
> symbols used in Scala libraries? Not that many. Take finitness of knowledge
> and the extensiveness and quality of documentation of Scala library - can
> your average programmer gain deep understanding of Scala concepts just by
> browsing the scaladocs? No? Then here enters cognitive bias, and the
> average programmer decides that the thing before him/her is ,well,
> incomprehensible.
>
> Let's also not forget the properties of human visual sensory system. The
> human visual sensory system at its most basic level works as a sort of
> visual "tokenizer and parser". This visual tokenizer and parser has ability
> to automatically recognize certain visual features and present them to a
> concious of a user. This kind of visual recognition is then perceived as
> effortless.  What the human visual sensory system reacts to is: change in
> contrast, distinctness of shape, and motion (not relevant here).  Now take
> the average chunk of a professional Scala code, for example one in a Scala
> library, what does one find there visually: no dots between member access,
> no parentheses around call arguments - meaning call chains are hard to
> parse visually (and load short term memory since meaning of chain members
> has to be tracked along the chain); multiple nested parentheses in huge
> type definitions - a visual blur and short term memory load; a blur of
> :;=>$_ in chained operator calls, and chained closure invocations - makes
> it hard to understand what is where and what the limits and scopes are just
> by visual inspection.
>
> The list of issues presented here is just of the top of my mind, thus by no
> way an extensive one. And I cannot point to exact papers of relevant
> research. However if someone wants to, they can find the relevant papers by
> searching with the terms mentioned.
>
> Linas
> On Nov 21, 2011 8:50 AM, "Bernd Johannes" wrote:
>
>
>
>
>
>
>
> > Hello Scalaist,
>
> > we've seen discussions about scala complexity, complex problems, fear of
> > complexity and related matters around scala (but not exclusively) for years
> > now.
>
> > This topic deals (among others) with something very fundamental: computer -
> > human - interaction.  A programming language along with its environment is
> > a
> > digital tool that interacts with humans (developers).
>
> > As I got the impression over time that all those discussions tend to run in
> > circles maybe something can be done to push things forward. Is there a
> > possiblity to use the often taunted "academic" roots of scala as it
> > strength
> > by inducing some real research?
>
> > There have been some language metric papers and mesurement approaches
> > around
> > for different computer languages - most of them dealing with procedural
> > and OO
> > metrics. So I think they are not simply applicable to scala and its
> > specific
> > constructs (ok - the very basic ones like "LOC" or "LLOC" can be used as
> > is,
> > but their value is very limited).
>
> > Can't somebody with an appropriate academic network try to inspire some
> > research in that direction? One of the basic questions would be "how to
> > measure scala code complexity" (and this simple question in itself will
> > unfold
> > into a considerable avalanche of more detailed questions and some real
> > problems to solve. I don't expect this to be a copy and paste work...).
>
> > In the end scala would gain some "hard facts" about complexity and how to
> > measure it. And this in turn will lead to appropriate tools to do those
> > measurements - which will help to dispell complexity fears ("look - the
> > metric
> > says: It's good" ... it's all about psychology ;-) )
>
> > Just some thoughts...
>
> > Greetings
> > Bernd

Matthew Pocock 3
Joined: 2010-07-30,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Human perception of complexity


On 30 November 2011 20:37, brad <brad [dot] buchsbaum [at] gmail [dot] com> wrote:
I'll just comment briefly on one point raised in this message:

"Limitations of memory are induced by: limited short term memory (up
to 9 items max on average, new items push out old ones)"

Actually, the number of distinct "items" that a person can maintain in
working memory is more like 3 or 4.

I am dyslexic. I have a working memory of 3-4 context-free items e.g. single digits, names, letters and so on, and have a tendency to get 'which item is in which slot' muddled. I don't think I'm unusual in being both dyslexic and a programmer - indeed, one effective coping mechanism is to consciously express rules for doing things as an aid to doing them.  
I can imagine
that things like multiple type parameters, implicit parameters, etc.
would "take up slots" in working memory before adequate learning has
occurred. And "taking up slots" is roughly same as saying: your brain,
especially the prefrontal cortex, is working hard.

I agree. By naming things like variables and type parameters explicitly or mnemonically, the burden is shifted from short-term towards associative memory, at least for knowing what the symbol represents. Other things that take up slots include: branching (if/else, case, while, recursive invocations), nesting (recursive curly brackets, dots, ellipse arg lists, ellipse precedence control), non-local context (names not declared directly above, implicits, imported symbols). For each of these, you have to remember something that you need to soon forget to be able to reason about the code you're looking at. After time, much of the grammar gets hard-wired. With time using a library, the non-local context injected by that library (e.g. DSL constructs, key types) are learned. But, initially you have to deal with all of this through an act of concious effort, and it is hard work.
Matthew 

Brad Buchsbaum






On Nov 26, 8:13 pm, Linas <vejobro [dot] [dot] [dot] [at] gmail [dot] com> wrote:
> I would like to offer another perspective of a similar kind. The usefulness
> and perceived complexity of a software tool are directly related to how
> well this tool manages to compensate for mental and physical limitations of
> a user or at least to not push the user to the bounds of his/her mental and
> physical ability.
>
> The bounds of a human mental ability are defined by the limitations of
> memory and limitations of knowledge. Limitations of memory are induced by:
> limited short term memory (up to 9 items max on average, new items push out
> old ones) and associative recall (hints are necessary to recall long term
> memories efficiently). Limitations of knowledge are induced by: the
> finitness of knowledge possesed by individuals and cognitive bias (making
> conclusions from incomplete information).
>
> The bounds of human physical ability are defined by the properties of
> physical body (two hands, two lengs, two eyes etc...) and the properties of
> the sensory and cognitive systems, eyes and visual perception being most
> relevant in this case.
>
> Now, how does this all relate to Scala? Take for example limited short term
> memory and implicits - tracking the scope and effects of implicits in a
> Scala code of a reasonable complexity will exhaust short term memory of
> most normal humans. Another name for this problem in software is
> "invisibility". Take associative recall and Scalas propensity to use
> cryptic symbols for method names in libraries, like ":+, ::, etc" - how
> many average programmers have associative memories to the meaning of those
> symbols used in Scala libraries? Not that many. Take finitness of knowledge
> and the extensiveness and quality of documentation of Scala library - can
> your average programmer gain deep understanding of Scala concepts just by
> browsing the scaladocs? No? Then here enters cognitive bias, and the
> average programmer decides that the thing before him/her is ,well,
> incomprehensible.
>
> Let's also not forget the properties of human visual sensory system. The
> human visual sensory system at its most basic level works as a sort of
> visual "tokenizer and parser". This visual tokenizer and parser has ability
> to automatically recognize certain visual features and present them to a
> concious of a user. This kind of visual recognition is then perceived as
> effortless.  What the human visual sensory system reacts to is: change in
> contrast, distinctness of shape, and motion (not relevant here).  Now take
> the average chunk of a professional Scala code, for example one in a Scala
> library, what does one find there visually: no dots between member access,
> no parentheses around call arguments - meaning call chains are hard to
> parse visually (and load short term memory since meaning of chain members
> has to be tracked along the chain); multiple nested parentheses in huge
> type definitions - a visual blur and short term memory load; a blur of
> :;=>$_ in chained operator calls, and chained closure invocations - makes
> it hard to understand what is where and what the limits and scopes are just
> by visual inspection.
>
> The list of issues presented here is just of the top of my mind, thus by no
> way an extensive one. And I cannot point to exact papers of relevant
> research. However if someone wants to, they can find the relevant papers by
> searching with the terms mentioned.
>
> Linas
> On Nov 21, 2011 8:50 AM, "Bernd Johannes" <bjoha [dot] [dot] [dot] [at] bacon [dot] de> wrote:
>
>
>
>
>
>
>
> > Hello Scalaist,
>
> > we've seen discussions about scala complexity, complex problems, fear of
> > complexity and related matters around scala (but not exclusively) for years
> > now.
>
> > This topic deals (among others) with something very fundamental: computer -
> > human - interaction.  A programming language along with its environment is
> > a
> > digital tool that interacts with humans (developers).
>
> > As I got the impression over time that all those discussions tend to run in
> > circles maybe something can be done to push things forward. Is there a
> > possiblity to use the often taunted "academic" roots of scala as it
> > strength
> > by inducing some real research?
>
> > There have been some language metric papers and mesurement approaches
> > around
> > for different computer languages - most of them dealing with procedural
> > and OO
> > metrics. So I think they are not simply applicable to scala and its
> > specific
> > constructs (ok - the very basic ones like "LOC" or "LLOC" can be used as
> > is,
> > but their value is very limited).
>
> > Can't somebody with an appropriate academic network try to inspire some
> > research in that direction? One of the basic questions would be "how to
> > measure scala code complexity" (and this simple question in itself will
> > unfold
> > into a considerable avalanche of more detailed questions and some real
> > problems to solve. I don't expect this to be a copy and paste work...).
>
> > In the end scala would gain some "hard facts" about complexity and how to
> > measure it. And this in turn will lead to appropriate tools to do those
> > measurements - which will help to dispell complexity fears ("look - the
> > metric
> > says: It's good" ... it's all about psychology ;-) )
>
> > Just some thoughts...
>
> > Greetings
> > Bernd



--
Dr Matthew PocockIntegrative Bioinformatics Group, School of Computing Science, Newcastle Universitymailto: turingatemyhamster [at] gmail [dot] com gchat: turingatemyhamster [at] gmail [dot] commsn: matthew_pocock [at] yahoo [dot] co [dot] uk irc.freenode.net: drdozerskype: matthew.pococktel: (0191) 2566550mob: +447535664143
Bernd Johannes
Joined: 2011-01-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Human perception of complexity

Am Mittwoch, 30. November 2011, 21:37:45 schrieb brad:
> I'll just comment briefly on one point raised in this message:
>
> "Limitations of memory are induced by: limited short term memory (up
> to 9 items max on average, new items push out old ones)"
>
> Actually, the number of distinct "items" that a person can maintain in
> working memory is more like 3 or 4. There are many caveats and
> complexities associated with this (i.e can information be "chunked"?
> etc.). The definitive modern discussion of this topic maybe be found
> here*, if you're curious.
>
> *http://web.missouri.edu/~cowann/docs/articles/2001/Cowan%20BBS
> %202001.pdf
>
> Learning involves the formation of long-term memory structures that
> shift the burden from limited capacity working memory. But when we
> first learn something complex, before these memory schema have been
> formed, the information must be held in working memory. I can imagine
> that things like multiple type parameters, implicit parameters, etc.
> would "take up slots" in working memory before adequate learning has
> occurred. And "taking up slots" is roughly same as saying: your brain,
> especially the prefrontal cortex, is working hard.
>
> Brad Buchsbaum

It's one of the not so good understood mysteries of our brain to be able to
automate many - even complex things. When we get "fluent" in some domain the
brain is able to shift some of the hard work to some "deeper" machinery which
does not require so much conscious effort.
Reading is one of those abilities which is incredible taxing for beginners but
works like magic once we are sufficiently trained. Driving a car is another
example.
And I firmly believe (I have no evidence nor data - so I have to believe) that
programming is also partially delegated to the helpfull automaton in the
cellar.
With java this happened quite quickly - perhaps because of the mentally low
effort boilerplate. In scala I am not fluent enough yet. But I suppose that
scala takes more focus to code. But I definitely need more practise to be able
to compare it.

Greetings
Bernd

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland