What is highest priority for Scala to succeed in corporate world (Should be in scala-debate?) ?

Another thing that mitigiates the pains of binary incompatibility is
the the cross building capacity of SBT,
if I declare a dependency on, for example :

"com.weiglewilczek.slf4s" %% "slf4s" % "1.0.7"

then when I updrade scala version, things will happen painlessly,

provided that :

1) the maintainers of the libraries are quick to release their jars
when new Scala versions come out
2) the maintainers of the libraries use proper version naming scheme
to not break the magic

It's a big if, but I think most maintainers of libraries are aware of this.

Also what makes binary incompatibility less painfull for me as an
eclipse user is that I use SBT's eclipse plugin
to generate and update the project definition. My build config is the
single source of
truth in terms of versions, that are all agnostic of the Scala release.

I take the time to mention this, because I think there could be more
documentation out there
that explains the availability of a tool chain that can go a long way
in making the binary incompatibility
much more acceptable.

Right now the doc is on thres sites :

- ScalaIDE's site
- SBT's site
- SBT's eclipse plugin github page

I'm sure that most seasoned Scala devs are aware of this, but someone
starting Scala will
loose a lot of time if they don't discover this quickly.

Cheers

2011/11/11 Mirco Dotta :
> Francois, don't take it personally, I was simply reporting a fact.
>
> Going back to binary compatibility, I think that an important step has been made: Scala minor releases
> are now binary compatible.
> That demonstrates the problem is (1) relevant, and (2) work is being done to mitigate it.
>
> I agree with you when you say that a Java dev is not used to deal with binary incompatibilities, hence we
> need to make his (and our!) life easier.
>
> How can we do that? In my opinion, the first thing that is needed is a way to tell programmatically if a
> library is binary incompatible wrt the used scala library (the example you reported for Play! is a perfect
> use case).
>
> Just imagine the Scala IDE issuing an error when you add in the classpath a library that is not binary compatible.
> I bet that it would make binary compatibility a much simpler problem to deal with (for everyone).
> But for making this real we need help from the compiler. Specifically, we need to know with what Scala version
> a binary class is compiled.
>
> An additional important aspect is improving the documentation and understanding of how to evolve Scala
> code to ensure release-to-release binary compatibility. If library maintainers are given the tools to strive for
> binary compatible releases, then the whole problem will simply go away.
>
> Cheers,
>  Mirco
>
> On Nov 11, 2011, at 10:27 PM, Francois wrote:
>
>> On 11/11/2011 21:55, Mirco Dotta wrote:
>>>> and Scala IDE came with 2.9.
>>> For the record, the Scala IDE supports both 2.8 and 2.9.
>>>
>>> http://download.scala-ide.org/releases-28/stable/site
>>> http://download.scala-ide.org/releases-29/stable/site
>>
>> I do know that, as I think any Scalaiste more than some days old do. What I wanted to report is that complettly newcomers, especially Java one, it was a surprise to have to care about binary compatibility between Scala version.
>>
>> Well, and it was just a report, I don't want to have to argue against fact. You can take care of them of not, it was just to contribute what might be an "highest priority for Scala to succeed in corporate world".
>>
>> Thanks,
>>
>> --
>> Francois ARMAND
>> http://fanf42.blogspot.com
>> http://www.normation.com
>>
>
>

On Thu, Nov 10, 2011 at 9:24 AM, jherber wrote:
> In a big company, it is not uncommon to find millions of lines of code
> written in Java, with huge chunks of it pre generics.

(replying on scala-debate, not scala-user; I hope others will follow)

In 2050, when you and I are dead or our beards are long and gray, all
that Java code will still be Java and still maintained, just as today
in 2011 millions of lines of COBOL and Fortran code from decades past
is still with us.

Java is too big to fail. It's normal and not something Scala should be
wringing its hands over. We should look for opportunities where they
exist. Gradually Java will fade and new languages will grow. Let's
find fresh soil to grow in and not to be too obsessed with pulling up
existing plants.

(I'm agreeing, not disagreeing.)

WOW. I hear you and used to think so, but then I also find it amazing that
these "big companies" don't use Cobol that much anymore... that builds kind
of a contradiction there...

Our customers - many orders of magnitude larger than us, have vey funky
environments. I have seen all sorts of scripting environments known to man
(and a few in-house) in production all over the place.

Also - the moment I stopped pushing for it, some guys here started picking
it up... now we have at least one "official" project using an internal scala
DSL :)

Maybe the most important thing we all need to do is just chill... of course
a great UI won't hurt the chilling process!

------

About Big data - yeah. Kind'a easy to explain to someone: map-reduce this
sh!t. somehow that sounds better to the masses than "flat-map this sh!t"?

That however will be just a stepping stone kicking in new paradigms... Yo,
Greg - how's your book coming along?

------

To answer the original "what scala needs to succeed in the enterprise" ?

Karma!

Cheers,
RAzie

-----Original Message-----
From: scala-user [at] googlegroups [dot] com [mailto:scala-user [at] googlegroups [dot] com] On
Behalf Of jherber
Sent: November-10-11 9:24 AM
To: scala-user
Subject: [scala-user] Re: What is highest priority for Scala to succeed in
corporate world (Should be in scala-debate?) ?

I don't post often, but when I do...

This thread is becoming an IDE flame war, but in the meantime, take a look
at this article just published:

http://www.informationweek.com/news/software/info_management/231902645

The Big Data Stack is probably the next big thing to work its way into large
enterprises. What offering does Scala have here? Scala ecosystem recently
bet big on parallel collections, lang virtualization, and absorbing AKKA
(all truly amazing stuff). But, these are just different directions than
big enterprises are taking.
With the Big Data Stack, numerous industries are realizing the value of a
generic, supported framework for the storage and analysis of unlimited data
in structured and unstructured forms. This solves a huge industry problem
as general trend is more real-time results, more introspection of your
customer's data so you can sell or offer more value than competitor, and of
course it solves the data explosion problem in an affordable, predictable
way without vendor lock in.

I am consulting with a big financial center currently, so let me give some
background and context of the environment. There might be a dozen or two of
us who could write code in Scala right now. But the other 10,000 developers
will not have a clue how to read it, how to get trained, how to get started,
how to get Management to pay for all their training and lost productivity
while they train, rewrite, and learn on the job. We'd no doubt see the
range of Scala as it is very flexible, making it very difficult to maintain
and read. This contradicts the process and control lock down big companies
must have to make sure strategy flows from top to bottom as it is translated
into execution. From a ground up perspective, the first time a Manager who
codes "Enterprise Style Java", risks his reputation (and budget and bonus)
on some smart consultant's Scala recommendation, only to experience first
hand a random long pauses using an IDE or has an unexpected class explosion
from specialization or a code review with someone pushing the type system,
he will shut it down and vocalize his perception to other managers and that
consultant is booted.

In a big company, it is not uncommon to find millions of lines of code
written in Java, with huge chunks of it pre generics. I don't see that code
going anywhere. It is simple to read, buggy as heck to modify beyond its
intended extension points, but in Management's mind it has powered the
company for almost a decade so why change it? They see so many other areas
that would have a higher return on investment.

I think Scala could make inroads in another way. Big companies buy smaller
companies all the time. If eventually enough small companies are ingested
that are using Scala, the base make up of the big company changes. Big
companies also go with proven advantages from vendors or open source, but
they typically do not make big risky bets, and they only change tooling and
paradigms when competition forces change.
There's also ingestion in the small - great libraries.

But this would take a long time, and I suspect great progress is not happing
in this direction. Why? I'm not sure, but I don't see a great migration in
open source to replace Java for enterprise tooling with Scala (I am aware of
the contributions flowing from startups). I do not see dozens of Apache
projects banging out new libraries and Java Enterprise API implementations
using Scala, even where they are small, lightweight subsets like JAXB and
JAX-RS. So something is amiss with the value proposition of Scala, such
that it is not getting ingested by the Java ecosystem in the small. There
is some progress, but not as much as I thought there would be at the
"Apache" level.

How about in the large? The Big Data Stack is the next big thing, and it
has been incubating on the JVM almost as long as Scala has, yet I see very
little to no influence by Scala. If Scala is such a great productive
language, why are not all the new tools around the Big Data Stack and pieces
of the Big Data Stack written in Scala? Again, I'm just not sure why this
didn't happen, just that it appears not to be, and thus ingestion as a
mechanism to break out appears not to be happening.

Then there is the operational requirements of big enterprise. How do we
constrain Scala for Enterprise? Enterprises follow a model that is the
opposite of Academia in terms of the composition and use of labor (many cogs
following directed, repeatable process to generate globally usable artifacts
vs Acadmia's accretion of new knowledge using search and investigation to
generate narrow use artifacts.) How would you get 10,000 or maybe 1000
developers or 100 or 10 to follow an approved subset of the language?
Trust? Ugh. Scala has turing complete type system (side stepping
tractability issues) - amazing for research and exploration of the possible
- not so good for 9 to 5 coder who might be proficient in 2004-2008 era Java
writing code that any of those other 10,000 devs must be able to read and
modify.

I used to think Scala was going to crack the enterprise by 2009-2010 (been
an avid follower since 2007). Then the great collection rewrite showed us
that type abstraction in Scala would allow us to do things more correctly
and keep type safety. However that type safety had a cost and the
collection API is now non-navigable or extendable by new users. How do we
make the same kinds of jumps in usability abstraction?

In many ways Scala's direction appears to be the antithesis of big
enterprise tooling. But, every consumer having a ubiquitous computing
device that is always connected in their hand is forcing big enterprises to
increase capabilities or lose their relationship with those customers. This
is an opportunity for Scala, but maybe not the Scala we all know and love.
Maybe it is time to evaluate a Scala "light"?

Scala's hypothesis of a scalable language fusing FP and OO was and continues
to be needed (thanks Martin), but what has been learned here can certainly
feed into a subset language that leverages Scala, but answers another
hypothesis: How do we bridge a scalable language to a constrained
environment to deliver value to the masses that don't grok type theory,
category theory/abstract algebra, and that don't have much time to learn
anything that does not fit their current analogies or at least have obvious,
predictable value model? I used to think DSLs were the vehicle by which
Scala would crack into the enterprise.
While they are targeted, and offer high levels of productivity - in the end
they require you must understand all of Scala to extend, grow, and therefore
require passing the enterprise test before they can be used to bring the
language in.

Is working with the constrained big enterprise something the Scala community
even wants to solve? It cuts into resources and that cuts into all the
great edge problems Scala allows us to interact with.
You could think of Scala as an incubator - somewhere between edge research
and somewhere between getting real stuff done at tomorrows big companies
(startups). These environments have a great deal in common, and it isn't
surprising Scala can thrive there. But enterprise is a re-think if you want
to target them. I don't think providing more documentation or a flawless
GUI based environment solves the problem outright. Imagine one small team
using modern day fighter jet fighter for missions. The training, the
documentation, the hours to effectively use it are unreal so you have to be
a top student - but the result is very capable, fast, and far reaching
instrument a handful can use. Well, enterprises are more like big
battleships. slow to turn. lots of hands on deck taking orders in a
hierarchy to accomplish things. Sure battleships might have some cool
bolt-on technology, and some really smart people on board (and even more
9-5er types). But, I don't think providing more doc, training, or a
flawless heads up display will turn battleship hands into jet fighter
pilots, let alone battleships into aircraft carriers. Maybe it is time to
reframe the problem.

Just some food for thought on this issue.

Kind Regards,

Jim Herber

On 09/11/2011 23:30, Francois Armand wrote:

> A "Scala Cookbook" could be intersting, but I would much more prefer a
> repository of Scala scripts, if they were usable for real.

I'd find this useful too. I'd also be happy to contribute - for example
at the moment I'm writing a simple stand-alone J2EE servlet that allows
you to browse Mercurial history without running 'hg serve' for each
repo. Part of that involves parsing XML via a pipe and processing the
resulting scala.xml.Document - not rocket science but there's a couple
of bits that took some hunting around and reading of the library source
to figure out - for example the fact that BufferedSource sometimes
scribbles on stderr and that ProcessIO leaks file descriptors.

My concern is that I'm still learning Scala so I'm not sure my code is
exactly 'Best Scala Practice'. If such a script repository was set up I
think there'd need to be some vetting process for the content. Ideally
it would be possible to annotate the scripts with comments and
suggestions - for people learning Functional concepts, "why" is every
bit as important as "how".

> 3. Being able to use Scala for scripting. Than implies 1 and a better
> starting time + something to include automagically dependencies,
> essentially. And it will be so good to show how cool and powerful Scala
> is in some little lines. I really would *love* to convert the sysadmin I
> work with to use it as possible replacement of perl or python (or bash),
> but for now, the lauching time are killing that possibility.

My 2c.

System administrators, in general, have rather particular
requirements.

Here's a couple that I've observed:

1) Simplicity.
2) Stability.
3) Speed, but simplicity is much more important.
4) An API, (where applicable) which wraps OS level functionality in
simple fashion.
5) Can be installed, without playing version bingo, from the OS
package management system.
6) Fast code/test/deploy cycle, for a 200 line program, compile/
startup time of more than 2 seconds is too long.
7) Is Bourne, Python, Ruby or for those hangers on from the dot-com
boom, Perl.

System administrators, in general, don't like the following things.

1) Dynamic behaviour redefinition, think implicits.
2) Factory classes, design patterns, types, static typing.
3) Monads, Monoids, things that make more than 2 minutes to make sense
of.
4) Stuff that needs to constantly auto-update itself from some server
in Switzerland, every time you do a build.

I could be wrong....

On Wed, Nov 9, 2011 at 21:30, Francois Armand wrote:
> Le 05/11/2011 22:51, Ken McDonald a écrit :
> [...]
>>
>> The question is simple: List, in descending order of priority, what you
>> feel needs to be addressed/fixed/whatever for Scala to succeed in the
>> corporate (and hence IMHO ultimately the "real") world.
>>
>> My list:
>>
>> 1. IDE Support. [...]
>>
>> 2. Documentation. [...]
>>
>> 3. A "Scala Cookbook". [...]
>>
>
>
> My shop list would be:
>
> 1. A compiler as fast as the java one. And a modular scala-lib. Yeah, I
> now...
> 2. An as powerful and great library to deals with immutable data structure
> modification as the collection one is for collection (lens, zipper, etc)
> 3. Being able to use Scala for scripting. Than implies 1 and a better
> starting time + something to include automagically dependencies,
> essentially. And it will be so good to show how cool and powerful Scala is
> in some little lines. I really would *love* to convert the sysadmin I work
> with to use it as possible replacement of perl or python (or bash), but for
> now, the lauching time are killing that possibility.

You mean like this?

#!/usr/bin/env scalas
!#

/***
scalaVersion := "2.9.0-1"

libraryDependencies ++= Seq(
"net.databinder" %% "dispatch-twitter" % "0.8.3",
"net.databinder" %% "dispatch-http" % "0.8.3"
)
*/

import dispatch.{ json, Http, Request }
import dispatch.twitter.Search
import json.{ Js, JsObject }

def process(param: JsObject) = {
val Search.text(txt) = param
val Search.from_user(usr) = param
val Search.created_at(time) = param

"(" + time + ")" + usr + ": " + txt
}

Http.x((Search("#scala") lang "en") ~> (_ map process foreach println))

See https://github.com/harrah/xsbt/wiki/Scripts. Works like a charm.

>
> IDE support is reaching the point where Eclipse does not go in my way
> anymore and is of real help for boring things (imports, completion of long
> method names, formatting, things like that).
>
> Documentation is OK, especially with so many books and blogs post online.
>
> A "Scala Cookbook" could be intersting, but I would much more prefer a
> repository of Scala scripts, if they were usable for real.
>
> Cheers,
>
> --
> Francois Armand
> http://fanf42.blogspot.com
>

mainly because I judged the compiler to be
unreliable.