This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Guardians "Open Platform" Uses Scala

On the 20th May the Guardian announced that their "Open Platform", an API to access the vast repository of Guardian media with over a million articles, video clips, photographs and audio tracks, was "open for business". With 36 million people making regular use of the repository, the Open Platform was created to service the rapidly growing demand and provide a high performance interface for media application developers. Over 2000 of them registered to use the API and 200 application products built to use it during the Beta evaluation phase. This exciting API was implemented in Scala using Lucene/Solr for media storage.

Graham Tackley, the Web Platform Development Team Lead for guardian.co.uk, explained, slides here, at Eurocon 2010 how they represent their complex 350 table relational database model in Solr for the media storage and used Scala to meet the demanding real-time content searching, indexing or updating. Using actors for example, he explains how they were able to reduce the search index build time from 20 hours to just one. Request patterns, he says, are hard to predict so the ability to easily scale the services was essential. His presentation slides contain more detail on the system architecture and you can find Grahams impressions of Scala on his blog.

The developers can create client facing "MicroApps" using Google App Engine. This give the developers a way to create new, highly scalable applications that interface via the Open Platform API to the Guardian repository.

More information on Open Platform can be found on the Guardian Open Platform blog and a slide presentation here.

Cryptic method names - Take a look at Python!

Graham Tackley says in his article: "Scala gets some bad press because it's possible to get carried away with some its shiny new toys, especially the ability to create methods with non-alpha names. Some of this I agree with - methods like >:> really doesn't aid readability for me."

 

I agree with him and think that cryptic method names and operators are a big disadvantage of Scala! A programming language should be easy to read and understand what implies that the reader shouldn't have to expand cryptic expressions in his head. Method names like "/:" (foldLeft) are not commonly understood and ugly! They are perlish. As a rule of thumb I would only use those operator method names that are widely understood (+, -, *, ...). It's a bad idea to save a few characters at the expense of readability.

 

I suggest the Scala community to take a good look at Python that is concise and mostly self-explanatory.

Re: Cryptic method names - Take a look at Python!

Hi Christian,

while I agree that using unnecessary cryptic method names is a bad idea regardless of the language used, I don't agree with you that this is Scala's disadvantage.

 

Fact is that it's always hard to "define" a set of "allowed" operators, because people tend to come from a different background. For instance ":" or "::" are widely understood in the functional community, would you allow it or not?

Erlangs "!" or the factorial "!" are quite common, too. Allow or disallow? What about all the things from Haskell?

 

I personally prefer a language which doesn't think i'm not intelligent enough to choose a good method name. If I would choose a bad method name why should the language allow "def dsfighsdkjgbdag" but disallow "def *->:"?

It is my personal duty as a developer to choose method names which are understood by its users. Restricting the range of allowed characters doesn't help me here at all.

If you don't like the shortcurts of foldLeft or foldRight, you are not forced to use them. But most people like them, because they visualize the shape of the tree that gets evaluated.

 

Thanks for your kind hint to Python! Although it is a bit better than PHP in my opinion, it is still a dynamically typed language and therefore more a tool for hobbyists.

 

Re: Cryptic method names - Take a look at Python!

I perfectly agree with you. I like the way scala doesn't limit you in any way 

 

Re: Cryptic method names - Take a look at Python!

Using cryptic method names is not a matter of syntax, but a matter of convention. Even Java allows to develop overly complicated code, like methods with to many if statements or classes with to many dependencies and tangles. Modern IDEs together with code inspection and analysis tools have drastically improved the code quality in recent years. There should be tool supported conventions for cryptic method names as well.

Re: Cryptic method names - Take a look at Python!

This discussion is of course subjective, but maybe not as subjective as many people think. In my opinion there is something like API or language usability. The key to a usable API/language are intuitive names. Ideally a program can be understood by a programmer no matter what programming background he has. And I think API/language usability could be tested!

Compare Python's

  for element in colletion:

to Scala's:

  for(element <- collection) {

Ok, "<-" is not that hard to understand, but "in" is obvious.

I've heard more than once that Scala is as readable as Perl!

Python was only an example for readable language; wether Python is dynamically typed or not isn't relevant here. But to blaspheme Python as a tool for hobbyists sounds ignorant to me, even if I like statically typed languages better too.

Re: Cryptic method names - Take a look at Python!

Well, I don't think your example is ideal, given that the "<-" is not really an API thing, but one of Scala's few "real" keywords as far as i know.

Considering that most people don't even use that for-loop, but prefer the possibilities of the collection API instead, I have to wonder:

When I would have to look up how Pythons for loop works, how would i do that? Considering I have a collection how would I even discover that I have the possibility to loop through it?

 

In Scala this is quite obvious, consider the following example:

<code>

val collection = List(1,2,3,4,5)

collection foreach prinltn

<i> => Prints each number to a new line.</i>

</code>

Basically this is a method of the collection, which can be shown by an IDE or a commandline tool, like all other methods.

So instead of using some custom handcrafted control syntax, just use the possibilities of the libraries.

Instead of using some imperative multi-line code to do a simple thing, you can use Scala's functions of higher order and be done on the same line you started. (I'm wondering why Python has some functional methods (filter, map, reduce), but didn't add them to the collection. Why they force people to use a different syntax for it is beyond me.)

So speaking of cryptic things: Would you agree to remove things like [1,2,3], set([1,2,3]) or {'foo' : 1, 'bar' : 2} from Python?

In my opinion this is far more worse than any "cryptic" method names. There is no way an IDE or another tool can help you here. In more regular languages can write down "List", "Set" or "Map" and then look at the description and the available methods.

Regarding consistent code and a good API design, just look for instance at Python's list again:

Some things use angle brackets, like the list constructor ([1,2,3]), the selection of an item of the list (l[1], slicing the list (l[1:3]).

Some things use <element> <some words> <list>, like the Python contains method and it's negation ("in" / "not in").

Some things use <list> <operator> <element>, some <method>(<list>).

I wouldn't call that consistent or a good API, it's more a bunch of things happily slapped together. This wouldn't be bad at all, if we wouldn't have had enough languages doing this already. But I guess everyone can live with that.

I just personally prefer Scala's way to have an object, its methods and maybe arguments: it's always this way and only this way. No need to learn some special syntax or anything. If I have a question, I look at the methods.

 

I think everything can be seen from different viewpoints. Maybe there will be some alternative parser in the future, which allows people to program Scala with a Python-like syntax! I would love that.

 

 

Re: Cryptic method names - Take a look at Python!

You are right, Python is not as consistent as it should be, but Python code is very readable though. I think it is so because Python uses words where other languages are using (cryptic) operator symbols. Another factor is Python's minimalism; for example the absence of curly braces for blocks.

My intention was not to start another language flamewar, but to suggest that the scala community should take care of API and language usability. Why are there usability tests for GUIs but not for APIs and programming languages? Scala could do better!

Re: Guardians "Open Platform" Uses Scala

If /: seems like gobbledygook to you and you are open to feeling differently about it, this bit of intuition might help.

I actually quite like /: now -- it is much more obvious to me than foldLeft.

 

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland