This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Scalable Component Abstractions, and runtime configuration

3 replies
David Hall 4
Joined: 2009-08-21,
User offline. Last seen 42 years 45 weeks ago.

(long post, sorry. tl;dr we have this beautiful cake pattern, but it
doesn't play nice with runtime configuration. What to do?)

In my last push for a paper deadline, I ended up needing to do a kind
of "grid search" on a number of parameters and configurations.
Basically, I needed to run my system under the crossproduct of number
of parameters (baseline or no, one of K kinds of pruning, learning or
no, different thresholds, initial starting points, etc. etc.). Because
I was in a rush, I ended up making a mess of my code, but now that
it's done, I'm trying to go back and figure out how to do it right.

Some of these parameters are easy to get right: the booleans and
doubles, and probably are best solved with some kind of config file.
It's the

Being a zealous convert to Scala, I may have overapplied the Cake
pattern, which I think in this case compounded the problem. Basically,
I had a class "Algorithm" something along the lines of this:

abstract class Algorithm(params:...) {
type MFactors <: Factors
def initialFactors(...):MFactors
def reestimateFactors(f: MFactors,...): MFactors
// actually do stuff with the factors somewhere else.
}

I had two possible configurations for MFactors, Baseline and Clever,
which were provided to Algorithm using mixins. Along the same lines,
the Clever version has another abstract type, which can be filled in
one of three ways. Clever also can be learned or not, and learning has
3 x 3 possible configurations. I tied all of these things together
with traits, and it made coding much more pleasant and--for the most
part--easy to understand.

But then came experiment time, and it seemed like I needed to create a
different entry point object for each configuration of traits I
wanted.

In Java-land, the answer is some kind of dependency injection or
factories, either homebrewed or using some kind of framework. With
Scala, complex Cake-objects can't be dynamically stitched together at
runtime by the JVM, and so you have to rely on the interpreter some
how (which seems to make things like distribution harder) or abandon
Cake or generate the cross product at compile time...

What's the answer for us in Scala-land? Scripts? Do I replace my
abstract types and abstract methods with dependent types and
forwarding methods? Am I missing something obvious?

Thanks for any insight!

David Hall 4
Joined: 2009-08-21,
User offline. Last seen 42 years 45 weeks ago.
Re: Scalable Component Abstractions, and runtime configuration

Hit send too early... s/It's the/The problem is what to do with mixins./

On Sat, Feb 20, 2010 at 12:43 AM, David Hall wrote:
> (long post, sorry. tl;dr we have this beautiful cake pattern, but it
> doesn't play nice with runtime configuration. What to do?)
>
> In my last push for a paper deadline, I ended up needing to do a kind
> of "grid search" on a number of parameters and configurations.
> Basically, I needed to run my system under  the crossproduct of number
> of parameters (baseline or no, one of K kinds of pruning, learning or
> no, different thresholds, initial starting points, etc. etc.). Because
> I was in a rush, I ended up making a mess of my code, but now that
> it's done, I'm trying to go back and figure out how to do it right.
>
> Some of these parameters are easy to get right: the booleans and
> doubles, and probably are best solved with some kind of config file.
> It's the
>
> Being a zealous convert to Scala, I may have overapplied the Cake
> pattern, which I think in this case compounded the problem. Basically,
> I had a class "Algorithm" something along the lines of this:
>
> abstract class Algorithm(params:...) {
>  type MFactors <: Factors
>  def initialFactors(...):MFactors
>  def reestimateFactors(f: MFactors,...): MFactors
>  // actually do stuff with the factors somewhere else.
> }
>
> I had two possible configurations for MFactors, Baseline and Clever,
> which were provided to Algorithm using mixins. Along the same lines,
> the Clever version has another abstract type, which can be filled in
> one of three ways. Clever also can be learned or not, and learning has
> 3 x 3 possible configurations. I tied all of these things together
> with traits, and it made coding much more pleasant and--for the most
> part--easy to understand.
>
> But then came experiment time, and it seemed like I needed to create a
> different entry point object for each configuration of traits I
> wanted.
>
> In Java-land, the answer is some kind of dependency injection or
> factories, either homebrewed or using some kind of framework. With
> Scala, complex Cake-objects can't be dynamically stitched together at
> runtime by the JVM, and so you have to rely on the interpreter some
> how (which seems to make things like distribution harder) or abandon
> Cake or generate the cross product at compile time...
>
> What's the answer for us in Scala-land? Scripts? Do I replace my
> abstract types and abstract methods with dependent types and
> forwarding methods? Am I missing something obvious?
>
> Thanks for any insight!

Dean Wampler
Joined: 2008-12-26,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Scalable Component Abstractions, and runtime configura
Nik Kallen (Scala user at Twitter) and Yahuda Katz (Ruby on Rails core committer at Engine Yard) had an interesting exchange recently on twitter and their blogs about these sorts of issues in statically-typed languages like Scala and dynamically-typed languages like Ruby. Here's Nik's blog post that started it all:
http://magicscalingsprinkles.wordpress.com/2010/02/08/why-i-love-everything-you-hate-about-java/
Here's a post by Yahuda:
http://yehudakatz.com/
It's true that dynamic configuration of traits-like modules is easier in languages like Ruby. In Scala, I would look for ways to reduce the number of permutations of traits combinations. For example, can some of the behaviors for which you now have separate traits be merged into fewer traits whose behaviors are more data driven?
Also, can some traits be replaced with delegation? That is, replacing "with" relationships with fields? Some of those fields might be functions. Then configuration becomes assignment. Actually, most dependency injection tools, like the Spring Framework, are mostly about setting fields in objects, not inheritance relationships, anyway.
Hope this helps.dean 
On Sat, Feb 20, 2010 at 2:44 AM, David Hall <dlwh [at] cs [dot] berkeley [dot] edu> wrote:
Hit send too early... s/It's the/The problem is what to do with mixins./

On Sat, Feb 20, 2010 at 12:43 AM, David Hall <dlwh [at] cs [dot] berkeley [dot] edu> wrote:
> (long post, sorry. tl;dr we have this beautiful cake pattern, but it
> doesn't play nice with runtime configuration. What to do?)
>
> In my last push for a paper deadline, I ended up needing to do a kind
> of "grid search" on a number of parameters and configurations.
> Basically, I needed to run my system under  the crossproduct of number
> of parameters (baseline or no, one of K kinds of pruning, learning or
> no, different thresholds, initial starting points, etc. etc.). Because
> I was in a rush, I ended up making a mess of my code, but now that
> it's done, I'm trying to go back and figure out how to do it right.
>
> Some of these parameters are easy to get right: the booleans and
> doubles, and probably are best solved with some kind of config file.
> It's the
>
> Being a zealous convert to Scala, I may have overapplied the Cake
> pattern, which I think in this case compounded the problem. Basically,
> I had a class "Algorithm" something along the lines of this:
>
> abstract class Algorithm(params:...) {
>  type MFactors <: Factors
>  def initialFactors(...):MFactors
>  def reestimateFactors(f: MFactors,...): MFactors
>  // actually do stuff with the factors somewhere else.
> }
>
> I had two possible configurations for MFactors, Baseline and Clever,
> which were provided to Algorithm using mixins. Along the same lines,
> the Clever version has another abstract type, which can be filled in
> one of three ways. Clever also can be learned or not, and learning has
> 3 x 3 possible configurations. I tied all of these things together
> with traits, and it made coding much more pleasant and--for the most
> part--easy to understand.
>
> But then came experiment time, and it seemed like I needed to create a
> different entry point object for each configuration of traits I
> wanted.
>
> In Java-land, the answer is some kind of dependency injection or
> factories, either homebrewed or using some kind of framework. With
> Scala, complex Cake-objects can't be dynamically stitched together at
> runtime by the JVM, and so you have to rely on the interpreter some
> how (which seems to make things like distribution harder) or abandon
> Cake or generate the cross product at compile time...
>
> What's the answer for us in Scala-land? Scripts? Do I replace my
> abstract types and abstract methods with dependent types and
> forwarding methods? Am I missing something obvious?
>
> Thanks for any insight!
> -- David
>



--
Dean Wampler
coauthor of "Programming Scala" (O'Reilly)
-  http://programmingscala.com

twitter: @deanwampler, @chicagoscala
blog: http://blog.polyglotprogramming.com
Chicago-Area Scala Enthusiasts (CASE):
-  http://groups.google.com/group/chicagoscala
-  http://www.meetup.com/chicagoscala/ (Meetings)
http://www.linkedin.com/in/deanwampler
http://www.polyglotprogramming.com
http://aquarium.rubyforge.org
http://www.contract4j.org
David Hall 4
Joined: 2009-08-21,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Scalable Component Abstractions, and runtime configura

On Sat, Feb 20, 2010 at 11:37 AM, Dean Wampler wrote:
> Nik Kallen (Scala user at Twitter) and Yahuda Katz (Ruby on Rails core
> committer at Engine Yard) had an interesting exchange recently on twitter
> and their blogs about these sorts of issues in statically-typed languages
> like Scala and dynamically-typed languages like Ruby. Here's Nik's blog post
> that started it all:
> http://magicscalingsprinkles.wordpress.com/2010/02/08/why-i-love-everyth...
> Here's a post by Yahuda:
> http://yehudakatz.com/
> It's true that dynamic configuration of traits-like modules is easier in
> languages like Ruby. In Scala, I would look for ways to reduce the number of
> permutations of traits combinations. For example, can some of the behaviors
> for which you now have separate traits be merged into fewer traits whose
> behaviors are more data driven?
> Also, can some traits be replaced with delegation? That is, replacing "with"
> relationships with fields? Some of those fields might be functions. Then
> configuration becomes assignment. Actually, most dependency injection tools,
> like the Spring Framework, are mostly about setting fields in objects, not
> inheritance relationships, anyway.
> Hope this helps.

Thanks for your reply. Basically all of my traits, in reality, can be
replaced by delegation. I saw the Cake pattern as somehow a better
form of delegation/dependency injection. (Less boilerplate, composes
nicely with other traits, you can control who can use which trait,
etc.) But it's becoming clear that in the--fairly
common--circumstance where you need to explore a large space of
possible combinations of components, the Cake pattern breaks down:
it's really for places where you either have a relatively small space
of possible combinations, or you care about only a sparse subset of
that space. So, it's great for the Scala compiler, but maybe not so
much my grid-search experiments.

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland