- About Scala
- Documentation
- Code Examples
- Software
- Scala Developers
questioning FP
body
p { margin-bottom: 0cm; margin-top: 0pt; }
Hi All,
(I welcome any sort of replies, even containing foul language, as
long as there's also information inside.)
I'm about to give a lecture on FP. Now, some concepts in FP I love, but for some I have a hard time finding a good angle. Specifically, the IO monad.
Consider the following imperative code:
service.fetchData
service.closeObviously, both method calls create side effects.
Now, (discarding iteratees), lets consider making this with the IO monad:
for (data <- service.fetchData;
_ <- service.close)
yield data
where all methods are changed to return IO[Data], IO[Unit] and also the calling code must now return an IO[X] for some X.
The question is, what have we gained?
Instead of sequencing by line order, we're explicitly sequencing in the for comprehension. Not much of a gain here, since any mistake that causes code to be written in the first order in the imperative case can also be done in the functional case.
I also don't see anything that makes the code less prone to other mistakes. Such as forgetting to close or calling close twise. Obviously, there are ways to fix this such as service.withData(f: Data => X), but again, the more imperative style of silently calling close inside withData doesn't loose to the functional style that must return IO[X].
What am I missing? I'd appreciate any other examples which show how the use of IO is superior by making code that is more robust.
Regards,
Ittay










Re: Re: questioning FP
Of course, there is the distinct possibility that this desire could be nothing more than wishful thinking.
I'm just a complete moron, but in my moronic search to find that which you desire above, I've basically come to the following conclusions:
- Most of the programming world either cannot cope with the likes of the IO monad and therefore ignore it at their peril. OR
- Some languages embrace models that work based on decades of research that bring useful tools to bear on reasoning about programs with interacting with "the real world" (IO/effects).
Are there even any hints of research that promises a coherent model of (IO) effects outside of the Monad model? The current work begin done by Lukas Rytz is, er, less than convincing.--
Jim Powers
Re: Re: questioning FP
2011/10/10 martin odersky <martin [dot] odersky [at] epfl [dot] ch>
Re: Re: questioning FP
This thread is epic.
On 10/10/2011 9:31 PM, "martin odersky" <martin [dot] odersky [at] epfl [dot] ch> wrote:Re: questioning FP
Tony Morris tmorris.net> writes:
> This thread is epic.
Tony, I was just wondering when you would chime in. Care to enlighten us with
your views on the epicness of this thread and the IO monad?
/Jesper Nordenberg
Re: questioning FP
Ittay Dror skrev 2011-10-10 07:16:
> The title is probably too much of a troll. Sorry about that.
>
> My questioning is specifically about making functions pure. And the
> hoops one need to jump through to get that (at least the way I know
> which is the IO monad). More specifically:
> 1. what does it buy me? in theory, pure functions are nice since they
> behave better, but i want to find a use case that appeals to everyday
> java developers. That is, a case where making a function pure removes
> pitfalls (while the function remains basically the same)
That's basically the same question as "what does a type system buy me?".
It enables automatic detection of programming errors at compile time
rather than at runtime.
It also opens up opportunities for performance optimizations and makes
your code a whole lot easier to test.
> 2. what are the alternatives to IO monad? an implicit World argument?
The ones I know are uniqueness typing (for example Clean) and effects
tracking (for example D, DDC, upcoming Scala effects system). So, either
you track effects using the type system, or you build a separate system
for it.
/Jesper Nordenberg
Re: Re: questioning FP
On Mon, Oct 10, 2011 at 1:49 AM, Jesper Nordenberg wrote:
> Ittay Dror skrev 2011-10-10 07:16:
>>
>> ...
> That's basically the same question as "what does a type system buy me?". It
> enables automatic detection of programming errors at compile time rather
> than at runtime.
>
> It also opens up opportunities for performance optimizations and makes your
> code a whole lot easier to test.
>
> …
Types also document aspects of the program for the reader.
Re: questioning FP
> Types also document aspects of the program for the reader.
Documentation which cannot get out of sync with the code and can be
reasoned about. The latter might seem like a totally academic idea, but
is extremely useful.
Re: Re: questioning FP
It enables automatic detection of _some_ errors; one still would like to know how big of a hassle it is (much more in Java than Scala, to the point of motivating dynamically typed languages!) and how frequent those errors are (not very, when it comes to IO, from what I've seen).
I could buy the part about testing, but performance is usually better when you pay _less_ attention, not more, to what is allowed and not. In particular, writing wrappers that are instantiated with real objects in addition to the actual IO you need to do is not a good way to have higher performance.
I'm quite skeptical of the value of _any_ of these approaches since the world outside your program does not have type information. Thus, you end up having to watch for unexpected effects and do run-time identification of your types regardless. That your type system will catch you if you, say, fail to have a file handle closed at a certain point is usually not a big deal (and that has lighter-weight solutions anyway for the large majority of cases).
--Rex
Re: questioning FP
Jesper Nordenberg wrote:
I'd love to see a code example where the introduction of IO[_] makes more errors detectable. In the code example I used in the first message, I couldn't find where it helps.
Performance how? Because of laziness? This can also work against you. In the lowest level it looks to me like working through a bind function is less efficient than working with the value directly.
How is code easier to test? I initially thought that when I have a pure function, it will behave the same every time. So once I test it I'm sure it works the same in production. But then I thought: Ok, lets assume that I have a function that closes a socket. Further, assume that closing it a second time creates an error. If I have code that does 'for (_ <- close(socket); _ <- close(socket))', it will fail just the same as the (simpler) imperative code "socket.close; socket.close", wouldn't it? Side effects are not gone. They are just abstracted over, the question is whether the abstraction helps.
Again, would love some code examples.
Re: Re: questioning FP
Consider the following code outline, and assume a purely functional implementation of Scala (even if strict!)
object A { def foo(String):Int = ... def bar(String):Int = ...} Then the program:
object B { def run(f:String,b:String):Int = A.foo(f)+A.bar(b) }
At the very least, in our purely functional version of Scala we know that the evaluation of A#foo and A#bar cannot possibly affect one-another, we are guaranteed referential transparency.
Now, drop back into *real* Scala. Without actually knowing what are inside foo and bar we cannot be assured that evaluating foo will not alter the results of bar. If there is referential transparency between foo and bar, to quote Adrian Moors elsewhere in this thread, it is implicit.
OK, let's switch back to the purely functional version of Scala for a moment and rewrite things like this:
object A { def foo(String):Int = ... def bar(String):IO[Int] = ... }
then:
object B { def run(f:String,b:String):IO[Int] = { val x = A.foo(f) A.bar(b) }}
Since we are back in purely functional land we can observe two things:
- The evaluation of A.foo(f) cannot possibly effect the results of A.bar(b) and therefore can trivially be eliminated (optimization!) as dead code
- The evaluation of A.bar(b) cannot possibly alter the result returned bu A.foo(f) otherwise A#foo type signature would indicate that
Unless I'm grossly mistaken we are already in a better world given the above.Finally, let's write(still in a purely functional Scala universe):
object A { def foo(String):IO[Int] = ... def bar(String):IO[Int] = ... def baz(String):Option[Int] = ...}
then:
object B { def run(f:String,b:String,z.String):IO[Option[Int]] = for { a <- A.foo(f) b <- A.bar(b) } yield A.baz.map(_+a+b)}
We know a bunch of information merely from the types alone to help us track down a possible bug in the above program. We know, from the types that A#baz is a pure function and we can test it's behavior with deterministic outcomes. On the other hand A#foo and A#bar may interact in strange ways. In the case of an error in the above program the search space to look for the bug is easily partitioned on types alone.
Extend this to a "real-world" example as necessary.
This doesn't come across as progress? Using the IO monad can guide us as to where to look for "action at a distance" kinds of bugs.
--
Jim Powers
Re: Re: questioning FP
In particular, you have not given an example that shows why it is a bad idea to inject new state with IO but deal with that state functionally thereafter, or to output existing state, treated functionally prior to that point, with IO. Just leave off the IO marker and all your points remain; you can do all the same things with unwrapped and/or Option-wrapped types (depending on whether or not the result could fail to exist).
The minimal example where the IO monad is actually better than the _most sensible alternative_ (not a mythically messy imperative situation) seems to me to require an awfully complicated setup, if it even exists at all. (I suspect based on the logic of the arguments that it does exist, but I have yet to see an example that would not have been simpler without the IO monad specifically.)
--Rex
On Mon, Oct 10, 2011 at 5:36 PM, Jim Powers <jim [at] casapowers [dot] com> wrote:
Re: Re: questioning FP
Actually I did ( :-) )! What particular IO actions were intended by my functions wasn't relevant to the discussion.
Well, no, not really. Another try - here's a variant of the first example (recall, we have an imaginary purely functional version of Scala):
object A { def foo(String):Option[Int] = ... def bar(String):Option[Int] = ... } Then the program:
object B { def run(f:String,b:String):Int = for { a <- A.foo(f) b <- A.bar(b) } yield a+b }
Is the exact same program as:
object B { def run(f:String,b:String):Int = for { b <- A.bar(b) a <- A.foo(f) } yield a+b }
Because in the case of a purely functional language I know that I can always commute evaluation order. Further, the computer knows this too! Substitute in the IO monad for the Option and the guarantees about commutativity go out the window. Granted that using functions in an IO context like above where I didn't take steps to eliminate the possibility of a bug by forcing the correct order (say through the use of a Reader or State monad) is possible. This is to say that while I can substitute IO for Option above the guarantees of correctness do not come along for the ride. (And, as I've said there are clean ways to enforce a "correct" evaluation order of side-effecting code. Overall, once these approaches are learned, your "cognitive load" actually goes down not up).
No, this claim is false as indicated by the above example: in general, IO cannot be substituted for other Monads and maintain the same correctness guarantees.
I'm not aware of a "sensible alternative", I'd be happy to hear of one. Martin is claiming such a thing exists, it would be nice if is work was fruitful. I would suspect that if he is he would be ushering in an whole new branch of mathematics :-) -- Jim Powers
Re: Re: questioning FP
Granted, but your earlier code didn't do anything to guarantee order, so it wasn't of any use except as a tag. And it's not a very useful tag because the tag is at the _declaration site_ not the _use site_. At least if you had to call something like Console.readLine it would be clear where the action was. If you mark inputs as being IO[String] and then unwrap them somewhere buried in the code, you don't have to know which sort of thing you're unwrapping (maybe it was an Option or some other sort of wrapper).
Indeed. But if one does things this way, it is a huge hassle (of the major-rewrite-to-add-a-println variety). For specialized cases, encoding a finite state machine into your types (for instance) can be exactly what you need to keep things straight. But from what I've seen, this is a niche case when it comes to IO.
The sensible alternative is untagged side-effectful code that immediately moves data into an appropriate typesafe and perhaps immutable form.
--Rex
Re: questioning FP
On Oct 11, 2011, at 10:17 AM, Rex Kerr wrote:
> Indeed. But if one does things this way, it is a huge hassle (of the major-rewrite-to-add-a-println variety). For specialized cases, encoding a finite state machine into your types (for instance) can be exactly what you need to keep things straight. But from what I've seen, this is a niche case when it comes to IO.
I feel like I should point out that adding a println to something that isn't already an IO action should be done for debugging only (and that's likely all people want when they bring this point up). It /is/ absurd to change signatures just to do this. In Haskell there is Debug.Trace[1] for these purposes. There's no reason not to create a similar "escape hatch" for use in Scala.
1: http://www.haskell.org/ghc/docs/6.12.1/html/libraries/base/Debug-Trace.html
Re: Re: questioning FP
>
> I'm not aware of a "sensible alternative", I'd be happy to hear of one.
> Martin is claiming such a thing exists, it would be nice if is work was
> fruitful. I would suspect that if he is he would be ushering in an whole
> new branch of mathematics :-)
>
I don't recall claiminig that it existed. I said I was looking for it.
It's called research.
Re: Re: questioning FP
Thank you for this excellent example.
All your points are valid, and very true. The question is not whether we gain something by using the IO monad. It is whether we gain enough. Since using it comes with its own price of trickier code, and mental hoops to jump through, spending brain CPU on how to deal with the monad, rather than the business problem.
Normally, one does not work with functions without knowing what they do. When I know what a function is supposed to do, I usually know what side effects it is doing, or, I don't care because they are an implementation detail. If they were not, then all I know is that something is going on under there, but can't do anything about it.
Ittay
Jim Powers wrote:Re: Re: questioning FP
Well, that's true with anything, that you have to decide for yourself if it's worth the trouble to understand. Before you can say "this is beneficial" you have to ask "beneficial for whom and for what purpose?"
We are using an IO-like monad to isolate database effects where I work. We're also using a separate State-like monad to compose computations that have state (instead of using mutable state). We find this to be a huge benefit. Especially when we can re-use the same code for both things that seemingly have nothing to do with each other. The code that is IO-specific is very short and confined to one or two files.
In practice, once you have factored out this concern, almost all of your code is pure. I.e. You don't see "IO" in very many places. It's not at all true that most code has to have side-effects.
The argument I'm hearing is that IO is somehow special. A "cross-cutting concern". Bullshit. It's just another DSL.
I'm kind of done with "oh, but don't you want to convince me?!" No, I really don't. The best you can do is try it both ways, know what you're talking about first hand, and then decide for yourself. There's no magic in monads. It's just separation of concerns and combination of components according to a simple interface.
Re: Re: questioning FP
On Tue, Oct 11, 2011 at 8:33 AM, Runar Bjarnason <runarorama [at] gmail [dot] com> wrote:
Re: Re: questioning FP
Runar Bjarnason wrote: Interesting. Maybe I missed something. I thought that when you use IO you need to propagate it up to 'main' (or 'run'). So one function that does IO (a println) pollutes your whole program. Where am I wrong?
Re: Re: questioning FP
On Oct 11, 2011 7:56 AM, "Ittay Dror" <ittay [dot] dror [at] gmail [dot] com> wrote:
>
>
>
> Runar Bjarnason wrote:
>>
>> Well, that's true with anything, that you have to decide for yourself if it's worth the trouble to understand. Before you can say "this is beneficial" you have to ask "beneficial for whom and for what purpose?"
>>
>> We are using an IO-like monad to isolate database effects where I work. We're also using a separate State-like monad to compose computations that have state (instead of using mutable state). We find this to be a huge benefit. Especially when we can re-use the same code for both things that seemingly have nothing to do with each other. The code that is IO-specific is very short and confined to one or two files.
>
> Interesting. Maybe I missed something. I thought that when you use IO you need to propagate it up to 'main' (or 'run'). So one function that does IO (a println) pollutes your whole program. Where am I wrong?
As someone who hasn't gotten much further then LYAH, I'll see if I can explain (as much for my benefit as anything else):
Just like the fact that you can use the value stored within an Option using map or flatMap, the value within IO can be passed to pure functions as well.
If you consider the previous example of adding 2 numbers that are retrieved by some IO, you do not need '+' to be aware of the source of these numbers. Ideally you would keep the IO type as close to your 'main' method as possible, using pure functions to actually do something with the results of the IO.
This results in a program where you naturally tend to separate your IO from the functions that process that data, hopefully leaving the majority of your program IO free.
--
On Oct 11, 2011 7:56 AM, "Ittay Dror" <ittay [dot] dror [at] gmail [dot] com> wrote:Derek
Re: Re: questioning FP
Derek Williams wrote:
The difference is that for Option I have a catamorphism. So I can always "get out" of using Option. With IO, once I use it, for no matter how small an issue, I need to use it in all the call hierarchy and change everything to work through for comprehension.
A sane developer will do that anyway (a bad developer will muck any codebase).
Returning to my original question, using Options for example (and monads in general) is easy to explain: you have an alternative, more semantically true, to returning null, so your code becomes safer (a function that returns nothing is clearly documented with the type), and remains clean since the use of Option is usually local (at some point you have a default value, and use getOrElse to reduce the Option to a value). With IO, it is trickier to explain:
First, because IO is not necessarily dangerous (unlike trying to invoke a method on null).
Second, because the IO monad does not prevent me from doing a dangerous action, just wraps it (if I try to read a non existent file, my app will fail, with or without IO[_]).
Third, it is pervasive (no catamorphism)
So the question is, what clear gain can I show?
Re: Re: questioning FP
On Tue, Oct 11, 2011 at 11:36 AM, Ittay Dror <ittay [dot] dror [at] gmail [dot] com> wrote:
You can get out, it's called running the program and it yields () - your program is just one big catamorphism over "actions" (how these actions are evaluated is entirely up to the implementation of IO that you use). :-D
You don't need for comprehensions you can map/flatMap directly (you can use >>= [and friends] from Scalaz ;-) ).Having something like IO (or monads in general) helps you to stay sane in an insane world where "bad" developers will muckity-muck your code. Again, it's really just being clear to the compiler what your intentions are; no harm can come from that. In fact, it helps your bad developers learn how to better organize their code so they become better developers. This is not merely discipline, it's computer-assisted proof development.
In the end your (our) profession is really only going to become more productive and, er, effective if we work hand-in-hand with software that can better reason about our intentions. Today that means FP for clarity and concision and types to help with correctness and documentation. Tomorrow this will mean (just guessing here) dependent types and theorem-proving systems (take a look at ATS: http://www.ats-lang.org/, or Agda: http://wiki.portal.chalmers.se/agda/pmwiki.php for some examples). Either way there are really only upsides to better (machine enforced) clarity. This will come at a cost of having to be explicit more and implicit less. Well even "pure" functions can be dangerous: division by zero, non-termination, etc. With an improved ability to express intent via the type (and proof) system (including monadic types like IO) we gain confidence in the correctness of our programs. At the very least enforced partitions are created to help isolate defects. And purity alone can't guarantee termination, avoid of resource exhaustion (stack/heap), not stepping outside of array bounds, eliminating division by zero, treating partial functions as total, etc. One of the hallmarks of FP is a tendency towards concision, and often that concision helps to identify/avoid some of the aforementioned problems. Types like IO may not prevent mistakes associated with side-effecting code, but it does isolate and label such side-effecting code. That clarity of intent is beneficial. That isolating components into composable parts, including side-effecting code, encourages better comprehension and reuse of code. --
Jim Powers
Re: Re: questioning FP
On Tue, Oct 11, 2011 at 11:36 AM, Ittay Dror <ittay [dot] dror [at] gmail [dot] com> wrote:And if I take the head of a List, my app will fail too. If I divide an Int by zero, my app will fail. This doesn't mean List and Int are useless types.
That having been said, I am working on getting regional IO into Scalaz's IO monad. This will guarantee, in the type system, that you can never read, write, or close a file that isn't open. Having kind polymorphism in Scala would make this much easier than it is (Martin, Adriaan, et al, take note).
Re: Re: questioning FP
Runar Bjarnason wrote:
I thought that similar to how Option guards us from nulls, IO guards us from side-effects (Int is not supposed to guard us from division.). If it doesn't, then again, what is the point? Just to tag a function as doing IO? And then tag all its callers? And make their code a bit awkward (for comprehension)? Isn't this like checked exceptions?
cool! This is something which has tangible benefits.
Re: Re: questioning FP
On Tue, Oct 11, 2011 at 12:23 PM, Ittay Dror <ittay [dot] dror [at] gmail [dot] com> wrote:
I don't think it's useful to think of it as a tag. It is a data type. Again, a value of type IO[A] is a program, written (supposedly) in a DSL that talks about reading and writing some opaque IO subsystem. In practise, the instruction set of this DSL is very small indeed (open file, close file, read file, write file, and maybe a handful more). But you're right in that a value of this type can generally not be converted to anything else. This is why IO should form only the outermost shell of your program, a very thin layer that talks to the outside world. I think you'll want to do that whether or not you choose to take advantage of monadic composition.
All I can say is "try it". I don't think you will run into all the problems you think you might, and it's not as awkward as all that (certainly not as awkward as, say, trying to retrofit to pure code a function that assumes the database is configured and open and tries to read from it). You don't ever need to "get out" of IO. You can always take a function of type A => B and turn it into a function of type IO[A] => IO[B].
Re: Re: questioning FP
On Tue, Oct 11, 2011 at 9:55 AM, Ittay Dror <ittay [dot] dror [at] gmail [dot] com> wrote:
Right there.
In practise, you compose effectful things in two ways:
for {
x <- effectsHere
y <- effectsHereToo(x)
} yield noEffectsHere(y)
or...
(effectsHere |@| effectsHereToo)((a, b) => noEffectsHere(a, b))
You'll find that most of your code is in "noEffectsHere". It doesn't need to be aware that it can be lifted into IO. This is what separation of concerns is all about.
Re: Re: questioning FP
On Tuesday, October 11, 2011 4:11:01 PM UTC+2, Runar Bjarnason wrote:
I don't know about that. Maybe in trivial cases it can be arranged that the effects are calculated upfront, but in a complex system?
Let's say I want to use an event bus. So it has a method 'EventBus#pubish(event: Event): IO[Unit]', right?
Now in every piece of my code where I want to publish an event, I need to return that IO instance.
So whenever I have a new event I want to publish (say after taking some step in the business flow), all methods leading to that point need to change to return IO[_], and all their client code.
Suddenly, my business code has changed because I wanted to publish an event: abstraction and modularity went down the drain.
Also, I have a new source of bugs to deal with. Namely that one of these methods will forget to return the IO instance (if I use IO[_] a lot, then they already need to deal with some instances, so it is easy to forget)
Perhaps if I'm already skilled with IO[_] I can arrange my code cleverly so that introducing event publishing does not change it significantly (can it? how?). But then wouldn't this code be more complex to begin with? Creating yet another source for bugs?
Ittay
Re: Re: questioning FP
On Thu, Oct 13, 2011 at 01:01:22PM -0700, Ittay Dror said
> I don't know about that. Maybe in trivial cases it can be arranged that the
> effects are calculated upfront, but in a complex system?
>
> Let's say I want to use an event bus. So it has a method
> 'EventBus#pubish(event: Event): IO[Unit]', right?
>
> Now in every piece of my code where I want to publish an event, I need to
> return that IO instance.
>
> So whenever I have a new event I want to publish (say after taking some step
> in the business flow), all methods leading to that point need to change to
> return IO[_], and all their client code.
>
> Suddenly, my business code has changed because I wanted to publish an event:
> abstraction and modularity went down the drain.
The business logic should be viewed as producing a stream of events to
be published instead of code that publishes events and produces a
unitary value. This is a good thing from the perspective of testable
code. You don't have to stub or mock the event bus to observe the
produced events, you can just inspect the result of calling the business
logic and check that the events are those you expect. Abstraction and
modularity have now been enhanced not eliminated.
> Also, I have a new source of bugs to deal with. Namely that one of these
> methods will forget to return the IO instance (if I use IO[_] a lot, then
> they already need to deal with some instances, so it is easy to forget)
>
> Perhaps if I'm already skilled with IO[_] I can arrange my code cleverly so
> that introducing event publishing does not change it significantly (can it?
> how?). But then wouldn't this code be more complex to begin with? Creating
> yet another source for bugs?
I hope it is clear that if the code is refactored as I suggested above
then the business logic code does not get any more complex and in fact
may become simpler. You might consider this to be a form of inversion of
control. Dependency injection inverts control of arguments, the pattern
I'm proposing inverts the control of results.
Re: Re: questioning FP
Geoff Reedy wrote:
It will also be producing values to be cached, log statements to be written, etc. In short, a complex result that should now be handled in the root entry of the program, even though some things may be implementation details. why should the top of the application care that I'm creating events as an architectural decision? Why should my bootstrap module be modified because some other module decided to do things like caching or events or write logs?
Yes, when my code is not longer doing complex interactions it is easier to test. But now the caller of this code must do these and be harder to test.
With DI and proper interfaces, I don't find mocking is hard. It is harder than matching the result values, but considering the drawbacks of this approach, maybe it is a price to pay.
Well, it is a good suggestion (with pros and cons), but note that when I asked a while ago about how to implement such an event bus in the scalaz mailing list a while ago, the reply was to use IO[_]. I like your suggestion much better.
ittay
Re: Re: questioning FP
First of all, thanks for starting this thread because IO is an inevitable question to anyone trying to do FP.
I've been reading this thread on and off trying to setup my mind about how I should feel concerned about the issue you describe. Then I realized that I am completely concerned with specs2!
Since the code is public, this might be a good example to support the discussion. Basically in specs2, when you execute a specification, you follow a (simplified) chain of actions like that:
run(spec) = select |> execute |> report
In the latest version of specs2, I've added a feature doing some IO:
run(spec) = select(IO[uses previous stats]) |> execute |> storeStats(does IO) |> report
This allows the user to select only examples which previously failed based on results stored on the disk. This also means that the whole chain should be more or less all "tainted" by IO types, whereas now the fact that I'm using IO is buried inside the components doing the selection and the storage.
I take it, from the current discussion, that a more functional way to architecture things (and this why I was mentioning "architecture" in one of my tweets) would be to be something like this:
run(spec) = readStats |> select |> execute |> report |> storeStats
This way the IO is left at the periphery of the system and not buried in the middle.
That being said, I think that one of your concerns is that we've propagated to the whole system something which is an "implementation detail": the way stats are stored and retrieved in order to provide the functionality. And you write "why should the top of the application care that I'm creating events as an architectural decision?".
Maybe we would benefit from a change of views here, as Geoff proposes. Writing logs is an important functionality of the application. It serves a user (the ops team, the developers). In that case, it's not a bad idea to make this kind of output explicit in the system. This might even have some influence on how we treat log statements as a first-class feature in a system.
Eric.
PS: new messages coming up to this thread as I write. I can't keep up,...
Re: Re: questioning FP
Runar Bjarnason wrote:
Lets take a more concrete example. I have a function fetchUser: String => IO[User]. What you're suggesting is that this method needs to be used at 'main', to get the User instance and pass it to the rest of my application ('noEffectsHere'), right? So I do all my IO upfront, and then call the rest of my code? It would be hard for me to arrange my code to work like this.
Re: Re: questioning FP
i don't think there is a clean solution.
you'll pollute the hierarchy because of that single println deep down there or you have to make a dirty cut at some point.
doing all sideeffect-stuff before or after all pure stuff is not always possible, and most likely very ugly for complex operations. i tend to just ignore side effects that "shouldn't matter much"
> -------- Original-Nachricht --------
> Datum: Tue, 11 Oct 2011 16:20:42 +0200
> Von: Ittay Dror
> An: Runar Bjarnason
> CC: scala-debate [at] googlegroups [dot] com
> Betreff: Re: [scala-debate] Re: questioning FP
>
> Runar Bjarnason wrote:
> >
> >
> > On Tue, Oct 11, 2011 at 9:55 AM, Ittay Dror
>
> > wrote:
> >
> > > Interesting. Maybe I missed something. I thought
> > > that when you use IO you need to propagate it up to 'main'
> > > (or 'run'). So one function that does IO (a println)
> > > pollutes your whole program. Where am I wrong?
> > >
> >
> > Right there.
> >
> > In practise, you compose effectful things in two ways:
> >
> > for {
> > x <- effectsHere
> > y <- effectsHereToo(x)
> > } yield noEffectsHere(y)
> >
>
> Lets take a more concrete example. I have a function fetchUser:
> String => IO[User]. What you're suggesting is that this method needs to
> be used at 'main', to get the User instance and pass it to the rest of
> my application ('noEffectsHere'), right? So I do all my IO upfront, and
> then call the rest of my code? It would be hard for me to arrange my
> code to work like this.
>
>
> >
> > or...
> >
> > (effectsHere |@| effectsHereToo)((a, b) => noEffectsHere(a, b))
> >
> > You'll find that most of your code is in "noEffectsHere". It
> > doesn't need to be aware that it can be lifted into IO. This is
> what
> > separation of concerns is all about.
> >
>
Re: Re: questioning FP
On Tuesday, October 11, 2011 10:20:42 AM UTC-4, Ittay Dror wrote:
No, that's not being suggested at all, and I'm not sure what gives you that idea. Consider:
fetchUser(s) map f
Both s and f can be pure and neither one of them needs to know anything at all about IO. The vast majority of code in real applications falls in this category. In fact, I'll bet that almost none of the code invoked by "fetchUser" has anything at all to do with IO either.
Re: Re: questioning FP
Runar Bjarnason wrote: Sure, then fetchUser is mostly non-IO, but needs to return IO[User]. Similarly, the functions that use fetchUser will also need to return IO[Something]. So a small implementation detail in a function can change its signature and forces it to be used differently (inside a for comprehension).
Re: Re: questioning FP
On Tuesday, October 11, 2011 11:26:43 AM UTC-4, Ittay Dror wrote:
It sounds like your apprehension is entirely syntactic. And the fact that a function requires IO is no small implementation detail. It's a very real dependency, a cause of nondeterminism, and a barrier to composition. But go ahead and use null instead of Option, exceptions instead of Either, mutable iterators instead of lists, and identity instead of IO. It's super dynamic and pragmatic.
Re: Re: questioning FP
If the compiler catches the bug, I have to fix it before I show the customer and get paid...If the bug only manifests itself in some low probability runtime condition, I may have already been paid and it's either someone else's problem or an opportunity to be paid again...
Clearly dynamic is very pragmatic. :-)
On Tuesday, October 11, 2011 12:02:59 PM UTC-4, Runar Bjarnason wrote:
Re: Re: questioning FP
Runar Bjarnason wrote: My points:
1. With Option and Either I have catamorphism, so using them in a function does not necessarily escape the function. And they avoid the alternative (meaning, if I use Option, i will never get an NPE)
2. Using the IO monad does not prevent non-determinism. The actions will run just the same (meaning if I use IO[_] i will still get FileNotFoundException). It only adds a tagging
3. The tagging is cumbersome to use.
Re: Re: questioning FP
On Tue, Oct 11, 2011 at 6:02 PM, Runar Bjarnason wrote:
>
>
> On Tuesday, October 11, 2011 11:26:43 AM UTC-4, Ittay Dror wrote:
>>
>> Sure, then fetchUser is mostly non-IO, but needs to return IO[User].
>> Similarly, the functions that use fetchUser will also need to return
>> IO[Something]. So a small implementation detail in a function can change its
>> signature and forces it to be used differently (inside a for comprehension).
>>
>
> It sounds like your apprehension is entirely syntactic. And the fact that a
> function requires IO is no small implementation detail. It's a very real
> dependency, a cause of nondeterminism, and a barrier to composition. But go
> ahead and use null instead of Option, exceptions instead of Either, mutable
> iterators instead of lists, and identity instead of IO. It's super dynamic
> and pragmatic.
>
Let's try to keep the sarcasm down on that list please.
Re: Re: questioning FP
martin odersky wrote: For the record, I wasn't offended by Runar's reply. On the contrary, I'm glad he keeps on replying to my answers.
RE: Re: questioning FP
Cool - It is the first time someone points out that this is like using null or exceptions… BUT, isn’t the point that this kind of IO[] wrapping creates the same kind of headaches that a checked exception does?
From: scala-debate [at] googlegroups [dot] com [mailto:scala-debate [at] googlegroups [dot] com] On Behalf Of Ittay Dror
Sent: October-11-11 12:10 PM
To: martin odersky
Cc: scala-debate [at] googlegroups [dot] com; Runar Bjarnason
Subject: Re: [scala-debate] Re: questioning FP
martin odersky wrote:
On Tue, Oct 11, 2011 at 6:02 PM, Runar Bjarnason runarorama [at] gmail [dot] com (<runarorama [at] gmail [dot] com>) wrote:For the record, I wasn't offended by Runar's reply. On the contrary, I'm glad he keeps on replying to my answers.
Re: Re: questioning FP
On Tue, Oct 11, 2011 at 9:44 PM, Razvan Cojocaru wrote:
> Cool - It is the first time someone points out that this is like using null
> or exceptions… BUT, isn’t the point that this kind of IO[] wrapping creates
> the same kind of headaches that a checked exception does?
Very good observation. That's why you want polymorphism.
Re: questioning FP
martin odersky skrev 2011-10-11 22:42:
> Very good observation. That's why you want polymorphism.
Can you give an example of the polymorphism you want that the IO monad
cannot provide?
/Jesper Nordenberg
Re: questioning FP
On Tue, Oct 11, 2011 at 11:32 PM, Jesper Nordenberg wrote:
> martin odersky skrev 2011-10-11 22:42:
>>
>> Very good observation. That's why you want polymorphism.
>
> Can you give an example of the polymorphism you want that the IO monad
> cannot provide?
>
xs map f
should be effectful if f is.
Re: questioning FP
martin odersky skrev 2011-10-11 23:38:
> On Tue, Oct 11, 2011 at 11:32 PM, Jesper Nordenberg wrote:
>> martin odersky skrev 2011-10-11 22:42:
>>>
>>> Very good observation. That's why you want polymorphism.
>>
>> Can you give an example of the polymorphism you want that the IO monad
>> cannot provide?
>>
> xs map f
>
> should be effectful if f is.
I'm not sure I agree with that. If f doesn't have side effects the map
operation can for example be performed in parallel. If f has side
effects the map operation must be performed sequentially.
So, the lack of side effects gives you less implementation restrictions.
/Jesper Nordenberg
Re: Re: questioning FP
thus, `map f` should be considered as having the same effect as `f` (assuming a "traditional" implementation of `map`, which actually executes `f`)
it seems to me it would be unsound not to have this
you seem to be saying map should be ad-hoc polymorphic in terms of the purity of f (in the sense of having overloads for different effects). I think that's reasonable, but it does not make the above untrue.
Re: questioning FP
Adriaan Moors epfl.ch> writes:
> if `f` prints to the console (or throws an exception, or formats your
> drive, or all of the above), so will `map`
>
> thus, `map f` should be considered as having the same effect as `f`
> (assuming a "traditional" implementation of `map`, which actually
> executes `f`)
>
> it seems to me it would be unsound not to have this
>
> you seem to be saying map should be ad-hoc polymorphic in terms of the
> purity of f (in the sense of having overloads for different effects).
> I think that's reasonable, but it does not make the above untrue.
I'm saying that a map operation with side effects is not the same as a map
operation without side effects. Consider the Haskell definitions:
mapM :: Monad m => (a -> m b) -> [a] -> m [b]
map :: (a -> b) -> [a] -> [b]
For mapM the reasonable choice is to perform the map operations in sequence,
but for map you can for example perform the operations in parallel (not very
useful for lists but consider the generic collection case) and there will be
no observable difference. The contracts of map and mapM are different so
they are not the same function.
/Jesper Nordenberg
Re: questioning FP
On Wednesday, October 12, 2011 4:31:16 AM UTC-4, Jesper Nordenberg wrote:
With the constraint that m is a Monad, yes. But the implication of Applicative is that combination of effects is associative.
From Data.Traversable in the Haskell base libraries:
Re: Re: questioning FP
On Wed, Oct 12, 2011 at 10:31 AM, Jesper Nordenberg <megagurka [at] yahoo [dot] com> wrote:
and I'm saying it doesn't have to be that way -- you can have different implementations if you want to (using ad-hoc polymorphism)but it shouldn't be imposed on you by the type system, so that you can have one parametrically polymorphic map (which abstracts over the effect, just like it abstracts over the element type)
as the scalaz folks have shown with traverse, this restriction is not inherent to monads/applicative functors; it stems from a restriction in Haskell's type system (you can't abstract over class contexts, so that you can't write one map to traverse them all -- i.e., this signature is not expressible in Haskell: map :: M m => (a -> m b) -> [a] -> m [b] -- where M is abstract, but this is exactly what traverse achieves)
I'm not sure -- is the following really that big a leap:from... abstracting over the builder in order to map Lists to Lists and Sets to Sets without implementing map all over for each combination to... using the same kind of abstractions to be able to plug in different evaluation strategies (much like we are doing with the parallel collections, in fact)
abstraction is about factoring out commonalities, and varying only what is truly different
Re: Re: questioning FP
On Wednesday, October 12, 2011 4:55:49 AM UTC-4, Adriaan Moors wrote:
Traverse is very specific that it requires m to have class Applicative.
In Haskell:
traverse :: Applicative f => (a -> f b) -> t a -> f (t b)
In Scala (as a method on T[A]):
def traverse[M[_]:Applicative, B](f: A => M[B]): M[T[B]]
Re: Re: questioning FP
On Wed, Oct 12, 2011 at 2:36 PM, Runar Bjarnason <runarorama [at] gmail [dot] com> wrote:
okay, so to get effect polymorphism, all you need to do is abstract over the Applicative bound...
def traverse[Effectful[E[x]], M[_]: Effectful, B](f: A => M[B]): M[T[B]]
it looks a bit complicated though ^,^