This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Can't get remote actors to work (server throws a MatchError)

6 replies
Jens Alfke
Joined: 2009-01-30,
User offline. Last seen 42 years 45 weeks ago.

[apologies if this is a duplicate; having some mail issues today...]

I'm a Scala newbie, attempting to try out the remote-actor
functionality. Unfortunately my own handmade test code fails to
connect, and the pingpong example posted by dirkmeister fails in
exactly the same way.
http://www.dirkmeister.de/2008/12/25/remote-actors-in-scala
However, Dirk kindly verified his example on the same configuration
I'm on (Scala 2.7.3, Mac OS X 10.5.6) and it still works for him.

The short description: The listener thread throws a MatchError as soon
as an incoming connection is made, causing it to shut down. I think
the exception is thrown from TcpService.scala:238, in
TcpServiceWorker.readNode.

Here's what I get when I run the 'pong' (listener) side, if I enable
logging:

$ scala -cp . de.dirkmeister.pingpong.RemotePongApp 8012
Info: Thread[Thread-0,5,main]: corePoolSize = 4, maxPoolSize = 256
Info: Thread[Thread-6,5,main]: waiting for new connection...
Info: created service at Node(172.18.111.57,8012)
Info: Started new Thread[Thread-7,5,main]
Info: Thread[Thread-6,5,main]: caught scala.MatchError:
Node(172.18.111.57,8011)
Info: Thread[Thread-6,5,main]: shutting down...

The first three lines above are printed immediately, the rest after I
start the 'ping' app in another terminal window:

$ scala -cp . de.dirkmeister.pingpong.RemotePingApp 8011 localhost 8012
Info: Thread[Thread-0,5,main]: corePoolSize = 4, maxPoolSize = 256
Info: Thread[Thread-6,5,main]: waiting for new connection...
Info: created service at Node(172.18.111.57,8011)
Info: Thread[Thread-7,5,main]: transmitting data...
Info: Thread[Thread-7,5,main]: transmitting data...

As I said, the logs from the 'pong' app imply the exception is thrown
from the 'match' statement TcpServiceWorker.readNode. But that
statement expects a Node object, and according to the message in the
exception, the object that failed to match _is_ a Node object as
expected. So is something screwy going on with Java serialization such
that there are two incompatible Node classes? Or is it a subtlety of
Scala pattern matching that I don't know about yet?

Any help would be gratefully appreciated...

—Jens

PS: As I said above, my configuration is Scala 2.7.3, Java SE
1.5.0_16, Mac OS X 10.5.6, MacBook Pro.

Jens Alfke
Joined: 2009-01-30,
User offline. Last seen 42 years 45 weeks ago.
Re: Can't get remote actors to work (server throws a MatchError

To help track this down, I extracted the scala.actor.remote source
files from my scala 2.7.3 distribution, changed the package name to
'scala.actor.remote_snej', changed the app code to import from this
new package, and compiled everything together. That way I can modify
the classes to help track down the problem. The first thing I did was
to make some of the 'catch' handlers print exception backtraces.

The weird thing is that the app now fails slightly differently, even
though everything should be exactly the same (apart from package
names.) But as before, the exception message implies that there are
two incompatible classes with the same name:
> cannot assign instance of scala.Symbol to field
> scala.actors.remote_snej.Locator.name of type scala.Symbol
...which is nonsensical unless there are two different classes named
scala.Symbol. (Full dump is below.)

This problem, like the one I got yesterday using the regular remote
actor package, seems to stem from the Java object de-serialization.
It's been years since I did any serious Java work, but IIRC this can
occur if the same class is loaded through two different ClassLoaders.
But I have no idea why that would be happening…

Which JVM versions is Scala 2.7.3 compatible with? I've got Java SE
1.5.0_16, the default installation that comes with OS X 10.5.6.
Both sides of the connection are running as the same user on the same
machine, so there's no possibility of them loading different software.

—Jens

PS: Here's the full output from running the RemotePongApp:

Info: Thread[Thread-0,5,main]: corePoolSize = 4, maxPoolSize = 256
Info: Thread[Thread-6,5,main]: waiting for new connection...
Info: created service at Node(10.0.1.105,9000)
Info: Started new Thread[Thread-7,5,main]
Info: Thread[Thread-6,5,main]: waiting for new connection...
Info: Thread[Thread-7,5,main]: caught java.lang.ClassCastException:
cannot assign instance of scala.Symbol to field
scala.actors.remote_snej.Locator.name of type scala.Symbol in instance
of scala.actors.remote_snej.Locator
at java.io.ObjectStreamClass
$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2004)
at java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:
1184)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:
1914)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:
1832)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:
1719)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1305)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:
1908)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:
1832)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:
1719)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1305)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348)
at
scala
.actors.remote_snej.JavaSerializer.deserialize(JavaSerializer.scala:52)
at scala.actors.remote_snej.Serializer.readObject(Serializer.scala:41)
at scala.actors.remote_snej.TcpServiceWorker.run(TcpService.scala:265)
Info: Thread[Thread-7,5,main]: terminated

Jens Alfke
Joined: 2009-01-30,
User offline. Last seen 42 years 45 weeks ago.
Re: Can't get remote actors to work (server throws a MatchError

On Jan 30, 2009, at 11:02 AM, I wrote:

> This problem, like the one I got yesterday using the regular remote
> actor package, seems to stem from the Java object de-serialization.
> It's been years since I did any serious Java work, but IIRC this can
> occur if the same class is loaded through two different ClassLoaders.

Looks like I was right. I've tracked the problem down to the
CustomObjectInputStream class (in JavaSerializer.scala). If I turn the
class into a no-op by commenting out its resolveClass override, the
pingpong sample runs correctly.

The purpose of this class seems to be to give a different Java
ClassLoader first crack at loading classes while unserializing objects
received from the remote Actor. By instrumenting the code, I found
that the only class that gets loaded specially is ... scala.Symbol,
the one that causes the failure. If I check the getClassLoader
property of the various classes being resolved by this method,
scala.Symbol ends up with a different one (a sun.misc.Launcher
$AppClassLoader) than all the others (a java.net.URLClassLoader).

Where did this AppClassLoader come from? it's the system class loader
as returned by ClassLoader.getSystemClassLoader (called at
RemoteActor.scala:50). I have no idea why this isn't the same as the
ClassLoader used to load the Scala classes.

As the docs say, the initial setting of the ClassLoader chain is
platform-dependent. It appears there's something about my platform or
configuration that's incompatible with whatever it is this custom
resolveClass method is trying to do. So is this actually a bug /
compatibility problem in the Scala remote-actors library itself?

—Jens

PS: Someone else feel free to chime in, especially if you understand
the remoteactor package, or ClassLoaders, better than I do. I'm not
intentionally talking to myself here! :)

Meredith Gregory
Joined: 2008-12-17,
User offline. Last seen 42 years 45 weeks ago.
Re: Can't get remote actors to work (server throws a MatchErro
Jens,

Excellent sleuthing!

i've not run into this problem with Scala actors on my platform, but ran into it in spades using CAL as they were using class loaders to resolve all kinds of resources other than classes.

Did you file a ticket? i'm particularly interested in how the Scala folks respond to this issue as i'm in the process of making some recommendations and need good anecdotes regarding responsiveness.

Best wishes,

--greg

On Fri, Jan 30, 2009 at 12:21 PM, Jens Alfke <jens [at] mooseyard [dot] com> wrote:

On Jan 30, 2009, at 11:02 AM, I wrote:

This problem, like the one I got yesterday using the regular remote actor package, seems to stem from the Java object de-serialization. It's been years since I did any serious Java work, but IIRC this can occur if the same class is loaded through two different ClassLoaders.

Looks like I was right. I've tracked the problem down to the CustomObjectInputStream class (in JavaSerializer.scala). If I turn the class into a no-op by commenting out its resolveClass override, the pingpong sample runs correctly.

The purpose of this class seems to be to give a different Java ClassLoader first crack at loading classes while unserializing objects received from the remote Actor. By instrumenting the code, I found that the only class that gets loaded specially is ... scala.Symbol, the one that causes the failure. If I check the getClassLoader property of the various classes being resolved by this method, scala.Symbol ends up with a different one (a sun.misc.Launcher$AppClassLoader) than all the others (a java.net.URLClassLoader).

Where did this AppClassLoader come from? it's the system class loader as returned by ClassLoader.getSystemClassLoader (called at RemoteActor.scala:50). I have no idea why this isn't the same as the ClassLoader used to load the Scala classes.

As the docs say, the initial setting of the ClassLoader chain is platform-dependent. It appears there's something about my platform or configuration that's incompatible with whatever it is this custom resolveClass method is trying to do. So is this actually a bug / compatibility problem in the Scala remote-actors library itself?

—Jens

PS: Someone else feel free to chime in, especially if you understand the remoteactor package, or ClassLoaders, better than I do. I'm not intentionally talking to myself here! :)



--
L.G. Meredith
Managing Partner
Biosimilarity LLC
806 55th St NE
Seattle, WA 98105

+1 206.650.3740

http://biosimilarity.blogspot.com
Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: Can't get remote actors to work (server throws a MatchErro
Hopefully it isn't wait until java 7 and the classloader redesign....
It definitely sounds like a bug in how scala is wiring up classloaders (or maybe it expects you to place your classes elsewhere?)

Sent from my iPhone
On Jan 30, 2009, at 3:36 PM, Meredith Gregory <lgreg [dot] meredith [at] gmail [dot] com> wrote:

Jens,

Excellent sleuthing!

i've not run into this problem with Scala actors on my platform, but ran into it in spades using CAL as they were using class loaders to resolve all kinds of resources other than classes.

Did you file a ticket? i'm particularly interested in how the Scala folks respond to this issue as i'm in the process of making some recommendations and need good anecdotes regarding responsiveness.

Best wishes,

--greg

On Fri, Jan 30, 2009 at 12:21 PM, Jens Alfke <jens [at] mooseyard [dot] com (jens [at] mooseyard [dot] com" rel="nofollow">jens [at] mooseyard [dot] com)> wrote:

On Jan 30, 2009, at 11:02 AM, I wrote:

This problem, like the one I got yesterday using the regular remote actor package, seems to stem from the Java object de-serialization. It's been years since I did any serious Java work, but IIRC this can occur if the same class is loaded through two different ClassLoaders.

Looks like I was right. I've tracked the problem down to the CustomObjectInputStream class (in JavaSerializer.scala). If I turn the class into a no-op by commenting out its resolveClass override, the pingpong sample runs correctly.

The purpose of this class seems to be to give a different Java ClassLoader first crack at loading classes while unserializing objects received from the remote Actor. By instrumenting the code, I found that the only class that gets loaded specially is ... scala.Symbol, the one that causes the failure. If I check the getClassLoader property of the various classes being resolved by this method, scala.Symbol ends up with a different one (a sun.misc.Launcher$AppClassLoader) than all the others (a java.net.URLClassLoader).

Where did this AppClassLoader come from? it's the system class loader as returned by ClassLoader.getSystemClassLoader (called at RemoteActor.scala:50). I have no idea why this isn't the same as the ClassLoader used to load the Scala classes.

As the docs say, the initial setting of the ClassLoader chain is platform-dependent. It appears there's something about my platform or configuration that's incompatible with whatever it is this custom resolveClass method is trying to do. So is this actually a bug / compatibility problem in the Scala remote-actors library itself?

—Jens

PS: Someone else feel free to chime in, especially if you understand the remoteactor package, or ClassLoaders, better than I do. I'm not intentionally talking to myself here! :)



--
L.G. Meredith
Managing Partner
Biosimilarity LLC
806 55th St NE
Seattle, WA 98105

+1 206.650.3740

http://biosimilarity.blogspot.com
extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Can't get remote actors to work (server throws a MatchError

On Thu, Jan 29, 2009 at 03:05:41PM -0800, Jens Alfke wrote:
> So is something screwy going on with Java serialization such that there are two incompatible Node classes?

By all appearances, yes. The first failures I run into are in processMsg in NetKernel, where cases that should be
guaranteed to match, don't:

Jens Alfke
Joined: 2009-01-30,
User offline. Last seen 42 years 45 weeks ago.
Re: Can't get remote actors to work (server throws a MatchErro

On Jan 30, 2009, at 12:36 PM, Meredith Gregory wrote:
Did you file a ticket? i'm particularly interested in how the Scala folks respond to this issue as i'm in the process of making some recommendations and need good anecdotes regarding responsiveness.

FYI, I've just filed ticket #1686
In searching through Trac, I found that there was a prior ticket reporting similar problems, but which was closed by the originator as being a misunderstanding (though with no explanation given.) Also, a recently checked-in RemoteActor example sets the classLoader explicitly at initialization time, implying that the author was aware of this issue.
—Jens

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland