This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Make the scala Interpreter use a Java application's classloader?

7 replies
Johann Petrak
Joined: 2010-04-09,
User offline. Last seen 16 weeks 3 hours ago.

I am trying to understand how one could use the Scala interpreter
for scripting inside a Java application that has its own plugin
mechanism, so that the Interpreter can access and make use of the
plugin classes.
The Java application has its own class loader which is used to
access plugin classes and new classes might get accessible
at any time through that class loader.
So ... is there a way to make the Interpreter use one's own
classloader as the parent of the classloader it uses?
Or is there some other way to achieve the same result?

Michael
Joined: 2009-02-23,
User offline. Last seen 20 weeks 2 days ago.
Re: Make the scala Interpreter use a Java application's classloa

Johann Petrak writes:

>
> I am trying to understand how one could use the Scala interpreter
> for scripting inside a Java application that has its own plugin
> mechanism, so that the Interpreter can access and make use of the
> plugin classes.

Johann,

The problem here is that the Interpreter does on the fly compilation. That is it
will compile the script source to class files and then execute them. For this
compilation step the interpreter needs access to the actual class files. That
is, read the binary representation of the class files into memory instead of
loading them into the JVM via a class loader. So in order to make the
interpreter work, you need to make the class files exposed by your class loader
accessible to the compiler through a file system abstraction (i.e.
scala.tools.nsc.io.AbstractFile).

I have done this for OSGi. The scenario is very similar: OSGi provides its own
class loader for each bundle. I implemented AbstractFile on top of OSGi bundles.
See
http://svn.apache.org/viewvc/sling/trunk/contrib/scripting/scala/script/...

Johann Petrak
Joined: 2010-04-09,
User offline. Last seen 16 weeks 3 hours ago.
Re: Re: Make the scala Interpreter use a Java application's cla

Hi Micahel,

thank you for this information. I am not sure I understand everything
fully though :)

Unfortunately I am still more an apprentice with both Java and Scala
so please forgive me if my additional questions show a lack of
understanding of your first answer.

So, my original idea of just letting the Interpreter somehow use
the application's classloader does not work, even when we want to
just access Java (and not Scala) classes through it?

The problem is that the application is to be seen as a black box,
i.e. I cannot change the application's plugin loading mechanism.
(The loading mechanism is essentially the same as the for
the default classloader, with JARs getting added from the file
system.)

So, am I out of luck? Is there some other way e.g. in the interactive
Interpreter session to somehow "tell" the Interpreter which directories
or JAR files to add to its classpath?
I could live with that solution: instead of the interpreter
automatically getting access to each plugin loaded by the application,
the user has to carry out some action to make those plugin JARs
available to the Interpreter (after the Interpreter session has started)
too. Would that be possible? And if yes, how?

Cheers,
Johann

On 2010-10-27 10:09, Michael wrote:
> Johann Petrak writes:
>
>>
>> I am trying to understand how one could use the Scala interpreter
>> for scripting inside a Java application that has its own plugin
>> mechanism, so that the Interpreter can access and make use of the
>> plugin classes.
>
> Johann,
>
> The problem here is that the Interpreter does on the fly compilation. That is it
> will compile the script source to class files and then execute them. For this
> compilation step the interpreter needs access to the actual class files. That
> is, read the binary representation of the class files into memory instead of
> loading them into the JVM via a class loader. So in order to make the
> interpreter work, you need to make the class files exposed by your class loader
> accessible to the compiler through a file system abstraction (i.e.
> scala.tools.nsc.io.AbstractFile).
>
> I have done this for OSGi. The scenario is very similar: OSGi provides its own
> class loader for each bundle. I implemented AbstractFile on top of OSGi bundles.
> See
> http://svn.apache.org/viewvc/sling/trunk/contrib/scripting/scala/script/...
>
>
>

Razvan Cojocaru 3
Joined: 2010-07-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Make the scala Interpreter use a Java application's cla

1. you can have the java interpreter use the current java classpath, just
set useJavaCp = true

val env = new nsc.Settings(errLogger)
env.usejavacp.value = true
val p = new Interpreter(env)
p.setContextClassLoader

2. you can use custom classpath by using the boot or app properties, see the
Mark's proposal at http://gist.github.com/404272 which was implemented in
2.8.1...something

cheers,
Razvan

-----Original Message-----
From: Johann Petrak
Sent: Wednesday, October 27, 2010 6:26 AM
To: Michael
Cc: scala-user [at] listes [dot] epfl [dot] ch
Subject: Re: [scala-user] Re: Make the scala Interpreter use a Java
application's classloader?

Hi Micahel,

thank you for this information. I am not sure I understand everything
fully though :)

Unfortunately I am still more an apprentice with both Java and Scala
so please forgive me if my additional questions show a lack of
understanding of your first answer.

So, my original idea of just letting the Interpreter somehow use
the application's classloader does not work, even when we want to
just access Java (and not Scala) classes through it?

The problem is that the application is to be seen as a black box,
i.e. I cannot change the application's plugin loading mechanism.
(The loading mechanism is essentially the same as the for
the default classloader, with JARs getting added from the file
system.)

So, am I out of luck? Is there some other way e.g. in the interactive
Interpreter session to somehow "tell" the Interpreter which directories
or JAR files to add to its classpath?
I could live with that solution: instead of the interpreter
automatically getting access to each plugin loaded by the application,
the user has to carry out some action to make those plugin JARs
available to the Interpreter (after the Interpreter session has started)
too. Would that be possible? And if yes, how?

Cheers,
Johann

On 2010-10-27 10:09, Michael wrote:
> Johann Petrak writes:
>
>>
>> I am trying to understand how one could use the Scala interpreter
>> for scripting inside a Java application that has its own plugin
>> mechanism, so that the Interpreter can access and make use of the
>> plugin classes.
>
> Johann,
>
> The problem here is that the Interpreter does on the fly compilation. That
> is it
> will compile the script source to class files and then execute them. For
> this
> compilation step the interpreter needs access to the actual class files.
> That
> is, read the binary representation of the class files into memory instead
> of
> loading them into the JVM via a class loader. So in order to make the
> interpreter work, you need to make the class files exposed by your class
> loader
> accessible to the compiler through a file system abstraction (i.e.
> scala.tools.nsc.io.AbstractFile).
>
> I have done this for OSGi. The scenario is very similar: OSGi provides its
> own
> class loader for each bundle. I implemented AbstractFile on top of OSGi
> bundles.
> See
> http://svn.apache.org/viewvc/sling/trunk/contrib/scripting/scala/script/...
>
>
>

Michael
Joined: 2009-02-23,
User offline. Last seen 20 weeks 2 days ago.
Re: Make the scala Interpreter use a Java application's classloa

Razvan Cojocaru writes:

> val env = new nsc.Settings(errLogger)
> env.usejavacp.value = true
> val p = new Interpreter(env)
> p.setContextClassLoader
>

Alternatively you can set the class path throuh nsc.Settings.classpath.

Johannes Rudolph 2
Joined: 2010-02-12,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Make the scala Interpreter use a Java application's cla

To add a little summary here: The basic problem here is, that
ClassLoaders are generally opaque. So, you can neither enumerate
classes inside nor access the actual byte data of classes exposed
through a ClassLoader. This means for Scala:
* The current usejavacp mechanism does not work on ClassLoaders (that
should have become clear in this thread) but uses the
"java.class.path" system property to extract the classpath as
specified on the command line.
* The proposal (implementation?) from Mark introduces another
mechanism which makes it at least possible to specify a classpath *per
ClassLoader* by putting a well-known file (a resource) inside an
element of the classpath which is accessible through the particular
ClassLoader.

I want to add another method which may work in some cases: Since many
ClassLoader's are actually URLClassLoaders you can extract their URLs
by casting to URLClassLoader and use getURLs to get to the URLs and
then use new File(url.getURI) to try to access the actual files (or
cache them somewhere if they are directly not accessible as a file).
Then put all those recursively detected file paths on the classpath
for the interpreter.

I've actually used this technique in a JBoss application where I added
another special case for the RepositoryClassLoader to extract the
classpath for the current EAR deployment.

On Thu, Oct 28, 2010 at 10:28 AM, Michael wrote:
> Razvan Cojocaru writes:
>
>> val env = new nsc.Settings(errLogger)
>> env.usejavacp.value = true
>> val p = new Interpreter(env)
>> p.setContextClassLoader
>>
>
> Alternatively you can set the class path throuh nsc.Settings.classpath.
>
>

Michael
Joined: 2009-02-23,
User offline. Last seen 20 weeks 2 days ago.
Re: Make the scala Interpreter use a Java application's classloa

> I want to add another method which may work in some cases: Since many
> ClassLoader's are actually URLClassLoaders you can extract their URLs
> by casting to URLClassLoader and use getURLs to get to the URLs and
> then use new File(url.getURI) to try to access the actual files (or
> cache them somewhere if they are directly not accessible as a file).
> Then put all those recursively detected file paths on the classpath
> for the interpreter.

Instead of down casting the class loader it is usually also possible to read the
byte data of classes from the class loader using the getResource(s) methods. In
this way you don't depend on a specific type of the class loader (i.e.
URLClassLoader). AFAIK this method is used by Jasper and others. This is also
mostly the method I implemented for the Scala scripting engine for Apache Sling.
However this approach needs more work since you can't just extend the class
path. Rather you have to implement AbstractFile on top of the class loader and
pass this to the Scala compiler in addition to the rest of the class path.

Michael

Razvan Cojocaru 3
Joined: 2010-07-28,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: Make the scala Interpreter use a Java application's cla

Interesting. I assumed the usejavacp will instruct it to use whatever
is the current classloader. I also assumed the plugin loading thing
will modify the current classloader.

Sorry for the confusion. So I guess you have to remember the jars
and/or bundles you loaded for the plugins and pass them via the
"app.classpath"?

On 10/28/10, Johannes Rudolph wrote:
> To add a little summary here: The basic problem here is, that
> ClassLoaders are generally opaque. So, you can neither enumerate
> classes inside nor access the actual byte data of classes exposed
> through a ClassLoader. This means for Scala:
> * The current usejavacp mechanism does not work on ClassLoaders (that
> should have become clear in this thread) but uses the
> "java.class.path" system property to extract the classpath as
> specified on the command line.
> * The proposal (implementation?) from Mark introduces another
> mechanism which makes it at least possible to specify a classpath *per
> ClassLoader* by putting a well-known file (a resource) inside an
> element of the classpath which is accessible through the particular
> ClassLoader.
>
> I want to add another method which may work in some cases: Since many
> ClassLoader's are actually URLClassLoaders you can extract their URLs
> by casting to URLClassLoader and use getURLs to get to the URLs and
> then use new File(url.getURI) to try to access the actual files (or
> cache them somewhere if they are directly not accessible as a file).
> Then put all those recursively detected file paths on the classpath
> for the interpreter.
>
> I've actually used this technique in a JBoss application where I added
> another special case for the RepositoryClassLoader to extract the
> classpath for the current EAR deployment.
>
> On Thu, Oct 28, 2010 at 10:28 AM, Michael wrote:
>> Razvan Cojocaru writes:
>>
>>> val env = new nsc.Settings(errLogger)
>>> env.usejavacp.value = true
>>> val p = new Interpreter(env)
>>> p.setContextClassLoader
>>>
>>
>> Alternatively you can set the class path throuh nsc.Settings.classpath.
>>
>>
>
>
>
> --
> Johannes
>
> -----------------------------------------------
> Johannes Rudolph
> http://virtual-void.net
>

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland