Re: kawa and apache tomcat

[Matthieu Vachon <matthieu.o.vachon@gmail.com>]

Hi Damien,

We've face a similar issue in the past and here a start of
explanation, anyone can correct me if they see any mistake.

You're running inside a servlet container which is responsible of
executing request reaching it. To do so, the container held a thread
pool so it can serve request.

When a request reach the container, it picks a thread from the thread
pool which is then used to execute the request. In this thread, at
some point, you are reaching your Kawa code. The kawa code stores a
few `ThreadLocal` variables which are saved in the thread given by the
servlet container. However, Kawa does not release these thread locals
when code exits his scope of action.

The request completes and the container take back the thread given to
you. I guess then that the container now sees that some thread local
were added to thread without being discarded before reclaiming the
thread and warn you about it.

Depending on what is stored in the thread local storage and if the
threads in the thread pool are renewed or not, memory leak can be
occur because of this. For example, the `gnu.mapping.Environment` set
some stuff but does not remove what it adds (as least I did not find
the call).

This is not a problem in a standard JVM application because once the
application closes, nothing needs to be done. But in a container
environment, that's another story since thread a kinda re-used over
multiple requests. On high load, this can create huge memory leaks
depending on the what is actually stored.

In our case, we are in a rather old version of Kawa. We patched our
version so that all thread locals are stored in a global structure. In
a servlet context listener, we then call a clean up method just before
finishing the request.

We also add to work around a shutdown hook register which was causing
some leaking problems in a servlet container application because it
was registered but never removed
(https://github.com/maoueh/kawa-fork/blob/b7e574c4edeaedaf657960801791534f27e6216c/gnu/kawa/io/WriterManager.java#L86).

Since you are on the latest version of Kawa, I guess fixing Kawa could
be feasible. The inverse of register scheme environment should
probably added so that it can be called once the actual request as
been processed to clean up any resources still held by Kawa.

Overall, maybe it there could be a single register scheme once the
container is loaded and again a single unregister once the container
is shutdown. I don't enough of the Kawa bootstrap process to known if
this would be easy and even feasible.

Hope this helped you a bit.

Regards,
Matt

On Fri, Jan 15, 2016 at 5:45 AM, Damien MATTEI <Damien.Mattei@unice.fr> wrote:
>
> hi,
>
> i 'm developping and testing a servlet application in tomcat 8 using some kawa code byte-compiled for java, so there is pure java code  and kawa code hosted on the apache tomcat web server,
> all works fine but i noticed this when stopping the application (named Sidonie) in tomcat manager, tomcat complains about this in the log files:
>
> 15-Jan-2016 11:08:49.115 SEVERE [http-nio-8080-exec-12] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [Sidonie] created a ThreadLocal with key of type [gnu.mapping.Environment.InheritedLocal] (value [gnu.mapping.Environment$InheritedLocal@3dc806b8]) and a value of type [gnu.mapping.InheritingEnvironment] (value [#<environment http-nio-8080-exec-19>]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
> 15-Jan-2016 11:08:49.116 SEVERE [http-nio-8080-exec-12] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [Sidonie] created a ThreadLocal with key of type [java.lang.InheritableThreadLocal] (value [java.lang.InheritableThreadLocal@7927031f]) and a value of type [kawa.standard.Scheme] (value [kawa.standard.Scheme@3070d094]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
> 15-Jan-2016 11:08:49.116 SEVERE [http-nio-8080-exec-12] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [Sidonie] created a ThreadLocal with key of type [gnu.mapping.ThreadLocation.ThreadLocalWithDefault] (value [gnu.mapping.ThreadLocation$ThreadLocalWithDefault@599f5a94]) and a value of type [gnu.math.IntNum] (value [10]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
> 15-Jan-2016 11:08:49.116 SEVERE [http-nio-8080-exec-12] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [Sidonie] created a ThreadLocal with key of type [gnu.mapping.ThreadLocation.ThreadLocalWithDefault] (value [gnu.mapping.ThreadLocation$ThreadLocalWithDefault@6b51425b]) and a value of type [gnu.kawa.io.BinaryOutPort] (value [gnu.kawa.io.BinaryOutPort@4cf02060]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
>
> this message appear the same number of times as the kawa code is called ,for example if i reload the web page which call again the kawa code i will  have two instance of this message in log ...
>
> for info my kawa code are called after a Scheme.registerEnvironment();
>
> tomcat seems cannot be able to free some kawa ressource, perheaps i should free them by myself in  the code but scheme and java have their own garbage collector and normally deallocate ressource when no more in use, i use files with diffrent method too, when i do an
> open ,i close the file at end but not always when using some method such as gnu.list.blob...
>
> any ideas?
>
> regards,
> damien
> --
> Damien.Mattei@unice.fr, Damien.Mattei@oca.eu, UNS / OCA / CNRS