Tomcat停止前等待某个函数运行结束的问题

Question

在编写java ee应用的时候，在一个Servlet里的的初始化（init）里启动了一个多线程（用netty写的进行socket通信的），在Servlet里的销毁（destory）里手动显示的关闭netty的线程，但是却报如下错误，请问该怎么解决？

09-Jan-2017 14:24:07.384 严重 [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [electric] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@637604d1]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.util.internal.InternalThreadLocalMap@6c896b26]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
09-Jan-2017 14:24:07.393 信息 [main] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["http-nio-8080"]
09-Jan-2017 14:24:07.393 信息 [main] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["ajp-nio-8009"]
09-Jan-2017 14:24:07.393 信息 [main] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler ["http-nio-8080"]
09-Jan-2017 14:24:07.394 信息 [main] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler ["ajp-nio-8009"]
09-Jan-2017 14:24:09.468 信息 [nioEventLoopGroup-3-1] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [io.netty.buffer.PoolArena$1]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
 java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [io.netty.buffer.PoolArena$1]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
    at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1295)
    at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1283)
    at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1148)
    at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1109)
    at io.netty.buffer.PoolArena.freeChunk(PoolArena.java:279)
    at io.netty.buffer.PoolThreadCache$MemoryRegionCache.freeEntry(PoolThreadCache.java:456)
    at io.netty.buffer.PoolThreadCache$MemoryRegionCache.free(PoolThreadCache.java:426)
    at io.netty.buffer.PoolThreadCache$MemoryRegionCache.free(PoolThreadCache.java:418)
    at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:275)
    at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:266)
    at io.netty.buffer.PoolThreadCache.free0(PoolThreadCache.java:239)
    at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:235)
    at io.netty.buffer.PooledByteBufAllocator$PoolThreadLocalCache.onRemoval(PooledByteBufAllocator.java:359)
    at io.netty.buffer.PooledByteBufAllocator$PoolThreadLocalCache.onRemoval(PooledByteBufAllocator.java:345)
    at io.netty.util.concurrent.FastThreadLocal.remove(FastThreadLocal.java:226)
    at io.netty.util.concurrent.FastThreadLocal.removeAll(FastThreadLocal.java:67)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:146)
    at java.lang.Thread.run(Thread.java:745)

Exception in thread "nioEventLoopGroup-3-1" java.lang.NoClassDefFoundError: io/netty/buffer/PoolArena$1
    at io.netty.buffer.PoolArena.freeChunk(PoolArena.java:279)
    at io.netty.buffer.PoolThreadCache$MemoryRegionCache.freeEntry(PoolThreadCache.java:456)
    at io.netty.buffer.PoolThreadCache$MemoryRegionCache.free(PoolThreadCache.java:426)
    at io.netty.buffer.PoolThreadCache$MemoryRegionCache.free(PoolThreadCache.java:418)
    at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:275)
    at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:266)
    at io.netty.buffer.PoolThreadCache.free0(PoolThreadCache.java:239)
    at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:235)
    at io.netty.buffer.PooledByteBufAllocator$PoolThreadLocalCache.onRemoval(PooledByteBufAllocator.java:359)
    at io.netty.buffer.PooledByteBufAllocator$PoolThreadLocalCache.onRemoval(PooledByteBufAllocator.java:345)
    at io.netty.util.concurrent.FastThreadLocal.remove(FastThreadLocal.java:226)
    at io.netty.util.concurrent.FastThreadLocal.removeAll(FastThreadLocal.java:67)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:146)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: Illegal access: this web application instance has been stopped already. Could not load [io.netty.buffer.PoolArena$1]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
    at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1285)
    at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1148)
    at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1109)
    ... 14 more
Caused by: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [io.netty.buffer.PoolArena$1]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
    at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1295)
    at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1283)
    ... 16 more
Disconnected from server

tomcat 版本是 tomcat9=m14,netty用的netty-all 4.1.5.
请问该如何解决呢？

Answer 1

其实这个问题可以忽略的，我遇到过这个问题。当时找到的答案有：

1. https://github.com/relayrides...

2. https://github.com/netty/nett...

我的是Spring程序，所注册了一个SmartLifecycle，在程序stop的时候调用：

try {

  GlobalEventExecutor.INSTANCE.awaitInactivity(5L, TimeUnit.SECONDS);
  LOGGER.info("Close {}", GlobalEventExecutor.class.getName());

} catch (Exception e) {

  LOGGER.warn("Error happened when close {}", GlobalEventExecutor.class.getName());
  LOGGER.warn(ExceptionUtils.getStackTrace(e));

}

try {

  ThreadDeathWatcher.awaitInactivity(5L, TimeUnit.SECONDS);
  LOGGER.info("Close {}", ThreadDeathWatcher.class.getName());

} catch (Exception e) {

  LOGGER.warn("Error happened when close {}", ThreadDeathWatcher.class.getName());
  LOGGER.warn(ExceptionUtils.getStackTrace(e));

}

如果你是Servlet的话，应该也有类似的Listener，在Listener里干这件事情就行了。

Answer 2

非法访问看一下你的tomcat的权限是否打开

Answer 3

问题产生的原因是这样的，因为netty是异步关闭的，所以tomcat的servlet关闭了之后,netty其实还没有关闭，所以就爆出这个错误。只要把netty设置为同步关闭就可以了。但是现在同步关闭的时候还是会有一个警告：

严重 [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [electric] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@6c9a20a]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.util.internal.InternalThreadLocalMap@59404c7b]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

不知该从哪里入手。。。