That thread dump was captured during a hang, during the restart process. So starting to suspect it might be the unbounded GC threads, as we do not set any explicitly and there are multiple JVMs running on the same machine (11 cores). I will see if I can get the GC logs from our admin
Tomcat 9 instance is hanging every 3 days. So we are suspecting it might be the unbounded GC threads. Please advice!
How production Tomcat 9 instance is hanging every 3 days or so, requiring a restart. We are unsure of the exact cause but suspect NIO.
View(s)
21- 0Like(s)
- 0Dislike(s)
- 0Saved
0
Share(s)
Facebook
Twitter
Email
Copy Link
Ram Lakshmanan
Hello Brett!
Greetings.
In the thread dump attached, I don't see any major problem. It looks like you have captured the thread dump when application was healthy. Just one observation is: you have configured 69 Garbage Collection threads. Sometimes having too many threads can turn out to be counter productive of Garbage collection. However we can't confirm this is a problem, until we see the Garbage collection log.
Thread dump should have been captured during the problematic period, i.e. at the end of the 3rd day before restart the application. That too one snapshot of thread dump isn't enough, you might have to capture atleast 3 snapshots of thread dumps in a gap of 10 seconds.
Tomcat can hang because of several reasons: Long GC pause, memory leak, poor DB query, lack of threads, wrong connection pool configurations, device's capacity, network latency, inefficient code, ... Thread dump only gives small subset of information, to diagnose the problem. Inorder to diagnose tomcat hang, you need to capture 360-degree data such as: Thread dumps, Heap dumps, Garbage collection logs, vmstat, iostat, top, top -H, df, netstat... You can consider using 14-day trial of yCrash. It captures all of the above data, analyzes them and generates root cause analysis report of the slowdown.
Edit your Comment