Hello Elio!
Greetings. Upon reviewing your thread report, I could 72 threads in your application is stuck on the '
java.io.FileInputStream.readBytes()' method. If these many threads are stuck, it can cause unresponsiveness application. Here is the stacktrace of one of the stuck thread:
Thread-10298 THREAD ID : 17747 STATE : RUNNABLE stackTrace: at java.io.FileInputStream.readBytes(Native Method) at java.io.FileInputStream.read(Unknown Source) at com.xxxx.utils.Streams.copyStreamCount(Streams.java:64) at com.xxxx.utils.Streams.copyStream(Streams.java:48) at com.xxxx.utils.Streams$StreamCopyThread.run(Streams.java:115)
A thread will get stuck in the 'java.io.FileInputStream.readBytes()' method, when there is a disk I/O problem. I don't think DB CPU spikes is releated to it (unless you are running both the DB and application server on the same node/container).
I would recommend doing following as next steps:
a. Look for number of open file descriptors from your application & neighbouring applications that are running on the same machine and see whether they are getting closed properly after the usage.
b. Issue 'iostat' command and monitor the I/O statistics on the device
c. Issue 'dmesg' command and check whether any disk I/O errors are reported in the kernel logs
d. Look for disk, I/O related kernel parameters and see whether there is any anomoly.
You can do all of the above steps manually or you can use the open source yCrash script which will capture 360-degree application level artifacts (like GC logs, 3 snapshots of thread dumps, heap dumps) and system level artifacts (like top, top -H, netstat, vmstat, iostat, dmesg, diskusage, kernel parameters...). Once you have these data, either you can manually analyze them or upload it to yCrash tool, which will analyze all these artifacts and generate root cause analysis report. It has potential to indicate the root cause of the problem.


Edit your Comment