Profile Image
zoey.moonfish

What is blocking qtp threads?

JVM "hangs" for about 15 mins before starting the execution.
At the beginning, it seems to be related to the nofiles, which threads showed as 4096, but at OS they are set to unlimited. Not familiar with AIX.
Then, this also looks like an antivirus is causing the wait, but there are no AV active on the server.
Little lost on what else is causing the delay.


Report URL - https://fastthread.io/my-thread-report.jsp?p=c2hhcmVkLzIwMjQvMDUvOC9BcmNoaXZlIDMuemlwLS0xNy0xNS0yNg==

    Please Sign In or to post your comment or answer

    Profile Image

    Kousika M

    Hello Zoey,

     

    Greetings!

     

    The 'qtp-1291319943-80-acceptor-1@8ddcbb12-ServerConnector@61980f36{HTTP/1.1, (http/1.1)}{172.16.9.170:20914}' thread is stuck on accept0() method in sun.nio.ch.ServerSocketChannelImpl file. Before getting stuck, this thread obtained 1 lock (java/lang/Object lock) and never released it. Due to that 1 thread is BLOCKED as shown in the report. If threads are BLOCKED for a prolonged period, your application can become unresponsive. Please find the below screenshot for reference.

     

     

     

    You can examine 'qtp-1291319943-80-acceptor-1@8ddcbb12-ServerConnector@61980f36{HTTP/1.1, (http/1.1)}{172.16.9.170:20914}' stack trace to see why it is BLOCKED.

    Profile Image

    Ram Lakshmanan

    Hello Zoey!

     

     I reviewed your thread dump. I couldn't notice any noticeable bottlenecks that could cause the JVM hang for 15 minutes. There could be several reasons why application is going into this hang state. Some of the reasons are:

    • Garbage collection pauses
    • Threads getting BLOCKED
    • Network connectivity
    • Load balancer routing issue
    • Heavy CPU consumption of threads
    • Operating System running with old patches
    • Memory Leak
    • DB not responding properly
    • Kernel issues
    • Backend slow downs
    • Hypervisor/container orchestrator not allocating enough resources

    :

    :

     

     So just thread dump is not enough to diagnose the problem.                                                         

     

     You can use the open source yCrash script which will capture 360-degree application level artifacts (like GC logs, 3 snapshots of thread dumps, heap dumps) and system level artifacts (like top, top -H, netstat, vmstat, iostat, dmesg, diskusage, kernel parameters...). Once you have these data, either you can manually analyze them or upload it to yCrash tool, which will analyze all these artifacts and generate root cause analysis report. It has potential to indicate the root cause of the problem.

    Got something else on mind? Post Your Question