Profile Image
pandiashok99

therads are taking high cpu utilization

My application going to dead state, most of the threads are taking high cpu utilization. There is no issues from the os and DB end.
Requesting your help to get resolved.


Report URL - https://fastthread.io/my-thread-report.jsp?p=c2hhcmVkLzIwMjQvMDIvMjkvdGhyZWFkdW1wLTI0LnppcC0tNS0yNS04

    Please Sign In or to post your comment or answer

    Profile Image

    jzwadowski

    Would you join source code of  com.ccavenue.mars.api.utility.AesCryptUtil class?
    It's probable performance botlleneck.

    Profile Image

    Ram Lakshmanan

    Hello Pandia Ashok!

     

     I totally agree with jzwadoski's assessment. Your CPU performance bottleneck is origniating from the '

    com.ccavenue.mars.api.utility.AesCryptUtil.byteToHex()'. I could see various threads stuck on this method and consuming huge amount of CPU.  Below is the excerpt from one of your report, pointing out high CPU consuming threads.

     

     

     Most of these threads has the following stacktrace:

     

    java.lang.Thread.State: RUNNABLE
at java.util.Arrays.copyOfRange(Arrays.java:3664)
at java.lang.String.<init>(String.java:207)
at java.lang.StringBuilder.toString(StringBuilder.java:407)
at com.ccavenue.mars.api.utility.AesCryptUtil.byteToHex(AesCryptUtil.java:153)
at com.ccavenue.mars.api.utility.AesCryptUtil.encrypt(AesCryptUtil.java:90)
at com.ccavenue.mars.api.DoAPICall.doPost(DoAPICall.java:60)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:523)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at com.ccavenue.mars.api.utility.SecurityFilter.doFilter(SecurityFilter.java:50)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68)
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117)
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68)
at io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:275)
at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:79)
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:134)
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:131)
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction$$Lambda$993/604352767.call(Unknown Source)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1551)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction$$Lambda$994/728640875.call(Unknown Source)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1551)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction$$Lambda$994/728640875.call(Unknown Source)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1551)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction$$Lambda$994/728640875.call(Unknown Source)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1551)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction$$Lambda$994/728640875.call(Unknown Source)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:255)
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:79)
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:100)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:387)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:852)
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1280)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- None

     

    It looks like 'com.ccavenue.mars.api.utility.AesCryptUtil' is originating from CCAvenue SDK. Are you running on the latest version of this SDK?

     

    Sometimes if be could that your application is processing more traffic and it needs more compute power. Thus adding new servers (or containers) to give additional capacity can also resolve the problem.

     

     

    Profile Image

    pandiashok99

    Thank you for your promt responce, @jzwadowski and  @Ram Lakshmanan.


    I am very new to this threads concept,  I am expecting more explanation from you, to get further help from you.
    I am using jboss EAP 7.4 in my organization so we have monitoring tools like grafana and for logs quartzdesk.

     

    Would you join source code of  com.ccavenue.mars.api.utility.AesCryptUtil class?

    yes, AesCryptUtil.class file is in my application(war).

    It looks like 'com.ccavenue.mars.api.utility.AesCryptUtil' is originating from CCAvenue SDK. Are you running on the latest version of this SDK?

    Sorry, Ram lakshamanan, I not understand your query, requesting you to elaborate more.

     

    And for this application we have 5 standalone servers I have.
    Kindly, help me to understand and resolve the present issues?

     

     

     

     

    Got something else on mind? Post Your Question

    Edit your question

    Separate with comma, space or ENTER key. Special characters are not allowed in tag.

    Edit your Answer

    Edit your Comment