Profile Image
Ajay Pandey

JBoss freezing during peak hours

Hi Team,

Our application is deployed over jboss-eap7.1, jboss stops responding during peak hours. There is no out of memory error in Jboss. Heap allocation is 9GB, it goes max to 7 GB.

Collected thread dumps several times but it always says some error deducted and high thread count. 

Can we resolve this issue by increasing io-threads and task threads? currently it is 56 and 448>

 



Report URL - https://fastthread.io/my-thread-report.jsp?p=c2hhcmVkLzIwMjEvMDcvMzAvLS1oaWdoLWNwdS10ZHVtcC5vdXQtLTktMjUtNDQ=

  • jboss

  • outofmemoryerror

  • heapallocation

  • io-threads

Please Sign In or to post your comment or answer

Profile Image

Ram Lakshmanan

Hello Ajay!

 

 GReetings. I would like to highlight 2 issues from your thread dumps, which has potential to cause CPU spike. It can cause the application go unresponsive.

 

1.  Below is the excerpt from your thread dump:

 

java.lang.Thread.State: RUNNABLE
at sun.nio.fs.LinuxWatchService.poll(Native Method)
at sun.nio.fs.LinuxWatchService.access$600(LinuxWatchService.java:47)
at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:314)
at java.lang.Thread.run(Thread.java:748)

 There are 42 threads which are executing exact same code path. Here is a thread discussion which says File system watcher code can consume high CPU. Why does your application need 42 Linux file system watcher threads? Is it expected?

2. You have taken 6 thread dumps inbetween 4:57 and 4:59 (which is a good practice). However in all these 6 thread dumps, 2 threads (default task-23 and default task-9) are continously looping on the same lines of code. When thread loops continously then CPU will start to spike up. Those two threads stack trace are give below:

default task-23

java.lang.Thread.State: RUNNABLE
at javax.faces.component.UIComponent.popComponentFromEL(UIComponent.java:1959)
at com.sun.faces.facelets.tag.jsf.ComponentTagHandlerDelegateImpl.popComponentFromEL(ComponentTagHandlerDelegateImpl.java:319)
at com.sun.faces.facelets.tag.jsf.ComponentTagHandlerDelegateImpl.apply(ComponentTagHandlerDelegateImpl.java:206)
at javax.faces.view.facelets.DelegatingMetaTagHandler.apply(DelegatingMetaTagHandler.java:120)
at javax.faces.view.facelets.CompositeFaceletHandler.apply(CompositeFaceletHandler.java:98)
at com.sun.faces.facelets.tag.jsf.core.ViewHandler.apply(ViewHandler.java:164)
at javax.faces.view.facelets.CompositeFaceletHandler.apply(CompositeFaceletHandler.java:98)
at com.sun.faces.facelets.compiler.NamespaceHandler.apply(NamespaceHandler.java:93)
at javax.faces.view.facelets.CompositeFaceletHandler.apply(CompositeFaceletHandler.java:98)
at com.sun.faces.facelets.compiler.EncodingHandler.apply(EncodingHandler.java:86)
at com.sun.faces.facelets.impl.DefaultFacelet.apply(DefaultFacelet.java:152)
at com.sun.faces.application.view.FaceletViewHandlingStrategy.buildView(FaceletViewHandlingStrategy.java:769)
at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:100)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:139)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:410)
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at com.newgen.wfdesktop.filter.WDSessionCheckFilter.doFilter(WDSessionCheckFilter.java:345) 

:

:

:

default task-9

java.lang.Thread.State: RUNNABLE
at javax.faces.component.UIComponent.popComponentFromEL(UIComponent.java:1959)
at com.sun.faces.facelets.tag.jsf.ComponentTagHandlerDelegateImpl.popComponentFromEL(ComponentTagHandlerDelegateImpl.java:319)
at com.sun.faces.facelets.tag.jsf.ComponentTagHandlerDelegateImpl.apply(ComponentTagHandlerDelegateImpl.java:206)
at javax.faces.view.facelets.DelegatingMetaTagHandler.apply(DelegatingMetaTagHandler.java:120)
at javax.faces.view.facelets.CompositeFaceletHandler.apply(CompositeFaceletHandler.java:98)
at javax.faces.view.facelets.DelegatingMetaTagHandler.applyNextHandler(DelegatingMetaTagHandler.java:137)
at com.sun.faces.facelets.tag.jsf.ComponentTagHandlerDelegateImpl.apply(ComponentTagHandlerDelegateImpl.java:188)
at javax.faces.view.facelets.DelegatingMetaTagHandler.apply(DelegatingMetaTagHandler.java:120)
at javax.faces.view.facelets.CompositeFaceletHandler.apply(CompositeFaceletHandler.java:98)
at com.sun.faces.facelets.tag.jsf.core.ViewHandler.apply(ViewHandler.java:164)
at javax.faces.view.facelets.CompositeFaceletHandler.apply(CompositeFaceletHandler.java:98)
at com.sun.faces.facelets.compiler.NamespaceHandler.apply(NamespaceHandler.java:93)
at javax.faces.view.facelets.CompositeFaceletHandler.apply(CompositeFaceletHandler.java:98)
at com.sun.faces.facelets.compiler.EncodingHandler.apply(EncodingHandler.java:86)
at com.sun.faces.facelets.impl.DefaultFacelet.apply(DefaultFacelet.java:152)
at com.sun.faces.application.view.FaceletViewHandlingStrategy.buildView(FaceletViewHandlingStrategy.java:769)
at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:100)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:139)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:410)
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at com.newgen.wfdesktop.filter.WDSessionCheckFilter.doFilter(WDSessionCheckFilter.java:345)

:

:

:

Got something else on mind? Post Your Question

Not the answer you're looking for? Browse other questions tagged

  • jboss

  • outofmemoryerror

  • heapallocation

  • io-threads

Edit your question

Separate with comma, space or ENTER key. Special characters are not allowed in tag.

Edit your Answer

Edit your Comment