more threads are created in WAS container without a need for it

Hi Mohamed,

I don't know the exact answer to this. But in the attached fastThread report I could see that you have around 756 Daemon threads. These threads are mostly created by JVM for performing background tasks like Garbage collection and other housekeeping tasks. Check the "Deamon vs non-Daemon" section in your report. To learn about Daemon vs non-daemon threads visit - 

https://blog.fastthread.io/2017/05/25/less-known-facts-about-daemon-and-non-daemon-threads/

I have also noticed that there are a total 64 Garbage Collector (GC) threads. It may affect the application's performance. You may consider reducing GC threads.

Mahesh, thanks for the response.

We dont find an issue with the application performance.  the only problem is to find out the reason and fix why more threads (web container) are getting created.  If I check the PMI and the monitoring metrics, the active are only 20 - 30 threads wherein the created threads in pool is 500+.  

We've the configuration to have the max web container thread to be 600 based on the load which was working fine so far.  After upgrading to 8.5.5.19 with Java 8.0.6.25 we see these unusual behaviours.

Hello Mohamed!

 I could see your WAS container has created 520 threads. In this 520 threads only 7 of them are active, remaining 491 threads are in WAITING state. i.e. basically all these are doing nothing.

 I could see following reasons for this to happen:

a. Change in the Load Balancing algorithm: Did you made any change to the load balancer algorithm? Especially did you switch to round robin algorithm? Round robin algorithm tends to keep the connections/threads more active than the requirement.

b. Change WAS thread pool configuration: Did you change the 'min' thread pool count in the new server? If you keep min threads count high then more number of threads would be kept in WAITING state.

c. WAS settings default changed: May be you didn't do #b i.e., 'Change the WAS thread pool configuration', however default settings of WAS has changed and that is causing the threads to be kept active.

Thanks Ram, literally I was expecting a reply from you :)

Yes, I was looking for the cause on why so many threads were created when there isn't a need for it.  We've set the min as 200 and max as 600 based on the peak load and it was fine for almost a year.

1. LB algorithm - There isn't any change in plugin LB algorithm since last year, below is what it's set as in the plugin.

<ServerCluster CloneSeparatorChange="false" GetDWLMTable="false" IgnoreAffinityRequests="false" LoadBalance="Round Robin"

2. We've this pool set and per the configuration if its not used it should be set with the min of 200, but here its opened and not getting closed or coming back to min pool count.

3. No recent changes to the pool configurations. 

Only recent change is we applied FP for both WAS (8.5.5.19) & Java (8.0.6.25). 

Below is from the monitoring tool on the threads usage.

If we could find the reason that would be great.

Thanks Ram, but to mention here all the Web --> App communication is one-to-one.

Believe making changes to the LB algorithm will not make any difference.

Dear Ram,

LB is still with the Least Connections algorithm for years.  

As said earlier the only change which has been made is below

1.  Applied WAS FP 8.5.5.19 bundled with Java 8.0.6.20

2.  Applied IFPH34271

since the Java version (8.0.6.20) has a known issue of JVM crash and we experienced the DMGR getting crashed for no reason above iFix was applied which updated the Java version to 8.0.6.25