A lot of blocked SecureRandom threads

Hi Nexign,

1)  'http-nio-8080-exec-1595 url: /rest/api/2/issue/UCL-49693; user: UCellEvent' thread is stuck on line #215 of sun.security.provider.SecureRandom file in engineNextBytes() method. Before getting stuck, this thread obtained 2 locks (sun.security.provider.SecureRandom lock, org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper lock) and never released it. Due to that 37 threads are BLOCKED as shown in the below graph. 

Below are the few lines of thread stack trace:

http-nio-8080-exec-1595 url: /rest/api/2/issue/UCL-49693; user: UCellEvent
Stack Trace is:
java.lang.Thread.State: RUNNABLE
at sun.security.provider.SecureRandom.engineNextBytes(SecureRandom.java:215)
- locked <0x0000000080217460> (a sun.security.provider.SecureRandom)
at java.security.SecureRandom.nextBytes(SecureRandom.java:468)
at java.util.UUID.randomUUID(UUID.java:145)
at java_util_UUID$randomUUID.call(Unknown Source)
at com.onresolve.scriptrunner.canned.jira.workflow.model.AbstractWorkflowFunctionCommandObject.<init>(AbstractWorkflowFunctionCommandObject.groovy:18)
at com.onresolve.scriptrunner.canned.jira.workflow.listeners.model.CustomListenerCommand.<init>(CustomListenerCommand.groovy)
at sun.reflect.GeneratedConstructorAccessor1667.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.fasterxml.jackson.databind.introspect.AnnotatedConstructor.call(AnnotatedConstructor.java:119)
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createUsingDefault(StdValueInstantiator.java:270)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:282)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:156)
at com.fasterxml.jackson.databind.deser.BeanDeserializer$deserialize.call(Unknown Source)

Hello Nexign Team!

a. What version of JDK are you using? This issue has been reported as JDK bug: 

Bug ID: JDK-8098581 SecureRandom.nextBytes() hurts performance with small size requests (java.com). This bug has been fixed in JDK 8u112 and JDK 9 b105

b. You can consider passing the below system property to see whether it resolves the problem -Djava.security.egd=file:/dev/./urandom

c. Here is a great pointer in Adobe which is discussing the same problem:

SecureRandom.nextBytes hangs request threads in AEM (adobe.com). They are talking about this problem on May' 17, 2021. If you can't upgrade to the above mentioned JDK version, then you can consider implementing the solution #2 mentioned in the adobe forum: 

solution is to use /dev/urandom instead of /dev/random. However, it has the downside of reduced security due to less randomness.

• Edit $JAVA_HOME/jre/lib/security/java.security
• Modify this line:

securerandom.source=file:/dev/random

to

securerandom.source=file:/dev/urandom

d. Here is a stackOverflow question talking about this same problem as well: 

java - How to deal with a slow SecureRandom generator? - Stack Overflow