Do I need to increase heap size?

Hi Mohammad,

As per my observations on your heap memory, it is growing up but after some time it is dropping down all the way to the bottom. If you observe the below screenshot is taken from the attached GC report. 

This indicates to me that may be because of the traffic, memory is increasing at some particular time, first at 03:02 PM and then at 5:21 PM. If you confirm that at this particular time interval there will be heavy traffic then you might have to increase the heap size. If this is not by the traffic then follow the below steps to diagnose the memory problems:

You can diagnose Memory problems either through a manual or automated approach.

Manual approach

In the manual approach, you need to capture heap dump as the first step. A heap dump is a snapshot of memory, which shows all the objects in memory, values contained by those objects, and their references. You can capture heap dump using one of the 7 approaches given here. But an important criterion is: You need to capture the heap dump right before OutOfMemoryError is thrown. If you are going to capture a heap dump after OutOfMemoryError has occurred, then leaking objects can get garbage collected, and it will become hard (or even impossible) to diagnose the problem. A better way to collect heap dumps is by setting the below JVM parameters for your application. This will help you out when an OOM error gets it will create the heap dump at mentioned file path.

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=<filepath/heapdump.bin>

Once heap dumps are captured, you need to import the heap dumps from your production servers to your local machine. From your local machine, you can use heap dump analysis tools like jHat, HeapHero, Eclipse MAT to analyze the heap dumps. Here is a video tutorial, which may assist you with it.

Automated approach

You can use root cause analysis tools like yCrash - which automatically captures application-level data (thread dump, heap dump, Garbage Collection log) and system-level data (netstat, vmstat, iostat, top, top -H, dmesg, kernel parameters…). Besides capturing the data automatically, it marries these two datasets and generates an instant root cause analysis report. 

Thank you for the reply; the reason for the drop, the report is multiple GC files for one day, the drop is when we face out of memory and then the application crashed, and we had to restart it.

Why I am getting out of memory? Could be the server can not handle that much memory ( I am thinking of reducing the heap to 64).

Also, I am thinking of migrating to Java 1.8_202.

Regards

Hello Mohammad!

 I reviewed your GC report. Here are my observations/thoughts:

a. Based on your description & GC graph, it looks like your application is suffering from memory problems (since it's crashing with OutOfMemoryError). You need to fix this first. Tuning will not help you. You can use the approach outlined by Sainath (above) to diagnose the memory leak.

b. Inbetween here are few options/recommendations to reduce G1 Evacuation Pause events:

1. Evacuation failure might happen because of over tuning. So eliminate all the memory related properties and keep only min and max heap and a realistic pause time goal (i.e. Use only -Xms, -Xmx and a pause time goal -XX:MaxGCPauseMillis). Remove any additional heap sizing such as -Xmn, -XX:NewSize, -XX:MaxNewSize, -XX:SurvivorRatio, etc.

2. If the problem still persists then increase JVM heap size (i.e. -Xmx).

3. If you can't increase the heap size and if you notice that the marking cycle is not starting early enough to reclaim the old generation then reduce -XX:InitiatingHeapOccupancyPercent. The default value is 45%. Reducing the value will start the marking cycle earlier. On the other hand, if the marking cycle is starting early and not reclaiming, increase the -XX:InitiatingHeapOccupancyPercent threshold above the default value.

4. You can also increase the value of the '-XX:ConcGCThreads' argument to increase the number of parallel marking threads. Increasing the concurrent marking threads will make garbage collection run fast.

5. Increase the value of the '-XX:G1ReservePercent' argument. Default value is 10%. It means the G1 garbage collector will try to keep 10% of memory free always. When you try to increase this value, GC will be triggered earlier, preventing the Evacuation pauses. Note: G1 GC caps this value at 50%.

c. As you have suggested, upgradeing to Java 1.8+ is a good idea. Remember starting from 1.8, PermGen has been replaced by Metaspace. I could see you in your JVM arguments you have configured -XX:PermSize and -XX:MaxPermSize JVM arguments. They have to be replaced by -XX:MetaspaceSize and -XX:MaxMetaspaceSize arguments.

 Here is a video which might be of help to understand GC tunning little further.