Profile Image
Ethan

JBoss Server Memory Optimization Configuration

I am seeking expert advice regarding configuration changes to reduce memory consumption. Over the past two months, the memory utilization of our JBoss server has been gradually increasing. Despite implementing the following changes, there has been no improvement in behavior:

  1. Updated heap max-size from 25000m to 20000m.
  2. Changed -XX:MaxGCPauseMillis from 500 to 200.
  3. Added -XX:+ExplicitGCInvokesConcurrent.
  4. Added -XX:+UnlockExperimentalVMOptions.
  5. Added -XX:G1MixedGCLiveThresholdPercent=60.
  6. Added -XX:G1MixedGCCountTarget=14.

Report URL - https://gceasy.io/my-gc-report.jsp?p=c2hhcmVkLzIwMjUvMDEvMTQvMSBtZmNvbnNvbGUtZ2MubG9nLjAuY3VycmVudC0tMi0xMy05&channel=WEB

    Please Sign In or to post your comment or answer

    Profile Image

    Charlie Arehart

    Can you clarify first why, if your memory needs are increasing, you LOWERED the max heap from 25 to 20g?

     

    And second, if your goal is indeed to determine why your memory use is increasing, fiddling with jvm tuning knobs seems the wrong approach. You'd instead want to find either what specifically is using more memory, or what has changed over time.

     

    As for "what is using memory", have you confirmed if it's indeed the heap USE that is increasing? Meaning, if you do a gc, what does the usage fall to? Then some hours later (assuming jboss has not restarted), repeat that. Is the low point after a gc increasing or not? If it's not, then heap use is not increasing and so not seemingly a problem.

     

    You might instead be looking only at the OS tracking of the size of the jboss or Java process. Just beware that reflects memory ALLOCATED rather than used (see above). And that could well be non-heap memory, like meta space. There are ways to observe how THAT is being used over time. And addressing that would be different from dealing with increased heap use.

     

    Then as for what could be CAUSING the change in memory use you feel has happened over time, you may say also that "nothing has changed" in those months. But if you're referring to your configuration, I'd turn your attention to your web traffic. Since this is a jboss app, I'd ask: is ANY aspect of the site or sites open to the web? Even if a "login is required"?

     

    Assuming it IS open to the web, I've helped many folks determine that they're incoming traffic is dramatically changing in volume and nature due to spiders, bots, hackers, and more. (And even an intranet app could be attacked from within the network.) Just the increase in traffic--whether successful or failing--could drive up memory use as your app tries to respond to that traffic.

     

    And you can do such "traffic" investigation via web server logs, but using an APM would generally be more effective. Of course, the latter won't help in discerning a change from the past if you're just installing it, but it would help compare now to later, if you remain unable to resolve things readily. 

     

    Or maybe the ycrash folks or others will have different suggestions.

     

    I'd appreciate if you'd reply with any thoughts, to let us know if you've considered mine here. 

    Profile Image

    Unni Mana

     

    Hello Ethan,

    After going through the report, it is found that your application is very healthy.

    Memory consumption may increase because of the following reasons:
    - traffic increases
    - your application is creating more objects. if you take a look at the

      object creation rate, It is  154.55 mb/sec.it is very high.

      Check why application is creating lot of objects.

    Also, in the report, after a particular time, say, after 7:00, the memory keeps on increasing. There must be some reason behind it. Needs to investigate it.

    Also, go through the beginning of the report which gives some recommentation. Especially on the stringDeduplication argument.

    you may consider passing -XX:+UseStringDeduplication to your application. It will remove duplicate strings in your application and has potential to improve overall application's performance

     

     Note: Try running the application for more than 24 hours and run the GC report so that you will get more valuable insight.

     

    Profile Image

    Kousika M

    Hello Ethan,

    Greetings!

    The report says your application is very healthy with very good Throughput of 99.859%. 

    To investigate and address the gradual increase in memory usage, you can check the Memory Leaks.

    You can capture heap dumps at different time intervals and analyze the memory usage over time. This can help identify objects that are not being garbage collected and might be causing a memory leak. You can use the tool heaphero to investigate the memory related issues.

    Consider reviewing recent code changes that could have introduced higher memory usage, such as increased object creation or inefficient algorithms.



    Thanks.

    Got something else on mind? Post Your Question