JVM garbage collection using CMS
To ensure reliable server performance with Java, the PingDirectory server depends on Java’s Concurrent Mark and Sweep process (CMS) for background garbage collection.
Out of several garbage collection options, CMS is the ideal choice for consistent system availability. For the most part, the CMS collector runs as one or more background threads within the Java virtual machine (JVM), freeing up space in JVM Heap from an area called the old generation. One of the criteria used by CMS to determine when to start background garbage collection is a parameter called CMSInitiatingOccupancyFraction
. This percentage value, which applies to the Old Generation, is recommended for the JVM to initiate CMS when data occupancy in Old Generation reaches the threshold.
Ideally, the database cache takes less than 70% of the space available in the Old Generation, and the CMSInitiatingOccupancyFraction
value of 80 leaves plenty of headroom to prevent the JVM from running out of space in Old Generation because of an inability for CMS to keep up. Because CMS takes processing resources away from the PingDirectory server, do not set theCMSInitiatingOccupancyFraction
at or below the expected database cache size, which would result in the constant running of CMS in the background. For a description of determining Old Generation size, see the section on Memory Footprint and Database Cache.
When the CMS collection process cannot keep pace with memory demands in the Old Generation, the JVM resorts to pausing all application processing to allow a full garbage collection. This event, called a "stop-the-world pause", does not break existing TCP connections or alter the execution of the PingDirectory server requests. The goal in tuning CMS is to prevent the occurrence of these pauses. When one does occur, the server generates an alert after the pause and records the pause time in the error log.
Because determining an ideal CMSInitiatingOccupancyFraction
can be difficult, we warn if the PingDirectory server detects a garbage collection pause by generating a recommended value for the occupancy threshold based on the current amount of memory being consumed by the backend caches. Unfortunately, an administrator cannot determine the ideal occupancy threshold value in advance. To warn of any impending garbage collection pauses, the server calculates a recommended value for the CMSInitiatingOccupancyFraction
property and exposes it in the JVM Memory Usage monitor entry in the following attribute:
recommended-cms-initiating-occupancy-fraction-for-current-data-set
Also, when you start the server, an administrative alert indicates the current state of the CMSInitiatingOccupancyFraction
and its recommended value.
$ bin/start-server [20/April/2012:10:35:25 -0500] category=CORE severity=NOTICE msgID=458886
msg="PingDire Server
8.0.0.0 (build 20120418135933Z, R6226) starting up"
... (more output) ...
[20/April/2012:10:35:53 -0500] category=UBID_EXTENSIONS severity=NOTICE
msgID=1880555580 msg="Memory-intensive PingDirectory server
components are configured to consume 71750382 bytes of memory:
['userRoot local DB backend' currently consumes 26991632 bytes and
can grow to a maximum of 64323584 bytes, 'changelog cn=changelog backend'
currently consumes 232204 bytes and can grow to a maximum of 2426798 bytes,
'Replication Changelog Database' currently consumes 376661 bytes and can
grow to a maximum of 5000000 bytes]. The configured value of
CMSInitiatingOccupancyFraction is 36 which is less than the minimum
recommended value (43) for the server's current configuration. Having
this value too low can cause the Concurrent Mark and Sweep garbage
collector to run too often, which can cause a degradation of throughput
and response time. Consider increasing the CMSInitiatingOccupancyFraction
value to at least the minimum value, preferably setting it to the
recommended value of 43 by editing the config/java.properties file,
running dsjavaproperties, and restarting the PingDirectory server.
If the server later detects that this setting actually leads to a
performance degradation, a separate warning message will be logged.
If this server has not yet been fully loaded with data, then you
can disregard this message"
[20/April/2012:10:35:53 -0500] category=CORE severity=NOTICE msgID=458887
msg="The PingDirectory server has started successfully"
The PingDirectory server only makes a recommendation if all of the backends are preloaded and the CMSInitiatingOccupancyFraction
JVM property is explicitly set, which is done automatically. For example, if you installed the PingDirectory server and specified that the database be preloaded (or "primed") at startup, then the server can make a good recommendation when a pause occurs. If the backend database cache is not full and has not been preloaded, then the recommended value might be an inaccurately low value.
The generated value for the PingDirectory server’s property could change over time with each server build, Java release, or changes in data set. If the current value is close to the recommended value, then you do not need to change the property unless the server experiences a JVM pause. If the PingDirectory server experiences a JVM garbage collection pause, you can retrieve the recommended value from the server, reset the property, run |