Database cleaner

Production environments that have a high volume of write operations might require cleaner thread tuning to control the on-disk database size as log files with inactive nodes wait to be cleaned and deleted.

The PingDirectory server stores its Oracle Berkeley DB Java Edition (JE) database files on-disk in the db directory. Each JE database log file is labeled nnnnnnnn.jdb, where nnnnnnnn is an 8-digit hexadecimal number that starts at 00000000 and is increased by 1 for each file written to disk. JE only appends data to the end of each file and does not overwrite any existing data. JE uses one or more cleaner threads that run in the background to compact the number of JE database (db) files.

The cleaner threads begin by scanning the records in each db file, starting with the file that contains the smallest number of active records. Next, the cleaner threads append any active records to the most recent database file. If a record is no longer active because of modifications or deletions, the cleaner threads leave it untouched. After the db file no longer has active records, the cleaner threads can either delete the file or rename the discarded file.

The local DB backend configuration object has two properties that control database cleaning: db-cleaner-min-utilization and db-num-cleaner-threads. The db-cleaner-min-utilization property determines, by percentage, when to begin cleaning out inactive records from the database files. By default, the property is set to 75, which indicates that database cleaning ensures that at least 75% of the total log file space is devoted to live data.

The db-num-cleaner-threads property determines how many threads are configured for db cleaning. The default single cleaner thread is normally sufficient. However, environments with a high volume of write traffic might need to increase this value to ensure that database cleaning can keep up.

If the number of database files grow beyond your expected guidelines or if the PingDirectory server is experiencing an increased number of update requests, you can increase the number of cleaner threads using the dsconfig tool by going to Backend → Select Advanced Properties → db-num-cleaner-threads.