Logs and log publishers

PingDirectory servers support different types of log publishers that can be used to provide the monitoring information for operations, access, debug, and error messages that occur during normal server processing. The server provides a standard set of default log files as well as mechanisms to configure custom log publishers with their own log rotation and retention policies.

Types of log publishers

Several types of log publishers can be used to log processing information about the server, including:

Audit loggers

Provide information about actions that occur within the server. Specifically, this type of log records:

All changes applied, detected or failed
Dropped operations that were not completed
Changes dropped because of being out of scope
No changes needed for an operation

The log also shows the entries that were involved in a process.

Error loggers

Provide information about warnings, errors, or significant events that occur within the server.

Debug loggers

Provide detailed information about processing performed by the server, including:

Any exceptions caught during processing
Detailed information about data read from or written to clients
Accesses to the underlying database

Access loggers

Provide information about Lightweight Directory Access Protocol (LDAP) operations processed within the server. This log only applies to operations performed in the server. This includes:

Configuration changes
Searches of monitor data
Bind operations for authenticating administrators using the command-line tools and the administrative console

View the list of log publishers

View the list of log publishers on each server using the dsconfig tool:

$ bin/dsconfig list-log-publishers
Log Publisher                      : Type              : enabled
-----------------------------------:-------------------:--------
Debug ACI Logger                   : debug-access      : false
Expensive Operations Access Logger : file-based-access : false
Failed Operations Access Logger
File-Based Access Logger
File-Based Audit Logger
: file-based-access : true
: file-based-access : true
: file-based-audit  : false
File-Based Debug Logger
File-Based Error Logger
Replication Repair Logger
: file-based-debug  : false
: file-based-error  : true
: file-based-error  : true

Log compression

PingDirectory servers support the ability to compress log files as they are written. Because of the inherent problems with mixing compressed and uncompressed data, compression can only be enabled when the logger is created. Compression cannot be turned on or off once the logger is configured. If the server encounters an existing log file at startup, it will rotate that file and begin a new one rather than attempting to append it to the previous file.

Compression is performed using the standard gzip algorithm. Because it can be useful to have an amount of uncompressed log data for troubleshooting, having a second logger defined that does not use compression might be preferred.

Configure compression by setting the compression-mechanism property to have the value of gzip when creating a new logger. See Creating a new log publisher for details.

Configuring log file encryption

About this task

The server supports the ability to encrypt log files as they are written. The encrypt-log configuration property controls whether encryption will be enabled for the logger. Enabling encryption causes the log file to have an .encrypted extension (and if both encryption and compression are enabled, the extension will be .gz.encrypted). Any change that affects the name used for the log file could prevent older files from getting properly cleaned up.

Like compression, encryption can only be enabled when the logger is created. Encryption cannot be turned on or off after the logger has been configured. For any log file that is encrypted, enabling compression is also recommended to reduce the amount of data that needs to be encrypted. This will also reduce the overall size of the log file. The encrypt-file command (or custom code, using the LDAP SDK’s com.unboundid.util.PassphraseEncryptedInputStream) is used to access the encrypted data.

To enable encryption, at least one encryption settings definition must be defined in the server. Use the one created during setup, or create a new one with the encryption-settings create command. By default, the encryption will be performed with the server’s preferred encryption settings definition. To explicitly specify which definition should be used for the encryption, the encryption-settings-definition-id property can be set with the ID of that definition. You should create the encryption settings definition from a passphrase so that the file can be decrypted by providing that passphrase, even if the original encryption settings definition is no longer available. A randomly generated encryption settings definition can also be created, but the log file can only be decrypted using a server instance that has that encryption settings definition.

When using encrypted logging, a small amount of data can remain in an in-memory buffer until the log file is closed. The encryption is performed using a block cipher, and it cannot write an incomplete block of data until the file is closed. This is not an issue for any log file that is not being actively written. To examine the contents of a log file that is being actively written, use the rotate-log command to force the file to be rotated before attempting to examine it.

The following commands can be used to set log file encryption:

Steps

Use dsconfig to enable encryption for a Log Publisher. In this example, the File-based Access Log Publisher "Encrypted Access" is created, compression is set, and rotation and retention policies are set.

$ bin/dsconfig create-log-publisher-prop --publisher-name "Encrypted
Access" \
  --type file-based-access \
  --set enabled:true \
  --set compression-mechanism:gzip \
  --set encryption-settings-definition-
id:332C846EF0DCD1D5187C1592E4C74CAD33FC1E5FC20B726CD301CDD2B3FFBC2B \
  --set encrypt-log:true \
  --set log-file:logs/encrypted-access \
  --set "rotation-policy:24 Hours Time Limit Rotation Policy" \
  --set "rotation-policy:Size Limit Rotation Policy" \
   --set "retention-policy:File Count Retention Policy" \
  --set "retention-policy:Free Disk Space Retention Policy" \
   --set "retention-policy:Size Limit Retention Policy"

To decrypt and decompress the file:

$ bin/encrypt-file --decrypt \
  --decompress-input \
  --input-file logs/encrypted-access.20180216040332Z.gz.encrypted \
  --output-file decrypted-access
Initializing the server's encryption framework...DoneWriting decrypted
data to file '/ds/PingDirectory/decrypted-access' using a key generated
from encryption settings definition
'332c846ef0dcd1d5187c1592e4c74cad33fc1e5fc20b726cd301cdd2b3ffbc2b'Success
fully wrote 123,456,789 bytes of decrypted data