PingDirectory

Ingesting logs into a log management system

Many organizations use a centralized log management system, such as Splunk or DataDog. In these cases, product log messages can be ingested into that system to make them accessible from a common location, and to provide improved support for analytics and taking other actions upon log content.

There are several ways that log content can make it into the log management system. Some of these options include:

  • Most log management software provides agent software that can read data from log files and send it to the centralized system. In some cases, the agent can stream log data as it is written, which reduces the chance that an attacker has a chance to alter it. In other cases, it can copy the data at configured intervals.

  • For cases in which the target application is running in a container like Docker, a common practice is to have that application write log messages to standard output or standard error, and to forward those streams to the log management software. To help support this, the PingDirectory server provides an option to write access and error log messages, with each message formatted as a JSON object for greater parsability, to standard output or standard error. For more information about logging to standard output or standard error, see the config/sample-dsconfig-batch-files/enable-console-based-logging.dsconfig batch file.

  • The log management software can provide an API that applications can use to write log messages directly to that service. Although the PingDirectory software does not provide support for this out of the box, it is possible to use the Server SDK to write a custom log publisher to take advantage of this API.