PingAuthorize Server troubleshooting information
PingAuthorize Server has a comprehensive default set of log files and monitor entries that are useful when troubleshooting a particular server problem.
Error log
By default, this log file is available at logs/errors
below the server install root, and it provides information about warnings, errors, and other significant events that occur within the server. A number of messages are written to this file on startup and shutdown, but while the server is running, there is normally little information written to it. In the event that a problem does occur, the server writes information about that problem to this file.
The following is an example of a message that might be written to the error log:
[11/Apr/2011:10:31:53.783 -0500] category=CORE severity=NOTICE msgID=458887 msg="The {pingauthorize} Server has started successfully."
The category field provides information about the area of the server from which the message was generated. Available categories include:
ACCESS_CONTROL, ADMIN, ADMIN_TOOL, BACKEND, CONFIG, CORE, DSCONFIG, EXTENSIONS, PROTOCOL, SCHEMA, JEB, SYNC, LOG, PLUGIN, PROXY, QUICKSETUP, REPLICATION, RUNTIME_INFORMATION, TASK, THIRD_PARTY, TOOLS, USER_DEFINED, UTIL, VERSION
The severity field provides information about how severe the server considers the problem to be. Available severities include:
DEBUG
-
Used for messages that provide verbose debugging information and do not indicate any kind of problem. Note that this severity level is rarely used for error logging, because the server provides a separate debug logging facility.
INFORMATION
-
Used for informational messages that can be useful from time to time but are not normally something that administrators need to see.
MILD_WARNING
-
Used for problems that the server detects, which can indicate something unusual occurred, but the warning does not prevent the server from completing the task it was working on. These warnings are not normally something that should be of concern to administrators.
MILD_ERROR
-
Used for problems detected by the server that prevented it from completing some processing normally but that are not considered to be a significant problem requiring administrative action.
NOTICE
-
Used for information messages about significant events that occur within the server and are considered important enough to warrant making available to administrators under normal conditions.
SEVERE_WARNING
-
Used for problems that the server detects that might lead to bigger problems in the future and should be addressed by administrators.
SEVERE_ERROR
-
Used for significant problems that have prevented the server from successfully completing processing and are considered important.
FATAL_ERROR
-
Used for critical problems that arise which might leave the server unable to continue processing operations normally.
The messages written to the error log can be filtered based on their severities in two ways. First, the error log publisher has a default-severity
property, which can be used to specify the severity of messages logged regardless of their category. By default, this includes the NOTICE
, SEVERE_WARNING
, SEVERE_ERROR
, and FATAL_ERROR
severities.
You can override these severities on a per-category basis using the override-severity
property. If this property is used, then each value should consist of a category name followed by an equal sign and a comma-delimited set of severities that should be logged for messages in that category. For example, the following override severity would enable logging at all severity levels in the PROTOCOL
category:
protocol=debug,information,mild-warning,mild-error,notice,severe-warning,severe-error,fatal-error
Note that for the purposes of this configuration property, any underscores in category or severity names should be replaced with dashes. Also, severities are not inherently hierarchical, so enabling the DEBUG
severity for a category will not automatically enable logging at the INFORMATION
, MILD_WARNING
, or MILD_ERROR
severities.
The error log configuration can be altered on the fly using tools like dsconfig
, the Administrative Console, or the LDIF connection handler, and changes will take effect immediately. You can configure multiple error logs that are active in the server at the same time, writing to different log files with different configurations. For example, a new error logger can be activated with a different set of default severities to debug a short-term problem, and then that logger can be removed after the problem is resolved, so that the normal error log does not contain any of the more verbose information.
server.out log
The server.out
file holds any information written to standard output or standard error while the server is running. Normally, it includes a number of messages written at startup and shutdown, as well as information about any administrative alerts generated while the server is running. In most cases, this information is also written to the error log. The server.out
file can also contain output generated by the JVM. For example, if garbage collection debugging is enabled, or if a stack trace is requested via kill -QUIT
as described in a later section, then output is written to this file.
Debug log
The debug log provides a means of obtaining information that can be used for troubleshooting problems but is not necessary or desirable to have available while the server is functioning normally. As a result, the debug log is disabled by default, but it can be enabled and configured at any time.
Some of the most notable configuration properties for the debug log publisher include:
enabled
-
Indicates whether debug logging is enabled. By default, it is disabled.
log-file
-
Specifies the path to the file to be written. By default, debug messages are written to the
logs/debug
file. debug-level
-
Specifies the minimum log level for debug messages that should be written. The default value is
error
, which only provides information about errors that occur during processing (for example, exception stack traces). Other supported debug levels includewarning
,info
, andverbose
. Note that unlike error log severities, the debug log levels are hierarchical. Configuring a specified debug level enables any debugging at any higher levels. For example, configuring theinfo
debug level automatically enables thewarning
anderror
levels. debug-category
-
Specifies the categories for debug messages that should be written. Some of the most useful categories include
caught
(provides information and stack traces for any exceptions caught during processing),database-access
(provides information about operations performed in the underlying database),protocol
(provides information about ASN.1 and LDAP communication performed by the server), anddata
(provides information about raw data read from or written to clients).
As with the error and access logs, multiple debug loggers can be active in the server at any time with different configurations and log files to help isolate information that might be relevant to a particular problem. Debug targets can be used to further pare down the set of messages generated. For example, you can specify that the debug logs be generated only within a specific class or package.
Enabling one or more debug loggers can have a significant impact on server performance. Enable debug loggers only when necessary, and scope them so that only pertinent debug information is recorded. If you need to enable the debug logger, you should work with your authorized support provider to best configure the debug target and interpret the output. |
Config audit log and the configuration archive
The configuration audit log provides a record of any changes made to the server configuration while the server is online. This information is written to the logs/config-audit.log
file and provides information about the configuration change in the form that can be used to perform the operation in a non-interactive manner with the dsconfig
command. Other information written for each change includes:
-
Time that the configuration change was made
-
Connection ID and operation ID for the corresponding change, which can be used to correlate it with information in the access log
-
DN of the user requesting the configuration change and the method by which that user authenticated to the server
-
Source and destination addresses of the client connection
-
Command that can be used to undo the change and revert to the previous configuration for the associated configuration object
In addition to information about the individual changes that are made to the configuration, the server maintains complete copies of all previous configurations. These configurations are provided in the config/archived-configs
directory and are gzip-compressed copies of the config/config.ldif
file in use before the configuration change was made. The file names contain timestamps that indicate when that configuration was first used.
Setup log
The setup
tool writes a log file providing information about the processing that it performs. By default, this log file is written to logs/setup.log
although a different name may be used if a file with that name already exists, because the setup
tool has already been run. The full path to the setup log file is provided when the setup
tool has completed.
Tool log
Many of the administrative tools provided with the server (for example, import-ldif
, export-ldif
, backup
, restore
, etc.) can take a significant length of time to complete writing information to standard output or standard error (or both) while the tool is running. They also write additional output to files in the logs/tools
directory (for example, logs/tools/ import-ldif.log
). The information written to these log files can be useful for diagnosing problems encountered while they were running. When running using the server tasks interface, log messages generated while the task is running can alternately be written to the server error log file.