Writing logs in JSON format

You can configure PingAccess to write logs in JavaScript Object Notation (JSON) format using the log4j2 logging library. JSON is a common logging format for security information and event management (SIEM) tracking systems and is easily human-readable.

About this task

PingAccess includes JSON log templates, which you can find in the <PA_HOME>/conf/log4j/json-templates directory, for the following log files:

pingaccess.log
pingaccess_api_audit.log
pingaccess_agent_audit.log
pingaccess_engine_audit.log
pingaccess_sideband_audit.log
pingaccess_sideband_client_audit.log

The jvm-garbage-collection.log file isn’t log4j2-enabled, so PingAccess doesn’t provide a JSON log template for it.

The log4j2.xml file contains RollingFile appenders that produce output in both standard and JSON format. By default, the appenders for both formats output to the same filename.

To output only one format, comment out the appender reference for the other format. Learn more in step 3.
To output logs in both standard and JSON format, configure different filenames for each format in the RollingFile appender configurations. Otherwise, both formats display in the same file. Learn more in step 4.

Steps

Open the <PA_HOME>/conf/log4j2.xml file in a text editor.

Uncomment the JSON appender references in the root, apiaudit, engineaudit, agentaudit, sidebandclientaudit, and sidebandaudit logger configurations.

Example:

In the Set up the Root logger section of the log4j2.xml file, uncomment the File-JSON appender reference:

Code

<!-- ======================= -->
<!-- Set up the Root logger  -->
<!-- ======================= -->
<AsyncRoot level="INFO" includeLocation="false">
    <AppenderRef ref="File"/>
    <AppenderRef ref="File-JSON" />
    <!--<AppenderRef ref="CONSOLE" />-->
    <!--<AppenderRef ref="CONSOLE-JSON" />-->
    <!--<AppenderRef ref="SYSLOG" />-->
</AsyncRoot>

Repeat this in the Audit log configuration section with the ApiAuditLog-JSON, EngineAuditLog-JSON, AgentAuditLog-JSON, SidebandClientAuditLog-JSON, and SidebandAuditLog-JSON appender references.

If you want to write the logs to the console instead of, or in addition to a file, uncomment the CONSOLE-JSON, CONSOLE-ApiAuditLog-JSON, CONSOLE-EngineAuditLog-JSON, CONSOLE-AgentAuditLog-JSON, CONSOLE-SidebandClientAuditLog-JSON, and CONSOLE-SidebandAuditLog-JSON appender references as necessary.

If you want JSON output only, comment out the appender references for the non-JSON format output.

Doing so prevents PingAccess from writing both standard and JSON formats to the same log file.

Example:

In the Set up the Root logger section of the log4j2.xml file, comment out the File appender reference:
Code
   <AsyncRoot level="INFO" includeLocation="false">  <AppenderRef ref="File-JSON" />    </AsyncRoot>
Repeat this in the Audit log configuration section with the ApiAuditLog-File, EngineAuditLog-File, AgentAuditLog-File, SidebandClientAuditLog-File, and SidebandAuditLog-File appender references.

If you want to output two separate log files for standard and JSON format, change the name of the output file in the RollingFile appender configurations for the JSON format.

Example:

In the API auditing file logging configuration section, go to the ApiAuditLog-JSON RollingFile appender configuration. Modify the fileName and filePattern:

Code

<!-- API Audit log : JSON format file logging configuration -->
        <RollingFile name="ApiAuditLog-JSON"
                     fileName="${sys:pa.home}/log/pingaccess_json_api_audit.log"
                     filePattern="${sys:pa.home}/log/pingaccess_json_api_audit.%d{yyyy-MM-dd}.log"
                     ignoreExceptions="false">
            <JsonTemplateLayout eventTemplateUri="${sys:pa.log4j.json.templates.uri}/api-audit-log.json"/>
            <Policies>
                <TimeBasedTriggeringPolicy />
            </Policies>
        </RollingFile>

Repeat this with the EngineAuditLog-JSON, AgentAuditLog-JSON, SidebandAuditClientLog-JSON, and SidebandAuditLog-JSON appender configurations.

(Optional) For each JsonTemplateLayout value, designate the URI location of the desired JSON templates.

The ${sys:pa.log4j.json.templates.uri} URI designates the default location where the JSON log file templates are stored. You can replace this with a custom URI filepath. Otherwise, log files are stored in their default location of <PA_HOME>/conf/log4j/json-templates.

Using the same example from the previous step, you can find JsonTemplateLayout after the RollingFile details:

Code

<!-- API Audit log : JSON format file logging configuration -->
        <RollingFile name="ApiAuditLog-JSON"
                     fileName="${sys:pa.home}/log/pingaccess_json_api_audit.log"
                     filePattern="${sys:pa.home}/log/pingaccess_json_api_audit.%d{yyyy-MM-dd}.log"
                     ignoreExceptions="false">
            <JsonTemplateLayout eventTemplateUri="${sys:pa.log4j.json.templates.uri}/api-audit-log.json"/>
            <Policies>
                <TimeBasedTriggeringPolicy />
            </Policies>
        </RollingFile>

Save and close the log4j2.xml file.

Custom log patterns

To create custom log patterns in log4j2-enabled logs using JSON format, you must use special syntax.

For example, if a log file appender references a custom HTTP header using %clientrequestheader to log x-myheader:

Example pattern

<RollingFile ... >
<PatternLayout>
<pattern>%d | %header{x-myheader} | %m%n</pattern>
</PatternLayout>
...
</RollingFile>

In the corresponding JSON template (for example, api-audit-log.json), you must refer to the %clientRequestHeader{x-myheader} using the following JSON object:

JSON object

"myheader": {
    "$resolver": "pattern",
    "pattern": "%clientRequestHeader{x-myheader}"
},

You can also use MDC as the $resolver to get the value directly. For example:

Using MDC to get the value directly

"myheader": {
    "$resolver": "mdc",
    "key": "AUDIT.http-client-request-header-x-myheader",
    "stringified": true
},

If you want to use MDC but aren’t sure what options are available, you can print out all available values in MDC:

Printing out all values in MDC

"allvalues": {
    "$resolver": "mdc",
    "stringified": true
},

You can find the reference to the relevant JSON template in the RollingFile appender configurations in the log4j2.xml file. The JSON file appender names include a -JSON suffix. The associated eventTemplateUri value indicates the relevant JSON template name.

JSON template reference

<RollingFile name="ApiAuditLog-JSON"
             fileName="${sys:pa.home}/log/pingaccess_api_audit.log"
             filePattern="${sys:pa.home}/log/pingaccess_api_audit.%d{yyyy-MM-dd}.log"
             ignoreExceptions="false">
     <JsonTemplateLayout eventTemplateUri="${sys:pa.log4j.json.templates.uri}/api-audit-log.json"/>
     <Policies>
         <TimeBasedTriggeringPolicy />
     </Policies>
</RollingFile>