Directory Services 7.3.6

Log HTTP access to files

JSON format

When you install DS using procedures from Installation, the default JSON-based HTTP access log file is logs/http-access.audit.json. The name of the access log publisher in the configuration is Json File-Based HTTP Access Logger.

The sample DS Docker image logs to standard output instead of files. This makes it easy to read log messages with the docker logs command, and is a pattern you should follow when creating your own DS Docker images. The name of the LDAP access log publisher configuration in the sample image is Console HTTP Access Logger:

  1. Decide whether to trust transaction IDs sent by client applications, used to correlate requests as they traverse multiple servers.

    Client applications using the ForgeRock Common Audit event framework send transaction IDs with their requests. The transaction IDs correlate audit events, tracing the request through multiple applications.

    Transaction IDs are sent over LDAP using an internal DS request control. They are sent over HTTP in an HTTP header.

    By default, DS servers do not trust transaction IDs sent with client application requests.

    When a server trusts transaction IDs from client application requests, outgoing requests reuse the incoming ID. For each outgoing request in the transaction, the request’s transaction ID has the form original-transaction-id/sequence-number, where sequence-number reflects the position of the request in the series of requests for this transaction. For example, if the original-transaction-id is abc123, the first outgoing request has the transaction ID abc123/0, the second abc123/1, the third abc123/2, and so on. This lets you distinguish specific requests within a transaction when correlating audit events from multiple services.

    To trust transactions, set the advanced global server property, trust-transaction-ids:true:

    $ dsconfig \
 set-global-configuration-prop \
 --advanced \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --set trust-transaction-ids:true \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

  2. Edit the default HTTP access log publisher as necessary.

    The following example enables the default log publisher for DS installed locally, not in a Docker image:

    $ dsconfig \
 set-log-publisher-prop \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --publisher-name "Json File-Based HTTP Access Logger" \
 --set enabled:true \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

CSV format

A CSV handler sends messages to a comma-separated variable (CSV) file.

The interface stability of this feature is Deprecated.

The CSV handler does not sanitize messages when writing to CSV log files.

Do not open CSV logs in spreadsheets and other applications that treat data as code.

The default CSV HTTP access log file is logs/http-access.csv:

  1. Decide whether to trust transaction IDs sent by client applications, used to correlate requests as they traverse multiple servers.

    Client applications using the ForgeRock Common Audit event framework send transaction IDs with their requests. The transaction IDs correlate audit events, tracing the request through multiple applications.

    Transaction IDs are sent over LDAP using an internal DS request control. They are sent over HTTP in an HTTP header.

    By default, DS servers do not trust transaction IDs sent with client application requests.

    When a server trusts transaction IDs from client application requests, outgoing requests reuse the incoming ID. For each outgoing request in the transaction, the request’s transaction ID has the form original-transaction-id/sequence-number, where sequence-number reflects the position of the request in the series of requests for this transaction. For example, if the original-transaction-id is abc123, the first outgoing request has the transaction ID abc123/0, the second abc123/1, the third abc123/2, and so on. This lets you distinguish specific requests within a transaction when correlating audit events from multiple services.

    To trust transactions, set the advanced global server property, trust-transaction-ids:true:

    $ dsconfig \
 set-global-configuration-prop \
 --advanced \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --set trust-transaction-ids:true \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

  2. Create an enabled CSV file HTTP access logger with optional rotation and retention policies:

    $ dsconfig \
 create-log-publisher \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --publisher-name "Common Audit Csv File HTTP Access Logger" \
 --type csv-file-http-access \
 --set enabled:true \
 --set "rotation-policy:24 Hours Time Limit Rotation Policy" \
 --set "rotation-policy:Size Limit Rotation Policy" \
 --set "retention-policy:File Count Retention Policy" \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

  3. For tamper-evident logs, follow these steps.

    Tamper-evident logging relies on digital signatures and regularly flushing messages to the log system. In high-volume directory deployments with heavy access patterns, signing log messages has a severe negative impact on server performance, reducing throughput by orders of magnitude.

    Be certain to test the performance impact with realistic access patterns for your deployment before enabling the feature in production.

    1. Prepare a keystore.

      For details, refer to Make tampering evident.

    2. Enable the tamper-evident capability:

      $ dsconfig \
 set-log-publisher-prop \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --publisher-name "Common Audit Csv File HTTP Access Logger" \
 --set tamper-evident:true \
 --set key-store-file:config/audit-keystore \
 --set key-store-pin:"&{audit.keystore.pin}" \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

      In this example, AUDIT_KEYSTORE_PIN is an environment variable containing the keystore PIN.

Standard HTTP format

For HTTP requests, you can configure an access logger that uses the Extended Log File Format, a W3C working draft. The default log file is logs/http-access:

  1. Enable the standard format HTTP access logger:

    $ dsconfig \
 set-log-publisher-prop \
 --hostname localhost \
 --port 4444 \
 --bindDN uid=admin \
 --bindPassword password \
 --publisher-name "File-Based HTTP Access Logger" \
 --set enabled:true \
 --usePkcs12TrustStore /path/to/opendj/config/keystore \
 --trustStorePassword:file /path/to/opendj/config/keystore.pin \
 --no-prompt

    The following example shows an excerpt of an HTTP access log with space reformatted:

    - <client-ip> bjensen  <datestamp> GET  /users/bjensen HTTP/1.1 200 <user-agent> 3  40
- <client-ip> bjensen  <datestamp> GET  /users/scarter HTTP/1.1 200 <user-agent> 4   9
- <client-ip> -        <datestamp> GET  /users/missing HTTP/1.1 401 <user-agent> 5   0
- <client-ip> kvaughan <datestamp> POST /users         HTTP/1.1 200 <user-agent> 6 120

    Missing values are replaced with -. Tabs separate the fields, and if a field contains a tab character, then the field is surrounded with double quotes. DS software repeats double quotes in the field to escape them.

    Configure the log-format property to set the fields. The default fields are shown here in the order they occur in the log file:

    Field Description

    cs-host

    Client hostname.

    c-ip

    Client IP address.

    cs-username

    Username used to authenticate.

    x-datetime

    Completion timestamp for the HTTP request.

    Configure with the log-record-time-format property.

    cs-method

    HTTP method requested by the client.

    cs-uri

    URI requested by the client.

    cs-uri-stem

    URL-encoded path requested by the client.

    cs-uri-query

    URL-encoded query parameter string requested by the client.

    cs-version

    HTTP version requested by the client.

    sc-status

    HTTP status code for the operation.

    cs(User-Agent)

    User-Agent identifier.

    x-connection-id

    Connection ID used for DS internal operations.

    When using this field to match HTTP requests with internal operations in the LDAP access log, set the access log advanced property, suppress-internal-operations:false. By default, internal operations do not appear in the LDAP access log.

    x-etime

    Execution time in milliseconds needed by DS to service the HTTP request.

    x-transaction-id

    ForgeRock Common Audit event framework transaction ID for the request.

    This defaults to 0, unless you configure the server to trust transaction IDs.

    The following additional fields are supported:

    Field Description

    c-port

    Client port number.

    s-computername

    Server name writing the access log.

    s-ip

    Server IP address.

    s-port

    Server port number.