Retrieve log entries using REST API
| This method retrieves logs directly by querying an API endpoint and polling Advanced Identity Cloud for new entries. For real-time log streaming to an external monitoring tool or SIEM, you can export logs to an external monitoring tool. |
You can retrieve log events directly from the /monitoring/logs REST API endpoint. With this method, you access each of your tenant environments individually and use REST API filters to refine log results.
Retrieve the log entries
To retrieve the stored Advanced Identity Cloud log entries for a source, use the /monitoring/logs endpoint, specifying the source as a parameter.
Advanced Identity Cloud stores logs for 30 days.
| You need to get an API key and secret before you can authenticate to the endpoint. |
Example request:
$ curl -G \
--request GET 'https://<tenant-env-fqdn>/monitoring/logs' \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-authentication'Example response:
{
"result": [{
"payload": "<payload>",
"timestamp": "<dateTime>",
"type": "application/json",
"source": "am-authentication"
}, {
"...": "..."
}],
"resultCount": "1000",
"pagedResultsCookie": "<pagedResultsCookie>",
"totalPagedResultsPolicy": "NONE",
"totalPagedResults": -1,
"remainingPagedResults": -1
}Advanced Identity Cloud returns the available log entries in the result array. Results are in JSON format or
plaintext, depending on the source you request.
To reduce the size of the output, log query results are by default restricted to the last 24 hours, unless you add beginTime and/or endTime query parameters. Learn more in Get log results for a time period.
Get log results for a time period
Use the beginTime and endTime query parameters to return entries created between two ISO 8601 formatted times.
Example request:
$ curl -G \
--request GET 'https://<tenant-env-fqdn>/monitoring/logs' \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-authentication' \
--data 'beginTime=2023-03-01T12:45:00Z' \
--data 'endTime=2023-03-01T12:50:00Z'The beginTime and endTime query parameters are subject to these rules:
-
If
endTimeis not specified, it defaults to the current time. -
If
beginTimeis not specified, it defaults to 24 hours beforeendTime. -
If
beginTimeis specified, it must be 24 hours or less beforeendTime.
Tail logs
To tail, or get the latest entries in the stored logs for a source, use the
/monitoring/logs/tail endpoint with the source as a parameter.
The first call to the tail endpoint returns log entries from the last 15 seconds. Subsequent calls return log entries in a range that starts from the last returned log entry in the previous result (inclusive) and ends with the latest log entry but one. If calls to the tail endpoint are not frequent enough to match the rate at which the log entries are produced, the result may not include all available log entries.
The format of the log results depends on the source or sources specified in your request. Some
sources return only JSON formatted log entries and some sources return only plaintext log entries.
Some sources, such as am-everything, can return log entries in both formats.
Example request:
$ curl -G \
--request GET 'https://<tenant-env-fqdn>/monitoring/logs/tail' \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-everything'Example response:
{
"result": [{
"payload": "<payload>",
"timestamp": "<dateTime>",
"type": "<type>",
"source": "am-core"
}, {
"...": "..."
}],
"resultCount": "100",
"pagedResultsCookie": "<pagedResultsCookie>",
"totalPagedResultsPolicy": "NONE",
"totalPagedResults": -1,
"remainingPagedResults": -1
}You can specify multiple sources in a single call. Example request:
$ curl -G \
--request GET 'https://<tenant-env-fqdn>/monitoring/logs/tail' \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-access,idm-access,idm-sync,idm-activity'|
To keep tailing, pass the Example request: |
View logs for a specific request
All log events for an external request into Advanced Identity Cloud are assigned the same unique transaction ID.
The x-forgerock-transactionid response header holds the transaction ID:
$ curl \
--request POST 'https://<tenant-env-fqdn>/am/json/realms/root/realms/alpha/authenticate' \
--include \
--header 'Content-Type: application/json' \
--header 'X-OpenAM-Username: bjensen' \
--header 'X-OpenAM-Password: Passw0rd!' \
--header 'Accept-API-Version: resource=2.0, protocol=1.0' \
...
x-forgerock-transactionid: <transaction-id>
...To filter the logs for a specific transaction ID, add the transactionId parameter to your API request; for example:
$ curl -G \
--request GET 'https://<tenant-env-fqdn>/monitoring/logs' \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-authentication' \
--data 'transactionId=<transaction-id>'Example response:
{
"result": [{
"payload": "<payload>",
"timestamp": "<dateTime>",
"type": "application/json",
"source": "am-authentication"
}, {
"...": "..."
}],
"resultCount": "8",
"pagedResultsCookie": null,
"totalPagedResultsPolicy": "NONE",
"totalPagedResults": -1,
"remainingPagedResults": -1
}Filter log results
Use the _queryFilter parameter to filter log results on any field or combination of
fields in a payload. You can add the parameter to the /monitoring/logs and
/monitoring/logs/tail endpoints.
The benefits of the _queryFilter parameter are:
-
Lets you iteratively refine queries to remove extraneous results and find the specific log entries you are interested in. This is useful when searching logs to debug a production issue.
Use the /monitoring/logsendpoint for iterative searching as the/monitoring/logs/tailendpoint only returns results from the last 15 seconds. -
Lets you tune queries to reduce Advanced Identity Cloud log volume, making integration with external log tools such as Splunk or Elastic Stack more efficient and potentially reducing storage costs.
The _queryFilter parameter takes a URL-encoded filter expression:
$ curl -G \
--request GET 'https://<tenant-env-fqdn>/monitoring/logs' \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=<source>' \
--data-urlencode '_queryFilter=<filter-expression>'Learn more about constructing a filter expression in the
filter expression rules for _queryFilter.
Here are some basic examples:
| Example filter expression | Description |
|---|---|
|
Search plaintext results for a particular string. |
|
Search for JSON results containing a particular client IP address. |
|
Search for JSON results with a particular log level. |
|
Search for JSON results with a particular event name. |
|
Search for JSON results containing a particular event name. |
|
Search for JSON results with a particular timestamp. |
|
Search for JSON results with a timestamp that starts with a particular datetime. |
|
Search for JSON results containing a particular client IP address and also containing a particular log level. |
|
Search for JSON results where an authentication node type is present. |
Filter array items in log results
To filter on array items, do not include an array index in your filter expression.
For example, to search for JSON results where the authentication node type is ScriptedDecisionNode:
-
Wrong:
/payload/entries/0/info/nodeType eq "ScriptedDecisionNode" -
Right:
/payload/entries/info/nodeType eq "ScriptedDecisionNode"
where a log entry for an authentication node looks like this:
{
"payload": {
"_id": "7ae37a4b-f22b-4c5e-8621-2130d5bc603c-9310858",
"component": "Authentication",
"entries": [
{
"info": {
"authLevel": "0",
"displayName": "Using Invite?",
"nodeId": "15edd2f7-22f1-4f32-bf0a-8ca3f98af850",
"nodeOutcome": "False",
"nodeType": "ScriptedDecisionNode",
"treeName": "Login"
}
}
],
"eventName": "AM-NODE-LOGIN-COMPLETED",
...
}Filter log results between two dates
To filter log results between two dates, use the beginTime and endTime query parameters with
ISO 8601 datetime values:
$ curl -G \
--request GET 'https://<tenant-env-fqdn>/monitoring/logs' \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=<source>' \
--data 'beginTime=<begin-datetime>' \
--data 'endTime=<end-datetime>' \
--data-urlencode '_queryFilter=<filter-expression>'For example, to filter log results between two specific dates for a specific user :
$ curl -G \
--request GET 'https://<tenant-env-fqdn>/monitoring/logs' \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-authentication' \
--data 'beginTime=2023-05-24T12:40:00.00Z' \
--data 'endTime=2023-05-24T12:45:00.00Z' \
--data-urlencode '_queryFilter=/payload/principal eq "user.name@example.com"'Add response fields
Authentication events
You can use a script to add extra information to log entries for authentication events. Learn more in Add information to authentication audit log entries.
Identity object events
You can configure audit log results to include additional identity object events. For example, you may want to log the before and after values of specific activities, such as changes to a user’s last name or email address.
To include additional identity object event fields, add them to the includeIf property in the audit configuration. Make these changes in your development environment and then promote them.
| By default, Advanced Identity Cloud audits identity object event fields that are safe to log. When adding audit event fields, be mindful of the type of information that you intend to expose in the logs. For example, you may need to keep personally identifiable information (PII) out of the logs. |
Add identity object event fields to audit logging
-
Get the current audit configuration.
Example request:
$ curl \ --request GET \ --header 'Authorization: Bearer <access-token>' \ --header 'Content-Type: application/json' \ 'https://<tenant-env-fqdn>/openidm/config/audit'Learn more in the IDM REST API reference.
Show response
{ "_id": "audit", "auditServiceConfig": { "availableAuditEventHandlers": [ "org.forgerock.audit.handlers.csv.CsvAuditEventHandler", "org.forgerock.audit.handlers.elasticsearch.ElasticsearchAuditEventHandler", "org.forgerock.audit.handlers.jms.JmsAuditEventHandler", "org.forgerock.audit.handlers.json.JsonAuditEventHandler", "org.forgerock.audit.handlers.json.stdout.JsonStdoutAuditEventHandler", "org.forgerock.openidm.audit.impl.RouterAuditEventHandler", "org.forgerock.audit.handlers.splunk.SplunkAuditEventHandler", "org.forgerock.audit.handlers.syslog.SyslogAuditEventHandler" ], "caseInsensitiveFields": [ "/access/http/request/headers", "/access/http/response/headers" ], "filterPolicies": { "value": { "excludeIf": [ "/access/http/request/cookies/&{com.iplanet.am.cookie.name}", "/access/http/request/cookies/session-jwt", "/access/http/request/headers/&{com.sun.identity.auth.cookieName}", "/access/http/request/headers/&{com.iplanet.am.cookie.name}", "/access/http/request/headers/accept-encoding", "/access/http/request/headers/accept-language", "/access/http/request/headers/Authorization", "/access/http/request/headers/cache-control", "/access/http/request/headers/connection", "/access/http/request/headers/content-length", "/access/http/request/headers/content-type", "/access/http/request/headers/proxy-authorization", "/access/http/request/headers/X-OpenAM-Password", "/access/http/request/headers/X-OpenIDM-Password", "/access/http/request/queryParameters/access_token", "/access/http/request/queryParameters/IDToken1", "/access/http/request/queryParameters/id_token_hint", "/access/http/request/queryParameters/Login.Token1", "/access/http/request/queryParameters/redirect_uri", "/access/http/request/queryParameters/requester", "/access/http/request/queryParameters/sessionUpgradeSSOTokenId", "/access/http/request/queryParameters/tokenId", "/access/http/response/headers/Authorization", "/access/http/response/headers/Set-Cookie", "/access/http/response/headers/X-OpenIDM-Password" ], "includeIf": [] } }, "handlerForQueries": "json" }, "eventHandlers": [ { "class": "org.forgerock.audit.handlers.json.stdout.JsonStdoutAuditEventHandler", "config": { "name": "json", "topics": [ "access", "activity", "sync", "authentication", "config" ] } } ], "eventTopics": { "activity": { "filter": { "actions": [ "create", "update", "delete", "patch", "action" ] }, "passwordFields": [ "password" ], "watchedFields": [] }, "config": { "filter": { "actions": [ "create", "update", "delete", "patch", "action" ] } } }, "exceptionFormatter": { "file": "bin/defaults/script/audit/stacktraceFormatter.js", "type": "text/javascript" } } -
Make a backup of the audit configuration.
-
Update the
includeIfproperty (underfilterPolicies) in the audit configuration to include the fields you want to add.The following example updates the audit configuration to include before and after values of a user’s last name and email address:
$ curl \ --request PUT \ --header 'Authorization: Bearer <access-token>' \ --header 'Content-Type: application/json' \ --data-raw ' { "_id": "audit", ... "filterPolicies": { "value": { "excludeIf": [ "/access/http/request/cookies/&{com.iplanet.am.cookie.name}", "/access/http/request/cookies/session-jwt", ... ], "includeIf": [ "/activity/before/sn", (1) "/activity/after/sn", (2) "/activity/before/mail", (3) "/activity/after/mail" (4) ] } }, ... }' \ 'https://<tenant-env-fqdn>/openidm/config/audit'Fields added:
1 Logs the user’s last name before change. 2 Logs the user’s last name after change. 3 Logs the user’s email address before change. 4 Logs user’s email address after change.
Once updated, idm-activity and idm-everything audit logs will include the fields you have added.
For example, the following entry in a sample idm-activity log shows before and after values for changes to a user’s last name and email address from "Brown" to "Granger":
{
"payload": {
"message": "",
"runAs": "bd220328-9762-458b-b05a-982ac3c7fc54",
"transactionId": "1630683558570-abec9e9304c84ad368ba-28676/0",
"before": {
"sn": "Brown",
"mail": "jbrown@example.com"
},
"operation": "PATCH",
"passwordChanged": false,
"_id": "52f7cea0-285d-4ef6-bda3-83256dda71c5-1300250",
"revision": "00000000412cae36",
"eventName": "activity",
"userId": "bd220328-9762-458b-b05a-982ac3c7fc54",
"status": "SUCCESS",
"objectId": "managed/alpha_user/ce7492dc-8759-47b3-b4ee-eda8d4de4ab1",
"timestamp": "2023-09-03T15:39:42.862Z",
"changedFields": [],
"after": {
"sn": "Granger",
"mail": "jgranger@example.com"
}
"type": "application/json",
"timestamp": "<date-time>"
}|
You can also exclude fields from audit logging by adding them to the For example, to prevent audit logs from showing target object attributes for synchronization and reconciliation events, add the following entries to the |
Rate limiting
Logs endpoint
To reduce unwanted stresses on the system, Advanced Identity Cloud limits the number of requests you can make
to the /monitoring/logs endpoint in a certain timeframe:
-
The page-size limit is 1000 logs per request.
Ping Identity recommends you do not override the page-size limit with a greater value as it could increase request throttling and reduce the overall number of logs you can request per minute. -
The request limit is 60 requests per minute.
-
The theoretical upper rate limit is therefore 60,000 logs per minute.
These limits apply per environment, so your development, staging, and production environments each have their own quota.
The following rate limit notification response headers are sent for each request to the
/monitoring/logs endpoint:
- X-RateLimit-Limit
-
The maximum number of requests allowed in the current rate limit window.
- X-RateLimit-Remaining
-
The number of requests remaining in the current rate limit window.
- X-RateLimit-Reset
-
The time in seconds since Jan. 1, 1970, UTC when the rate limit window resets.
A 429 HTTP status code on the /monitoring/logs endpoint indicates that the rate limit has been exceeded, and no results are returned.
Logs tail endpoint
The /monitoring/logs/tail endpoint has the same limits and response headers as the
/monitoring/logs endpoint described above. However, the endpoint also has a limit of 15,000
lines per request, which supersedes the page-size limit of 1000 logs per request.
Because calls to the /monitoring/logs/tail endpoint do not always fetch all logs, use this
endpoint for debugging only. Use the /monitoring/logs endpoint when you need to fetch all logs.
Troubleshooting
Update audit configuration
Sometimes a log source is shown in the available sources in Advanced Identity Cloud but returns no results when you query the Advanced Identity Cloud logging endpoints. In this case, check the underlying IDM audit configuration to ensure that the corresponding audit topic for the source is enabled.
The following example shows how to enable the recon event handler so that reconciliation events
appear in the audit logs:
-
Get the current audit configuration.
Example request:
$ curl \ --request GET \ --header 'Authorization: Bearer <access-token>' \ --header 'Content-Type: application/json' \ 'https://<tenant-env-fqdn>/openidm/config/audit'Learn more in IDM REST API reference.
-
Update the audit configuration as needed. The following example enables the reconciliation audit event handler.
Example update:
$ curl \ --request PUT 'https://<tenant-env-fqdn>/openidm/config/audit' \ --header 'Authorization: Bearer <access-token>' \ --header 'Content-Type: application/json' \ --data-raw ' { "_id": "audit", ... "eventHandlers": [ { "class": "org.forgerock.audit.handlers.json.stdout.JsonStdoutAuditEventHandler", "config": { "elasticsearchCompatible": false, "enabled": true, "name": "json", "topics": [ "access", "activity", "sync", "authentication", "config", "recon" ] } } ], ... }' \ 'https://<tenant-env-fqdn>/openidm/config/audit'
Include large log entries in filter log results
Some Advanced Identity Cloud log output is too large to be stored as a single log entry, so is stored across two log entries instead. When this happens, any log output in JSON format is stored as two plaintext log entries rather than a single JSON log entry. Consequently, any filter expression that filters on a specific JSON field will not find any of these plaintext log entries.
To work around this, you can combine a specific field filter with a plaintext filter. For example, if you were searching for log results containing a particular transaction ID using the filter expression:
/payload/transactionId co "<transaction-id>"
you could add a plaintext filter as follows:
/payload/transactionId co "<transaction-id>" or /payload co "<transaction-id>"
to include both JSON and plaintext log entries in the log results.
Import cURL commands into Postman
If you use Postman’s import cURL commands feature to convert cURL
commands into Postman requests, you might have problems with the example commands on this page that
use a GET request and include a --data parameter, such as this one:
$ curl -G \
--request GET 'https://<tenant-env-fqdn>/monitoring/logs'
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-authentication'When you import the above command into Postman and run the request, it produces this error:
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>400 Bad Request</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Bad Request</h1>
<h2>Your client has issued a malformed or illegal request.</h2>
<h2></h2>
</body></html>This occurs because Postman adds the value of --data parameter twice—as a query string but also
incorrectly as a message body. The equivalent incorrect cURL query of the import would be:
curl \
--request GET 'https://<tenant-env-fqdn>/monitoring/logs?source=am-authentication' \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'source=am-authentication'To resolve, convert any --data parameters into a query string before importing into Postman:
$ curl \
--request 'https://<tenant-env-fqdn>/monitoring/logs?source=am-authentication'
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>'