PingAM 7.5.1

Monitoring metrics

This section describes the monitoring metrics for common REST, JMX, or Graphite interfaces as well as the Prometheus monitoring metrics.

Metric types

The following metric types are available.

Summary

The summary metric samples observations, providing a count of observations, sum total of observed amounts, average rate of events, and moving average rates across sliding time windows.

Field Description

_id

The metric ID.

_type

The metric type.

count

The number of events recorded for this metric.

total

The sum of the values of events recorded for this metric.

As the increment is always 1, the total and the count are always equal.

m1_rate

The one-minute average rate.

m5_rate

The five-minute average rate.

m15_rate

The fifteen-minute average rate.

mean_rate

The average rate.

units

A description of the units the metric is presented in.

Example

{
  "_id" : "authentication.success",
  "_type" : "summary",
  "count" : 2,
  "total" : 2.0,
  "m1_rate" : 3.2668341885586836E-14,
  "m5_rate" : 7.794695663154025E-5,
  "m15_rate" : 0.01377545747021923,
  "mean_rate" : 8.238608027596704E-4,
  "units" : "events/second"
}
json

Timer

The timer metric combines rate and duration information.

Field Description

_id

The metric ID.

_type

The metric type.

count

The number of events recorded for this metric.

total

The sum of the durations recorded for this metric.

min

The minimum duration recorded for this metric.

max

The maximum duration recorded for this metric.

mean

The mean average duration recorded for this metric.

stddev

The standard deviation of durations recorded for this metric.

duration_units

The units used for measuring the durations in the metric.

p50

50% of the durations recorded are at or below this value.

p75

75% of the durations recorded are at or below this value.

p95

95% of the durations recorded are at or below this value.

p98

98% of the durations recorded are at or below this value.

p99

99% of the durations recorded are at or below this value.

p999

99.9% of the durations recorded are at or below this value.

m1_rate

The one-minute average rate.

m5_rate

The five-minute average rate.

m15_rate

The fifteen-minute average rate.

mean_rate

The average rate.

rate_units

The units used for measuring the rate of the metric.

Duration-based values, such as min, max, and p50, are weighted towards newer data. By representing approximately the last five minutes of data, the timers make it easier to see recent changes in behavior, rather than a uniform average of recordings since the server was started.

Example

{
  "_id" : "cts.connection.success",
  "_type" : "timer",
  "count" : 486,
  "total" : 80.0,
  "min" : 0.0,
  "max" : 1.0,
  "mean" : 0.1905615495053855,
  "stddev" : 0.39274399467782056,
  "duration_units" : "milliseconds",
  "p50" : 0.0,
  "p75" : 0.0,
  "p95" : 1.0,
  "p98" : 1.0,
  "p99" : 1.0,
  "p999" : 1.0,
  "m1_rate" : 0.1819109974890356,
  "m5_rate" : 0.05433445522996721,
  "m15_rate" : 0.03155662103953588,
  "mean_rate" : 0.020858521722211427,
  "rate_units" : "calls/second"
}
json

Gauge

The gauge metric is a numerical value that can increase or decrease. The value for a gauge is calculated when requested, and represents the state of the metric at that specific time.

Field Description

_id

The metric ID.

_type

The metric type.

value

The current value of the metric.

Example

{
  "_id" : "jvm.used-memory",
  "_type" : "gauge",
  "value" : 2.13385216E9
}
json

Distinct counter

Metric providing an estimate of the number of unique values recorded.

For example, this could be used to estimate the number of unique users who have authenticated, or unique client IP addresses.

The DistinctCounter metric is calculated per instance of AM, and can’t be aggregated across multiple instances to get a site-wide view.

Field Description

_id

The metric ID.

_type

The metric type. Note that the distinctCounter type is reported as a gauge type. The output formats are identical.

value

The calculated estimate of the number of unique values recorded in the metric.

Example

{
  "_id" : "authentication.unique-uuid.success",
  "_type" : "gauge",
  "value" : 3.0
}
json

Authentication metrics

AM exposes the following authentication-related monitoring metrics:

Name Type Description

authentication.outcome

Summary

Rate of successful/unsuccessful/timed-out authentication flows.

authentication.module.module.outcome

Summary

Rate of successful/unsuccessful authentication module outcomes.

authentication.unique-uuid.success

DistinctCounter

Count of unique identities that have successfully logged in.

Authentication metrics labels
Label Values

module

  • Application

  • DataStore

outcome

  • success

  • failure

  • timeout

Authorization metrics

AM exposes the following authorization-related monitoring metrics after a policy evaluation takes place:

Name Type Description

authorization.policy-set.policy-set.evaluate.action.action.outcome

Summary

Rate of policy evaluation allowed/denied actions returned under a given policy set

authorization.policy-set.policy-set.evaluate.advice.advice-type

Summary

Rate of policy evaluation advice types returned under a given policy set.

authorization.policy-set.evaluate.subject-cache.size

Gauge

Number of cached subject membership relationships.

authorization.policy-set.policy-set.evaluate.outcome

Timer

Rate of successful/unsuccessful policy evaluation calls under a given policy set and time taken to perform this operation.

authorization.policy-set.policy-set.policy.policy-operation

Gauge

Number of policies created/updated/deleted under a given policy set since this AM instance started.

Authorization metrics labels
Label Values

action

Name of the action as specified in the policy, for example:

  • GET

  • GRANT

  • MODIFY

  • DELEGATE

  • READ

advice-type

Name of the policy condition advice, for example:

  • AuthSchemeConditionAdvice

  • AuthenticateToServiceConditionAdvice

  • AuthLevelConditionAdvice

outcome

  • success

  • allow

policy-operation

Type of operation performed on the policy, for example:

  • create

  • delete

  • update

policy-set

Name of the policy set, for example:

  • iPlanetAMWebAgentService

  • oauth2Scopes

quantile

Refer to Timer for quantile values.

Denylisting metrics

AM exposes the following denylisting monitoring metrics:

Name Type Description

denylist type.blacklist.bloomfilter.check.outcome

Summary

Rate of bloom filter denylist checks.

denylist type.blacklist.cts.search.outcome

Timer

Tracks time to search CTS for denylist entries

denylist type.blacklist.cts.search.result

Summary

Rate of denylist entries returned by searches.

denylist type.blacklist.cache.check.cache outcome

Summary

Rate of cache hits/misses of the denylist cache layer.

denylist type.blacklist.check.check outcome

Summary

Rate of denylist checks.

Denylisting metrics labels
Label Values

denylist type

  • session_client_based

  • oauth2

outcome

  • success

  • failure

cache outcome

  • hit

  • miss

check outcome

  • true The token is denylisted

  • false The token is not denylisted

quantile

Refer to Timer for quantile values.

CTS metrics

AM exposes the following CTS-related monitoring metrics:

Name Type Description

cts.connection.outcome

Timer

Rate of successful/unsuccessful CTS connections to DS and time taken to obtain the connection.

cts.connection.state.status

Counter

The number of connections in each state.

cts.reaper.cache.size

Gauge

Number of entries in the token reaper cache.

cts.reaper.cache.cts reaper token type.deletion.outcome

Summary

Rate of successful/unsuccessful token deletions from cache by token type.

cts.reaper.search.cts reaper token type.deletion.outcome

Summary

Rate of successful/unsuccessful token deletions from search by token type.

cts.reaper.search.outcome

Timer

Rate of successful/unsuccessful searches and time taken to perform this operation.

cts.task.cts reaper token type.operation.outcome

Summary

Rate of successful/unsuccessful CTS operation types, by token type and time taken to perform them.

CTS metrics labels
Label Values

cts reaper token type

  • cluster-notification

  • session

operation

  • create

  • delete

  • partial-query

  • patch

  • query

  • read

  • update

  • upsert

outcome

  • success

  • failure

quantile

Refer to Timer for quantile values.

status

  • out

  • pending

token_type

  • authentication-whitelist

  • back-channel-authentication-state

  • cluster-notification

  • logout-user

  • oauth2-blacklist

  • oauth2-csrf-protection

  • oauth2-grant-set

  • oauth2-stateless-grant

  • oauth2-stateless

  • oauth2

  • push-notification

  • request-uri-object

  • resource-set

  • rest

  • saml2

  • session-blacklist

  • session

  • sts

  • suspended-auth-session

  • transaction

  • uma-audit-entry

  • uma-pending-request

  • uma-permission-ticket

  • uma-requesting-party

  • unknown

JVM metrics

AM exposes the JVM-related monitoring metrics covered in this section.

These metrics may depend on the JVM version and configuration. In particular, garbage-collector-related metrics depend on the garbage collector that the server uses. The garbage-collector metric names are unstable, and can change even in a minor JVM release.

The following JVM metrics are all Gauge metrics.

Name Description

jvm.available-cpus

Number of processors available to the Java virtual machine.

jvm.class-loading.loaded

Number of classes loaded since the Java virtual machine started.

jvm.class-loading.unloaded

Number of classes unloaded since the Java virtual machine started.

jvm.free-used-memory

Amount of free memory.

jvm.used-memory

Amount of used memory.

jvm.max-memory

Maximum amount of memory.

jvm.garbage-collector.Copy.count

Number of collections performed by the "copy" garbage collection algorithm.

jvm.garbage-collector.Copy.time

Approximate accumulated time taken by the "copy" garbage collection algorithm.

jvm.garbage-collector.MarkSweepCompact.count

Number of collections performed by the "mark sweep" garbage collection algorithm.

jvm.garbage-collector.MarkSweepCompact.time

Approximate accumulated time taken by the "mark sweep" garbage collection algorithm.

jvm.memory-usage.heap.init

Amount of heap memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.heap.max

Maximum amount of heap memory the Java virtual machine attempts to use.

jvm.memory-usage.heap.committed

Amount of heap memory committed for the Java virtual machine to use.

jvm.memory-usage.heap.used

Amount of heap memory used by the Java virtual machine.

jvm.memory-usage.heap.usage

Percentage of heap memory used out of the maximum available.

jvm.memory-usage.non-heap.init

Amount of non-heap memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.non-heap.max

Maximum amount of non-heap memory the Java virtual machine attempts to use.

jvm.memory-usage.non-heap.committed

Amount of non-heap memory that is committed for the Java virtual machine to use.

jvm.memory-usage.non-heap.used

Amount of non-heap memory used by the Java virtual machine.

jvm.memory-usage.non-heap.usage

Percentage of non-heap memory used out of the maximum available.

jvm.memory-usage.pools.Compressed-Class-Space.init

Amount of "compressed class space" memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.Compressed-Class-Space.max

Maximum amount of "compressed class space" memory the Java virtual machine attempts to use.

jvm.memory-usage.pools.Compressed-Class-Space.committed

Amount of "compressed class space" memory committed for the Java virtual machine to use.

jvm.memory-usage.pools.Compressed-Class-Space.used

Amount of "compressed class space" memory used by the Java virtual machine.

jvm.memory-usage.pools.Compressed-Class-Space.usage

Percentage of "compressed class space" memory used out of the maximum available.

jvm.memory-usage.pools.CodeHeap-'non-nmethods'.init

Amount of CodeHeap "non-nmethods" memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.CodeHeap-'non-nmethods'.max

Maximum amount of CodeHeap "non-nmethods" memory the Java virtual machine attempts to use.

jvm.memory-usage.pools.CodeHeap-'non-nmethods'.committed

Amount of CodeHeap "non-nmethods" memory committed for the Java virtual machine to use.

jvm.memory-usage.pools.CodeHeap-'non-nmethods'.used

Amount of CodeHeap "non-nmethods" memory used by the Java virtual machine.

jvm.memory-usage.pools.CodeHeap-'non-nmethods'.usage

Percentage of CodeHeap "non-nmethods" memory used out of the maximum available.

jvm.memory-usage.pools.CodeHeap-'non-profiled-nmethods'.init

Amount of CodeHeap "non-profiled-nmethods" memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.CodeHeap-'non-profiled-nmethods'.max

Maximum amount of CodeHeap "non-profiled-nmethods" memory the Java virtual machine attempts to use.

jvm.memory-usage.pools.CodeHeap-'non-profiled-nmethods'.committed

Amount of CodeHeap "non-profiled-nmethods" memory committed for the Java virtual machine to use.

jvm.memory-usage.pools.CodeHeap-'non-profiled-nmethods'.used

Amount of CodeHeap "non-profiled-nmethods" memory used by the Java virtual machine.

jvm.memory-usage.pools.CodeHeap-'non-profiled-nmethods'.usage

Percentage of CodeHeap "non-profiled-nmethods" memory used out of the maximum available.

jvm.memory-usage.pools.CodeHeap-'profiled-nmethods'.init

Amount of CodeHeap "profiled-nmethods" memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.CodeHeap-'profiled-nmethods'.max

Maximum amount of CodeHeap "profiled-nmethods" memory the Java virtual machine attempts to use.

jvm.memory-usage.pools.CodeHeap-'profiled-nmethods'.committed

Amount of CodeHeap "profiled-nmethods" memory committed for the Java virtual machine to use.

jvm.memory-usage.pools.CodeHeap-'profiled-nmethods'.used

Amount of CodeHeap "profiled-nmethods" memory used by the Java virtual machine.

jvm.memory-usage.pools.CodeHeap-'profiled-nmethods'.usage

Percentage of CodeHeap "profiled-nmethods" memory used out of the maximum available.

jvm.memory-usage.pools.Metaspace.init

Amount of "metaspace" memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.Metaspace.max

Maximum amount of "metaspace" memory the Java virtual machine attempts to use.

jvm.memory-usage.pools.Metaspace.committed

Amount of "metaspace" memory committed for the Java virtual machine to use.

jvm.memory-usage.pools.Metaspace.used

Amount of "metaspace" memory used by the Java virtual machine.

jvm.memory-usage.pools.Metaspace.usage

Percentage of "metaspace" memory used out of the maximum available.

jvm.memory-usage.pools.Eden-Space.init

Amount of "eden space" memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.Eden-Space.max

Maximum amount of "eden space" memory (young generation) the Java virtual machine attempts to use.

jvm.memory-usage.pools.Eden-Space.committed

Amount of "eden space" memory (young generation) committed for the Java virtual machine to use.

jvm.memory-usage.pools.Eden-Space.used-after-gc

Amount of "eden space" memory (young generation) after the last time garbage collection recycled unused objects in this memory pool.

jvm.memory-usage.pools.Eden-Space.used

Amount of "eden space" memory (young generation) used by the Java virtual machine.

jvm.memory-usage.pools.Eden-Space.usage

Percentage of "eden space" memory (young generation) used out of the maximum available.

jvm.memory-usage.pools.Tenured-Gen.init

Amount of "tenured generation" (old generation) memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.Tenured-Gen.max

Maximum amount of "tenured generation" (old generation) memory the Java virtual machine attempts to use.

jvm.memory-usage.pools.Tenured-Gen.committed

Amount of "tenured generation" memory (old generation) committed for the Java virtual machine to use.

jvm.memory-usage.pools.Tenured-Gen.used-after-gc

Amount of "tenured generation" memory (old generation) after the last time garbage collection recycled unused objects in this memory pool.

jvm.memory-usage.pools.Tenured-Gen.used

Amount of "tenured generation" memory (old generation) used by the Java virtual machine.

jvm.memory-usage.pools.Tenured-Gen.usage

Percentage of "tenured generation" memory (old generation) used out of the maximum available.

jvm.memory-usage.pools.Survivor-Space.init

Amount of "survivor space" memory (young generation) the Java virtual machine initially requested from the operating system.

jvm.memory-usage.pools.Survivor-Space.max

Maximum amount of "survivor space" memory (young generation) the Java virtual machine attempts to use.

jvm.memory-usage.pools.Survivor-Space.committed

Amount of "survivor space" memory (young generation) committed for the Java virtual machine to use.

jvm.memory-usage.pools.Survivor-Space.used-after-gc

Amount of "survivor space" memory (young generation) after the last time garbage collection recycled unused objects in this memory pool.

jvm.memory-usage.pools.Survivor-Space.used

Amount of "survivor space" memory (young generation) used by the Java virtual machine.

jvm.memory-usage.pools.Survivor-Space.usage

Percentage of "survivor space" memory (young generation) used out of the maximum available.

jvm.memory-usage.total.committed

Amount of memory committed for the Java virtual machine to use.

jvm.memory-usage.total.init

Amount of memory the Java virtual machine initially requested from the operating system.

jvm.memory-usage.total.max

Maximum amount of memory the Java virtual machine attempts to use.

jvm.memory-usage.total.used

Amount of memory used by the Java virtual machine.

jvm.thread-state.blocked.count

Number of threads in the BLOCKED state.

jvm.thread-state.count

Number of live threads including both daemon and non-daemon threads.

jvm.thread-state.daemon.count

Number of live daemon threads.

jvm.thread-state.new.count

Number of threads in the NEW state.

jvm.thread-state.runnable.count

Number of threads in the RUNNABLE state.

jvm.thread-state.terminated.count

Number of threads in the TERMINATED state.

jvm.thread-state.timed_waiting.count

Number of threads in the TIMED_WAITING state.

jvm.thread-state.waiting.count

Number of threads in the WAITING state.

OAuth 2.0 metrics

AM exposes the following OAuth 2.0 monitoring metrics:

Name Type Description

oauth2.grant.grant-type

Summary

Rate of OAuth 2.0 grant completion by grant type.

oauth2.grant.revoke

Summary

Rate of OAuth 2.0 grant revocation

oauth2.token.token-type.issue

Summary

Rate of OAuth 2.0 token issuance by token type.

oauth2.token.access-token.revoke

Summary

Rate of OAuth 2.0 access token revocation.

oauth2.token.read-as-jwt.outcome

Timer

Rate of successfully/unsuccessfully reading OAuth 2.0 JSON Web Tokens (JWT).

OAuth 2.0 metrics labels
Label Values

grant-type

  • authorization-code

  • back-channel

  • client-credentials

  • device-code

  • implicit

  • jwt-bearer

  • refresh

  • resource-owner-password

  • saml2

  • token-exchange

  • uma2

outcome

  • success

  • failure

token-type

  • access-token

  • authorization-code

  • device-code

  • id-token

  • ops

  • permission-ticket

  • refresh-token

Session metrics

AM exposes the following session-related monitoring metrics:

Name Type Description

session.session-type.session-operation.outcome

Timer

Rate of successful/unsuccessful outcomes for this particular operation type and the time taken to perform this operation.

session.session-type.lifetime

Timer

Rate of session lifetimes.

session.authentication-in-memory.store.size

Gauge

Number of authentication sessions stored in the in-memory authentication session store.

session.cts-based.cache.eviction

Summary

Rate of evictions from the session cache. (Summary)

session.cts-based.cache.session-outcome

Summary

Rate of cache hits/misses for the session cache.

session.cts-based.cache.size

Gauge

Number of sessions in the session cache.

Session 2.0 metrics labels
Label Values

operation

  • check-exists

  • create

  • dereference-restricted-token-id

  • destroy

  • get-matching-sessions

  • get-restricted-token-id

  • get-valid-sessions

  • is-applicable

  • logout

  • refresh

  • register-listener

  • register-pll-listener

  • resolve

  • set-external-property

  • set-property

  • validate

outcome

  • success

  • failure

`session-operation

  • idle-timeout

  • max-timeout

session-outcome

  • hit

  • miss

session-type

  • authentication-client-based

  • authentication-cts-based

  • authentication-in-memory

  • client-based

  • cts-based