---
title: Monitoring
description: IDM includes the following tools for monitoring metrics:
component: pingidm
version: 7.3
page_id: pingidm:monitoring-guide:monitoring
canonical_url: https://docs.pingidentity.com/pingidm/8.1/monitoring-guide/monitoring.html
llms_txt: https://docs.pingidentity.com/pingidm/llms.txt
docs_for_agents: https://developer.pingidentity.com/build-with-ai/docs-for-agents.md
superseded_by: https://docs.pingidentity.com/pingidm/8.1/monitoring-guide/monitoring.html
section_ids:
  enable_metrics: Enable metrics
  dropwizard: Dropwizard widget
  prometheus: Prometheus endpoint
  disable_prometheus: Disable Prometheus
  configure-prometheus: Configure Prometheus
  configure-grafana: Configure Grafana
  create_a_grafana_dashboard: Create a Grafana dashboard
---

# Monitoring

IDM includes the following tools for monitoring metrics:

* A Dropwizard dashboard widget, for viewing metrics within IDM.

* A Prometheus endpoint, for viewing metrics through external resources such as Prometheus and Grafana.

## Enable metrics

IDM does not collect metrics by default. To enable metrics collection, open `conf/metrics.json` and set the `enabled` property to `true`:

```json
{
    "enabled" : true
}
```

After you have enabled metrics, the following command returns all collected metrics:

Request

```
curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request GET \
'http://localhost:8080/openidm/metrics/api?_queryFilter=true'
```

> **Collapse: Show example response**
>
> Response
>
> ```json
> {
>   "result": [
>     {
>       "_id": "repo.ds.get-connection",
>       "count": 87,
>       "max": 18.273865999999998,
>       "mean": 2.0228276744117877,
>       "min": 0.64178,
>       "p50": 1.487925,
>       "p75": 2.248557,
>       "p95": 4.115021,
>       "p98": 5.620608,
>       "p99": 8.533398,
>       "p999": 18.273865999999998,
>       "stddev": 1.9332103308881228,
>       "m15_rate": 8.653295680258273,
>       "m1_rate": 5.1162605576082285,
>       "m5_rate": 8.004328242732594,
>       "mean_rate": 1.8443204365017944,
>       "duration_units": "milliseconds",
>       "rate_units": "calls/second",
>       "total": 178.917206,
>       "_type": "timer"
>     },
>     {
>       "_id": "jvm.memory-usage.pools.G1-Old-Gen.committed",
>       "value": 794820608,
>       "_type": "gauge"
>     },
>     {
>       "_id": "jvm.max-memory",
>       "value": 2147483648,
>       "_type": "gauge"
>     },
>     {
>       "_id": "jvm.memory-usage.pools.CodeHeap-'non-profiled-nmethods'.init",
>       "value": 2555904.0,
>       "_type": "gauge"
>     },
>     {
>       "_id": "scheduler.persisted.add-job",
>       "count": 2,
>       "max": 19.246833,
>       "mean": 13.2243465,
>       "min": 7.20186,
>       "p50": 19.246833,
>       "p75": 19.246833,
>       "p95": 19.246833,
>       "p98": 19.246833,
>       "p99": 19.246833,
>       "p999": 19.246833,
>       "stddev": 6.022486499999999,
>       "m15_rate": 0.38261149564121166,
>       "m1_rate": 0.2053668476130369,
>       "m5_rate": 0.350069327617179,
>       "mean_rate": 0.042679419036704344,
>       "duration_units": "milliseconds",
>       "rate_units": "calls/second",
>       "total": 26.448693,
>       "_type": "timer"
>     },
>     {
>       "_id": "jvm.memory-usage.pools.G1-Old-Gen.init",
>       "value": 2.03423744E+9,
>       "_type": "gauge"
>     },
>     {
>       "_id": "internal.user.relationship.validate-relationship-fields",
>       "count": 4,
>       "max": 0.00233,
>       "mean": 0.00192725,
>       "min": 0.00153,
>       "p50": 0.002253,
>       "p75": 0.00233,
>       "p95": 0.00233,
>       "p98": 0.00233,
>       "p99": 0.00233,
>       "p999": 0.00233,
>       "stddev": 0.0003660105018985111,
>       "m15_rate": 0.7652229912824233,
>       "m1_rate": 0.4107336952260738,
>       "m5_rate": 0.700138655234358,
>       "mean_rate": 0.08670015551801163,
>       "duration_units": "milliseconds",
>       "rate_units": "calls/second",
>       "total": 0.007709,
>       "_type": "timer"
>     },
>     {
>       "_id": "jvm.memory-usage.total.max",
>       "value": 2147483647,
>       "_type": "gauge"
>     },
>     {
>       "_id": "jvm.memory-usage.total.committed",
>       "value": 2386423808,
>       "_type": "gauge"
>     },
>     {
>       "_id": "repo.ds.create.internal_user",
>       "count": 4,
>       "max": 18.306962,
>       "mean": 5.7264095,
>       "min": 1.4013419999999999,
>       "p50": 1.6358979999999999,
>       "p75": 18.306962,
>       "p95": 18.306962,
>       "p98": 18.306962,
>       "p99": 18.306962,
>       "p999": 18.306962,
>       "stddev": 7.2638797944374565,
>       "m15_rate": 0.7652229912824233,
>       "m1_rate": 0.4107336952260738,
>       "m5_rate": 0.700138655234358,
>       "mean_rate": 0.08670216068319382,
>       "duration_units": "milliseconds",
>       "rate_units": "calls/second",
>       "total": 22.905638,
>       "_type": "timer"
>     },
>     {
>       "_id": "jvm.memory-usage.heap.init",
>       "value": 2147483648,
>       "_type": "gauge"
>     },
>     {
>       "_id": "repo.ds.update.cluster",
>       "count": 8,
>       "max": 3.679064,
>       "mean": 2.9518007249923834,
>       "min": 2.3344139999999998,
>       "p50": 2.9840329999999997,
>       "p75": 3.1550309999999997,
>       "p95": 3.679064,
>       "p98": 3.679064,
>       "p99": 3.679064,
>       "p999": 3.679064,
>       "stddev": 0.3739151611284433,
>       "m15_rate": 0.3923533692947412,
>       "m1_rate": 0.30850549439674596,
>       "m5_rate": 0.3778195015144161,
>       "mean_rate": 0.2133903215631551,
>       "duration_units": "milliseconds",
>       "rate_units": "calls/second",
>       "total": 23.459754,
>       "_type": "timer"
>     },
>     {
>       "_id": "repo.ds.read.cluster",
>       "count": 8,
>       "max": 2.3707599999999998,
>       "mean": 1.9349245566320208,
>       "min": 1.192601,
>       "p50": 1.9848709999999998,
>       "p75": 2.204475,
>       "p95": 2.3707599999999998,
>       "p98": 2.3707599999999998,
>       "p99": 2.3707599999999998,
>       "p999": 2.3707599999999998,
>       "stddev": 0.3540316157820945,
>       "m15_rate": 0.2,
>       "m1_rate": 0.2,
>       "m5_rate": 0.2,
>       "mean_rate": 0.21335481478933183,
>       "duration_units": "milliseconds",
>       "rate_units": "calls/second",
>       "total": 15.501201,
>       "_type": "timer"
>     },
>     {
>       "_id": "jvm.memory-usage.pools.CodeHeap-'non-nmethods'.usage",
>       "value": 0.3455020215633423,
>       "_type": "gauge"
>     },
>     {
>       "_id": "jvm.memory-usage.pools.Metaspace.init",
>       "value": 0.0,
>       "_type": "gauge"
>     },
>     {
>       "_id": "repo.ds.create.scheduler",
>       "count": 4,
>       "max": 5.671175,
>       "mean": 3.72545225,
>       "min": 1.702539,
>       "p50": 4.15715,
>       "p75": 5.671175,
>       "p95": 5.671175,
>       "p98": 5.671175,
>       "p99": 5.671175,
>       "p999": 5.671175,
>       "stddev": 1.4309134647621733,
>       "m15_rate": 0.7652229912824233,
>       "m1_rate": 0.4107336952260738,
>       "m5_rate": 0.700138655234358,
>       "mean_rate": 0.08536364328879296,
>       "duration_units": "milliseconds",
>       "rate_units": "calls/second",
>       "total": 14.901809,
>       "_type": "timer"
>     },
>     {
>       "_id": "jvm.memory-usage.pools.G1-Survivor-Space.committed",
>       "value": 49283072,
>       "_type": "gauge"
>     },
>     {
>       "_id": "jvm.memory-usage.heap.usage",
>       "value": 0.18285735324025154,
>       "_type": "gauge"
>     },
>     {
>       "_id": "jvm.garbage-collector.G1-Old-Generation.count",
>       "value": 4.0,
>       "_type": "gauge"
>     },
>     {
>       "_id": "jvm.thread-state.waiting.count",
>       "value": 54.0,
>       "_type": "gauge"
>     },
>     {
>       "_id": "jvm.available-cpus",
>       "value": 16.0,
>       "_type": "gauge"
>     },
>     {
>       "_id": "jvm.class-loading.loaded",
>       "value": 22506.0,
>       "_type": "gauge"
>     },
>     {
>       "_id": "null-array-filter.augmentation.read",
>       "count": 10,
>       "max": 0.03385,
>       "mean": 0.0205321,
>       "min": 0.012830999999999999,
>       "p50": 0.022903999999999997,
>       "p75": 0.02393,
>       "p95": 0.03385,
>       "p98": 0.03385,
>       "p99": 0.03385,
>       "p999": 0.03385,
>       "stddev": 0.006245525749686731,
>       "m15_rate": 1.913057478206058,
>       "m1_rate": 1.0268342380651845,
>       "m5_rate": 1.7503466380858956,
>       "mean_rate": 0.21664554977539055,
>       "duration_units": "milliseconds",
>       "rate_units": "calls/second",
>       "total": 0.205321,
>       "_type": "timer"
>     },
>     {
>       "_id": "filter.scripted.on-request.d6fc81179beaca37094a23c2fcd00aaf54bb3ef9:router:onRequest",
>       "count": 1,
>       "max": 27.052286,
>       "mean": 27.052286,
>       "min": 27.052286,
>       "p50": 27.052286,
>       "p75": 27.052286,
>       "p95": 27.052286,
>       "p98": 27.052286,
>       "p99": 27.052286,
>       "p999": 27.052286,
>       "stddev": 0.0,
>       "m15_rate": 0.0,
>       "m1_rate": 0.0,
>       "m5_rate": 0.0,
>       "mean_rate": 30.733696941652937,
>       "duration_units": "milliseconds",
>       "rate_units": "calls/second",
>       "total": 27.052286,
>       "_type": "timer"
>     },
>     {
>       "_id": "repo.ds.read.internal_user",
>       "count": 4,
>       "max": 0.893355,
>       "mean": 0.7506262499999999,
>       "min": 0.667094,
>       "p50": 0.733529,
>       "p75": 0.893355,
>       "p95": 0.893355,
>       "p98": 0.893355,
>       "p99": 0.893355,
>       "p999": 0.893355,
>       "stddev": 0.08575225314058808,
>       "m15_rate": 0.7652229912824233,
>       "m1_rate": 0.4107336952260738,
>       "m5_rate": 0.700138655234358,
>       "mean_rate": 0.08670579236948879,
>       "duration_units": "milliseconds",
>       "rate_units": "calls/second",
>       "total": 3.002505,
>       "_type": "timer"
>     },
>     {
>       "_id": "jvm.memory-usage.pools.CodeHeap-'non-profiled-nmethods'.committed",
>       "value": 7864320.0,
>       "_type": "gauge"
>     },
>     {
>       "_id": "scheduler.job-store.repo.query-list.triggers",
>       "count": 18,
>       "max": 16.820667999999998,
>       "mean": 3.70247311020658,
>       "min": 1.068907,
>       "p50": 2.205507,
>       "p75": 3.3469379999999997,
>       "p95": 15.960484,
>       "p98": 16.820667999999998,
>       "p99": 16.820667999999998,
>       "p999": 16.820667999999998,
>       "stddev": 4.114367764764707,
>       "m15_rate": 1.5402060021782797,
>       "m1_rate": 0.9277075719931202,
>       "m5_rate": 1.4281843370491416,
>       "mean_rate": 0.38150669470151066,
>       "duration_units": "milliseconds",
>       "rate_units": "calls/second",
>       "total": 70.044336,
>       "_type": "timer"
>     },
>     ...
> }
> ```

|   |                                                                                                         |
| - | ------------------------------------------------------------------------------------------------------- |
|   | Metrics are only collected after they have been triggered by activity in IDM, such as a reconciliation. |

Learn more:

* [Metrics reference](metrics.html)

* [Load testing](load-testing.html)

## Dropwizard widget

The Dropwizard widget creates a graph of metrics based on server activity and is useful for lightweight, live monitoring of IDM. The widget has the following limitations:

* The graph created by the widget does not persist. If you reload or navigate away from the page, the graph restarts.

* The widget only works with time-based metrics.

To add the Dropwizard widget:

1. From the navigation bar, click Dashboards > Dashboard Name.

2. On the Dashboard Name page, click Add Widget.

3. In the Add Widget window, from the Select a Widget drop-down list, select Dropwizard Table with Graph.

   ![Add widget window with dropwizard selected](_images/add-widget-window-dropwizard.png)

4. To preview any metric on the graph, click Add to Graph adjacent to any metric.

5. Click Add.

   The Dropwizard widget now displays on the dashboard.

## Prometheus endpoint

This topic describes how to configure Prometheus and Grafana to collect IDM metrics. These third-party tools are *not* supported by ForgeRock. Refer to the [Prometheus documentation](https://prometheus.io/docs/introduction/overview/).

Prometheus is a third-party tool used for gathering and processing monitoring data. Prometheus uses the `openidm/metrics/prometheus` endpoint to gather information. This endpoint is protected by a basic authentication filter, using the following credentials, set in the `resolver/boot.properties` file:

```properties
openidm.prometheus.username=username
openidm.prometheus.password=password
```

### Disable Prometheus

To disable IDM's Prometheus handler, comment out or remove `openidm.prometheus.username` and `openidm.prometheus.password` from the `resolver/boot.properties` file. If these properties are not set, IDM does not enable the Prometheus handler.

### Configure Prometheus

1. Download [Prometheus](https://prometheus.io/).

2. Create a `prometheus.yml` configuration file. For more information, refer to the [Prometheus configuration documentation](https://prometheus.io/docs/prometheus/latest/configuration/configuration/). An example `prometheus.yml` file:

   ```yaml
   global:
     scrape_interval: 15s
     external_labels:
       monitor: 'my_prometheus'

   # https://prometheus.io/docs/operating/configuration/#scrape_config
   scrape_configs:
     - job_name: 'openidm'
       scrape_interval: 15s
       scrape_timeout: 5s
       metrics_path: 'openidm/metrics/prometheus'
       scheme: http
       basic_auth:
         username: 'prometheus'
         password: 'prometheus'
       static_configs:
         - targets: ['localhost:8080']
   ```

   This example configures Prometheus to poll the `openidm/metrics/prometheus` endpoint every 5 seconds (`scrape_interval: 5s`), receiving metrics in a plain text format (`_fields: ['text']` and `_mimeType: ['text/plain;version=0.0.4']`). For more information about reporting formats, refer to the Prometheus documentation on [Exposition Formats](https://prometheus.io/docs/instrumenting/exposition_formats/).

3. Verify the configuration returns metric results:

   Request

   ```console
   curl \
   --user prometheus:prometheus \
   --header "Accept-API-Version: resource=1.0" \
   --request GET \
   'http://localhost:8080/openidm/metrics/prometheus'
   ```

   > **Collapse: Show example response**
   >
   > Response
   >
   > ```console
   > # HELP idm_jvm_available_cpus Automatically generated
   > # TYPE idm_jvm_available_cpus gauge
   > idm_jvm_available_cpus 10.0
   > # HELP idm_jvm_class_loading_loaded Automatically generated
   > # TYPE idm_jvm_class_loading_loaded gauge
   > idm_jvm_class_loading_loaded 24876.0
   > # HELP idm_jvm_class_loading_unloaded Automatically generated
   > # TYPE idm_jvm_class_loading_unloaded gauge
   > idm_jvm_class_loading_unloaded 1.0
   > # HELP idm_jvm_free_used_memory_bytes Automatically generated
   > # TYPE idm_jvm_free_used_memory_bytes gauge
   > idm_jvm_free_used_memory_bytes 9.77543264E8
   > # HELP idm_jvm_garbage_collector_g1_old_generation_count Automatically generated
   > # TYPE idm_jvm_garbage_collector_g1_old_generation_count gauge
   > idm_jvm_garbage_collector_g1_old_generation_count 0.0
   > # HELP idm_jvm_garbage_collector_g1_old_generation_time Automatically generated
   > # TYPE idm_jvm_garbage_collector_g1_old_generation_time gauge
   > idm_jvm_garbage_collector_g1_old_generation_time 0.0
   > # HELP idm_jvm_garbage_collector_g1_young_generation_count Automatically generated
   > # TYPE idm_jvm_garbage_collector_g1_young_generation_count gauge
   > idm_jvm_garbage_collector_g1_young_generation_count 82.0
   > # HELP idm_jvm_garbage_collector_g1_young_generation_time Automatically generated
   > # TYPE idm_jvm_garbage_collector_g1_young_generation_time gauge
   > idm_jvm_garbage_collector_g1_young_generation_time 2127.0
   > # HELP idm_jvm_max_memory_bytes Automatically generated
   > # TYPE idm_jvm_max_memory_bytes gauge
   > idm_jvm_max_memory_bytes 2.147483648E9
   > ...
   > ```

4. Start Prometheus with the `prometheus.yml` configuration file:

   ```console
   prometheus --config.file=/path/to/prometheus.yml
   ```

5. To confirm that Prometheus is gathering data from IDM, go to the Prometheus monitoring page (default `http://localhost:9090`).

   ![prometheus main page](_images/prometheus-main-page.png)

### Configure Grafana

Prometheus lets you monitor and process information provided by IDM. If you need deeper analytics, you can use tools such as Grafana to create customized charts and graphs based on Prometheus data. For information on installing and running Grafana, refer to the [Grafana website](https://grafana.com).

You can also monitor aspects of IDM's performance using Prometheus to plug JVM metrics into a Grafana dashboard. For more information on using metrics to observe the system under load, refer to [Load testing](load-testing.html).

|   |                                                                                                                                                                                                                                                                      |
| - | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|   | Before you get started, download the *Monitoring Dashboard Samples* from the [Backstage download site](https://backstage.forgerock.com/downloads). Open `monitoring.dashboard.json` from the downloaded .zip file, as you'll need it during the following procedure. |

To set up a Grafana dashboard with IDM metrics using Prometheus:

1. In a browser, go to the main Grafana page (default `http://localhost:3000`) and log in.

   |   |                                                           |
   | - | --------------------------------------------------------- |
   |   | The default username and password for Grafana is `admin`. |

2. To add your Prometheus installation to Grafana as a data source, click the toggle menu button ![grafana toggle menu](_images/grafana-toggle-menu.svg), and click Connections > Data sources.

3. On the Data sources page, click Add data source.

4. On the Add data source page, select Prometheus.

   ![grafana add prometheus source](_images/grafana-add-prometheus-source.png)

5. Enter information and select options, as needed. The information you enter here should match the settings in the `monitoring.dashboard.json` file:

   1. Give your data source a name; for example, `ForgeRockIDM`.

   2. Set the URL (default `http://localhost:9090`).

   3. Enable Basic auth.

      1. Enter the User (default `prometheus`).

      2. Enter the Password (default `prometheus`).

6. Click Save & test.

   If the test succeeds, Grafana displays Data source is working.

### Create a Grafana dashboard

After Prometheus has been configured as a data source in Grafana, you can create a dashboard with IDM metrics:

1. In Grafana, click the toggle menu button ![grafana toggle menu](_images/grafana-toggle-menu.svg), and click Dashboards.

2. Click New, and do one of the following:

   * Select Import.

     1. On the Import dashboard page, drag the `monitoring.dashboard.json` file from its location on your system to the Upload dashboard JSON file area.

     2. Enter information in the Options area, and select the Prometheus data source you previously created.

     3. Click Import.

   * Select New dashboard.

     1. Click Add visualization.

     2. Select the Prometheus data source you previously created.

     3. Configure the panel.

        |   |                                                                                                                                                                                                                                              |
        | - | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
        |   | For more information, refer to:- [Prometheus query language](https://prometheus.io/docs/prometheus/latest/querying/basics/)

        - [Panel editor overview](https://grafana.com/docs/grafana/latest/panels-visualizations/panel-editor-overview/) |