Monitoring server metrics with Prometheus
Prometheus is an open-source monitoring application that can be used to track numeric metrics over time. It uses HTTP GET requests to retrieve the metric data in the OpenMetrics text-based format.
The server includes an HTTP servlet extension that can publish the values of a configured set of numeric monitor attributes in this format so that they can be consumed by a Prometheus server.
Enabling Prometheus support in the server
The server is preconfigured with an HTTP servlet extension that can publish a wide variety of metrics for use by Prometheus.
To enable the servlet extension, add it to an HTTP or HTTPS connection handler, and either disable and re-enable the connection handler or restart the server. For example:
dsconfig set-connection-handler-prop \ --handler-name "HTTPS Connection Handler" \ --add "http-servlet-extension:Prometheus Monitoring" \ --set enabled:false dsconfig set-connection-handler-prop \ --handler-name "HTTPS Connection Handler" \ --set enabled:true
The servlet extension offers several configuration properties that can be used to customize its behavior. They include:
base-context-path
-
The path that the Prometheus server can use to consume any published metrics. By default, this is
/metrics
. include-instance-name-label
-
Indicates whether all published metrics should include an
instance
label whose value is the name of the server instance. include-product-name-label
-
Indicates whether all published metrics should include a
product
label whose value is the name of the server product, such asPing Identity Directory Server
for the PingDirectory server. include-location-name-label
-
Indicates whether all published metrics should include a
location
label whose value is the name of the configured location for the server instance. always-include-monitor-entry-name-label
-
Indicates whether all published metrics should include a
monitor_entry
label whose value is the name of the monitor entry (the value of thecn
attribute) from which the metric was taken.
Even if this property is set to false, the server will still add this label to metrics in cases where there are multiple monitor entries with the same object class. |
include-monitor-object-class-name-label
-
Indicates whether all published metrics should include a
monitor_object_class
label whose value is the name of the object class for the monitor entry from which the metric was taken. include-monitor-attribute-name-label
-
Indicates whether all published metrics should include a
monitor_attribute
label whose value is the name of the monitor attribute from which the metric was taken. label-name-value-pair
-
An optional set of name-value pairs for labels that should be included in all metrics published by the server. If provided, each item should use an equal sign to separate the label name from the value, such as
label_name=label_value
.
Label names must:
Label values can be any non-empty string. |
After the servlet extension has been enabled, you can verify that the metric information is available by retrieving it with a simple HTTP GET request using a web browser or a command-line tool like curl
or wget
.
For example, to retrieve the Prometheus metrics from the server ds.example.com
using an HTTPS connection handler listening on port 8443 and the default base-context-path
value of /metrics
, use a URL of https://ds.example.com:8443/metrics
.
Customizing published metrics
The server is preconfigured with a wide variety of definitions that publish the values of specified monitor attributes as metrics that can be consumed by Prometheus. However, you can also configure additional metrics by creating Prometheus monitor attribute metrics.
Each of these definitions includes the following configuration properties:
metric-name
-
The name of the metric as it should be published for use in Prometheus. The metric name must start with a letter or an underscore and must contain only letters, digits, and underscores. This is required, and all metric definitions must have a unique name.
monitor-attribute-name
-
The name of the monitor attribute whose value should be published as a Prometheus metric. This is required, and only single-valued attributes are supported. The value must be an integer or floating-point number.
monitor-object-class-name
-
The name of the object class for the monitor entry that contains the attribute to publish as a Prometheus metric. This is required.
metric-type
-
The data type for the metric to be published. This is required, and the value should be one of the following:
-
boolean
— Indicates that the value of the attribute represented bymonitor-attribute-name
has one of the following values:true
,false
,enabled
,disabled
,yes
,no
,on
,off
,1
, or0
. The server sends a gauge metric to Prometheus with a value of:-
1
to represent any of the following:true
,enabled
,yes
,on
, or1
-
0
to represent any of the following:false
,disabled
,no
,off
, or0
-
-
counter
— Indicates that the value is one that increases over time as certain events occur in the server. This can include values that simply increment each time the event occurs, such as counting the number of connections accepted by the server since it started, but it can also include values that increase by larger amounts, such as counting the total number of bytes transferred in responses. -
gauge
— Indicates that the value is one that can go up or down over time and whose value represents the current state of some aspect of the server, such as the number of connections that are currently established.
-
filter
-
An optional search filter that can be used to indicate that the metric should only be published for monitor entries that match this filter. If this is not specified, then the metric is published for any monitor entry with the specified attribute and object class.
metric-description
-
An optional human-readable string that describes the purpose for the metric or that provides some other additional information about it.
label-name-value-pair
-
An optional set of name-value pairs for labels that should be included in the definition for the metric. If provided, each item should use an equal sign to separate the label name from the value, such as
label_name=label_value
. Label names must start with a letter or underscore and must only contain letters, digits, and underscores. Label values can be any non-empty string.
Metric-specific labels override server-wide labels, so if a metric-specific label is defined with the same name as a label that would otherwise be applied to all metrics, then the metric-specific value will be used instead of the server-wide value. |
For example, to create a Prometheus metric that exposes the value of the currentConnections
attribute of the general monitor entry, you could use a configuration change like:
dsconfig create-prometheus-monitor-attribute-metric \ --extension-name "Prometheus Monitoring" \ --metric-name general_monitor_current_connections \ --set monitor-attribute-name:currentConnections \ --set monitor-object-class-name:ds-general-monitor-entry \ --set metric-type:gauge \ --set "metric-description:The number of connections currently established"
If you want to remove any of the metrics that are included in the out-of-the-box configuration, then remove the definition for that metric from the server configuration.
Consuming metrics with Prometheus
After the server has been configured to publish its metrics, you can configure Prometheus to consume them.
The basic process is to update the server’s prometheus.yml
file to add a job
element to the scrape_configs
section for the server from which the metrics should be retrieved, as in the following example:
- job_name: "ds.example.com:8443" metrics_path: "/metrics" scheme: "https" tls_config: ca_file: ds.example.com-ca-certificate.pem static_configs: - targets: ["ds.example.com:8443"]
See the Prometheus documentation for complete details.