PingDirectory

Profiling server performance using the Stats Logger

The PingDirectory server includes a built-in Stats Logger that is useful for profiling server performance for a given configuration.

At a specified interval, the Stats Logger can write server statistics to a JSON file or to a log file in a .csv file, which can be read by spreadsheet applications. The logger has a negligible impact on server performance unless the log-interval property is set to a value of less than 1 second. You can customize the statistics logged and their verbosity.

The Stats Logger can also be used to view historical information about server statistics, including:

  • Replication

  • LDAP operations

  • Host information

  • Gauges

To update the configuration of the existing Stats Logger Plugin, you can either:

  • Set the advanced gauge-info property to basic/extended to include this information.

  • Create a dedicated Periodic Stats Logger for information about statistics of interest.

For information about how to enable this logging and where to find the log file, see Enabling the Stats Logger.

Enabling the Stats Logger

About this task

By default, the PingDirectory server’s built-in Stats Logger is disabled. To enable the Stats Logger:

Steps

  1. Run dsconfig in interactive mode.

    When you are prompted, enter the LDAP or LDAPS connection parameters.

    Example:

    $ bin/dsconfig
  2. To change to the Advanced Objects menu, enter o.

  3. On the main menu, enter the selection for Plugins.

  4. On the Plugin menu, enter the selection for View and edit an existing plugin.

  5. From the Plugin selection list, enter the selection for Stats Logger.

  6. On the Stats Logger Plugin menu, enter the selection to set the enabled property to TRUE. To save and apply the configuration, enter f.

    Result:

    The default logger logs information about the server every second to <server-root>/logs/dsstats.csv. If the server is idle, nothing is logged.

    To change the idle logging behavior, set the suppress-if-idle property to FALSE.

  7. Run the PingDirectory server.

    Example:

    For example, if you are running in a test environment, you can run the search-and-mod-rate tool to apply some searches and modifications to the server.

    To see an example command, run search-and-mod-rate --help.

  8. View the Stats Logger output at <server-root>/logs/dsstats.csv.

    You can open the file in a spreadsheet.

Configuring multiple Periodic Stats Loggers

About this task

You can create multiple Periodic Stats Loggers to:

  • Log different statistics

  • View historical information about gauges

  • To create a log at different intervals, such as logging cumulative operations statistics every hour

To create a new log, use the existing Stats Logger as a template to get reasonable settings, including rotation and retention policy.

Steps

  1. Run dsconfig in interactive mode.

    When prompted, enter the LDAP or LDAPS connection parameters.

    Example:

    $ bin/dsconfig
  2. To change to the Advanced Objects menu, enter o.

  3. In the main menu, enter the selection for Plugins.

  4. In the Plugin management menu, enter the selection for Create a new plugin.

  5. To use an existing plugin as a template, in the Create a New Periodic Stats Logger Plugin menu, enter t.

  6. Enter the number corresponding to the existing stats logger as a template.

  7. Enter a descriptive name for the new stats logger.

    For this example, enter Stats Logger-10s.

  8. Enter the log file path to the file.

    For this example, enter logs/dsstats2.csv.

  9. In the menu, make any other changes to the logger. To save and apply the configuration, enter f.

    Example:

    For this example, change the log-interval to 10s, and the suppress-if-idle to FALSE.

Result

You should see two loggers, dsstats.csv and dsstats2.csv, in the logs directory.

Adding custom logged statistics to a Periodic Stats Logger

Add custom statistics based on any attribute in any entry under cn=monitor with the Custom Logged Stats object.

This configuration object provides powerful controls for how monitor attributes are written to the log. For example, you can extract a value from a monitor attribute using a regular expression. New Custom Logged Stats are automatically included in the Periodic Stats Logger output.

In addition to allowing a straight pass-through of the values using the raw statistic-type, you can configure attributes to be treated as any of the following:

  • A counter, where the interval includes the difference in the value since the last interval

  • An average value held by the attribute during the specified interval

  • A minimum value held by the attribute during the specified interval

  • A maximum value held by the attribute during the specified interval

The value of an attribute can also be scaled by a fixed value or by the value of another monitor attribute.

Custom third-party server extensions written using the Server SDK can also expose interval statistics using a Periodic Stats Logger. The extension must first implement the SDK’s MonitorProvider interface and register with the server. The monitor attributes produced by this custom MonitorProvider are then available to be referenced by a Custom Logged Stats object.

To illustrate how to configure a Custom Logged Statistics Logger, the following section reproduces the built-in Consumer Total GB column that displays in the output when the included-resource-stat property is set to memory-utilization on the Periodic Stats Logger.

The column is derived from the total-bytes-used-by-memory-consumers attribute of the cn=JVM Memory Usage,cn=monitor entry as follows.

dn: cn=JVM Memory Usage,cn=monitor
objectClass: top
objectClass: ds-monitor-entry
objectClass: ds-memory-usage-monitor-entry
objectClass: extensibleObject
cn: JVM Memory Usage
...
total-bytes-used-by-memory-consumers: 3250017037

Configuring a custom logged statistic using dsconfig interactive

Steps

  1. Run dsconfig and, when prompted, enter the LDAP or LDAPS connection parameters.

    Example:
    $ bin/dsconfig
  2. In the server’s Advanced Objects menu, enter the selection for Custom Logged Stats.

  3. In the Custom Logged Stats menu, enter the selection for Create a new Custom Logged Stats.

  4. Select the Stats Logger Plugin from the list if more than one is present on the system. If you only have one stats logger, press Enter to confirm that you want to use the existing plugin.

  5. Enter a descriptive name for the Custom Logged Stats.

    For this example, enter Memory Usage.

  6. From the monitor-objectclass property menu, enter the objectclass attribute to monitor.

    For this example, enter ds-memory-usage-monitor-entry.

    To view the entry, run ldapsearch using the base distinguished name (DN) cn=JVM Memory Usage,cn=monitor entry.

  7. Enter the attributes of the monitor entry that you want to log in the stats logger, and then press Enter again to continue.

    For this example, enter total-bytes-used-by-memory-consumers.

  8. Specify the type of statistics for the monitored attribute to appear in the log file.

    For this example, enter the option for raw statistics as recorded by the logger.

  9. In the Custom Logged Stats menu, review the configuration.

  10. To set up a column name that lists the memory usage, enter the option to change the column-name property.

  11. To add a specific label for the column name, enter the option to add a value, and then enter Memory Consumer Total (GB). To continue, press Enter.

  12. Confirm that you want to use the column-name value that you entered in the previous step. To use the value, press Enter.

  13. To scale the Memory Consumer Totals (GB) by 1 gigabyte (GB), in the Custom Logged Stats menu, enter the option to change the divide-value-by property.

  14. In the divide-value-by property menu, enter the option to change the value, and then enter 1073741824.

    1073741824 bytes is equivalent to 1 gigabyte.

  15. In the Custom Logged Stats menu, review your configuration. To save and apply the settings, enter f.

    Example:
    >>>> Configure the properties of the Custom Logged Stats
     >>>> via creating 'Memory Usage' Custom Logged Stats
    
             Property                   Value(s)
             ---------------------------------------------------------------
        1)   description                -
        2)   enabled                    true
        3)   monitor-objectclass        ds-memory-usage-monitor-entry
        4)   include-filter             -
        5)   attribute-to-log           total-bytes-used-by-memory-consumers
        6)   column-name                Memory Consumer Total (GB)
        7)   statistic-type             raw
        8)   header-prefix              -
        9)   header-prefix-attribute    -
        10)  regex-pattern              -
        11)  regex-replacement          -
        12)  divide-value-by            1073741824
        13)  divide-value-by-attribute  -
        14)  decimal-format             #.##
        15)  non-zero-implies-not-idle  false
    
        ?)   help
        f)   finish - create the new Custom Logged Stats
        a)   hide advanced properties of the Custom Logged Stats
        d)   display the equivalent dsconfig arguments to create this object
        b)   back
        q)   quit
    
    Enter choice [b]:
    Result:

    A message of The Custom Logged Stats was created successfully is returned.

    After the Custom Logged Stats configuration change is completed, the new stats value should immediately show up in the Stats Logger output file.

Configuring a custom stats logger using dsconfig non-interactive

About this task

The following task replicates the previous procedure using dsconfig in non-interactive mode.

To create a custom stats logger:

Steps

  • Run the dsconfig non-interactive command with the create-custom-logged-stats option.

    Example:

    In this example, the command produces a column named Memory Consumer Total (GB) that contains the value of the of total-bytes-used-by-memory-consumers attribute pulled from the entry with the ds-memory-usage-monitor-entry objectclass. This value is scaled by 1073741824 to get to a value represented in GBs.

    $ bin/dsconfig create-custom-logged-stats --plugin-name "Stats Logger" \
      --stats-name "Memory Usage" --type custom \
      --set monitor-objectclass:ds-memory-usage-monitor-entry \
      --set attribute-to-log:total-bytes-used-by-memory-consumers \
      --set "column-name:Memory Consumer Total (GB)" --set statistic-type:raw \
      --set divide-value-by:1073741824