PingDS release notes

Requirements

Ping Identity supports customers deploying DS in Docker containers and Kubernetes platforms as on bare metal or in virtual machines (VMs), provided you have the requisite skills and follow the hardware and software requirements specified here.

Other versions and alternative environments might work as well. When opening a support ticket for an issue, however, make sure you can reproduce the problem on a combination covered here.

Hardware

Thanks to the underlying Java platform, DS software runs well on a variety of processor architectures. Many directory service deployments meet their service-level agreements without the latest or fastest hardware.

This section covers basic requirements, which might be enough for your deployment.

Before and after deploying in production, test whether the deployment meets your service-level objectives. If performance is acceptable, good.

When you uncover problems, troubleshoot performance issues.

Memory

When installing DS for evaluation, give the server at least 1 GB of available RAM.

For production systems, memory requirements depend on your data and performance expectations.

Required cache

DS must cache the following data in memory:

ACIs

If ACIs make up a significant percentage of the directory data, make sure DS has enough RAM to keep the ACIs cached. This can be the case, for example, in deployments where applications routinely create ACIs programmatically.

Static groups

If you have many static groups, make sure DS has enough RAM to keep them cached. With DS 7.3 and later, you can read group statistics over LDAP or over HTTP.

LDAP subentries

If your deployment has significant numbers of subentries, such as replicated password policies, make sure DS has enough RAM to keep them cached. With DS 7.3 and later, you can read subentry statistics over LDAP or over HTTP.

Argon2

If your deployment relies on the Argon2 password storage scheme, the default settings use less than 1 GB of available RAM.

When you change the Argon2 settings to strengthen the algorithm or to handle more authentications in parallel, increase the memory available to DS.

Database cache

DS caches JE backend data in memory according to the database cache settings.

Make sure DS has enough RAM to cache at least all internal nodes.

Disk space

When installing DS for evaluation, make sure you have 10 GB free disk space for the software and example data.

For installation in production, the more data you have, the more disk space you need:

  • Plan for four times the disk space needed for initial production data in LDIF format.

    Leave space for growth in database size as client applications change and add entries over time. A replicated directory server stores data, indexes for the data, operational attribute data, and historical information for replication.

    DS servers trade disk space for performance and resilience by compacting and purging data.

  • For a better estimate of the required disk space:

    1. Configure DS for production.

    2. Import a known fraction of the initial LDIF.

    3. Simulate realistic traffic to estimate change and growth in directory data.

    4. Extrapolate from the actual space occupied in testing to estimate the disk space required in production.

  • To improve performance, use quality solid state disk drives with fast I/O and high throughput.

CPU architectures

Processor architectures with fast single thread execution help DS deliver the lowest response times. For top-end performance with sub-millisecond response times and thousands of operations per second, the latest x86/x64 architecture chips perform better than others.

  • When deploying DS servers with replication enabled, provision at least two CPU cores per server. In high-volume deployments, provision more cores. Single CPU systems limit server performance.

  • If your deployment relies on CPU-intensive password storage scheme, such as Bcrypt and PBKDF2, provision enough CPU cores per server. When you increase the number of iterations, increase the CPU cores available to DS.

Network

On systems with fast processors and enough memory to cache directory data completely, the network can become a bottleneck. Even if a single 1 Gb Ethernet interface offers plenty of bandwidth to handle your average traffic load, it can be too small for peak traffic loads. Consider using separate interfaces for administrative traffic and for application traffic.

To estimate the network hardware required, calculate the size of the data returned to applications during peak load. For example, if you expect to have a peak load of 100,000 searches per second, each returning a full 8 KB entry, you require a network that can handle 800 MB/sec (3.2 Gb/sec) throughput, not counting other operations, such as replication traffic.

Storage

The directory server does not support network file systems such as NFS for database storage.

Provide sufficient disk space on local storage such as internal disk or an attached disk array.

For a directory server, storage hardware must house both directory data, including historical data for replication, and server logs. On a heavily used server, you might improve performance by putting access logs on dedicated storage.

Storage must keep pace with throughput for write operations. Write throughput can arise from modify, modify DN, add, and delete operations, and from bind operations when a login timestamp is recorded, and when account lockout is configured, for example.

In a replicated topology, a directory server writes entries to disk when they are changed, and a replication server writes changelog entries. The server also records historical information to resolve potential replication conflicts.

For network throughput, base storage throughput required on peak loads rather than average loads.

FQDNs

DS replication requires the use of fully qualified domain names (FQDNs).

Hostnames like localhost or my-laptop.local are acceptable for evaluation.

When setting up and configuring production servers, use FQDNs, and ensure DNS is set up correctly to provide FQDNs.

As a workaround when demonstrating across multiple host systems, you can update the hosts file, /etc/hosts or C:\Windows\System32\drivers\etc\hosts, to specify FQDNs.

Examples in the documentation use the hostname localhost to contact local DS servers. Trust in the examples depends on the use of a deployment ID and password when setting up servers. A server certificate generated from a deployment ID and password has localhost as the default hostname. By using the --hostname localhost option with a DS command-line tool, you simplify the secure connection process. When the tool validates the specified hostname against the hostname in the server certificate, they match. There is no need to add the server’s hostname to the server certificate.

When making a secure connection to a remote server, be sure the FQDN in the --hostname fqdn option matches a valid hostname in the server certificate. If the server certificate is generated with a deployment ID and password, you can easily renew the certificate to change or add a hostname. For examples, refer to Replace a TLS key pair or Generate a key pair (wildcard certificate).

Adapt the examples as necessary when using your own certificates, keys, and PKI.

Clock synchronization

Before using DS replication, set up synchronization between server system clocks.

To keep the system clocks synchronized, use a process that adjusts time to eventual clock consistency, such as ntpd. NTP adjusts the size of a second to move time to eventual clock consistency.

Once you have enabled replication, avoid moving the system clock in large increments, such as more than half a day at a time, or possibly less for systems under high load.

Certificates

For secure network communications with client applications that you do not control, install a properly signed digital certificate that your client applications recognize, such as one that works with your organization’s PKI, or one signed by a recognized CA.

To use the certificate during installation, the certificate must be located in a file-based keystore supported by the JVM (JKS, JCEKS, PKCS#12), or on a PKCS#11 token. To import a signed certificate into the server keystore, use the Java keytool command.

For details, refer to Key management.

HSM

DS 7.2.0 and later supports the standard PKCS#11 interface and uses hardware security modules (HSMs) through Java APIs for PKCS#11. Your HSM and client libraries must support access through standard Java security APIs, such as those following the PKCS#11 standard v2.20 or later. DS servers do not support vendor-specific interfaces.

DS servers use an HSM only to hold asymmetric key pairs and, optionally, CA certificates. DS does not store symmetric (secret) keys on the HSM.

Instead, DS uses a shared master key, which is an asymmetric key pair, to encrypt and decrypt the symmetric keys. DS stores symmetric keys (encrypted) with the data they protect. If you use an HSM for the shared master key, the HSM must share the identical master key with all DS servers in the deployment. Otherwise, DS servers cannot decrypt symmetric keys from another server, and therefore replicas cannot decrypt each other’s data.

Also, if you use an HSM for the shared master key, read the documentation carefully before you install DS. When you set up the server, you must avoid accidentally encrypting data while using the wrong shared master key.

For details, refer to PKCS#11 hardware security module.

Operating systems

DS software is supported on the following operating systems:

Operating system DS 5.5 DS 6.0 DS 6.5 DS 7.0 DS 7.1 DS 7.2 DS 7.3 DS 7.4 DS 7.5

Amazon Linux

2017.03

2, 2017.09

2, 2017.09, 2018.03

2018.03

2, 2018.03

2, 2018.03, 2023(1)

CentOS

6, 7

7, 8(2)

(Not supported)

Debian Linux

(Not supported)

11

Microsoft Windows Server(3)

2008, 2008 R2, 2012, 2012 R2, 2016

2016, 2019

2016, 2019, 2022

Oracle Solaris 10, 11 (SPARC, x64)

(Not supported)

10, 11(4)

(Not supported)

Red Hat Enterprise Linux

6, 7

7, 8(2)

7, 8(2),(5)

7, 8, 9(2),(5)

Rocky Linux

(Not supported)

8.5, 9(2)

9(2)

SuSE

11

12

12, 15

Ubuntu

14.04 LTS, 16.04 LTS

14.04 LTS, 16.04 LTS, 18.04 LTS

18.04 LTS, 20.04 LTS

18.04 LTS, 20.04 LTS, 22.04 LTS

(1) Support for Amazon Linux 2023 in DS 7.4.x starts with 7.4.2. Support for Amazon Linux 2023 in DS 7.3.x starts with 7.3.5.

(2) Write barriers for the default XFS file system are permanently enabled.

(3) PingDS is fully supported on Windows; however, it has been specifically optimized for Linux environments. Customers who require advanced integration with the operating system, including but not limited to service management, DNS, and other system-level functions, will find a Linux-based OS provides a more seamless and robust experience. Therefore, for optimal performance and enhanced integration capabilities, use a Linux-based operating system.

(4) After 5, 6.5.x was the only release to support Solaris.

(5) Red Hat Enterprise Linux 8 and OpenJDK with FIPS mode enabled doesn’t support the PBKDF2WithHmacSHA256 SecretKeyFactory algorithm and is incompatible with DS.

Write barriers (Linux)

Write barriers and journaling mode for Linux file systems help avoid directory database file corruption. They make sure writes to the file system are ordered even after a crash or power failure. Make sure these features are enabled.

Some Linux distributions permanently enable write barriers. There is no administrative action to take.

Other Linux systems leave the decision to you. If your Linux system lets you configure write barriers and journaling mode for the file system, refer to the options for your file system in the mount command manual page for details on enabling them.

Maximum open files

DS servers must open many file descriptors when handling thousands of client connections.

Linux systems often set a limit of 1024 per user. That setting is too low to accept thousands of client connections.

Make sure the server can use at least 64K (65536) file descriptors. For example, when running the server as user opendj on a Linux system that uses /etc/security/limits.conf to set user level limits, set soft and hard limits by adding these lines to the file:

opendj soft nofile 65536
opendj hard nofile 131072

The example above assumes the system has enough file descriptors available overall. Check the Linux system overall maximum as follows:

$ cat /proc/sys/fs/file-max
204252

Maximum watched files

A directory server backend database monitors file events. On Linux systems, backend databases use the inotify API for this purpose. The kernel tunable fs.inotify.max_user_watches indicates the maximum number of files a user can watch with the inotify API.

Make sure this tunable is set to at least 512K:

$ sysctl fs.inotify.max_user_watches
fs.inotify.max_user_watches = 524288

If this tunable is set lower than that, update the /etc/sysctl.conf file to change the setting permanently, and use the sysctl -p command to reload the settings:

$ echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
[sudo] password for admin:

$ sudo sysctl -p
fs.inotify.max_user_watches = 524288

Antivirus interference

Prevent antivirus and intrusion detection systems from interfering with DS software.

Before using DS software with antivirus or intrusion detection software, consider the following potential problems:

Interference with normal file access

Antivirus and intrusion detection systems that perform virus scanning, sweep scanning, or deep file inspection are not compatible with DS file access, particularly write access.

Antivirus and intrusion detection software have incorrectly marked DS files as suspect to infection, because they misinterpret normal DS processing.

Prevent antivirus and intrusion detection systems from scanning DS files, except these folders:

/path/to/opendj/bat/

Windows command-line tools

/path/to/opendj/bin/

Linux command-line tools

/path/to/opendj/extlib/

Optional additional .jar files used by custom plugins

/path/to/opendj/lib/

Scripts and libraries shipped with DS servers

Port blocking

Antivirus and intrusion detection software can block ports that DS uses to provide directory services.

Make sure that your software does not block the ports that DS software uses. For details, refer to Administrative access.

Negative performance impact

Antivirus software consumes system resources, reducing resources available to other services including DS servers.

Running antivirus software can therefore have a significant negative impact on DS server performance. Make sure that you test and account for the performance impact of running antivirus software before deploying DS software on the same systems.

Java

  • Always use a JVM with the latest security fixes.

  • Make sure you have a required Java environment installed on the system.

    If your default Java environment is not appropriate, use one of the following solutions:

    • Edit the default.java-home setting in the opendj/config/java.properties file.

    • Set OPENDJ_JAVA_HOME to the path to the correct Java environment.

    • Set OPENDJ_JAVA_BIN to the absolute path of the java command.

  • When running the dskeymgr and setup commands, use the same Java environment everywhere in the deployment.

    Due to a change in Java APIs, the same DS deployment ID generates different CA key pairs with Java 11 and Java 17.

DS software is supported on the following Java environments:

Vendor DS 5.5 DS 6.0 DS 6.5 DS 7.0(1) DS 7.1(1) DS 7.2(1) DS 7.3(1) DS 7.4(1) DS 7.5(1)

OpenJDK(2)

8

8, 11(3)

11(3)

11(3), 17(4)

17(5), 21

Oracle Java

(1) DS and the dskeymgr and setup commands require support for the PBKDF2WithHmacSHA256 SecretKeyFactory algorithm. The FIPS settings on some operating systems limit the available algorithms. For example, Red Hat Enterprise Linux 8 and OpenJDK in FIPS mode uses incompatible settings.

(2) DS supports OpenJDK-based distributions, including:

  • AdoptOpenJDK/Eclipse Temurin Java Development Kit (Adoptium)

  • Amazon Corretto

  • Azul Zulu

  • Red Hat OpenJDK

Ping Identity tests most extensively with AdoptOpenJDK/Eclipse Temurin. Ping Identity recommends using the HotSpot JVM.

(3) DS requires Java 11.0.6 or later. Earlier Java 11 updates lack required cryptography fixes.

(4) DS requires Java 17.0.3 or later. Earlier Java 17 updates lack required cryptography fixes.

(5) DS requires Java 17.0.8 or later.

TLS cipher support depends solely on the JVM. For details, refer to TLS settings.

Application containers

The gateway applications support the following web application containers:

Container DS 5.5 DS 6.0 DS 6.5 DS 7.0 DS 7.1 DS 7.2 DS 7.3 DS 7.4 DS 7.5

Apache Tomcat

8.5, 9

IBM WebSphere Liberty

(Not supported)

20.0.0.1

22.0.0.4

JBoss Enterprise Application Platform

(Not supported)

7.2

7.3

7.4

Wildfly

(Not supported)

12, 19

15, 19

15, 26

Kubernetes deployments

Deploying PingDS in Kubernetes poses unique challenges you must navigate carefully. A successful deployment depends on well-prepared teams with the necessary skills and experience. Thorough planning, appropriate recovery strategies, and proper configuration for storage and data replication are all essential for a stable, high-performance deployment.

Your team members must have expertise in DS and in managing stateful applications in Kubernetes, successfully meeting all the challenges described in this section.

Does your team lack expertise in DS, Kubernetes, or the challenges for stateful applications described here? If so, choose a simpler cloud deployment option with less complexity and fewer administrative challenges, such as using PingOne Advanced Identity Cloud or deploying DS in VMs.

A stateful database in a stateless environment

Kubernetes is designed to serve ephemeral and stateless workloads it can easily spin up, duplicate, and destroy. DS is a distributed database—​a very stateful application—​that relies heavily on persistent data storage.

Make sure you properly manage replicas and their data to keep the service running across pod restarts and failures.

High availability (HA)

  • Spread replicas across physical nodes.

    Use anti-affinity to ensure Kubernetes schedules DS pods on different nodes. Anti-affinity helps prevent a single point of failure where all replicas in a region run on the same physical system and all crash together when that system fails.

  • Spread replicas across availability zones.

    In multi-zone deployments, use anti-affinity to schedule replicas in different availability zones or failure domains. This improves HA and facilitates disaster recovery. Failures in one zone don’t bring down the entire service.

Fault tolerance and resilience

  • Mitigate hardware, network, and node failures.

    Use anti-affinity to make Kubernetes schedule DS pods on different nodes. When the hardware fails, the network goes out, or a node crashes, it should affect one replica at a time.

    Beyond HA, keeping DS pods on separate nodes prevents competition for system resources.

Costs of anti-affinity

  • Higher resource demands.

    Anti-affinity requires additional nodes to satisfy the scheduling constraints. If the Kubernetes cluster doesn’t have enough nodes, pods remain unscheduled.

    Don’t let DS pods remain unscheduled. DS requires multiple replicas to ensure HA of the identity data storage layer.

  • Workload balancing.

    Anti-affinity rules can lead to underutilized nodes when the cluster can’t balance workloads effectively. Tune your anti-affinity rules for both availability and efficient use of resources.

  • Configuration complexity.

    Configuring anti-affinity policies requires careful planning. A policy that is too strict leads to scheduling failures. A policy that is too lenient defeats the purpose of fault tolerance.

    Anti-affinity prevents pods from being scheduled on the same node, but it can require affinity rules to ensure Kubernetes schedules pods in certain zones or regions. This dual configuration of affinity and anti-affinity adds complexity to the scheduling strategy.

  • Pod scheduling complexity.

    When a node fails, Kubernetes attempts to reschedule the pod on another node that satisfies the anti-affinity rules. If no suitable nodes are available, the pod can remain unscheduled for an extended period, affecting HA.

Persistent storage management

It is a major challenge to ensure Kubernetes stores data reliably across node restarts, pod failures, and rescheduling.

Data persistence

Kubernetes uses PersistentVolumeClaims (PVCs) and PersistentVolumes (PVs) for storage. Managing these storage resources, especially across different environments or cloud providers, adds significant complexity.

Storage I/O performance

The performance of the underlying storage system varies depending on the Kubernetes deployment. Ensuring consistent I/O performance for DS is more difficult in containerized environments.

Storage abstraction

While Kubernetes abstracts storage, DS requires low-latency, high-performance access to storage. The additional layers of abstraction can introduce latency, affecting the overall performance of DS.

Networking and performance

Network latency

Kubernetes networks introduce additional latency between DS nodes, especially in multi-node or multi-region deployments. This can negatively affect replication performance.

Load balancing

Kubernetes networking and load balancers optimize for stateless services.

DS requires stable, long-lived TCP connections across all DS servers to maintain a stable, shared state. For multi-cluster deployments, you must adapt Kubernetes networking and load balancing settings to ensure effective replication.

Data consistency and high availability

Replication and consistency

Ensuring data consistency across DS replicas can be challenging in distributed environments. Kubernetes can reschedule pods across different nodes or regions, potentially leading to inconsistencies in replication and eventual consistency issues.

Failover

For an HA database system like DS, automatic failover is critical. In Kubernetes, especially in a multi-node or multi-region setup, configuring automatic failover adds complexity.

Data recovery

When a database node fails, it’s critical to recover the lost data while the service continues without interruption. This can be challenging in Kubernetes, which is designed for ephemeral containers.

Scaling and resource allocation

Horizontal scaling

While Kubernetes excels at scaling stateless applications, scaling a stateful set horizontally is much more complex. Sharding data, managing replication, and ensuring consistent data across multiple pods adds operational complexity.

Vertical scaling limitations

DS requires consistent CPU, memory, and I/O resources. Scaling vertically by increasing resources for a single pod is more challenging in Kubernetes than in traditional server environments with dedicated systems allocated for intensive workloads.

When scaling vertically, you can reach limits for nodes or need to adjust settings for all DS pods to avoid resource imbalances.

Dynamic resource allocation

Kubernetes is optimized to assign resources to pods dynamically. As a database, DS has strict requirements for memory, CPU, and disk I/O.

In a shared Kubernetes cluster, you must overcome the challenges of resource constraints and noisy neighbor issues to ensure consistent DS performance.

Administrative tasks

Complex backups

Performing consistent backups in a Kubernetes environment can be challenging as it involves multiple pods and volumes. The backup procedures must capture the database state, accounting for any differences in data stored across different pods or persistent volumes. This requires careful orchestration when some DS pods are configured differently from others.

Disaster recovery

In Kubernetes, node failures or cluster outages are expected. Recovering DS data when such events happen while ensuring data integrity is more challenging than simply restarting a stateless service. Ensuring quick recovery times and data integrity in multi-node or multi-region setups can be complex.

You must make sure DS recovers the correct data. When you restore data only on some replicas, the data must be more recent than the replication purge delay.

Resources

DS commands like backendstat, export-ldif, import-ldif, rebuild-index, and verify-index require additional resources when running on the pod. Size pods with this in mind to prevent Kubernetes from terminating the pod due to insufficient resources when performing administrative tasks.

In addition, make sure a pod is only marked operational for the deployment after the appropriate ramp-up time. Make sure the pod stays online during any ramp-down time required for administrative tasks.

Observability and health monitoring

Log files

Collect the logs for all pods in a central location.

Performance monitoring

Automate performance monitoring to react before imminent failure and for capacity planning, not just for dynamic scaling.

Operational excellence

Kubernetes is versatile and dynamic. Running a database like DS demands a high level of operational excellence. Key operations like rescheduling, scaling, backups, failover, and recovery are particularly challenging in Kubernetes unless all your team members have a solid grasp of the concepts and the requisite expertise.

Pod restarts and rescheduling

Kubernetes restarts and reschedules pods at any time, which can be disruptive for a stateful database like DS. Unlike stateless services, DS must maintain data integrity and consistency during these operations. Ensuring smooth restarts without data loss or corruption adds operational overhead.

Database upgrades

Rolling updates and upgrades are standard for stateless applications in Kubernetes. To upgrade DS with schema changes and index rebuilds is more complex. You must plan carefully to ensure data remains consistent during each upgrade.

If your team has the required skills and experience, use ForgeOps as a guide. ForgeOps demonstrates deploying the Ping Identity Platform, including DS, on Kubernetes.

Third-party software

Ping Identity provides support for using the following third-party software when logging Common Audit events:

Software Version

Java Message Service (JMS)

2.0 API

MySQL JDBC Driver Connector/J

8 (at least 8.0.19)

Splunk

8.0 (at least 8.0.2)

Elasticsearch and Splunk have native or third-party tools to collect, transform, and route logs. Examples include Logstash and Fluentd.

Ping Identity recommends you consider these alternatives. These tools have advanced, specialized features focused on getting log data into the target system. They decouple the solution from the Ping Identity Platform systems and version, and provide inherent persistence and reliability. You can configure the tools to avoid losing audit messages if a Ping Identity Platform service goes offline, or delivery issues occur.

These tools can work with Common Audit logging:

  • Configure the server to log messages to standard output, and route from there.

  • Configure the server to log to files, and use log collection and routing for the log files.

Ping Identity provides support for using the following third-party software when monitoring Ping Identity servers:

Software Version

Grafana

5 (at least 5.0.2)

Graphite

1

Prometheus

2.0