ForgeOps architecture
After you perform a ForgeOps deployment, the Ping Identity Platform is fully
operational in a Kubernetes cluster. forgeops
artifacts provide preconfigured
JVM settings, memory, CPU limits, and other configurations.
Here are some of the characteristics of ForgeOps deployments:
- Cluster and deployment sizes
-
When you use the Terraform artifacts in the
forgeops-extras
repository to create a Kubernetes cluster on Google Cloud, AWS, or Azure, you specify one of three sizes:-
A small cluster with capacity to handle 1,000,000 test users
-
A medium cluster with capacity to handle 10,000,000 test users
-
A large cluster with capacity to handle 100,000,000 test users
When you use the forgeops-minikube script to create a Kubernetes cluster on Minikube, you don’t specify a cluster size.
When you perform a ForgeOps deployment, you specify a deployment size. This deployment size should be the same as your cluster size, except when you perform single-instance ForgeOps deployments.
Single-instance deployments are special deployments that you use to configure AM and IDM and build custom Docker images for the Ping Identity Platform. They are called single-instance deployments because unlike small, medium, and large deployments, they have only single pods that run AM and IDM. They are only suitable for developing the AM and IDM configurations and must not be used for testing performance, monitoring, security, and backup requirements in production environments.
You can perform one or more single-instance deployments on small, medium, and large GKE, EKS, and AKS clusters. Each single-instance deployment resides in its own namespace.
You can perform one (and only one) single-instance deployment on a Minikube cluster.
-
- Multi-zone Kubernetes cluster
-
In small, medium, and large ForgeOps deployments, Ping Identity Platform pods are distributed across three zones for high availability.
(In single-instance deployments, Ping Identity Platform pods reside in a single zone.)
Go here for a diagram that shows the organization of pods in zones and node pools in small, medium, and large ForgeOps deployments.
- Third-party deployment and monitoring tools
-
-
NGINX Ingress Controller for Kubernetes ingress support.
-
HAProxy Ingress Controller for Kubernetes ingress support.[1]
-
Prometheus for monitoring and notifications.[1]
-
Prometheus Alertmanager for setting and managing alerts.[1]
-
Certificate Manager for obtaining and installing security certificates.
-
Helm for deploying Helm charts.
-
- Ready-to-use Ping Identity Platform components
-
-
Multiple DS instances are deployed for higher availability. Separate instances are deployed for Core Token Service (CTS) tokens and identities. The instances for identities also contain AM and IDM run-time data.
-
The AM configuration is file-based, stored at the path
/home/forgerock/openam/config
inside the AM Docker container (and in the AM pods). -
Multiple AM instances are deployed for higher availability.[2]
-
AM instances are configured to access DS data stores.
-
Multiple IDM instances are deployed for higher availability.[2]
-
IDM instances are configured to access DS data stores.
-
- Highly available, distributed deployment[1] [2]
-
Deployment across three zones ensures that the ingress controller and all Ping Identity Platform components are highly available.
Pods that run DS are configured to use soft anti-affinity. Because of this, Kubernetes schedules DS pods to run on nodes that don’t have any other DS pods whenever possible.
The exact placement of all other ForgeOps pods is delegated to Kubernetes.
Pods are organized across three zones in a single node pool with six nodes. Pod placement among the nodes might vary, but the DS pods should run on nodes without any other DS pods.
- Ingress controller
-
The NGINX Ingress Controller provides load balancing services for ForgeOps deployments. Ingress controller pods run in the
nginx
namespace. Implementation varies by cloud provider.Optionally, you can deploy HAProxy Ingress as the ingress controller instead of NGINX Ingress Controller.[1]
- Secret generation and management
-
The open source Secret Agent operator generates Kubernetes secrets for Ping Identity Platform deployments. It also integrates with Google Cloud Secret Manager, AWS Secrets Manager, and Azure Key Vault, providing cloud backup and retrieval for secrets.
- Secured communication
-
The ingress controller is TLS-enabled. TLS is terminated at the ingress controller. Incoming requests and outgoing responses are encrypted.
Inbound communication to DS instances occurs over secure LDAP (LDAPS).
For more information, refer to Secure HTTP.
- Stateful sets
-
ForgeOps deployments use Kubernetes stateful sets to manage the DS pods. Stateful sets protect against data loss if Kubernetes client containers fail.
On small-, medium- and large- deployments, CTS data stores are configured for affinity load balancing for optimal performance.
AM policies, application data, and identities reside in the
idrepo
directory service. Small-, medium- and large- deployments use a singleidrepo
master configured to fail over to one of two secondary directory services. - Authentication
-
IDM is configured to use AM for authentication.
- DS replication[2]
-
All DS instances are configured for full replication of identities and session tokens.
- Backup and restore[1]
-
Backup and restore can be performed using several techniques. You can:
-
Use the volume snapshot capability in GKE, EKS, or AKS. The cluster where the ForgeOps deployment resides must be configured with a volume snapshot class before you can take volume snapshots, and persistent volume claims must use a CSI driver that supports volume snapshots.
-
Use the ds-backup utility.
-
Use a "last mile" backup archival solutions, such as Amazon S3, Google Cloud Storage, and Azure Cloud Storage that is specific to the cloud provider.
-
Use a Kubernetes backup and restore product, such as Velero, Kasten K10, TrilioVault, Commvault, or Portworx PX-Backup.
For more information, refer to Backup and restore overview.
-
- Initial data loading
-
After the first AM instance in a ForgeOps deployment has started, an
amster
job runs. This job loads application data, such as OAuth 2.0 client definitions, to theidrepo
DS instance.