Troubleshooting

Kubernetes deployments are multi-layered and often complex.

Errors and misconfigurations can crop up in a variety of places. Performing a logical, systematic search for the source of a problem can be daunting.

Here are some techniques you can use to troubleshoot problems with ForgeOps deployments:

Problem Troubleshooting Technique

Problem	Troubleshooting Technique
Some pods don’t start.	Review Kubernetes logs and other diagnostics. Verify if your cluster is resource-constrained. Check for under configured clusters by using the `kubectl describe nodes` and `kubectl get events -w` commands. Pods killed with out of memory (OOM) conditions indicate that your cluster is under configured. Make sure that you’re using tested versions of third-party software. Stage your installation. Install Ping Identity Platform components separately, instead of installing all the components with a single command. Staging your installation lets you make sure each component works correctly before installing the next component.
All the pods have started, but you can’t reach the services running in them.	Make sure you don’t have any ingress issues.
AM doesn’t work as expected.	Set the AM logging level, recreate the issue, and analyze the AM log files. Turn on audit logging in AM.
IDM doesn’t work as expected.	Set the IDM logging level, recreate the issue, and analyze the IDM log files. Turn on audit logging in IDM.
Your JVM crashed with an out of memory error, or you suspect that you have a memory leak.	Collect and analyze Java thread dumps and heap dumps.
Changes you’ve made to ForgeOps’s Kustomize files don’t work as expected.	Fully expand the Kustomize output, and then examine the output for unintended effects.
Your minikube deployment doesn’t work.	Make sure that you don’t have a problem with virtual hardware requirements.
You’re having name resolution or other DNS issues.	Use diagnostic tools in the debug tools container.
You want to run DS utilities without disturbing a DS pod.	Use the bin/ds-debug.sh script or DS tools in the debug tools container.
You want to keep the `amster` pod running to diagnose AM configuration issues.	Use the amster command.
You want to troubleshoot AM configuration upgrade issues.	Use the config --no-upgrade option.
The `kubectl` command requires too much typing.	Enable kubectl tab autocompletion.

Some pods don’t start.

Review Kubernetes logs and other diagnostics.

Verify if your cluster is resource-constrained. Check for under configured clusters by using the kubectl describe nodes and kubectl get events -w commands. Pods killed with out of memory (OOM) conditions indicate that your cluster is under configured.

Make sure that you’re using tested versions of third-party software.

Stage your installation. Install Ping Identity Platform components separately, instead of installing all the components with a single command. Staging your installation lets you make sure each component works correctly before installing the next component.