PingIntelligence Health Check Guide

This document provides administrators with a list of commands that can be used to perform health checks on different PingIntelligence components.

There are multiple methods explained for each component. You can automate the steps or use them in manual mode. The document also captures information on log files, PID details, and port details of following PingIntelligence component:

ASE
ABS AI Engine
PingIntelligence Dashboard

Perform health checks on ASE

This section discusses the commands that can be used to check the health status of ASE. You can use the following options to conduct a health check on ASE nodes :

Health check URL

Enable the ASE health check URL in the /pingidentity/ase/config/ase.conf file. To do this set the enable_ase_health config property to true. The default value ofenable_ase_health is false.

If the configuration is modified on a running ASE node, restart the node after modifying the configuration. For more information, see Start and stop ASE. In a clustered ASE environment, stop the ASE cluster and update the ase.conf file of the primary node and restart the other ASE nodes. For more information, see Restart ASE cluster.

Once the enable_ase_health is set to true, hit thhe following URLs and do a health check. If ASE is receiving the traffic, the response will be 200 OK.
```
http://<ase-hostname/ip>:<http_port>/ase
https://<ase-hostname/ip>:<https_port>/ase
```
Status command

Use the following CLI command to know the status of an ASE process, the running status of http or https process, and port number. It also gives basic configuration information.
```
$./bin/cli.sh status
```
shell
ABS Info command

This command shows the status of communication between ABS and all the ASE nodes in a cluster. It shows last log upload and attack fetch information from ABS. If ASE is having any issues in uploading logs to ABS or connecting to ABS it will be reported in the output of the abs_info command.
```
$ ./bin/cli.sh -u admin -p admin abs_info
```
shell
Process status

If ASE is running as a systemctl service, use the following command to check the status of the service.
```
$ systemctl status pi-ase.service
```
shell

Perform health checks on ABS AI Engine

This section discusses the commands that can be used to check the health status of ABS AI Engine. You can use the following options to conduct a health check :

ABS Admin API

Use the ABS Admin REST API either from the Postman Collection or use curl command.

$ curl -k -X GET 'https://<ABS Hostname/IP:8080/v4/abs/admin' -H 'x-abs-ak: <ABS access key>' -H 'x-abs-sk: <ABS ssecret key>'

shell

Process status

If ABS AI Engine is running as a systemctl service, use the following command to check the status of the service.
```
$ systemctl status pi-abs.service
```
shell
Check ABS log for job failures

Use the following command to check the ABS log for any job failures. If any failures are detected, reach out to Ping Identity support team.
```
$ grep allocated logs/abs/abs.log | grep failure
```
shell
Check ABS log for MongoDB heartbeat

The /logs/abs/abs.log file reports the status of MongoDB heart beats at regular intervals. This is a good indicator to check ABS to MongoDB connectivity issues.

Perform health checks on PingIntelligence Dashboard

This section discusses the commands that can be used to check the health status of PingIntelligence Dashboard and its components.

== Dashboard data engine

Status command

This command shows the status of the dashboard process.
```
$ ./bin/cli.sh status
```
shell
It returns the status as Running or Not Running.
Process status

If the dashboard data engine is running as a systemctl service, use the following command to check the status of the service.
```
$ systemctl status pi-data-engine
```
shell
Check dashboard log file for errors or exceptions

To detect the connectivity issues between dashboard data engine and ABS or Elasticsearch verify the /pingidentity/dataengine/logs/admin/dataengine.log file.
```
$ tail logs/admin/dataengine.log
```
shell

== Web GUI

Health check URL

The following URL provides a 200 OK response if WebGUI component is up and running. You can use curl command or browser to check the status.
```
https://<WebGUI Hostname/IP>:<port>/status
```
```
$ curl -k -o /dev/null -s -w "%{http_code}\n" https://<webgui>:8030/status
200
```
shell
Status command

The following command shows the status of the WebGUI process.
```
$ ./bin/cli.sh status
```
shell
Process status

If the WebGUI is running as a systemctl service, use the following command to check the status of the service.
```
$ systemctl status pi-webgui.service
```
shell
Check WebGUI admin log file for errors or exceptions

To detect the connectivity issues between WebGUI and ABS or Elasticsearch verify the/pingidentity/webgui/admin/logs/ admin.log file.
```
$ tail logs/admin/admin.log
```
shell

== Elasticsearch

Health check URL

There are three ways to check the health of Elasticsearch using a health check URL
- Using anonymous access - To enable access for anonymous user, add the following line to theelasticsearch.yaml.
  xpack.security.authc.anonymous.roles: monitoring_user
  You can update this during initial setup or later. You must restart Elasticsearch if you are making the change on a running instance. After updating the elasticsearch.yaml hit the following URL to check the status of Elasticsearch. You can use curl command or browser. A 200 OK response indicates a running Elasticsearch.
  https://<Elasticsearch Hostname/IP>:9200/
  $ curl -k -o /dev/null -s -w "%{http_code}\n" https://<Elasticsearch Hostname/IP>:9200/
  
  shell
- Using a health check user- Add a health check user to Elasticsearch using the following command.
  curl -u elastic:<elastic user password> -k -X POST "https://localhost:9200/_xpack/security/user/<health_check_user>?pretty" -H 'Content-Type: application/json' -d' { "password" : "<password for health_check_user>", "roles": ["monitoring_user"] } '
  
  shell
  After adding the health check user, hit the following URL to check the status of Elasticsearch. You can use curl command or browser. A 200 OK response indicates a running Elasticsearch.
  https://<health_check_user>:<password>@<Elastcisearch hostname/IP>:9200/
  $ curl -k -o /dev/null -s -w "%{http_code}\n" https://<health_check_user>:<password>@<Elastcisearch hostname/IP>:9200/
  
  shell
  This approach doesn’t require an Elasticsearch restart.
- Using Elasticsearch username and password - You can query the health status of Elasticsearch using the elastic user and its password to see a more comprehensive output, which also reports the state of the cluster. Use the following curl command.
  $ curl -XGET -k -H 'content-type: application/json; charset=UTF-8' -u "elastic:<password>" 'https://<elasticsearch hostname/IP>:9200/_cluster/health?pretty'
  
  shell
Process status

If Elasticsearch is running as a systemctl service, use the following command to check the status of the service.
```
$ systemctl status pi-elasticsearch.service
```
shell
Check Elasticsearch log for errors or exceptions

Verify the Elasticsearch log for any exceptions or errors.
```
$ tail logs/elasticsearch.log
```
shell

== Kibana

Health check URL

There are two ways to check the health of Kibana using a health check URL:
- Using anonymous access - To enable access, add the following line to thekibana.yaml.
  status.allowAnonymous: true
  You can update this during initial setup or later. You must restart Kibana if you are making the change on a running instance. After updating the kibana.yaml hit the following URL to check the status. You can use curl command or browser. A 200 OK response indicates a running Kibana instance.
  https://<Kibana Hostname/IP>:5601/pi/ui/dataengine/api/status
  $ curl -k -o /dev/null -s -w "%{http_code}\n" https://<Kibana Hostname/IP>:5601/pi/ui/dataengine/api/status
  
  shell
- Using health check user - Add a health check user to Kibana with the following command
  curl -u elastic:<elastic user password> -k -X POST "https://localhost:9200/_xpack/security/user/<health_check_user>?pretty" -H 'Content-Type: application/json' -d' { "password" : "<password for health_check_user>", "roles": ["monitoring_user"] } '
  
  shell
  After adding the health check user, hit the following URL to check the status of Kibana. You can use curl command or browser. A 200 OK response indicates a running Kibana.
  https://<health_check_user>:<password>@<Kibana hostname/IP>:5601/pi/ui/dataengine/api/status
  $ curl -k -o /dev/null -s -w "%{http_code}\n"https://<health_check_user>:<password>@<Kibana hostname/IP>:5601/pi/ui/dataengine/api/status
  
  shell
Process Status

If Kibana is running as a systemctl service, use to check the status of the service.
```
$ systemctl status pi-kibana.service
```
shell
Check Kibana log for errors or exceptions

Verify the Kibana log for any exceptions or errors.
```
$ tail logs/kibana.log
```
shell

Logs, port numbers, PIDs

This section covers supplementary information like log file details, important port numbers, and PID information of PingIntelligence for APIs components.

Log files

The following table shows the main log files of PingIntelligence components.

ASE ABS AI Engine PingIntelligence Dashboard

ASE management, access and audit logs

ABS logs

abs.log must be the first place for debugging any issues on the ABS. The log has information about each machine learning job on the host. All incoming communication from ASE or PingIntelligence Dashboard or REST API requests are logged in this file. It also has a periodic log on heartbeat to MongoDB.

Dashboard data engine :/pingidentity/dataengine/logs/dataengine.log
WebGUI:/pingidentity/webgui/logs/admin.log and /pingidentity/webgui/logs/sso.log
Elasticsearch: /pingidentity/elasticsearch/logs/elasticsearch.log
Kibana:/pingidentity/kibana/logs/kibana.log

Port numbers

The following table shows important port numbers used by PingIntelligence components.

ASE	ABS AI Engine	PingIntelligence Dashboard
ASE ports	ABS ports	PingIntelligence Dashboard server: 8030. Port number 8030 should be exposed to public internet. Make sure that your organization’s firewall allows access to this port. Elasticsearch: 9200 Kibana: 5601 H2 database: 9092. H2 database is installed and runs as a part of PingIntelligence Dashboard.

ASE

ABS AI Engine

PingIntelligence Dashboard

ASE ports

ABS ports

PingIntelligence Dashboard server: 8030. Port number 8030 should be exposed to public internet. Make sure that your organization’s firewall allows access to this port.
Elasticsearch: 9200
Kibana: 5601
H2 database: 9092. H2 database is installed and runs as a part of PingIntelligence Dashboard.

PID information

All the PingIntelligence components have their respective PID files. Refer these files for monitoring or for getting the PID information of the processes.

ASE ABS AI Engine PingIntelligence Dashboard

ASE	ABS AI Engine	PingIntelligence Dashboard
The ASE PID file contains the PID for the controller process and the http balancer and https balancer processes. `/pingidentity/ase/logs/ase.pid`	The `/pingidentity/abs/data/abs.pid` file contains the PID for the main ABS process.	There are separate PID files for the different components of PingIntelligence Dashboard. `/pingidentity/dataengine/data/dataengine.pid` `/pingidentity/webgui/logs/webgui.pid` `/pingidentity/elasticsearch/logs/elasticsearch.pid` `/pingidentity/kibana/logs/kibana.pid`

The ASE PID file contains the PID for the controller process and the http balancer and https balancer processes. /pingidentity/ase/logs/ase.pid

The /pingidentity/abs/data/abs.pid file contains the PID for the main ABS process.

There are separate PID files for the different components of PingIntelligence Dashboard.

/pingidentity/dataengine/data/dataengine.pid
/pingidentity/webgui/logs/webgui.pid
/pingidentity/elasticsearch/logs/elasticsearch.pid
/pingidentity/kibana/logs/kibana.pid