ABS alerts

Email alerts are sent based on the following category of events. These events are also logged in the abs.log file. The threshold values for these events are pre-set. If you want to change the threshold values after the system is running, then you have to manually change the values in MongoDB.

Dynamic Rate Limit: alert sent when CPU, disk, or memory crosses the configured threshold value.
ABS Node: alert sent when ABS cluster nodes are added or removed.
MongoDB: alert sent when a MongoDB node is added or becomes inaccessible.
Percentage Disk Usage Limit: alert sent when the disk usage reaches the configured percentage_diskusage_limit value. When this limit is reached, ABS stops accepting any new access log files from ASE. The alert is also logged in the abs.log file. You can use update.sh script in/abs/util/ directory to update the thresholds for Percentage Disk Usage Limit.
License: The following license related alerts are sent:
- ABS license invalid: alert is sent if the ABS license is found to be invalid. In this case ABS shuts down.
- ABS license expiration: alert sent when ABS license is expired.
- Transaction limit reached: alert sent when ABS reaches the licensed monthly transaction limit.
Scale Up and Scale Down: alert sent when a system resource, such as CPU, memory, or disk utilization, is above or below its threshold value for a specified interval of time. If the value is above the threshold value, add ABS nodes to distribute the load. If the resource utilization is below the lower threshold, you may remove an ABS node from the ABS cluster.
DDoS attack alert: ABS sends alerts for multi-client Login Attacks and for API DDoS Attack Type 1. The email alert provides a time period for the attack along with a URL to access information on all client IPs participating in the attack.

Following is a template for alerts:

Event:  <the type of event>
Value:  <the specific trigger for the event>
When:  <the date and time of the event>
Where:  <the IP address of the server where the event occured>

For example,

Event: Scale Down ABS Node
Value : 192.168.11.166
CPU scale down threshold reached.
When : 2019-Jun-05 18:02:33 UTC
Where: 192.168.11.166

The following table describes the various email alerts sent by ABS and their possible resolution. The resolution provided is only a starting point to understand the cause of the alert. If ABS is reporting an alert even after the following the resolution provided, contact Ping Identity support.

Email alert	Possible cause and resolution
File System Maxed Out - Rate Limit Alert	Cause: A possible reason for this alert could be that historical access log files from ASE have accumulated on the storage disk. Resolution: Purge or archive the old access log files from storage disk.
ABS node added to cluster	ABS sends an email alert when a node joins an ABS cluster. Confirm: ABS admin should verify whether the correct ABS node joined the ABS cluster.
ABS node removed from cluster	ABS sends an email alert when a node is removed from an ABS cluster. Confirm: ABS admin should check the reason for removal of ABS node from the cluster. ABS node could disconnect from cluster because of network issues, a manual stop of ABS, or change in IP address of the ABS machine.
Memory scale up or scale down	Cause: ABS sends an email alert when the ABS node reaches the memory scale up or scale down limits in the configuration. The reason for reaching scale up limit can be because of large number of access log files coming from ASE. Scale down limit could be reached because of low number of access logs coming from ASE. Resolution: If ABS reaches scale up limit, add another ABS node to the cluster. If the system utilization is low, you can remove an ABS node from the cluster.
CPU scale up or scale down	Cause: ABS sends an email alert when the ABS node reaches the CPU scale up or scale down limits in the configuration. The reason for reaching scale up limit can be because of large number of access log files coming from ASE. Scale down limit could be reached because of low number of access logs coming from ASE. Resolution: If ABS reaches scale up limit, add another ABS node to the cluster. If the system utilization is low, you can remove an ABS node from the cluster.
Disk scale up or scale down	Cause: ABS sends an email alert when the ABS node reaches the disk scale up or scale down limits in the configuration. The reason for reaching scale up limit can be because of large number of access log files coming from ASE. Scale down limit could be reached because of low number of access logs coming from ASE. Resolution: If ABS reaches scale up limit, add another ABS node to the cluster. If the system utilization is low, you can remove an ABS node from the cluster.
License <path> is invalid. ABS will shut down now	Cause: ABS sends this email alert when ABS does not have correct permissions to read the license file from the configured path, or there is a typing error in the name of the license file. Resolution: Validate current license file path. Also check for file permissions of the license file.
ABS license at <path> has expired. Please renew your license.	Cause: ABS sends this email alert when ABS license has expired. The license expires at the end of the license period. Resolution: Renew your ABS license.
Maximum transaction limit reached for the current month	ABS sends this warning message when ABS crosses the licensed monthly transaction limit.
API DDoS Attack Type 1 or Login DoS detected between <timestamp> and <timestamp> on node <value>	ABS sends this warning message when it detects an API DDoS attack type 1 or a Login DoS attack.
MongoDB primary node is down	Cause: ABS sends this email alert when MongoDB process is unavailable due to a shortage in memory or CPU. This alert can also trigger because of network issues for MongoDB node. Resolution: Check MongoDB Primary node status to bring it back online or add additional secondary node if needed.

Email alert

Possible cause and resolution

File System Maxed Out - Rate Limit Alert

Cause: A possible reason for this alert could be that historical access log files from ASE have accumulated on the storage disk.

Resolution: Purge or archive the old access log files from storage disk.

ABS node added to cluster

ABS sends an email alert when a node joins an ABS cluster.

Confirm: ABS admin should verify whether the correct ABS node joined the ABS cluster.

ABS node removed from cluster

ABS sends an email alert when a node is removed from an ABS cluster.

Confirm: ABS admin should check the reason for removal of ABS node from the cluster. ABS node could disconnect from cluster because of network issues, a manual stop of ABS, or change in IP address of the ABS machine.

Memory scale up or scale down

Cause: ABS sends an email alert when the ABS node reaches the memory scale up or scale down limits in the configuration. The reason for reaching scale up limit can be because of large number of access log files coming from ASE. Scale down limit could be reached because of low number of access logs coming from ASE.

Resolution: If ABS reaches scale up limit, add another ABS node to the cluster. If the system utilization is low, you can remove an ABS node from the cluster.

CPU scale up or scale down

Cause: ABS sends an email alert when the ABS node reaches the CPU scale up or scale down limits in the configuration. The reason for reaching scale up limit can be because of large number of access log files coming from ASE. Scale down limit could be reached because of low number of access logs coming from ASE.

Resolution: If ABS reaches scale up limit, add another ABS node to the cluster. If the system utilization is low, you can remove an ABS node from the cluster.

Disk scale up or scale down

Cause: ABS sends an email alert when the ABS node reaches the disk scale up or scale down limits in the configuration. The reason for reaching scale up limit can be because of large number of access log files coming from ASE. Scale down limit could be reached because of low number of access logs coming from ASE.

Resolution: If ABS reaches scale up limit, add another ABS node to the cluster. If the system utilization is low, you can remove an ABS node from the cluster.

License <path> is invalid. ABS will shut down now

Cause: ABS sends this email alert when ABS does not have correct permissions to read the license file from the configured path, or there is a typing error in the name of the license file.

Resolution: Validate current license file path. Also check for file permissions of the license file.

ABS license at <path> has expired. Please renew your license.

Cause: ABS sends this email alert when ABS license has expired. The license expires at the end of the license period.

Resolution: Renew your ABS license.

Maximum transaction limit reached for the current month

ABS sends this warning message when ABS crosses the licensed monthly transaction limit.

API DDoS Attack Type 1 or Login DoS detected between <timestamp> and <timestamp> on node <value>

ABS sends this warning message when it detects an API DDoS attack type 1 or a Login DoS attack.

MongoDB primary node is down

Cause: ABS sends this email alert when MongoDB process is unavailable due to a shortage in memory or CPU. This alert can also trigger because of network issues for MongoDB node.

Resolution: Check MongoDB Primary node status to bring it back online or add additional secondary node if needed.

PingIntelligence

ABS alerts