--- title: Tune thresholds for false positives description: ABS automatically generates attack thresholds which are used by the machine learning system to identify attacks and anomalies. Initial attack thresholds are determined based on training and production traffic in your API ecosystem. At the end of the training period, ABS calculates the first set of system-generated threshold values and uses these values to detect attacks. component: pingintelligence version: 5.1 page_id: pingintelligence:abs_ai_engine:pingintelligence_tune_thresholds_false_positives canonical_url: https://docs.pingidentity.com/pingintelligence/5.1/abs_ai_engine/pingintelligence_tune_thresholds_false_positives.html revdate: April 26, 2024 section_ids: check-threshold-values: Check threshold values change-attack-thresholds: Change attack thresholds change-threshold-value-tn-only: Change threshold value Tn only changing-threshold-value-tx-only: Changing Threshold Value Tx only change-threshold-value-tn-and-tx-together: Change threshold value Tn and Tx together how-to-configure-threshold-value: How to configure threshold value manually_set_thresholds: Manually set thresholds --- # Tune thresholds for false positives ABS automatically generates attack thresholds which are used by the machine learning system to identify attacks and anomalies. Initial attack thresholds are determined based on training and production traffic in your API ecosystem. At the end of the training period, ABS calculates the first set of system-generated threshold values and uses these values to detect attacks. By default, system generated threshold values are updated every 24-hours. This frequency can be changed at start-up by modifying `attack_update_interval` using [Global configuration update REST API](pingintelligence_global_configuration_update_rest_api.html) or anytime by using the `update.sh` script available in the `util` directory. The minimum value is 1-hour as sufficient traffic is required to update the model. You can change the threshold period at anytime by running the `update.sh` script. The value of the updated threshold period is applicable immediately. For example, if the current threshold update period is 10 hours and the new threshold period is 12 hours, then the AI model updates the threshold at the 12th hour. Access script help by logging into the MongoDB machine and running the following command: ``` /opt/pingidentity/mongo/update.sh help ``` **Example:** **change the training period and threshold interval together** ``` /opt/pingidentity/mongo/update.sh -u absuser -p abs123 --attack_initial_training 24 --attack_update_interval 24 updating attack_initial_training to 24 WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 }) updating attack_update_interval to 24 WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 }) The current values of the variables are: attack_initial_training=24 attack_update_interval=24 api_discovery=true discovery_initial_interval=48 ``` ## Check threshold values Threshold values can be checked using the ABS Threshold API. For each attack type, one or more variables (for example, Var A, B) is used by the machine learning process during attack detection. All variables have a Normal Threshold Value (tn), and some variables also have an Extreme Threshold Value (tx). These values are used during the attack detection process and automatically update over time to provide improved accuracy. To view the current threshold settings, use the [GET method](pingintelligence_get_threshold.html) with the following ABS `threshold` API: `https://:/v4/abs/attack/threshold?api=;` The IP address and port corresponding to the host ABS machine. The API payload returned is a JSON file which shows the threshold values for each attack type. See [Get Threshold API](pingintelligence_get_threshold.html) for an example. ## **Change attack thresholds** Ping Identity recommends using the automatically generated system thresholds in your production operations. However, if attacks are detected for legitimate traffic (i.e. false positives), then manual tuning options are provided. An administrator has two choices: * Change the system generated threshold value to a larger user-generated value. * Disable the variable to stop detecting attacks (see [Disabling Attacks](pingintelligence_disable_attack_detection.html)) To identify settings to change, generate an [attack report](pingintelligence_attack_types_rest_websocket_apis.html), which includes attacks known to be false positives. For each identified attack, an Attack Code (for example, "varA (Tn), varB (Tn)") is listed with the threshold variable(s) that triggered the attack. The Attack Code includes the responsible variables (for example, A, B) and threshold types (for example, Tn, Tx); the threshold type can be manually adjusted. Ping Identity recommends slowly increasing the triggered threshold value(s) using user-generated thresholds. After each update, evaluate the new setting to see if false positives are reduced. The process can be repeated until the issue is addressed. The [Threshold API PUT method](pingintelligence_put_threshold.html) is used to manually override the system generated setting with a user-defined value. When configuring the threshold manually, the normal threshold (tn), the extreme threshold (tx), or either threshold can be individually set. You can also use [Attack management](../pingintelligence_dashboard/pingintelligence_attack_management.html) in Dashboard to tune threshold values for a specific client identifier. | | | | - | ------------------------------------------------------------------------------------------------ | | | Make sure that you are in `Manually set thresholds` mode before changing the threshold manually. | ## Change threshold value Tn only The `Tn` threshold value can be changed for each attack type for a specific API. The initial `Tx` value is automatically calculated based on the gap between the values of `Tn` and `Tx`. This gap is determined at the end of the [training period](pingintelligence_training_abs_model.html). The minimum gap is 1, and the value of `Tx` always bigger than `Tn`. Here is an example: Values at end of training period: * `Tn = 12` * `Tx = 16` * `Gap = 4 (Tx-Tn)` Threshold API is used to set `Tn=13`for an API variable. * `Tx = 17 `(Gap value of 4 is automatically added to new `Tn `value)`` This difference between the value of `Tn` and `Tx` is maintained when only `Tn` is moved. However, the difference between the value of `Tn` and `Tx` can be changed when only Tx is changed. | | | | - | ----------------------------------------------------------- | | | The value of `Tn` can never be more than the value of `Tx`. | ## **Changing Threshold Value Tx only** Change the `Tx` value to adjust the gap between the normal and extreme threshold setting for an attack type on a specific API. The value of `Tx` defines the gap which ranges from a minimum of 1 to the maximum value defined in [Threshold range for Tn and Tx](pingintelligence_threshold_range_tn_tx.html). When `Tx` is moved, the system calculated gap calculated at the end of the training period is no longer used. For the attack types where `Tx` is not applicable to the variable, "`na`" is displayed in the threshold API. | | | | - | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | If the value of only `Tn` is moved without modifying `Tx`, then the new gap between the value of `Tn` and `Tx` is used until the value of `Tx` is changed again. | ## Change threshold value Tn and Tx together Both `Tn` and `Tx` can be changed for an attack type on a specific API. When `Tn` and `Tx` are moved simultaneously, the newly defined value of `Tn` and gap for `Tx` are changed. The ranges of `Tn` and `Tx` values are detailed in [Threshold range for Tn and Tx](pingintelligence_threshold_range_tn_tx.html). ## How to configure threshold value To manually set a threshold, use the PUT method with the following ABS `attack` API: `https://:/v5/abs/attack/threshold?api=;` The IP address and port correspond to the host ABS machine. The API input payload is a JSON file which sets the threshold value for attack types. The parameters include attack type and Normal Threshold (tn) value. When manually setting the threshold for a variable, ABS Threshold API displays both system generated and user configured threshold values. ABS applies the user configured threshold values until it is reconfigured to use system generated values (see below). ## Manually set thresholds The threshold API with PUT method sets the operation mode for the variable by configuring mode to `system` or `user`. The following snippet of Threshold API with PUT method shows how to change the threshold mode from system to user and change value of `tn`, `tx`, or both at the same time. If you do not wish to change the value for `tn` or `tx` in user mode, leave the field blank by putting `""` in the Threshold API body. In the following snippet, the value of `tn` and tx both are changed. ```json { "api_name" : "atmapp", "mode": "user", "ioc_threshold": [ { "type": "api_memory_attack_type_2", "variable": "A", "tn": "9", "tx": "12" }, { "type": "data_exfiltration_attack", "variable": "A", "tn": "18", "tx": "" }, { "type": "data_exfiltration_attack", "variable": "B", "tn": "18", "tx": "" }, { "type": "api_memory_attack_type_1", "variable": "A", "tn": "18", "tx": "" } ] } { "api_name" : "shop", "mode": "user", "ioc_threshold": [ { "type": "api_memory_attack_type_2", "variable": "A", "tn": "13" }, { "type": "api_memory_attack_type_2", "variable": "B", "tn": "10" } } ``` The API response is displayed below: ```json { "message": success: "Thresholds set to user mode for given variables.", "date": "Mon Jan 08 15:36:05 IST 2018" } ``` After a threshold value is manually set, ABS uses the updated user threshold values to detect attacks. When threshold mode is changed back to `system`, the user-configured values are no longer used or displayed in the threshold API output. The following snippet shows changing threshold to system mode from user mode for two variables associated with an API memory attack: ```json { "api_name" : "shop", "mode": "system", "ioc_threshold": [ { "type": "api_memory_attack_type_2", "variable": "A", }, { "type": "api_memory_attack_type_2", "variable": "B", } } ``` The API response is displayed below: ```json { "message": success: "Thresholds set to system mode for given variables.", "date": "Mon Jan 06 15:36:05 IST 2018" } ```