--- title: Purge MongoDB data description: The ABS MongoDB purge script dumps and/or deletes processed AI Engine and machine learning data from MongoDB. It is recommended to archive the data before purging it. The purge_mongo.sh script is available in the//pingidentity/abs/util directory. component: pingintelligence version: 5.1 page_id: pingintelligence:abs_ai_engine:pingintelligence_purge_mongodb_data canonical_url: https://docs.pingidentity.com/pingintelligence/5.1/abs_ai_engine/pingintelligence_purge_mongodb_data.html revdate: April 3, 2024 section_ids: purge-data: Purge data dump-data: Dump data purge-and-dump-data: Purge and dump data --- # Purge MongoDB data The ABS MongoDB purge script dumps and/or deletes processed AI Engine and machine learning data from MongoDB. It is recommended to archive the data before purging it. The `purge_mongo.sh` script is available in the`//pingidentity/abs/util` directory. The script offers three options: * Only purge data * Only dump data * Dump data into a specified directory and then purge it **Prerequisites-**Ensure that the following prerequisites are fulfilled, before using the script: * Execute the script from an ABS AI Engine node with connectivity to the Mongodb primary node. * The necessary write permissions are available on the directory where the data dump is stored. * Database names used as command line arguments like the data\_dbname and mldata\_dbname should be same as configured in [ABS configuration - abs.properties](pingintelligence_abs_configuration.html) file. | | | | - | ----------------------------------------------------------------------------------------- | | | It is recommended to execute the script during low load periods or during ABS down times. | The`purge_mongo.sh` script supports the following arguments: | Argument | Description | | -------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--data_db ` | Use this argument to specify the name of the ABS database. | | `--mldata_db ` | Use this argument to specify the name of the ML database. | | You must specify at least one database while executing the script. You can also specify both the databases in a single command. | | | `-d ` or` --days ` | Number of days of data to be retained. The default number of days is seven. The minimum number of days that can be specified is one and the maximum is 365. | | `-l ` or `--location < path_to_dump_dir>` | The directory path to store the MongoDB dump. | | `--purge_only` | Use this option when you only need to delete the data and not take a data dump. | | `--dump_only` | Use this option when you need to take the data dump without deleting the data. | | `--h` or `--help` | Use this argument for more information on the purge script parameters. | | `--gzip` | Use this argument to compress the data dump. It can be used with `dump only` and` purge and dump` options. This argument cannot be used with `purge only` option. | The following sections show the sample usage of`purge_mongo.sh` script with the three options: ## Purge data The following are a few sample commands to purge the MongoDB data: * ./purge\_mongo.sh --data\_db \ -d \ --purge\_only * ./purge\_mongo.sh --mldata\_db \ -d \ --purge\_only * ./purge\_mongo.sh --data\_db \ --mldata\_db \ d \ --purge\_only For example, the following command deletes ABS data older than 80 days. ``` ./purge_mongo.sh --data_db abs_data -d 80 --purge_only Starting the purge mongo tool This will delete the documents in abs_data database that are older than 80 days. Are you sure (yes/no): yes Deleting the documents in abs_data database that are older than 80 days. Please see /opt/pingidentity/abs/logs/purge/purge.log.2020-11-16-05-01-42 for more details ``` ## Dump data The following are a few sample commands to purge the MongoDB data: * ./purge\_mongo.sh --data\_db \ -d \ -l \ --dump\_only * ./purge\_mongo.sh --mldata\_db \ -d \ -l \ --dump\_only * ./purge\_mongo.sh --data\_db \ --mldata\_db \ d \ -l \ --dump\_only For example, the following command dumps data older than 80 days from ml\_database into a `/tmp` directory. It does not delete the data. `/tmp` is used as an example reference here, you can substitute `/tmp` with any other directory path in your environment. ``` ./purge_mongo.sh --mldata_db abs_mldata -d 80 -l /tmp --dump_only Starting the purge mongo tool Storing abs ml data from mongo at /tmp/ml_mongo_data.2020-11-16-06-09-06 Please see /opt/pingidentity/abs/logs/purge/purge.log.2020-11-16-06-10-42 for more details ``` ## Purge and dump data The following are a few sample commands to purge and dump the data: * ./purge\_mongo.sh --data\_db \ -d \ -l \ * ./purge\_mongo.sh --mldata\_db \ -d \ -l \ * ./purge\_mongo.sh --data\_db \ --mldata\_db \ -d \ -l \ For example, the following command dumps data older than 80 days from ml\_database into a `/tmp` directory and deletes the data. `/tmp` is used as an example reference here, you can substitute `/tmp` with any other directory path in your environment. ``` ./purge_mongo.sh --mldata_db abs_mldata -d 80 -l /tmp Starting the purge mongo tool Storing abs ml data from mongo at /tmp/ml_mongo_data.2020-11-16-06-12-14 This will delete the documents in abs_mldata database that are older than 80 days. Are you sure (yes/no): yes Deleting the documents in abs_mldata database that are older than 80 days. Please see /opt/pingidentity/abs/logs/purge/purge.log.2020-11-16-06-12-42 for more details ``` | | | | - | -------------------------------------------------------------------------------------------- | | | By default, the script dumps all data and then removes processed data older than seven days. | In case there is a failure in execution of `purge_mongo.sh` the script exits. You can retry executing the script. The execution details are logged in`//pingidentity/abs/logs/purge/`directory.