The ABS MongoDB purge script dumps and/or deletes processed AI Engine and machine learning data from MongoDB. It is recommended to archive the data before purging it. The purge_mongo.sh script is available in the /<pi-install-dir>/pingidentity/abs/util directory.
The script offers three options:
- Only purge data
- Only dump data
- Dump data into a specified directory and then purge it
Prerequisites-Ensure that the following prerequisites are fulfilled, before using the script:
- Execute the script from an ABS AI Engine node with connectivity to the Mongodb primary node.
- The necessary write permissions are available on the directory where the data dump is stored.
- Database names used as command line arguments like the data_dbname and mldata_dbname
should be same as configured in ABS configuration - abs.properties file. Note: It is recommended to execute the script during low load periods or during ABS down times.
The purge_mongo.sh script supports the following arguments:
Argument | Description |
---|---|
--data_db <abs database> |
Use this argument to specify the name of the ABS database. |
--mldata_db <ml database> |
Use this argument to specify the name of the ML database. |
Note: You must specify at least one database while executing the script. You
can also specify both the databases in a single command.
|
|
-d <days> or --days
<days> |
Number of days of data to be retained. The default number of days is seven. The minimum number of days that can be specified is one and the maximum is 365. |
-l <path_to_dump_dir> or --location <
path_to_dump_dir> |
The directory path to store the MongoDB dump. |
--purge_only |
Use this option when you only need to delete the data and not take a data dump. |
--dump_only
|
Use this option when you need to take the data dump without deleting the data. |
--h or --help |
Use this argument for more information on the purge script parameters. |
--gzip |
Use this argument to compress the data dump. It can be used with
dump only and purge and dump options.
This argument cannot be used with purge only
option. |
Purge data
The following are a few sample commands to purge the MongoDB data:
- ./purge_mongo.sh --data_db <abs database> -d <days> --purge_only
- ./purge_mongo.sh --mldata_db <ml database> -d <days> --purge_only
- ./purge_mongo.sh --data_db <abs_database> --mldata_db <ml database> d <days> --purge_only
./purge_mongo.sh --data_db abs_data -d 80 --purge_only
Starting the purge mongo tool
This will delete the documents in abs_data database that are older than 80 days.
Are you sure (yes/no): yes
Deleting the documents in abs_data database that are older than 80 days.
Please see /opt/pingidentity/abs/logs/purge/purge.log.2020-11-16-05-01-42 for more details
Dump data
- ./purge_mongo.sh --data_db <abs database> -d <days> -l <path> --dump_only
- ./purge_mongo.sh --mldata_db <ml database> -d <days> -l <path> --dump_only
- ./purge_mongo.sh --data_db <abs_database> --mldata_db <ml database> d <days> -l <path> --dump_only
./purge_mongo.sh --mldata_db abs_mldata -d 80 -l /tmp --dump_only
Starting the purge mongo tool
Storing abs ml data from mongo at /tmp/ml_mongo_data.2020-11-16-06-09-06
Please see /opt/pingidentity/abs/logs/purge/purge.log.2020-11-16-06-10-42 for more details
Purge and dump data
- ./purge_mongo.sh --data_db <abs database> -d <days> -l <path>
- ./purge_mongo.sh --mldata_db <ml database> -d <days> -l <path>
- ./purge_mongo.sh --data_db <abs_database> --mldata_db <ml database> -d <days> -l <path>
./purge_mongo.sh --mldata_db abs_mldata -d 80 -l /tmp
Starting the purge mongo tool
Storing abs ml data from mongo at /tmp/ml_mongo_data.2020-11-16-06-12-14
This will delete the documents in abs_mldata database that are older than 80 days.
Are you sure (yes/no): yes
Deleting the documents in abs_mldata database that are older than 80 days.
Please see /opt/pingidentity/abs/logs/purge/purge.log.2020-11-16-06-12-42 for more details
In case there is a failure in execution of purge_mongo.sh the script exits. You can retry executing the script. The execution details are logged in /<pi-install path>/pingidentity/abs/logs/purge/directory.