--- title: Upgrading PingIntelligence description: After upgrading Elasticsearch, upgrade PingIntelligence 5.1 to 5.2 and switch to RHEL 8. component: pingintelligence version: 5.2 page_id: pingintelligence:installing_pingintelligence_for_apis:pingintelligence_upgrading canonical_url: https://docs.pingidentity.com/pingintelligence/5.2/installing_pingintelligence_for_apis/pingintelligence_upgrading.html revdate: May 6, 2024 section_ids: before-you-begin: Before you begin about-this-task: About this task steps: Steps result: Result: result-2: Result: result-3: Result: result-4: Result: result-5: Result: --- # Upgrading PingIntelligence After upgrading Elasticsearch, upgrade PingIntelligence 5.1 to 5.2 and switch to RHEL 8. ## Before you begin Stop all PingIntelligence 5.1 components before starting the upgrade. ## About this task To upgrade PingIntelligence: ## Steps 1. Upgrade API Security Enforcer (ASE) from 5.1.1 to 5.1.3 in the corresponding RHEL 7.9 instance. | | | | - | ------------------------------------- | | | There is no 5.2 RHEL 8 build for ASE. | 1. Make sure that ASE is stopped. 2. Make a backup of the existing ASE base folder. 3. Copy the ASE 5.1.3 build. 4. Untar the new build. 5. Update the `ase.conf` and `abs.conf` with the required details, such as port, ASE mode, and API Behavioral Security (ABS) IP, by referring to backed-up `conf` files. 6. Add the actual passwords for the following passwords: ``` config/ase.conf: sender_password, keystore_password config/abs.conf: access_key, secret_key config/cluster.conf: cluster_secret_key ``` 7. Copy `ase.crt` from the backup folder to the `ase/config` directory. 8. Copy the `PingIntelligence.lic` license file to the `ase/config` directory. 9. Generate the master key. ``` /opt/pingidentity/ase/bin/cli.sh generate_obfkey -u admin -p ``` 10. Obfuscate the key. ``` /opt/pingidentity/ase/bin/cli.sh obfuscate_keys -u admin -p ``` 11. Copy the API JSON files to the `config/api` directory. 12. Start ASE. ``` /opt/pingidentity/ase/bin/start.sh ``` 2. In the already migrated RHEL 8 `mongo`, add the new DB `pi4api_dashboard` and grant `readWrite` roles for the `absuser` for this DB. | | | | - | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | Make sure the `pi4api_dashboard` collection does not exist before the 5.2 upgrade. The 5.2 upgrade will remove the data from the following tables, which are under the `pi4api_dashboard` collection, if it exists:``` api_groups api_state user_sessions users ``` | 1. Shut down the `mongo` primary and secondary. ``` mongod --shutdown --dbpath data/ ``` 2. Start the primary `mongo` without the `--auth` flag. ``` mongod --dbpath ./data/ --logpath ./logs/mongo.log --port 27017 --replSet absrs01 --fork -bind_ip 0.0.0.0 ``` 3. Sign on to `mongo` without specifying a user. ``` mongo ``` 4. Run the following commands: ``` create pi4api_dashboard DB use pi4api_dashboard ``` ``` Switch to admin db use admin ``` Execute the command to grant the `readWrite` role for `absuser` for `pi4api_dashboard`. ``` db.grantRolesToUser("absuser", ["readWrite",{ role: "readWrite", db: "pi4api_dashboard" }]); ``` 5. Shut down the `mongo` primary. 6. Restart `mongo` (both primary and secondary) with `--auth` enabled and with `--tlsMode`. ``` mongod --auth --dbpath ./data/ --logpath ./logs/mongo.log --port 27017 --replSet absrs01 --fork --keyFile ./key/mongodb-keyfile -bind_ip 0.0.0.0 --tlsMode requireTLS --tlsCertificateKeyFile ./key/mongodb.pem ``` 3. Make the following changes in Kafka: 1. Create the discovery topic. ``` /home/ec2-user/pingidentity/kafka/bin/kafka-topics.sh --bootstrap-server 172.16.40.244:9091 --create --topic pi4api.queuing.apis --partitions 1 --replication-factor 1 --command-config /home/ec2-user/pingidentity/kafka/config/client.properties ``` 2. Create the access control lists (ACL) for the ABS producer user for the discovery topic. ``` /home/ec2-user/pingidentity/kafka/bin/kafka-acls.sh --bootstrap-server 172.16.40.244:9091 --add --allow-principal User:abs_producer --operation Create --operation Read --operation Write --topic pi4api.queuing.apis --command-config /home/ec2-user/pingidentity/kafka/config/client.properties ``` 3. Create the ACLs for the ABS consumer user for the discovery topic. ``` /home/ec2-user/pingidentity/kafka/bin/kafka-acls.sh --bootstrap-server 172.16.40.244:9091 --add --allow-principal User:abs_consumer --operation Read --topic pi4api.queuing.apis --command-config /home/ec2-user/pingidentity/kafka/config/client.properties ``` 4. Create the ACLs for the data engine consumer user. ``` /home/ec2-user/pingidentity/kafka/bin/kafka-acls.sh --bootstrap-server 172.16.40.244:9091 --add --allow-principal User:pi4api_de_user --operation Create --operation Read --operation Write --topic pi4api.queuing.apis --command-config /home/ec2-user/pingidentity/kafka/config/client.properties ``` 5. Add the ACLs below in Kafka if they have not already been added: ``` Current ACLs for resource ResourcePattern(resourceType=TOPIC, name=pi4api.queuing.anomalies, patternType=LITERAL): (principal=Group:pi4api.abs, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=User:pi4api_de_user, host=, operation=READ, permissionType=ALLOW) (principal=User:abs_consumer, host=, operation=READ, permissionType=ALLOW) (principal=User:abs_consumer, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=Group:pi4api.abs, host=, operation=READ, permissionType=ALLOW) (principal=Group:pi4api.data-engine, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=User:pi4api_de_user, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=User:abs_producer, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=Group:pi4api.data-engine, host=, operation=READ, permissionType=ALLOW) (principal=User:abs_producer, host=, operation=WRITE, permissionType=ALLOW) Current ACLs for resource ResourcePattern(resourceType=GROUP, name=pi4api.abs, patternType=LITERAL): (principal=User:abs_consumer, host=, operation=READ, permissionType=ALLOW) (principal=User:abs_consumer, host=, operation=DESCRIBE, permissionType=ALLOW) Current ACLs for resource ResourcePattern(resourceType=TOPIC, name=pi4api.queuing.ioas, patternType=LITERAL): (principal=Group:pi4api.abs, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=User:pi4api_de_user, host=, operation=READ, permissionType=ALLOW) (principal=User:abs_consumer, host=, operation=READ, permissionType=ALLOW) (principal=User:abs_consumer, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=Group:pi4api.abs, host=, operation=READ, permissionType=ALLOW) (principal=Group:pi4api.data-engine, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=User:pi4api_de_user, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=User:abs_producer, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=Group:pi4api.data-engine, host=, operation=READ, permissionType=ALLOW) (principal=User:abs_producer, host=, operation=WRITE, permissionType=ALLOW) Current ACLs for resource ResourcePattern(resourceType=TOPIC, name=pi4api.queuing.apis, patternType=LITERAL): (principal=User:abs_producer, host=, operation=READ, permissionType=ALLOW) (principal=Group:pi4api.abs, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=User:pi4api_de_user, host=, operation=READ, permissionType=ALLOW) (principal=User:abs_consumer, host=, operation=READ, permissionType=ALLOW) (principal=User:abs_consumer, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=Group:pi4api.abs, host=, operation=READ, permissionType=ALLOW) (principal=Group:pi4api.data-engine, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=User:pi4api_de_user, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=User:pi4api_de_user, host=, operation=WRITE, permissionType=ALLOW) (principal=User:abs_producer, host=, operation=CREATE, permissionType=ALLOW) (principal=User:abs_producer, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=Group:pi4api.data-engine, host=, operation=READ, permissionType=ALLOW) (principal=User:abs_producer, host=, operation=WRITE, permissionType=ALLOW) (principal=User:pi4api_de_user, host=, operation=CREATE, permissionType=ALLOW) Current ACLs for resource ResourcePattern(resourceType=GROUP, name=pi4api.data-engine, patternType=LITERAL): (principal=User:pi4api_de_user, host=, operation=READ, permissionType=ALLOW) (principal=User:pi4api_de_user, host=, operation=DESCRIBE, permissionType=ALLOW) Current ACLs for resource ResourcePattern(resourceType=TOPIC, name=pi4api.queuing.transactions, patternType=LITERAL): (principal=Group:pi4api.abs, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=User:pi4api_de_user, host=, operation=READ, permissionType=ALLOW) (principal=User:abs_consumer, host=, operation=READ, permissionType=ALLOW) (principal=User:abs_consumer, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=Group:pi4api.abs, host=, operation=READ, permissionType=ALLOW) (principal=Group:pi4api.data-engine, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=User:pi4api_de_user, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=User:abs_producer, host=, operation=DESCRIBE, permissionType=ALLOW) (principal=Group:pi4api.data-engine, host=, operation=READ, permissionType=ALLOW) (principal=User:abs_producer, host=, operation=WRITE, permissionType=ALLOW) ``` 4. Upgrade the Dashboard from 5.1.0.2 to 5.1.1. | | | | - | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | Make sure the `discovered_apis` index does not exist before upgrading the Dashboard from 5.1.0.2. During the 5.2 Dashboard upgrade, all the documents related to this index will be cleaned up. | 1. Stop the `dataengine` and `webgui`. 2. Make a backup of `dataengine.jks`, `kafka_truststore.jks`, and `webgui.jks` files, and save them separately. 3. Delete the `dataengine` and `webgui` folders inside the Ping Identity directory. 4. Download the Dashboard 5.1.1 build to the Dashboard instance `/home/ec2-user` folder. 5. Untar the downloaded build. ### Result: The `dataengine` and `webgui` folders will be created inside the Ping Identity directory. 1. Make the following changes to `dataengine`: 1. Go to the `dataengine` folder. ``` cd /home/ec2-user/pingidentity/dataengine/config ``` 2. Copy the `dataenine.jks` (from the backup) file to the `dataengine/config` directory. 3. Copy `kafka_truststore.jks` (from the backup) file to the `dataengine/config` directory. 4. Update the properties below in `kafka.properties` with valid entries: ``` pi.kafka.bootstrap-servers=10.96.6.45:9093 pi.kafka.consumer.sslTruststoreLocation=/home/ec2-user/pingidentity/dataengine/config/kafka_truststore.jks pi.kafka.consumer.sslTruststorePassword=changeme pi.kafka.consumer.groupId=pi4api.data-engine pi.kafka.consumer.authentication.username=pi4api_de_user pi.kafka.consumer.authentication.password=changeme ``` 5. Update `dataengine.properties` with valid entries: ``` pi.dataengine.server.ssl.key-store-password=changeme pi.dataengine.server.ssl.key-alias= pi.dataengine.abs.url=https://10.96.6.82:8080 pi.dataengine.abs.access_key=abs_ak pi.dataengine.abs.secret_key=abs_sk pi.dataengine.elasticsearch.url=https://10.96.6.45:9200 pi.dataengine.elasticsearch.username=elastic pi.dataengine.elasticsearch.password=changeme ``` 2. Generate a new master key. ``` cd /home/ec2-user/dataengine ./bin/cli.sh generate_obfkey ``` 3. Obfuscate keys. ``` ./bin/cli.sh obfuscate_keys ``` 4. Start the `datenegine`. ``` ./bin/start.sh ``` 5. Make the following changes to `webgui`: 1. Go to the `webgui` folder. ``` cd /home/ec2-user/pingidentity/webgui/config ``` 2. Copy the `webgui.jks` (from the backup) file to the `webgui/config` directory. 3. Update the properties below in the `webgui.properties` file: ``` pi.webgui.server.ssl.key-store-password=changeme pi.webgui.server.ssl.key-alias= pi.webgui.abs.url=https://10.96.6.82:8080 pi.webgui.abs.api-service-url=https://10.96.6.82:8050 pi.webgui.abs.access-key=abs_ak pi.webgui.abs.secret-key=abs_sk pi.webgui.ase.url=https://10.96.6.80:8010 pi.webgui.ase.access-key=ase_ak pi.webgui.ase.secret-key=ase_sk pi.webgui.elasticsearch.url=https://10.96.6.45:9200 pi.webgui.elasticsearch.username=elastic pi.webgui.elasticsearch.password=changeme pi.webgui.datasource.username=sa pi.webgui.datasource.password=changeme pi.webgui.datasource.encryption-password=changeme ``` 4. Generate a new master key. ``` cd /home/ec2-user/webgui ./bin/cli.sh generate_obfkey ``` 5. Obfuscate keys. ``` ./bin/cli.sh obfuscate_keys ``` 6. Start the `webgui`. ``` ./bin/start.sh ``` | | | | - | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | | Use only RHEL 8 instances to run the remaining PingIntelligence 5.2 components (ABS, API Publish, machine learning (ML) service, data engine, and web GUI). | 5. Stop all PingIntelligence 5.1 components (ABS, API Publish, Dashboard) before starting the 5.2 upgrade. 6. To upgrade ABS, make sure you have a RHEL 8 instance ready to install the PingIntelligence ABS 5.2 build and then proceed with the following: 1. Install Java 11.0.2 and set `JAVA_HOME`. 2. Download the PingIntelligence ABS 5.2 build from the [PingIntelligence Downloads website](https://www.pingidentity.com//en/resources/downloads/pingintelligence/pingintelligence-software.html). 3. Copy the build to the RHEL 8 instance. 4. Untar the build. ### Result: An `abs` folder will be created inside the `pingidentity` folder. 5. Copy the `PingIntelligence.lic` license file to the `pingidentity/abs/config` directory. 6. Copy the `abs.jks` file from the old ABS (RHEL 7, ABS 5.1) to the new RHEL 8 ABS `ssl` directory. ``` /pingidentity/abs/config/ssl/ ``` You can alternatively create a new `abs.jks` file. 1. Copy the `kafka.truststore.jks` file from the old ABS (RHEL 7, ABS 5.1) to new RHEL 8 ABS corresponding path. ``` /opt/pingidentity/abs/config/kafka.truststore.jks ``` 2. Update the properties below in `Kafka.properties`: ``` pi.kafka.bootstrap-servers=10.96.6.196:9093 pi.kafka.sslTruststoreLocation=/opt/pingidentity/abs/config/kafka.truststore.jks pi.kafka.sslTruststorePassword= pi.kafka.consumer.authentication.password= pi.kafka.producer.authentication.password= pi.kafka.producer.min-insync-replicas=1 ``` 3. Update the `abs.properties` file with the details below: ``` jks_password= Mongo_rs=mongodb://10.96.6.242:27017,10.96.6.201:27017 mongo_username=absuser mongo_password=abs123 mongo_ssl=true email_password= ``` 4. Generate a new ABS master key. ``` /opt/pingidentity/abs/bin/cli.sh generate_obfkey -u admin -p admin ``` 5. Obfuscate keys. ``` /opt/pingidentity/abs/bin/cli.sh obfuscate_keys -u admin -p admin ``` ### Result: The following keys will be obfuscated: `config/abs.properties`: `mongo_password`, `jks_password`, and `email_password` `config/kafka.properties`: `pi.kafka.consumer.authentication.password`, `pi.kafka.producer.authentication.password`, and `pi.kafka.sslTruststorePassword` 6. Start ABS. ``` /opt/pingidentity/abs/bin/start.sh ``` 7. To upgrade API Publish, make sure you have a RHEL 8 instance ready to install the PingIntelligence ABS 5.2 build and then proceed with the following: 1. Install Java 11.0.2 and set `JAVA_HOME`. 2. Download the PingIntelligence API Publish 5.2 build from the [PingIntelligence Downloads website](https://www.pingidentity.com//en/resources/downloads/pingintelligence/pingintelligence-software.html). 3. Copy the build to the RHEL 8 instance. 4. Untar the build. ### Result: An `apipublish` folder will be created inside the `pingidentity` folder. 5. Copy the `apipublish.jks` file from the old API Publish (RHEL 7, API Publish 5.1) to the new RHEL 8 API Publish `ssl` directory. ``` /pingidentity/apipublish/config/ssl/ ``` You can alternatively create a new `apipublish.jks` file. 1. Update the `apipublish.properties` file with the details below: ``` pi.apipublish.ssl.key-store-password=api123 pi.apipublish.datasource.mongo_rs=mongodb://10.96.6.242:27017,10.96.6.201:27017 pi.apipublish.datasource.username=absuser pi.apipublish.datasource.password=abs123 pi.apipublish.datasource.mongo_ssl=true ``` 2. Generate a new API Publish master key. ``` /pingidentity/apipublish/bin/cli.sh generate_obfkey -u admin -p admin ``` 3. Obfuscate keys. ``` /pingidentity/apipublish/bin/cli.sh obfuscate_keys -u admin -p admin ``` ### Result: The following keys will be obfuscated: `config/apipublish.properties`: `pi.apipublish.ssl.key-store-password` and `pi.apipublish.datasource.password` 4. Start API Publish. ``` ../bin/start.sh ``` 8. Install the new ML service 5.2 build on the RHEL 8 instance by following the steps in [Installing the PingIntelligence machine learning service](pingintelligence_install_machine_learning_service.html). 9. To install `dataengine`, make sure you have a RHEL 8 instance with 8 core CPU, 16 GB, 1 TB hard disk drive (HDD). 1. Download the PingIntelligence 5.2 Dashboard build and extract it in the RHEL 8 instance. 2. Install Java 11.0.2 and set `JAVA_HOME`. 3. Copy the `data-engine.jks` file from the old `dataengine` and copy it to the new `dataengine` RHEL 8 instance in `dataengine/config directory`. 4. Add the Mongo certificate to `data-engine.jks`. 1. In the RHEL 8 `mongo` primary node, go to `mongo/key/mongo.pem`and copy the public key part. 2. Store the public key as `mongo.crt` in `dataengine/config`. 3. Run the following command: ``` keytool -import -keystore dataengine.jks -storetype JKS -storepass changeme -alias mongo -file mongo.crt -noprompt ``` 5. Copy the `kafka.truststore.jks` file to the `dataengine/config/` directory. 6. Update `Kafka.properties` with the details below: ``` pi.kafka.bootstrap-servers=:9093 pi.kafka.consumer.sslTruststoreLocation=/opt/pingidentity/dataengine/config/kafka_truststore.jks pi.kafka.consumer.sslTruststorePassword= pi.kafka.consumer.authentication.password= ``` 7. Update `Dataengine.properties` with the details below: ``` pi.dataengine.server.ssl.key-store-password= pi.dataengine.server.ssl.key-alias= # abs properties pi.dataengine.abs.url=https://:8080 pi.dataengine.abs.access_key=abs_ak pi.dataengine.abs.secret_key=abs_sk pi.dataengine.elasticsearch.url=https://:9200 pi.dataengine.elasticsearch.username=elastic pi.dataengine.elasticsearch.password= pi.dataengine.datasource.url=mongodb://:27017 pi.dataengine.datasource.username=absuser pi.dataengine.datasource.password=abs123 ``` 8. Generate `dataengine_master.key`. ``` ./bin/cli.sh generate_obfkey ``` 9. Obfuscate keys. ``` ./bin/cli.sh obfuscate_keys ``` 10. Start `dataengine`. ``` ./bin/start.sh ``` 10. Install `webgui`. 1. Copy the `h2-backup` folder (that was copied and saved in [Migrating Elasticsearch from RHEL 7.9 to 8](pingintelligence_migrating_elasticsearch.html) in step 13e) to the RHEL 8 instance under the `webgui/data` directory. 2. Copy `webgui.jks` from the old `webgui` instance to the RHEL 8 instance `webgui/config` directory. 3. Add the Mongo certificate to `webgui.jks`. 1. In the RHEL 8 `mongo` primary node, go to `mongo/key/mongo.pem` and copy the public key part. 2. Store the public key as `mongo.crt` in `webgui/config`. 3. Run the following command: ``` keytool -import -keystore webgui.jks -storetype JKS -storepass changeme -alias mongo -file mongo.crt -noprompt ``` 4. Update `webgui.properties` with the details below: ``` pi.webgui.server.ssl.key-store-password= pi.webgui.server.ssl.key-alias= pi.webgui.abs.url=https://10.96.6.242:8080 pi.webgui.abs.api-service-url=https://10.96.6.242:8050 pi.webgui.abs.access-key= pi.webgui.abs.secret-key= # ase properties pi.webgui.ase.url=https://10.96.6.217:8010 pi.webgui.ase.access-key= pi.webgui.ase.secret-key= # elasticsearch properties pi.webgui.elasticsearch.url=https://10.96.6.19:9200 pi.webgui.elasticsearch.username=elastic pi.webgui.elasticsearch.password= pi.webgui.datasource.url=mongodb://10.96.6.242:27017 pi.webgui.datasource.username=absuser pi.webgui.datasource.password=abs123 ``` 5. Generate `dataengine_master.key`. ``` ./bin/cli.sh generate_obfkey ``` 6. Obfuscate keys. ``` ./bin/cli.sh obfuscate_keys ``` 7. Start `webgui`. ``` ./bin/start.sh ```