--- title: Backup and restore using DS utilities description: The DS operator supports two DS utilities that back up directory data: component: forgeops version: 7.3 page_id: forgeops::how-to/backup/ds-utilities canonical_url: https://docs.pingidentity.com/forgeops/7.3/how-to/backup/ds-utilities.html keywords: ["Backup & Restore", "Directory Server", "dsbackup", "export-ldif"] section_ids: back_up_the_ds_idrepo_directory: Back up the ds-idrepo directory archive: Archive restore_the_ds_idrepo_directory: Restore the ds-idrepo directory --- # Backup and restore using DS utilities The DS operator supports two DS utilities that back up directory data: * [dsbackup](https://docs.pingidentity.com/pingds/7.3/maintenance-guide/backup-restore.html#backup) * [export-ldif](https://docs.pingidentity.com/pingds/7.3/tools-reference/export-ldif.html) To back up directory data using one of these DS utilities, update the DS operator's configuration to create a backup job. The backup job: * Takes a snapshot of the directory data volume. * Copies the data from the snapshot to an ephemeral persistent volume. * Runs the dsbackup or export-ldif utility on the data on the ephemeral volume. The DS utility writes the utility's output to another persistent volume. You then archive the backup to cloud storage, where you can maintain it for as long as you need to. ![ForgeOps Backup with DS utilities.](../../_images/ds-utilities-backup.svg) The next sections include example steps to back up and restore the `ds-idrepo` directory. To back up and restore the `ds-cts` directory, follow similar steps. ## Back up the `ds-idrepo` directory To back up the `ds-idrepo` directory using the dsbackup or export-ldif utility: 1. In a browser window, [log in to the Ping Identity Platform admin UI](../../cdm/access.html), and then create an example identity using the Identities > Manage option. You'll use this identity later to verify that backup and restore were successful. 2. Log out of the Ping Identity Platform admin UI. 3. Set the active namespace in your local Kubernetes context to the namespace in which the CDM is deployed. 4. Run the kubectl get pvc command to get the size of the volume that holds the `ds-idrepo` directory's data. The `CAPACITY` column contains the volume size: ``` $ kubectl get pvc NAME STATUS VOLUME CAPACITY ... ... data-ds-idrepo-0 Bound pvc-04293c38-05a8-44b0-b137-0db259854971 100Gi ... data-ds-idrepo-1 Bound pvc-04ab2617-a9a2-4f71-9094-6d3a4b7c0082 100Gi ... data-ds-idrepo-2 Bound pvc-19a9915e-46f4-4ba5-b3fa-7d1ff83f38aa 100Gi ... ... ``` 5. Update the /path/to/forgeops/etc/backup/ds-backup-restore-ds-operator/ds-backup.yaml file, which contains a default configuration for backing a CDM directory: 1. Set `volumeClaimSpec/resources/requests/storage` to the size of the volume that holds the `ds-idrepo` directory's data. 2. Set `env/value` to `tar,ldif` (for an LDIF backup) or to `tar,dsbackup` (for a standard DS backup). 3. Set the value of `claimToBackup` to `data-ds-idrepo-1`. 6. Apply your changes to the DS operator's backup configuration. Changing this configuration initiates the backup process: ``` $ cd /path/to/forgeops/etc $ kubectl apply -f backup/ds-backup-restore-ds-operator/ds-backup.yaml directorybackup.directory.forgerock.io/ds-backup created ``` 7. Verify that the backup process started: 1. Run the kubectl get jobs command. You should see a job named `ds-backup`. 2. Run the kubectl get pods command. You should see a pod whose name starts with the string, `ds-backup-`. 3. Run the kubectl get volumesnapshot command. You should see that the `temp-ds-backup` snapshot has been created. This snapshot is ephemeral, used only during the backup process. 4. Run the kubectl get pvc command. You should see that two new PVCs have been created: * The `temp-ds-backup` PVC, which is an ephemeral PVC, is used as input to the DS backup utility that you're running. * The `ds-backup` PVC, which is created from the `temp-ds-backup` PVC, contains the output from the backup. This PVC should be archived to cloud storage after the backup process has completed. 8. Verify that the backup process completed: 1. Review the value in the `COMPLETIONS` column of the kubectl get jobs ds-backup command output: ``` $ kubectl get jobs ds-backup NAME COMPLETIONS DURATION AGE ds-backup 1/1 3m13s 3m40s ``` 2. Review the `ds-backup` pod's log, which contains the output from the DS utility—either `dsbackup` or `export-ldif`. The text `done` in the last line of the log indicates that the backup completed. With the backup job completed, you're ready to archive the `ds-backup` PVC to cloud storage. ## Archive After you've backed up your data using one of the DS utilities, move the data to cloud storage. Archive your backup data by using a tool from your cloud provider or a third-party product. The CDM does not include a data archival tool. ## Restore the `ds-idrepo` directory To test restoring DS instances using DS utility backups: 1. Set the active namespace in your local Kubernetes context to the namespace in which the CDM is deployed. 2. Delete your CDM deployment. Be sure to reply `N` when you're prompted to delete PVCs, volume snapshots, and secrets: ``` $ cd /path/to/forgeops/bin $ ./forgeops delete "small" platform detected in namespace: "my-namespace". Uninstalling component(s): ['all'] from namespace: "my-namespace". OK to delete components? [Y/N] Y OK to delete PVCs? [Y/N] N OK to delete volume snapshots? [Y/N] N OK to delete secrets? [Y/N] N service "admin-ui" deleted ... ``` 3. Delete the `ds-idrepo` PVCs: ``` $ kubectl delete pvc data-ds-idrepo-0 data-ds-idrepo-1 data-ds-idrepo-2 persistentvolumeclaim "data-ds-idrepo-0" deleted persistentvolumeclaim "data-ds-idrepo-1" deleted persistentvolumeclaim "data-ds-idrepo-2" deleted ``` 4. Transfer your archived backup data from cloud storage to a persistent volume using a tool from your cloud provider, or a third-party product. Note that the CDM does not include a tool to transfer data from cloud storage to a persistent volume. 5. Update the /path/to/forgeops/etc/backup/ds-backup-restore-ds-operator/ds-restore.yaml file, which contains a default configuration for restoring a CDM directory: 1. Set `volumeClaimSpec/resources/requests/storage` to the size of the volume that holds the `ds-idrepo` directory's data. 2. Set `env/value` to `tar,ldif` (for an LDIF restore) or to `tar,dsbackup` (for a standard DS restore). Use the same value that you used when you backed up the DS instance. 3. Set the value of `sourcePvcName` to the name of the PVC that contains the backup that you transferred from cloud storage. 6. Apply your changes to the DS operator's restore configuration: ``` $ cd /path/to/forgeops/etc $ kubectl apply -f backup/ds-backup-restore-ds-operator/ds-restore.yaml directorybackup.directory.forgerock.io/ds-restore created ``` 7. Verify that the restore process started: 1. Run the kubectl get jobs command. You should see a job named `ds-restore`. 2. Run the kubectl get pods command. You should see a pod whose name starts with the string, `ds-restore-`. 3. Run the kubectl get volumesnapshots command. You should see a snapshot named `temp-ds-restore`: * The DS operator creates this snapshot from your backup PVC. * The `temp-ds-restore` snapshot will be used to initialize the `ds-idrepo` directory service's PVCs. 8. Verify that the restore process completed: 1. Review the value in the `COMPLETIONS` column of the kubectl get jobs ds-restore command output: ``` $ kubectl get jobs ds-restore NAME COMPLETIONS DURATION AGE ds-restore 1/1 3m13s 3m40s ``` 2. Review the `ds-restore` pod's log, which contains the output from the DS utility—either `dsbackup` or `export-ldif`. 3. Review output from the kubectl get volumesnapshots command. For the `temp-ds-restore` volume snapshot, verify that the value in the `READYTOUSE` column is `true`: ``` $ kubectl get volumesnapshots NAME READYTOUSE SOURCEPVC RESTORESIZE ... temp-ds-restore true temp-ds-restore 100Gi ... temp-ds-backup true data-ds-idrepo-1 100Gi ... ``` Do not proceed to the next step until the `temp-ds-restore` volume snapshot is ready to use. 9. Update the `ds-idrepo` overlay file for your deployment. Specify that the source of the `ds-idrepo` PVC should be the `temp-ds-restore` snapshot: 1. Determine the path to the `ds-idrepo` overlay file for your deployment. For example, the overlay file for a small CDM deployment is at the path, /path/to/forgeops/kustomize/overlay/small/ds-idrepo.yaml. 2. Add the following `dataSource` key to the `podTemplate/volumeClaimSpec` section of the overlay file: ``` dataSource: name: temp-ds-restore kind: VolumeSnapshot apiGroup: snapshot.storage.k8s.io ``` > **Collapse: Example of an updated overlay file for a small CDM deployment** > > ``` > # Sample DirectoryService deployment > apiVersion: directory.forgerock.io/v1alpha1 > kind: DirectoryService > metadata: > name: ds-idrepo > spec: > # The number of DS servers in the topology > replicas: 3 > # The resources assigned to each DS pod > podTemplate: > resources: > requests: > memory: 4Gi > cpu: 1500m > limits: > memory: 6Gi > volumeClaimSpec: > storageClassName: fast > resources: > requests: > storage: 100Gi > # The following triggers restoring from a snapshot: > dataSource: > name: temp-ds-restore > kind: VolumeSnapshot > apiGroup: snapshot.storage.k8s.io > ``` The values in the `dataSource` key tell the CDM which snapshot to use when restoring the `ds-idrepo` PVC. The PVC is restored from a snapshot if: * The PVC does not exist. * The snapshot backup configured in the `dataSource` section exists. 10. Reinstall the CDM using the forgeops install command you had used when you deployed the CDM, for example: ``` $ ./bin/forgeops install --small --fqdn cdm.example.com ``` 11. Run the kubectl describe pvc data-ds-idrepo-0 command and review the output under the label, `DataSource`: ``` DataSource: APIGroup: snapshot.storage.k8s.io Kind: VolumeSnapshot Name: temp-ds-restore ``` The `Kind` field should have a value of `VolumeSnapshot`, indicating that the source of the PVC was the ephemeral volume snapshot created by the restore job. The value in the `Name` field should be `temp-ds-restore`, which is the name of the ephemeral snapshot. 12. Run the kubectl describe pvc data-ds-idrepo-1 and kubectl describe pvc data-ds-idrepo-2 commands. The output should be similar to what you observed in the previous step. 13. Log back in to the Ping Identity Platform admin UI, and then select the Identities > Manage option. You should see the example identity you created before you took a backup.