Test Suite
Use the Test Suite to define tests, scenarios, and assertions to validate behavior for most Trust Framework and Policy Manager entities.
Policy writers can build a library of test cases to use as part of a test-driven development approach to policy and Trust Framework design. The library you develop can form a suite of regression checks that you run against each new version of policies or the Trust Framework.
The Test Suite has these components: tests, scenarios, and assertions. The following table highlights the similarities and differences. The components are very similar. However, with test cases, you specify a Trust Framework or Policy Manager entity to test. Scenarios do not use such entities and are instead for reuse across tests.
| Test cases | Scenarios |
|---|---|
A test case definition includes:
|
A scenario definition includes:
You can reuse a scenario within a test suite. |
Tests
In the Test Suite, use the Tests tab to view and manage tests and test groups. A test group is a collection of tests.
To add a test or test group, click .
When you create a test, keep the following items in mind:
| Field | Description | ||
|---|---|---|---|
Name |
A unique name that avoids the following characters: |
||
Description |
A description for the test to clarify its intention and usage. |
||
Tested Entity |
The entity to verify with the test. After you assign an entity, you can run the test on that entity using the Test tab in the Trust Framework or Policy Manager pages. |
||
Scenario Type |
The type of scenario to use:
|
When you create a test group, you need only provide a name and description.
Scenarios
In the Test Suite, use the Scenarios tab to view and manage scenarios and scenario groups. A scenario group is a collection of scenarios.
Scenarios define a decision request and optional attribute and service overrides to serve as input for a test. After you define a scenario, you can reference it by name in your tests. Also, on the Test tab in the Trust Framework or Policy Manager pages, you can load a scenario directly into the test by clicking the Load Scenario button in the lower-right corner.
To add a test or test group, click .
You can specify request and override data by hand, or by importing it in JSON format, by clicking the Import JSON button in the lower-right corner.
When you create a scenario group, you need only provide a name and description.
Assertions
After you define a test scenario, you can create assertions to verify content in the decision response generated by the scenario. Use assertions to ensure that a particular property in the response is behaving correctly.
In the Test Suite, use the Assertions tab to view and manage assertions and assertion groups.
To create an assertion, your options include:
-
Using the Assertions tab.
For assertions you create using the Assertions tab, use them in a test by clicking Add Assertion, setting Assertion Type to Referenced, and then selecting the assertion in the drop-down list.
-
Creating them inline when you define a test on the Tests tab.
When you define an assertion, you:
-
Provide a JSONPath accessor to extract information from the response.
-
Specify a matcher to indicate how to compare the extracted information against an expected value.
-
Specify the expected value type.
-
Specify the expected value.
The following image shows an assertion that checks whether the result value equals PERMIT:
Test execution
After you assign a testable entity, such as a policy or attribute, to a test case, you can run the test. To view and run the test, your options are:
-
In the test definition, after you add the tested entity to the test and save your changes, click the name of the tested entity to view the entity. Next, click Test and then Tests.
-
View the entity through a tab on the left, such as the Policies tab. Next, click Test and then Tests.
You now see a table of the tests available for the entity. Click a test’s Execute button to run that test. For longer running tests, you can go to other tasks in the Policy Editor and return to this page later to check progress.
If a test uses assertions, when you expand the row for the test case, an Assertions tab appears. Use this tab to see the results for the assertions.