Policy writers can build a library of test cases to use as part of a test-driven development approach to policy and Trust Framework design. The library you develop can form a suite of regression checks that you run against each new version of policies or the Trust Framework.

The Test Suite has these components: Tests, scenarios, and assertions. The following table highlights the similarities and differences. The components are very similar. However, with test cases, you specify a Trust Framework or Policy Manager entity to test. Scenarios do not use such entities and are instead for reuse across tests.

Test cases Scenarios

A test case definition includes:

  • A decision request
  • Optional overrides for attributes
  • Optional overrides for services
  • An entity to test
  • Optional assertions

A scenario definition includes:

  • A decision request
  • Optional overrides for attributes
  • Optional overrides for services

You can reuse a scenario within a test suite.

Tests

In the Test Suite, use the Tests tab to view and manage tests and test groups. A test group is a collection of tests.

To add a test or test group, click +.

When you create a test, keep the following items in mind:

Field Description

Name

A unique name avoiding the following characters: 

{ } | .

Description

A description for the test to clarify its intention and usage.

Tested Entity

The entity to verify with the test.

After you assign an entity, you can run the test on that entity using the Test tab in the Trust Framework or Policy Manager pages.

Scenario Type

The type of scenario to use:

  • Inline—you define the scenario on the same page where you define the test
  • Referenced—you select a scenario that you already defined in the Scenarios tab
Tip:

You can use a referenced scenario as a template for a new inline scenario by selecting that referenced scenario and then switching to Inline Scenario.

When you create a test group, you need only provide a name and description.

Scenarios

In the Test Suite, use the Scenarios tab to view and manage scenarios and scenario groups. A scenario group is a collection of scenarios.

Scenarios define a decision request and optional attribute and service overrides to serve as input for a test. After you define a scenario, you can reference it by name in your tests. Also, on the Test tab in the Trust Framework or Policy Manager pages, you can load a scenario directly into the test by clicking the Load Scenario button in the lower, right corner.

To add a test or test group, click +.

You can specify request and override data by hand or by importing it in JSON format by clicking the Import JSON button in the lower, right corner.

When you create a scenario group, you need only provide a name and description.

Assertions

After you define a test scenario, you can create assertions to verify content in the decision response generated by the scenario. Use assertions to ensure that a particular property in the response is behaving correctly.

In the Test Suite, use the Assertions tab to view and manage assertions and assertion groups.

To create an assertion, your options include:

  • Using the Assertions tab.
    Tip:

    For assertions you create using the Assertions tab, use them in a test by clicking + Add Assertion, setting Assertion Type to Referenced, and then selecting the assertion in the drop-down list.

  • Creating them inline when you define a test on the Tests tab.

When you define an assertion, you:

  1. Provide a JSONPath accessor to extract information from the response.
  2. Specify a matcher to indicate how to compare the extracted information against an expected value.
  3. Specify the expected value type.
  4. Specify the expected value.

The following image shows an assertion that checks whether result value equals PERMIT:

Screen capture of the assertion definition configured as specified

Test execution

After you assign a testable entity, such as a policy or attribute, to a test case, you can run the test. To view and run the test, your options are:

  • In the test definition, after you add the tested entity to the test and save changes, click the name of the tested entity to view the entity. Next, click Test and then Tests.
  • View the entity through a tab on the left, such as the Policies tab. Next, click Test and then Tests.

You see a table of the tests available for the entity. Click a test's Execute button to run that test. For longer running tests, you can go to other tasks in the Policy Editor and return to this page later to check progress.

If a test uses assertions, when you expand the row for the test case, an Assertions tab appears. Use this tab to see the results for the assertions.