Java Agents 2024.9

Not-enforced rules

When an agent is configured to protect a web application, it protects every resource in that application. Each access to any resource incurs the overhead of a policy evaluation request to AM; access 100 resources, and you incur 100 requests to AM, with all of their associated network overhead.

Some resources, such as the "public" directory of a web application, contain data that is not sensitive. It can be accessed by any, authenticated or unauthenticated, clients. The agent uses lists of not-enforced rules to identify these resources in the web application.

The agent matches incoming requests to the lists of not-enforced rules. When a request matches a not-enforced rule, the agent bypasses the call to AM:

  • If an unauthenticated user sent the request, the agent does not redirect the user to log in.

  • If an authenticated user sent the request, the agent does not request a policy evaluation from AM.

Use not-enforced rules to reduce the number of unnecessary calls to AM, and therefore improve the performance and speed of your application.

The following image shows the data flow when Java Agent evaluates not-enforced rules for a request, first searching for a match in the cache, then in the not-enforced lists:

not-enforced-flow

1. A client requests a resource.

2-4. If the not-enforced URI or IP cache is enabled, the Java Agent checks whether the request matches any cached results. If the same request from the client previously matched a not-enforced rule, the Java Agent passes the request without requiring the client to authenticate.

5. If the caches are not enabled, or the request doesn’t match a cached result, the Java Agent checks whether the request matches a rule in a not-enforced list.

The Java Agent evaluates every rule in the lists in order, until it finds the first match. When it finds a match, it stops evaluating, and does not consider other rules further down the list even if they provide a better match. Take care to place your most specific rules at or near the beginning of the list.

6-8. The Java Agent caches the result and passes the request without requiring the client to authenticate.

9-14. If the request doesn’t match a rule in a not-enforced list, the Java Agent checks whether rules are inverted, and responds as follows:

Not-enforced URI rules Not-enforced IP rules Pass request without requiring authentication?

Inverted

Inverted

Yes

Not inverted

Not inverted

No

Inverted

Not inverted

No

Not inverted

Inverted

No

Configure not-enforced rules

Configure not-enforced rules by using the properties listed in Not-enforced rules in the Properties reference, or on the Application tab of the AM admin UI. Configure the following lists of not-enforced rules:

Not-enforced URI rules

Rules in Not-Enforced URIs allow access to resources, such as images, stylesheets, or the HTML pages that provide the public front end of your site.

Not-enforced IP rules

Rules in Not-Enforced Client IP List allow access to your site from an administrative IP address, an internal network range, or a search engine.

Compound not-enforced URI and IP rules

Allow access based on a combination of resources and IPs.

When there are multiple lists of rules, the agent evaluates them in the following order:

Order Rule type Rule includes requirements for Cookie values or Header values Rule uses the DENY keyword to Deny access

1

Compound

Yes

Yes

2

Compound

Yes

No

3

Compound

No

Yes

4

Compound

No

No

5

IP

Yes

Yes

6

IP

Yes

No

7

IP

No

Yes

8

IP

No

No

9

URL

Yes

Yes

10

URL

Yes

No

11

URL

No

Yes

12

URL

No

No

Conventions for not-enforced rules

Use the following conventions to define not-enforced URI rules and not-enforced IP rules.

Path normalization

Java Agent normalizes resource paths before applying not-enforced rules.

During this process, %2F becomes /.

Invert rules

Invert specific rules

The NOT keyword is overridden by the DENY keyword. Refer to Deny access.

Invert any rule in a not-enforced list by preceding it with the keyword NOT, separated by a space (blank) character.

In the following example, requests for a .jpg file in the /private URI require authentication:

NOT /private/*.jpg

In the following example, the agent enforces authentication and requests policy evaluations for any request from the network specified by the 192.168.1.0/24 CIDR notation:

NOT 192.168.1.0/24

Invert all rules

For security considerations, do not invert all rules. Instead, use the NOT keyword to invert specific rules.
The NOT keyword and the properties in this section are overridden by the DENY keyword. Refer to Deny access.

Invert all rules by setting Invert Not-Enforced IPs or Invert Not-Enforced URIs to true.

Wildcards

For more information about using wildcards, refer to Wildcards.

Note that trailing forward slashes are not recognized as part of a resource name. Therefore, /images//, /images/, and /images are equivalent.

Multi-level wildcard (*)

The following list summarizes the behavior of the multi-level wildcard (*):

  • Matches zero or more occurrences of any character except for the question mark (?).

  • Spans multiple levels in a URL.

  • Cannot be escaped. Therefore, the backslash character (\) or other characters cannot be used to escape the asterisk, as such \*.

  • * Cannot be used in the same rule as the one-level wildcard (-*-) or a regular expression.

The following table gives examples of the multi-level wildcard * in rules defined in Not-Enforced Client IP List:

Multi-level wildcard for not-enforced IP rules
Rule Matches request IP Does not match request IP

192.168.1.*

192.168.1.0

192.168.1.0/24

192.168.0.1

The following table gives examples of the multi-level wildcard * in rules defined in Not-Enforced URIs:

Multi-level wildcard for not-enforced URI rules
Rule Matches request URL Does not match request URL

http://A-examp.com:8080/*

http://A-examp.com:8080/

http://A-examp.com:8080/index.html

http://A-examp.com:8080/x.gif

http://B-examp.com:8080/

http://A-examp.com:8090/index.html

http://A-examp.com:8080/a?b=1

http://A-examp.com:8080/*.html

http://A-examp.com:8080/index.html

http://A-examp.com:8080/pub/ab.html

http://A-examp.com:8080/pri/xy.html

http://A-examp.com/index.html

http://A-examp.com:8080/x.gif

http://B-examp.com/index.html

http://A-examp.com:8080/*/ab

http://A-examp.com:8080/pri/xy/ab/xy/ab

http://A-examp.com:8080/xy/ab

http://A-examp.com/ab

http://A-examp.com/ab.html

http://B-examp.com:8080/ab

http://A-examp.com:8080/ab/*/de

http://A-examp.com:8080/ab/123/de

http://A-examp.com:8080/ab/ab/de

http://A-examp.com:8080/ab/de/ab/de

http://A-examp.com:8080/ab/de

http://A-examp.com:8090/ab/de

http://B-examp.com:8080/ab/de/ab/de

One-level wildcard (-*-)

The following list summarizes the behavior of the one-level wildcard (-*-):

  • Matches zero or more occurrences of any character except for the forward slash (/) and the question mark (?).

  • Does not span multiple levels in a URL.

  • Cannot be escaped. Therefore, the backslash character (\) or other characters cannot be used to escape the hyphen-asterisk-hyphen, like this \-*-.

  • Cannot be used in the same rule as the multi-level wildcard (*) or a regular expression.

The following table gives examples of the one-level wildcard -*- in rules defined in Not-Enforced URIs:

One-level wildcard for not-enforced URI rules
Rule Matches request URL Does not match request URL

http://A-examp.com:8080/b/-*-

http://A-examp.com:8080/b/

http://A-examp.com:8080/b/cd

http://A-examp.com:8080/b/cd/

http://A-examp.com:8080/b

http://A-examp.com:8080/b/c?d=e

http://A-examp.com:8080/b/cd/e

http://A-examp.com:8090/b/

http://A-examp.com:8080/b/-*-/f

http://A-examp.com:8080/b/c/f

http://A-examp.com:8080/b/cde/f

http://A-examp.com:8080/b/c/e/f

http://A-examp.com:8080/f/

http://A-examp.com:8080/b/c-*-/f

http://A-examp.com:8080/b/cde/f

http://A-examp.com:8080/b/cd/f

http://A-examp.com:8080/b/c/f

http://A-examp.com:8080/b/c/e/f

http://A-examp.com:8080/b/c/

http://A-examp.com:8080/b/c/fg

Multiple wildcards

When multiple wildcards are included in the same rule of a Not-Enforced URIs, the agent matches the parameters in any order that they appear in a resource URI.

For example, the following rule applies to any resource URI that contains a member_level and location query parameter, in any order:

/customers/*?*member_level=*&location=*

The following requests would be not-enforced:

https://www.example.com/customers/default.jsp?member_level=silver&location=fr
https://www.example.com/customers/default.jsp?location=es&member_level=silver
https://www.example.com/customers/default.jsp?location=uk&vip=true&member_level=gold

Deny access

Use the DENY keyword with a regular expression or classic pattern to prevent access either to a resource or from an IP address. If the request matches the resource path or IP address, Java Agent denies access with an HTTP 403 Forbidden status.

When the DENY keyword is used, access to a resource and/or access from an IP address is always denied. If the NOT keyword or the following properties are used with the DENY keyword, they are ignored: Invert Not-Enforced IPs or Invert Not-Enforced URIs.

Consider the following examples:

  • Deny access to requests from the specified IP address:

    DENY 155.251.79.32

  • Deny access to all .jpg files:

    DENY /*.jpg

    Note here that the use of DENY causes NOT to be ignored:

    NOT,DENY /*.jpg

  • DENY access to incoming URLs containing dubious characters, such as percent characters remaining in the normalized path. Java Agent converts %2F to / and %5C to \ before applying not-enforced rules:

    DENY /*%*

Regular expressions

The Java Agent uses regular expression matching from the JDK. Make sure your expressions are evaluated in a way that is consistent with this.

Add the keyword REGEX or REGEXP followed by a blank (space) character before the URI or IP address. For example:

REGEX https?://www\.example\.com/([^/])+/.*\.jpg
REGEX 192\.168\.10\.\d+

Consider the following points when using regular expressions:

  • Wildcards and regular expressions cannot be used in the same rule.

  • Using netmask CIDR notation or IP address ranges and regular expressions is not supported. However, you can create a regular expression that matches a range of IP addresses, such as:

    REGEX 192\.168\.10\.(10|\d)

  • If an invalid regular expression is specified in a rule, the rule is dropped and an error message is logged.

HTTP Methods

Add one or more of the following keywords to the not-enforced rule to apply it when the incoming request uses a specific HTTP method: GET, HEAD, POST, PUT, PATCH, DELETE, OPTIONS, TRACE.

By default, no HTTP method is specified for a rule, and all methods are not-enforced for that rule. When one or more HTTP methods are specified, only those methods are not-enforced; methods that are not specified are enforced.

The following example does not require authentication for any request method to 192.168.10.*:

192.168.10.*

In the following example, the agent does not enforce authentication or request policy evaluations for GET requests to /public, but does for other HTTP methods:

GET /public/*

To specify a list of methods, add a comma-delimited list of methods, followed by a blank (space) character before the item to match.

GET,POST /public/*
GET,POST,PUT /examples/notenforced/*.jpg
GET,REGEX https?://www\.example\.com/([^/])+/.*\.jpg
NOT,GET,REGEX 192\.168\.10\.\d+
POST 192.168.10.*
GET 192.168.10.1-192.168.10.254 192.168.0.1
POST,PUT 192.168.1.0/24

To invert a method, precede it with an exclamation point ! character. In the following examples, the agent enforces authentication and requests policy evaluations for POST requests, but not for other HTTPS methods:

!POST /public/*
!POST 192.168.1.0/24

Unrecognized keywords in a rule are ignored and do not invalidate the rest of the rule.

Use the following syntax to require the incoming request to have a named cookie with a specified value:

COOKIE(Name/Value/Modifiers) Not Enforced URIs
COOKIE(Name/Value/Modifiers) Not Enforced IPs

  • Name: Cookie name

  • Value: Cookie value

  • Modifiers: One or more modifiers to change the lookup method:

    • c: (For not-enforced URI rules only) Perform a case-insensitive search for the cookie name. By default, the search is case-sensitive.

    • i: Perform a case-insensitive search for the cookie value. By default, the search is case-sensitive.

    • r: Treat the string in Value as a regular expression.

The following example does not require authentication for requests to /private/admin/images/, when the request contains a cookie with the case-insensitive name login_result, and case-insensitive value valid:

COOKIE(login_result/valid/ci) /private/admin/images/*

Because the search is case-insensitive, the following example is equivalent:

COOKIE(Login_result/VALID/ci) /private/admin/images/*

In the following example, the agent does not enforce authentication or request policy evaluations for requests to 192.168.*, when the request contains a cookie with the case-sensitive name login_result and the case-insensitive value VALID:

COOKIE(login_result/VALID/i) 192.168.*

Combine cookie filters with other filters, such as HTTP methods. Combining a HEADER and COOKIE expression in the same rule implies a logical AND; both expressions must match in order to apply. To apply the rules as a logical OR, create two separate rules.

In the following example, the agent does not enforce authentication or request policy evaluations for GET, POST, and PUT requests to the /other/records/ folder, when the request contains a cookie with the case-sensitive name internal, and a case-insensitive value that ends with .*ID:

GET,POST,COOKIE(internal/.*ID/ri),PUT /other/records/*.html

In the following example, the agent does not enforce authentication or request policy evaluations for GET, POST, and PUT HTTP requests from the client IP range 192.168.*, when the request contains a cookie with the case-sensitive name internal, and a case-sensitive value that ends with .*ID:

GET,POST,COOKIE(internal/.*ID/r),PUT 192.168.*

Header values

Use the following syntax to require the incoming request to have a named header with a specified value:

HEADER(Name/Value/Modifiers) Not Enforced URIs
HEADER(Name/Value/Modifiers) Not Enforced IPs

  • Name: Header name.

  • Value: Header value to search for.

  • Modifiers

    • i: Perform a case-insensitive search for the header value. By default, the search is case-sensitive.

    • r: Treat the string in Value as a regular expression.

In the following example, the agent does not enforce authentication or request policy evaluations for access to .txt files in /yearly/2021/ when the request contains a header with the case-sensitive name ID, and a case-insensitive value validated

HEADER(ID/validated/i) /yearly/2021/*.txt

In the following example, the agent does not enforce authentication or request policy evaluations for access to the IP range 192.168.* when the request contains a header with the case-sensitive name ID, and a case-insensitive value validated:

HEADER(ID/validated/i) 192.168.*

Combine cookie filters with other filters, such as HTTP methods. Combining a HEADER and COOKIE expression in the same rule implies a logical AND; both expressions must match. To apply the rules as a logical OR, create two separate rules.

In the following example, the agent does not enforce authentication or request policy evaluations for GET, POST, and PUT requests to HTML resources in the /other/records/ folder when the request contains a header with the case-insensitive name internal, and a case-insensitive value that ends with .*ID:

GET,POST,HEADER(internal/.*ID/ri),PUT /other/records/*.html

Compound rules

Configure compound not-enforced rules to combine not-enforced URI and IP rules in a single rule.

Configure rules in either Not-Enforced Client IP List or Not-Enforced URIs, using an IP rule or list of IP rules, a delimiter, and an URI rule or list of URI rules.

In the following example, the agent does not enforce authentication or request policy evaluations for HTTP requests from the IP range 192.168.1.1-192.168.4.3 to any file in the /images URI:

192.168.1.1-192.168.4.3 | /images/*

Consider the following points for compound rules:

  • Place keywords, such as HTTP methods, NOT, and REGEX, at the beginning of the compound rule. Keywords affect both the IP and the URI rules.

    In the following example, the agent does not enforce authentication or request policy evaluations for GET or POST HTTP requests from the IP range 192.168.1.1-192.168.4.3, to any file (*) in the /images URI.

    GET,POST 192.168.1.1-192.168.4.3 | /images/*

    In the following example, the agent enforces authentication and requests policy evaluations for any request to a method except POST, from any IP address in the 192.168.1 subnet, to any file in the /private URI.

    NOT,!POST 192.168.1.* | /private/*

  • Check that both sides of a rule using the REGEX keyword can be parsed as a regular expression.

    In the following example, the delimiter is &&, because the | character can lead to invalid regular expressions:

    POST,REGEX 192\.168\.10\.(10|\d) && \/images\/([^/])+\.*\.jpg

    For information about configuring a different delimiter, refer to Not-Enforced Compound Rule Separator.

  • The agent caches hits and misses for each resource accessed.

    Caching is enabled if either Enable Not-Enforced IP Cache or Enable Not-Enforced URIs Cache is true.

    The cache size takes the biggest value of Max Entries in Not-Enforced IP Cache or Max Entries in Not-Enforced URI Cache.

Extended characters

URLs defined in Not-Enforced URIs can contain any number of extended ASCII characters. The agent container automatically percent-encodes extended characters, before the agent is called.

Extended characters in the resource path of a not-enforced rule

By default, Java Agent uses UTF-8 to percent-encode extended characters in the resource paths of not-enforced rules. To change the character encoding, set Container Character Encoding.

In the following example, the agent does not enforce authentication or request policy evaluation for HTTP requests to the URL http://www.example.com/forstå:

org.forgerock.agents.notenforced.uri.list=http://www.example.com/forstå/*

Note how the extended ASCII character å can be entered without encoding.

Extended characters in HTTP query parameters of a not-enforced rule

By default, Java Agent uses ISO-8859-1 to encode extended characters in HTTP query parameters of not-enforced rules. To change the character encoding, set Container Parameter Encoding.