PingAuthorize

Tuning API security gateway performance

The API security gateway processes and enforces policy decisions for HTTP requests in real time. Although its default configuration is suitable for many environments, high-throughput scenarios, such as load tests or production systems handling hundreds of concurrent requests per second, often require tuning to maintain performance and stability.

In high-throughput scenarios, you might observe increased response times, timeouts, or degraded throughput. These issues typically stem from resource contention, such as an overwhelmed request processing queue or thread pool exhaustion. Because the gateway handles requests asynchronously using multiple internal thread pools, its performance depends on how well those resources are configured for the demands of your workload.

The following sections introduce the gateway’s asynchronous request flow and explain how to adjust key configuration parameters to align system behavior with your performance goals.

Gateway request flow

The security gateway uses a two-level thread model for asynchronous request processing. This model separates request acceptance from request processing, improving responsiveness under load.

HTTP connection handler threads

These threads accept incoming client connections, parse HTTP requests, and place them into an internal processing queue.

Gateway forward threads

These threads retrieve requests from the internal queue, evaluate access control policies, forward the requests to the backend API, and wait for responses.

If the backend API server is slow to respond, forward threads remain occupied longer, and the queue of waiting requests grows. If requests remain in the queue too long, they could time out.

Performance tuning ensures the system has enough processing threads to keep up with the rate of incoming requests and the latency of backend responses.

Tuning parameters

You should increase the number of HTTP connection handler threads and gateway forward threads if you need to serve higher throughput (more than 10 requests per second). Use the following configuration properties to tune gateway performance:

num-request-handlers

This property controls the size of the connection handler thread pool and determines how many concurrent client connections the server can accept.

Example configuration
dsconfig set-connection-handler-prop \
  --handler-name "HTTP Connection Handler"  \
  --set num-request-handlers:100
shell
num-forward-threads

This property controls the size of the worker thread pool and determines how many requests can be processed in parallel. If your backend API has a high response time, you should increase this value.

Example configuration
dsconfig set-http-servlet-extension-prop \
  --extension-name Gateway  \
  --set num-forward-threads:500
shell

For optimal performance, the num-forward-threads value should be 5 - 10 times greater than the num-request-handlers value.
response-timeout

If your backend API server has a long response time, you must ensure the gateway’s timeout for that external server is configured appropriately. This prevents the gateway from terminating a request too early.

The security gateway has a global request timeout of 30 seconds. You must set the external server’s response timeout to a value less than 30 seconds for proper error handling.
Example configuration
dsconfig set-external-server-prop \
  --server-name "Example server"  \
  --set "response-timeout:25 s"
shell

Monitoring security gateway performance

After tuning the security gateway for higher throughput, use the following monitor providers to observe gateway request processing:

Gateway Timing Monitor Provider

Reports the count and total duration for each phase of request processing in the gateway.

Gateway Request Type Timing Monitor Provider

Reports the count, total and average times of each gateway API endpoint, grouped by HTTP method (GET, POST). The monitor only lists executed gateway API endpoints.

Gateway Statistics Monitor Provider

Reports statistics related to gateway executions. Currently, this monitor only reports the gateway forward queue size.

Analyzing metrics from these providers can help you determine if you’ve resolved the slowdown or if further tuning is required. Learn more about server monitoring in Managing monitoring.