--- title: Detecting expensive HTTP threads description: The HTTP Connection Handler can detect expensive threads that spend an unusually long time processing HTTP requests. component: pingauthorize version: 11.0 page_id: pingauthorize:troubleshooting_pingauthorize_server:paz_detecting_expensive_threads canonical_url: https://docs.pingidentity.com/pingauthorize/11.0/troubleshooting_pingauthorize_server/paz_detecting_expensive_threads.html revdate: June 25, 2025 section_ids: how-it-works: How it works configuring-expensive-thread-detection: Configuring expensive thread detection steps: Steps steps-2: Steps analyzing-thread-dump-files: Analyzing thread dump files thread-dump-file-management: Thread dump file management --- # Detecting expensive HTTP threads The HTTP Connection Handler can detect expensive threads that spend an unusually long time processing HTTP requests. This helps identify potential performance bottlenecks and responsiveness issues in the PingAuthorize Server. When request processing takes longer than a configured threshold, the PingAuthorize Server can automatically generate a thread dump. A thread dump is a snapshot of all active application threads, indicating whether each thread is actively running, blocked waiting for a resource, or waiting for another task to complete. Thread dumps also include the sequence of method calls each thread is currently executing. This diagnostic information is helpful for troubleshooting slowdowns caused by issues such as: * Delays in connecting to external data sources * Long-running policy evaluations * Resource contention or deadlocks * Infrastructure problems affecting responsiveness Analyzing a thread dump enables you to pinpoint the specific operations responsible for performance degradation. ## How it works The PingAuthorize Server uses the following process to detect expensive threads: 1. At a regular interval, the server checks the status of all active HTTP threads. 2. If a thread is still handling the same request it was processing during the previous check, the server flags that thread as expensive. 3. If the number of concurrently expensive threads meets a configured threshold, the server creates a thread dump. 4. Each thread dump is saved as a separate file in the `/logs/thread-dumps` directory. To avoid excessive logging, the server waits for a configured interval before creating another thread dump. Thread dump files are named with the following format: `expensive-operation-dump-.log` | | | | - | ---------------------------------------------------------------- | | | The thread dump file path and naming format aren't configurable. | ## Configuring expensive thread detection The HTTP Connection Handler provides the following settings for configuring expensive thread detection: | Setting name | `dsconfig` parameter name | Description | Default value | | --------------------------------------------- | ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | | **Expensive Thread Check Interval** | `expensive-thread-check-interval` | Specifies how frequently the server checks thread activity. If a thread is processing the same request for two consecutive checks, the server flags it as expensive. You must specify a non-zero value for this setting to enable expensive thread detection. | `0 ms` | | **Expensive Thread Minimum Concurrent Count** | `expensive-thread-minimum-concurrent-count` | Specifies the minimum number of simultaneously expensive threads required to generate a thread dump. | `1` | | **Expensive Thread Hold Off Interval** | `expensive-thread-hold-off-interval` | Specifies the cooldown period the server waits after creating a thread dump before it can create another. This setting helps prevent excessive disk usage. | `60000 ms` | You can configure expensive thread detection with the administrative console or with the `dsconfig` command. * Admin console * dsconfig ### Steps 1. Go to **System > Connection Handlers**. 2. Click **HTTP Connection Handler** or **HTTPS Connection Handler**. 3. Scroll down and configure the following: * **Expensive Thread Check Interval** * **Expensive Thread Minimum Concurrent Count** * **Expensive Thread Hold Off Interval** ### Steps * Run the `dsconfig set-connection-handler-prop` command with the following arguments: ```shell dsconfig set-connection-handler-prop \ --handler-name "HTTP Connection Handler" \ --set "expensive-thread-check-interval: ms" \ --set "expensive-thread-minimum-concurrent-count:" \ --set "expensive-thread-hold-off-interval: ms" ``` | | | | - | ---------------------------------------------------------------------------------------------------------------------- | | | You must restart the server or the HTTP Connection Handler for the expensive thread detection settings to take effect. | ## Analyzing thread dump files Each thread dump file is organized into three main parts: * Header summary: Provides context about why the thread dump was generated. It includes the name of the detector that triggered the dump, the number of expensive threads identified, and the time interval over which they were detected. * Expensive thread list: Lists the threads flagged as expensive immediately after the summary. Each entry includes a thread ID, thread name, and correlation ID to help trace related request activity across other logs. | | | | - | -------------------------------------------- | | | Save the thread IDs for use during analysis. | * Full stack trace: Shows a complete snapshot of all threads running in the PingAuthorize Server's Java process at the time of the thread dump. This section provides detailed request context to help identify performance bottlenecks or blocked threads. Use the thread IDs from the expensive thread list to locate and analyze each expensive thread's state. For example, the following header summary and expensive thread list identify a single expensive thread, detected over an interval of 5000 milliseconds: ``` HTTP Connection Handler :80 worker Expensive HTTP Thread Detector (10.3.0.0-20250611152739.000Z-364a3822) detected that 1 threads were found to be processing the same HTTP request over an interval of at least 5000 ms: * Thread id=176 name='HTTP Connection Handler :80 worker 6 (10.3.0.0-20250611152739.000Z-364a3822) requestID=0 correlationID="3bf39de1-7bc4-4937-b99a-62785a41f55d" ``` The following is a truncated version of the expensive thread's stack trace entry: ``` "HTTP Connection Handler :80 worker 6 (10.3.0.0-20250611152739.000Z-364a3822)" priority=5 id=176 java.lang.Thread.State: WAITING at java.base@17.0.10/jdk.internal.misc.Unsafe.park(Native Method) at java.base@17.0.10/java.util.concurrent.locks.LockSupport.park(LockSupport.java:211) at java.base@17.0.10/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1864) at java.base@17.0.10/java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3465) at java.base@17.0.10/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3436) at java.base@17.0.10/java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1898) at java.base@17.0.10/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2072) at app//com.pingidentity.authorize.decisionengine2.evaluation.effects.Inner.get(Inner.java:32) ``` A thread state of `WAITING` indicates the thread is idle, waiting indefinitely for another thread to perform an action. Learn more about thread states in the [`Enum Thread.State` reference](https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.State.html) in the Java documentation. ## Thread dump file management The PingAuthorize Server doesn't automatically manage the thread dump files it creates. You must implement your own retention policy for these files, including any log rotation, compression, or cleanup. In a containerized deployment, make sure a mechanism is in place, such as a volume mount, to persist thread dump files and make them available for analysis outside the container.