Details
-
Bug
-
Resolution: Answered
-
High
-
None
-
8.4.0, 8.2.6, 8.5.3, 8.7.1
-
8.02
-
18
-
Severity 2 - Major
-
19
-
Description
This problem is caused by a bug in the Java runtime: JDK-8214418 HttpClient falls in running with 100% cpu usage after an error signalled on channel (Backported as JDK-8241054)
The fix has been verified to be available in AdoptOpenJDK 11.0.8 - https://github.com/AdoptOpenJDK/openjdk-jdk11u/commit/8d1b63a4db2c6348a97b3cf45bd4d2caa7cad6b5
Issue Summary
Similar to BSERV-12131, the HTTP Client will stop working on occasion whenever using Java 11.
Steps to Reproduce
- Configure Jira to run on Java 11
- Register and configure a webhook to the application.
- Execute the webhook.
Expected Results
The webhook will trigger successfully.
Actual Results
When using Java 11, webhooks will work briefly and eventually fail. Thread dumps will show these webhook publishers in a BLOCKED or WAITING state:
"Web-Hook-Publisher-0" daemon prio=5 tid=0x0000000000000127 nid=0 waiting on condition java.lang.Thread.State: WAITING (parking) at java.base/jdk.internal.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000a4586577> (a java.util.concurrent.locks.ReentrantLock$NonfairSync) owned by Web-Hook-Publisher-1 id=0x000000000000012f at java.base/java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown Source) at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(Unknown Source) at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(Unknown Source) at java.base/java.util.concurrent.locks.ReentrantLock.lock(Unknown Source) at org.apache.http.nio.pool.AbstractNIOConnPool.lease(AbstractNIOConnPool.java:278) at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.requestConnection(PoolingNHttpClientConnectionManager.java:295) at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.requestConnection(AbstractClientExchangeHandler.java:377) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.start(DefaultClientExchangeHandlerImpl.java:129) at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:141) at com.atlassian.httpclient.apache.httpcomponents.BoundedHttpAsyncClient.execute(BoundedHttpAsyncClient.java:49) at org.apache.http.impl.client.cache.CachingHttpAsyncClient.callBackend(CachingHttpAsyncClient.java:691) at org.apache.http.impl.client.cache.CachingHttpAsyncClient.execute(CachingHttpAsyncClient.java:323) at org.apache.http.impl.client.cache.CachingHttpAsyncClient.execute(CachingHttpAsyncClient.java:281) at com.atlassian.httpclient.apache.httpcomponents.SettableFuturePromiseHttpPromiseAsyncClient.execute(SettableFuturePromiseHttpPromiseAsyncClient.java:34) at com.atlassian.httpclient.apache.httpcomponents.ApacheAsyncHttpClient.doExecute(ApacheAsyncHttpClient.java:339) at com.atlassian.httpclient.apache.httpcomponents.ApacheAsyncHttpClient.execute(ApacheAsyncHttpClient.java:292) at com.atlassian.httpclient.apache.httpcomponents.DefaultRequest$DefaultRequestBuilder.execute(DefaultRequest.java:258) at com.atlassian.httpclient.apache.httpcomponents.DefaultRequest$DefaultRequestBuilder.post(DefaultRequest.java:226) at com.atlassian.webhooks.plugin.PublishTaskFactoryImpl$PublishTaskImpl.run(PublishTaskFactoryImpl.java:119) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source)
This issue is suspected to be caused by the following JDK bug:
- JDK-8214418 HttpClient falls in running with 100% cpu usage after an error signalled on channel
- https://bugs.openjdk.java.net/browse/JDK-8214418 is not public
- PR for the fix can be found here: https://hg.openjdk.java.net/jdk/jdk13/rev/5022a4915fe9
Solution
Upgrade AdoptOpenJDK to version 11.0.8 which has this issue solved.
Workaround
- Run Jira Server using Java 8 or;
- The issue is observed with Java 11 that uses TLS v1.3. TLS v1.2 is unaffected by this. It's possible to force Java 11 to use TLS v1.2 by adding the following JVM parameter: -Dhttps.protocols=TLSv1.2 to your startup properties.
Attachments
Issue Links
- relates to
-
-
- Closed
-
-
-
- Closed
-
-
JRASERVER-71409 Ability to support other version of AdoptOpenJDK in 8.5.x
- Gathering Interest
-
RAID-2043 Loading...
- causes
-
- mentioned in
-
-
-
-
-
-
-