Skip to content

CAMEL-23194: Fix JMS shutdown deadlock that hangs CI builds#22381

Open
Croway wants to merge 1 commit intomainfrom
fix/CAMEL-23194-jms-shutdown-deadlock
Open

CAMEL-23194: Fix JMS shutdown deadlock that hangs CI builds#22381
Croway wants to merge 1 commit intomainfrom
fix/CAMEL-23194-jms-shutdown-deadlock

Conversation

@Croway
Copy link
Copy Markdown
Contributor

@Croway Croway commented Apr 1, 2026

Summary

Claude Code on behalf of Federico Mariani

Fixes the JMS test deadlock that has been hanging CI builds, causing repeated ABORTs on camel-4.14.x and camel-4.18.x.

Root Cause

CAMEL-20199 (Jul/Oct 2024) migrated synchronized blocks to ReentrantLock for virtual thread support. This accidentally collapsed two independent locks into one:

  • Before: BaseService.stop() used synchronized(lock) (a private Object), while DestinationResolverDelegate used synchronized(QueueReplyManager.this) (the instance monitor) — two independent locks, no deadlock possible.
  • After: Both became QueueReplyManager.this.lock.lock() — the same ReentrantLock inherited from BaseService, creating a circular wait during shutdown.

Deadlock Sequence

Thread 1 (afterAll → context.stop()):
  BaseService.stop()             ← acquires BaseService.lock
    → ReplyManagerSupport.doStop()
      → listenerContainer.destroy()
        → doShutdown()
          → Object.wait()        ← waiting for listener threads to finish

Thread 2 (listener thread):
  DestinationResolverDelegate.resolveDestinationName()
    → BaseService.lock           ← BLOCKED, held by Thread 1

Fix

  • DestinationResolverDelegate now uses a dedicated ReentrantLock instead of BaseService.lock, restoring the original two-lock design
  • Added volatile + fast-path check on the destination field
  • Applied to both camel-jms and camel-sjms (identical pattern)
  • Added junit.jupiter.execution.timeout.default = 10m for camel-jms tests as extra safety

Test plan

  • Run JMS tests repeatedly: for i in $(seq 1 50); do mvn test -pl components/camel-jms -Dtest="JmsHttpJmsTest,AsyncJmsInOutTest,AsyncJmsProducerTest,JmsInOutPipelineWithBeanTest,JmsInOutRoutingSlipTest,AsyncConsumerInOutTest" -Dsurefire.rerunFailingTestsCount=0 -Dcamel.surefire.forkTimeout=120 -q || echo "FAILED on run $i"; done
  • CI Camel Core (Build and test) should no longer hang on JMS tests

🤖 Generated with Claude Code

@Croway Croway requested review from apupier, gnodet and oscerd April 1, 2026 14:42
Root cause: deadlock between BaseService.lock and Spring's
lifecycleMonitor during CamelContext shutdown in afterAll.

Thread 1 (afterAll -> context.stop()):
  AbstractCamelContext.doStop()
    -> JmsProducer.doStop()
      -> ReplyManagerSupport.doStop()
        -> DefaultMessageListenerContainer.doShutdown()
          -> Object.wait() <- blocked, waiting for listener thread

Thread 2 (QueueReplyManager listener):
  DestinationResolverDelegate.resolveDestinationName()
    -> BaseService.lock <- blocked, held by Thread 1

Introduced by CAMEL-20199 (Oct 2024) which migrated synchronized
blocks to ReentrantLock. Before: BaseService.stop() used
synchronized(lock) on a private Object, while the inner class used
synchronized(QueueReplyManager.this) on the instance monitor — two
independent locks. After: both became QueueReplyManager.this.lock
(the same inherited ReentrantLock), creating the circular wait.

Fix: DestinationResolverDelegate now uses a dedicated ReentrantLock
instead of BaseService.lock, restoring the original two-lock design.
Added volatile + fast-path for the destination field.

Applied to both camel-jms and camel-sjms which share the same
pattern via their respective QueueReplyManager classes.
@Croway Croway force-pushed the fix/CAMEL-23194-jms-shutdown-deadlock branch from 10d55fb to ec06502 Compare April 1, 2026 14:51
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 1, 2026

🧪 CI tested the following changed modules:

  • components/camel-jms
  • components/camel-sjms
All tested modules (84 modules)
  • Camel :: AMQP
  • Camel :: AMQP [jar]
  • Camel :: ActiveMQ 5.x
  • Camel :: ActiveMQ 5.x [jar]
  • Camel :: ActiveMQ 6.x
  • Camel :: ActiveMQ 6.x [jar]
  • Camel :: All Components Sync point
  • Camel :: All Components Sync point [pom]
  • Camel :: Assembly
  • Camel :: Assembly [pom]
  • Camel :: Catalog :: CSimple Maven Plugin (deprecated) [maven-plugin]
  • Camel :: Catalog :: CSimple Maven Plugin (deprecated) SUCCESS [ 1.429 s]
  • Camel :: Catalog :: Camel Catalog
  • Camel :: Catalog :: Camel Catalog [jar]
  • Camel :: Catalog :: Camel Report Maven Plugin
  • Camel :: Catalog :: Camel Report Maven Plugin [maven-plugin]
  • Camel :: Catalog :: Camel Route Parser
  • Camel :: Catalog :: Camel Route Parser [jar]
  • Camel :: Catalog :: Console
  • Camel :: Catalog :: Console [jar]
  • Camel :: Catalog :: Dummy Component
  • Camel :: Catalog :: Dummy Component [jar]
  • Camel :: Catalog :: Lucene (deprecated)
  • Camel :: Catalog :: Lucene (deprecated) [jar]
  • Camel :: Catalog :: Maven
  • Camel :: Catalog :: Maven [jar]
  • Camel :: Catalog :: Suggest
  • Camel :: Catalog :: Suggest [jar]
  • Camel :: Component DSL
  • Camel :: Component DSL [jar]
  • Camel :: Coverage
  • Camel :: Coverage [pom]
  • Camel :: Docs
  • Camel :: Docs [pom]
  • Camel :: Endpoint DSL
  • Camel :: Endpoint DSL [jar]
  • Camel :: Endpoint DSL :: Support
  • Camel :: Endpoint DSL :: Support [jar]
  • Camel :: Integration Tests
  • Camel :: Integration Tests [jar]
  • Camel :: JBang :: Core
  • Camel :: JBang :: Core [jar]
  • Camel :: JBang :: Integration tests
  • Camel :: JBang :: Integration tests [jar]
  • Camel :: JBang :: MCP
  • Camel :: JBang :: MCP [jar]
  • Camel :: JBang :: Main
  • Camel :: JBang :: Main [jar]
  • Camel :: JBang :: Plugin :: Edit
  • Camel :: JBang :: Plugin :: Edit [jar]
  • Camel :: JBang :: Plugin :: Generate
  • Camel :: JBang :: Plugin :: Generate [jar]
  • Camel :: JBang :: Plugin :: Kubernetes
  • Camel :: JBang :: Plugin :: Kubernetes [jar]
  • Camel :: JBang :: Plugin :: Route Parser
  • Camel :: JBang :: Plugin :: Route Parser [jar]
  • Camel :: JBang :: Plugin :: Testing
  • Camel :: JBang :: Plugin :: Testing [jar]
  • Camel :: JBang :: Plugin :: Validate
  • Camel :: JBang :: Plugin :: Validate [jar]
  • Camel :: JMS
  • Camel :: JMS [jar]
  • Camel :: Kamelet Main
  • Camel :: Kamelet Main [jar]
  • Camel :: Launcher
  • Camel :: Launcher [jar]
  • Camel :: Launcher :: Container
  • Camel :: Launcher :: Container [pom]
  • Camel :: MLLP
  • Camel :: MLLP [jar]
  • Camel :: Simple JMS
  • Camel :: Simple JMS [jar]
  • Camel :: Simple JMS2
  • Camel :: Simple JMS2 [jar]
  • Camel :: YAML DSL
  • Camel :: YAML DSL [jar]
  • Camel :: YAML DSL :: Deserializers
  • Camel :: YAML DSL :: Deserializers [jar]
  • Camel :: YAML DSL :: Maven Plugins
  • Camel :: YAML DSL :: Maven Plugins [maven-plugin]
  • Camel :: YAML DSL :: Validator
  • Camel :: YAML DSL :: Validator [jar]
  • Camel :: YAML DSL :: Validator Maven Plugin
  • Camel :: YAML DSL :: Validator Maven Plugin [maven-plugin]

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 1, 2026

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using build-all, build-dependents, skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants