fix: Add FIPS mode detection and auto-disable APR SSL Engine

[mbiuki]

Problem

The Tomcat Native APR library (libtcnative-1) version 1.2.35 is incompatible with OpenSSL 3.x when running in FIPS mode, causing JVM segmentation faults during startup on modern systems (Ubuntu 24.04+, RHEL 9+).

PR #34068 attempted to fix this by removing the native library entirely, but the reviewer (@wezell) requested that we maintain the native library by default for performance benefits and instead add FIPS detection or a configuration flag.

Solution

This fix implements the reviewer's recommendation:

1. Keeps native library installed by default - No changes to Dockerfiles; libtcnative-1 remains for performance
2. Automatic FIPS detection - Checks /proc/sys/crypto/fips_enabled at container startup
3. Configuration flags - Provides CMS_DISABLE_APR_SSL and CMS_SSL_ENGINE for manual control
4. SSL endpoint stays enabled - Only disables APR library when needed, not SSL itself

Implementation Details

New Files

• dotCMS/src/main/docker/original/ROOT/srv/15-detect-fips-and-set-ssl-engine.sh - FIPS detection script that runs at container startup
• FIPS_APR_SSL_FIX.md - Comprehensive documentation with configuration options and troubleshooting

Modified Files

• dotCMS/src/main/docker/original/ROOT/srv/entrypoint.sh - Sources the FIPS detection script
• dotCMS/src/main/resources/container/tomcat9/conf/server.xml - Added documentation about FIPS auto-detection

Configuration Options

Users have three ways to control APR SSL behavior:

# Option 1: Automatic (default) - FIPS auto-detection
docker run -p 8080:8080 dotcms/dotcms:latest

# Option 2: Manual disable flag
docker run -e CMS_DISABLE_APR_SSL=true -p 8080:8080 dotcms/dotcms:latest

# Option 3: Direct control
docker run -e CMS_SSL_ENGINE=off -p 8080:8080 dotcms/dotcms:latest

Testing Plan

• Build Docker image successfully
• Test container startup in non-FIPS environment (APR SSL should be enabled)
• Test container startup with CMS_DISABLE_APR_SSL=true (APR SSL should be disabled)
• Test container startup with explicit CMS_SSL_ENGINE=off (APR SSL should be disabled)
• Verify startup logs show correct FIPS detection messages
• Test SSL/TLS connectivity on port 8443

Impact

• User Impact: None for non-FIPS environments - APR SSL remains enabled by default
• Performance: No change - APR SSL still used by default for optimal performance
• Security: Improved - FIPS environments now work automatically without crashes
• Compatibility: Improved - Eliminates OpenSSL 3.x + FIPS crashes

Related Issues

• Addresses reviewer feedback on PR fix: Remove Tomcat Native APR library to resolve OpenSSL 3.x crash #34068
• Fixes JVM crash on startup due to Tomcat Native APR incompatibility with OpenSSL 3.x #34067 (if that issue exists)

🤖 Generated with Claude Code

[github-actions]

PRs linked to this issue

• fix: Add FIPS mode detection and auto-disable APR SSL Engine by @mbiuki