[Bug]: Update to 8.13.0 breaks ruby function openssl EC use

[darkphoenix]

Is this a critical security issue?

• This is not a security issue.

Describe the Bug

Upon updating our compiler to 8.13.0, we noticed failed runs on a substantial percentage of our nodes. Not all of them were affected, but those which were continuously failed to get a catalog, erroring out with a Java exception from the puppet server, and a >1k line stacktrace (attached). The cause appears to be a custom function calling into openssl functions from Ruby - specifically, it attempts to validate X.509 keys by attempting to instantiate a private key object based on them:

Puppet::Functions.create_function(:'x509::validate_key') do
  dispatch :validate_key do
    param 'X509::PEM::Key', :pem
    param 'X509::Algorithm', :type
  end

  def validate_key(pem, type)
    require 'openssl'
    if type == 'rsa'
      begin
        OpenSSL::PKey::RSA.new(pem)
      rescue => e
        raise Puppet::ParseError, "RSA private key invalid: #{e}"
      end
    else
      begin
        OpenSSL::PKey::EC.new(pem)
      rescue => e
        raise Puppet::ParseError, "ECC private key invalid: #{e}"
      end
    end
  end
end

This causes the following exception:
java.lang.NoSuchMethodError: 'org.bouncycastle.asn1.ASN1Primitive org.bouncycastle.asn1.sec.ECPrivateKey.getParameters()'
with the stack trace attached. Notably, this only occurs with the EC functions, there is no mention of errors relating to RSA anywhere in our logs.

Expected Behavior

The Ruby code continues to work, and nodes can request their catalogs from the compiler.

Steps to Reproduce

1. Create a custom function using Ruby's openssl library to load an EC private key
2. Attempt to request a catalog using this function from openvox-server 8.13.0

Environment

Version: 8.13.0
Platform: Debian 12

Additional Context

We suspect this to be related to the update of the bouncycastle dep in version 8.13.

Relevant log output

stacktrace_puppetserver.txt

[Sharpie]

Thanks for the report! The re-production steps will need some more detail on how to generate a failing file as I can't re-produce this with a simple P256 key:

$ podman run --rm -it debian:13 /bin/bash

$ apt update && apt install curl
...

$ curl -LO https://apt.voxpupuli.org/openvox8-release-debian13.deb
$ dpkg -i openvox8-release-debian-13.deb
$ apt update && apt install openvox-server
...
Setting up openvox-server (8.13.0-1+debian13) ...

$ openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out p256.pem
$ /opt/puppetlabs/server/bin/puppetserver ruby -ropenssl -e 'puts OpenSSL::PKey::EC.new(File.read("p256.pem")).inspect'
#<OpenSSL::PKey::EC:0x4bb05a4e>

[Sharpie]

One interesting bit in the versions is that OpenVox Server 8.13.0 started shipping BouncyCastle 1.84.0, but still uses jruby-openssl 0.15.4, which only advertises support for BouncyCastle versions 1.78-1.83:

https://github.com/jruby/jruby-openssl#compatibility

So, definitely potential for that upgrade to have resulted in this sort of issue.

[darkphoenix]

I sadly can't identify a difference with our specific key - however, I can confirm that the key in question breaks with the same exception in your test setup, and that
/opt/puppetlabs/server/bin/puppetserver gem update jruby-openssl
makes it parse fine afterwards. So the version drift probably is to blame.

[corporate-gadfly]

It doesn't look like 0.16.0 is bundled with jruby in any released version.

jruby version	Bundled jruby-openssl version
9.4.12.1	0.15.4
9.4.13.0	0.15.4
9.4.14.0	0.15.4
10.0.3.0	0.15.5
10.1.0.0	0.15.6

[corporate-gadfly]

@darkphoenix are you able to test an installer from this openvox-server package?

I tested it with lima and here are my results with the following ruby code in irb:

require 'openssl'

# The valid prime256v1 SEC1 format EC Private Key block
ecc_pem = <<~PEM
-----BEGIN EC PRIVATE KEY-----
MHcCAQEEIFGFA23QpFp3ZgFniwlICLAjIc3Zxrt6Dw5ElfETPkoxoAoGCCqGSM49
AwEHoUQDQgAEfAWhRpiAKNuIlXG54M/vSWN1kam9/1SKp8yeoHBN0YhpGDrh+JC0
gwPqy/cOYhzzKUleb4+bjvImBkTB/1jvlQ==
-----END EC PRIVATE KEY-----
PEM

# This mirrors the exact logic of your original validation function
puts "Testing original validation path using upgraded server jar..."

begin
  # This line previously crashed with java.lang.NoSuchMethodError
  key = OpenSSL::PKey::EC.new(ecc_pem)
  
  puts "--------------------------------------------------------"
  puts " VERIFICATION SUCCESSFUL"
  puts "--------------------------------------------------------"
  puts "Object successfully instantiated: #{key.class}"
  puts "Is it a valid EC key? #{key.is_a?(OpenSSL::PKey::EC)}"
  puts "Check curve name: #{key.group.curve_name}" rescue nil
rescue => e
  puts "Verification Failed! Code still threw an error: #{e.message}"
  puts e.backtrace.first(5) if e.respond_to?(:backtrace) && e.backtrace
rescue Exception => java_error
  puts "Verification Failed! Underlying JVM threw: #{java_error}"
end

And the output was:

root@lima-ubuntu-24-04:/tmp# /opt/puppetlabs/bin/puppetserver irb
irb(main):001:0> require 'openssl'
irb(main):002:0* 
irb(main):003:0* # The valid prime256v1 SEC1 format EC Private Key block
irb(main):004:0" ecc_pem = <<~PEM
irb(main):005:0" -----BEGIN EC PRIVATE KEY-----
irb(main):006:0" MHcCAQEEIFGFA23QpFp3ZgFniwlICLAjIc3Zxrt6Dw5ElfETPkoxoAoGCCqGSM49
irb(main):007:0" AwEHoUQDQgAEfAWhRpiAKNuIlXG54M/vSWN1kam9/1SKp8yeoHBN0YhpGDrh+JC0
irb(main):008:0" gwPqy/cOYhzzKUleb4+bjvImBkTB/1jvlQ==
irb(main):009:0" -----END EC PRIVATE KEY-----
irb(main):010:0> PEM
irb(main):011:0* 
irb(main):012:0* # This mirrors the exact logic of your original validation function
irb(main):013:0> puts "Testing original validation path using upgraded server jar..."
irb(main):014:0* 
irb(main):015:1> begin
irb(main):016:1*   # This line previously crashed with java.lang.NoSuchMethodError
irb(main):017:1>   key = OpenSSL::PKey::EC.new(ecc_pem)
irb(main):018:1*   
irb(main):019:1>   puts "--------------------------------------------------------"
irb(main):020:1>   puts " VERIFICATION SUCCESSFUL"
irb(main):021:1>   puts "--------------------------------------------------------"
irb(main):022:1>   puts "Object successfully instantiated: #{key.class}"
irb(main):023:1>   puts "Is it a valid EC key? #{key.is_a?(OpenSSL::PKey::EC)}"
irb(main):024:1>   puts "Check curve name: #{key.group.curve_name}" rescue nil
irb(main):025:1> rescue => e
irb(main):026:1>   puts "Verification Failed! Code still threw an error: #{e.message}"
irb(main):027:1>   puts e.backtrace.first(5) if e.respond_to?(:backtrace) && e.backtrace
irb(main):028:1> rescue Exception => java_error
irb(main):029:1>   puts "Verification Failed! Underlying JVM threw: #{java_error}"
irb(main):030:0> end
irb(main):031:0* 
Testing original validation path using upgraded server jar...
--------------------------------------------------------
 VERIFICATION SUCCESSFUL
--------------------------------------------------------
Object successfully instantiated: OpenSSL::PKey::EC
Is it a valid EC key? true
Check curve name: prime256v1
=> nil
irb(main):032:0>

[corporate-gadfly]

With 8.13.0, I get:

root@lima-ubuntu-24-04:/tmp# /opt/puppetlabs/bin/puppetserver irb
irb(main):001:0> require 'openssl'
irb(main):002:0* 
irb(main):003:0* # The valid prime256v1 SEC1 format EC Private Key block
irb(main):004:0" ecc_pem = <<~PEM
irb(main):005:0" -----BEGIN EC PRIVATE KEY-----
irb(main):006:0" MHcCAQEEIFGFA23QpFp3ZgFniwlICLAjIc3Zxrt6Dw5ElfETPkoxoAoGCCqGSM49
irb(main):007:0" AwEHoUQDQgAEfAWhRpiAKNuIlXG54M/vSWN1kam9/1SKp8yeoHBN0YhpGDrh+JC0
irb(main):008:0" gwPqy/cOYhzzKUleb4+bjvImBkTB/1jvlQ==
irb(main):009:0" -----END EC PRIVATE KEY-----
irb(main):010:0> PEM
irb(main):011:0* 
irb(main):012:0* # This mirrors the exact logic of your original validation function
irb(main):013:0> puts "Testing original validation path using upgraded server jar..."
irb(main):014:0* 
irb(main):015:1> begin
irb(main):016:1*   # This line previously crashed with java.lang.NoSuchMethodError
irb(main):017:1>   key = OpenSSL::PKey::EC.new(ecc_pem)
irb(main):018:1*   
irb(main):019:1>   puts "--------------------------------------------------------"
irb(main):020:1>   puts " VERIFICATION SUCCESSFUL"
irb(main):021:1>   puts "--------------------------------------------------------"
irb(main):022:1>   puts "Object successfully instantiated: #{key.class}"
irb(main):023:1>   puts "Is it a valid EC key? #{key.is_a?(OpenSSL::PKey::EC)}"
irb(main):024:1>   puts "Check curve name: #{key.group.curve_name}" rescue nil
irb(main):025:1> rescue => e
irb(main):026:1>   puts "Verification Failed! Code still threw an error: #{e.message}"
irb(main):027:1>   puts e.backtrace.first(5) if e.respond_to?(:backtrace) && e.backtrace
irb(main):028:1> rescue Exception => java_error
irb(main):029:1>   puts "Verification Failed! Underlying JVM threw: #{java_error}"
irb(main):030:0> end
irb(main):031:0* 
Testing original validation path using upgraded server jar...
Verification Failed! Underlying JVM threw: 'org.bouncycastle.asn1.ASN1Primitive org.bouncycastle.asn1.sec.ECPrivateKey.getParameters()'
=> nil
irb(main):032:0>

[darkphoenix]

Can confirm that with that snapshot build, our "broken" key reads fine.