-
Notifications
You must be signed in to change notification settings - Fork 10
openvox 8x: document custom CA for HTTPS-inspecting proxies #241
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
miharp
wants to merge
4
commits into
OpenVoxProject:master
Choose a base branch
from
miharp:docs/ssl-custom-ca-proxy
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+151
−0
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
623fa16
openvox 8x: document custom CA for HTTPS-inspecting proxies
miharp a0b75b4
openvox 8x: incorporate review feedback on custom CA proxy doc
miharp 5fd0c81
openvox 8x: remove concat approach from managing-with-puppet section
miharp 70d70db
openvox 8x: add symlink alternative for certs/ approach
miharp File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,149 @@ | ||
| --- | ||
| layout: default | ||
| title: "SSL configuration: Adding a custom CA for HTTPS-inspecting proxies" | ||
| --- | ||
|
|
||
| Some network environments use HTTPS-inspecting proxies (such as Squid with SSL Bump) that | ||
| re-sign outbound TLS connections using a local CA. OpenVox's Ruby runtime uses its own | ||
| bundled CA certificate store rather than the system trust store, so the proxy CA must be | ||
| added explicitly before OpenVox will trust connections the proxy intercepts. | ||
|
|
||
| Common symptoms: `gem install` or module downloads (`puppet module install`) fail with | ||
| certificate verification errors even though the system CA trust store already includes the | ||
| proxy CA. | ||
|
|
||
| ## How OpenVox validates outbound TLS | ||
|
|
||
| OpenVox ships its own Ruby runtime with a vendored CA bundle compiled in at | ||
| `/opt/puppetlabs/puppet/ssl/cert.pem`. This path is OpenVox's `OpenSSL::X509::DEFAULT_CERT_FILE` | ||
| and is what Ruby uses — not the OS trust store — when validating TLS certificates for | ||
| outbound connections such as gem downloads and `puppet module install`. | ||
|
|
||
| ## Quick fix: add the CA to the `certs/` directory | ||
|
|
||
| Copy your proxy CA into OpenVox's `certs/` directory and run `openssl rehash` to generate | ||
| the fingerprint symlinks that OpenSSL uses to look up certificates: | ||
|
|
||
| ```console | ||
| cp /path/to/proxy-ca.pem /opt/puppetlabs/puppet/ssl/certs/proxy-ca.pem | ||
| /opt/puppetlabs/puppet/bin/openssl rehash /opt/puppetlabs/puppet/ssl/certs/ | ||
| ``` | ||
|
|
||
| This directory is OpenVox Ruby's `DEFAULT_CERT_DIR` and is included by `set_default_paths` | ||
| on every connection. The directory is empty by default — the `openvox-agent` package does | ||
| not place any files there — so user-added files survive package upgrades. | ||
|
|
||
| > **Note:** `openssl rehash` is not supported on Windows as of OpenVox 8. Use the | ||
| > `SSL_CERT_FILE` approach below on Windows nodes. | ||
|
|
||
| If you prefer a one-liner that skips rehash, appending directly to `cert.pem` also works, | ||
| but that file is replaced on upgrade: | ||
|
|
||
| ```console | ||
| cat /path/to/proxy-ca.pem >> /opt/puppetlabs/puppet/ssl/cert.pem | ||
| ``` | ||
|
|
||
| ## Persistent fix: `ssl_trust_store` (module downloads only) | ||
|
|
||
| For `puppet module install` and https file sources, the cleanest option is the | ||
| `ssl_trust_store` setting in `puppet.conf`. OpenVox loads this file as an additional trust | ||
| store on top of the built-in bundle, so your proxy CA survives upgrades and no environment | ||
| variables are required: | ||
|
|
||
| ```console | ||
| puppet config set ssl_trust_store /etc/ssl/certs/proxy-ca.pem | ||
| ``` | ||
|
|
||
| **Important:** `ssl_trust_store` only applies to Puppet's own outbound HTTPS requests. It | ||
| does not affect gem installs performed via the `puppet_gem` package provider, because those | ||
| run `gem` as a subprocess that does not read `puppet.conf`. Use the `SSL_CERT_FILE` approach | ||
| below if you also need gem installs to work. | ||
|
|
||
| ## Persistent fix: `SSL_CERT_FILE` pointing at a merged bundle | ||
|
|
||
| The `SSL_CERT_FILE` environment variable overrides OpenSSL's default cert path and works | ||
| for both `puppet module install` and gem installs. Create a merged bundle that combines the | ||
| original Mozilla certs with your proxy CA: | ||
|
|
||
| ```console | ||
| cat /opt/puppetlabs/puppet/ssl/cert.pem /path/to/proxy-ca.pem \ | ||
| > /etc/ssl/certs/puppet-custom-bundle.pem | ||
| ``` | ||
|
|
||
| **Make it permanent for the Puppet agent service** by adding the variable to the service | ||
| environment. On systemd systems, create a drop-in: | ||
|
|
||
| ```console | ||
| mkdir -p /etc/systemd/system/puppet.service.d | ||
| cat > /etc/systemd/system/puppet.service.d/ssl_cert_file.conf <<'EOF' | ||
| [Service] | ||
| Environment=SSL_CERT_FILE=/etc/ssl/certs/puppet-custom-bundle.pem | ||
| EOF | ||
| systemctl daemon-reload | ||
| systemctl restart puppet | ||
| ``` | ||
|
|
||
| For one-off commands, export the variable in the same shell: | ||
|
|
||
| ```console | ||
| SSL_CERT_FILE=/etc/ssl/certs/puppet-custom-bundle.pem puppet agent -t | ||
| ``` | ||
|
|
||
| ## Managing with Puppet | ||
|
|
||
| Deploy the CA with a `file` resource and trigger `openssl rehash` on change: | ||
|
|
||
| ```puppet | ||
| file { '/opt/puppetlabs/puppet/ssl/certs/proxy-ca.pem': | ||
| ensure => file, | ||
| owner => 'root', | ||
| group => 'root', | ||
| mode => '0644', | ||
| content => lookup('profile::proxy_ca_cert'), | ||
| notify => Exec['rehash-puppet-ssl-certs'], | ||
| } | ||
|
|
||
| exec { 'rehash-puppet-ssl-certs': | ||
| command => '/opt/puppetlabs/puppet/bin/openssl rehash /opt/puppetlabs/puppet/ssl/certs/', | ||
| refreshonly => true, | ||
| } | ||
| ``` | ||
|
|
||
| Store the proxy CA certificate as a multiline string in Hiera: | ||
|
|
||
| ```yaml | ||
| profile::proxy_ca_cert: | | ||
| -----BEGIN CERTIFICATE----- | ||
| ... | ||
| -----END CERTIFICATE----- | ||
| ``` | ||
|
|
||
| If the certificate is already deployed on the node at a known path (for example, by the | ||
| [puppet/trusted_ca](https://forge.puppet.com/modules/puppet/trusted_ca) module into the OS | ||
| trust store), use a symlink instead to avoid keeping a second copy: | ||
|
|
||
| ```puppet | ||
| file { '/opt/puppetlabs/puppet/ssl/certs/proxy-ca.pem': | ||
| ensure => link, | ||
| target => '/etc/pki/ca-trust/source/anchors/proxy-ca.pem', | ||
| notify => Exec['rehash-puppet-ssl-certs'], | ||
| } | ||
|
|
||
| exec { 'rehash-puppet-ssl-certs': | ||
| command => '/opt/puppetlabs/puppet/bin/openssl rehash /opt/puppetlabs/puppet/ssl/certs/', | ||
| refreshonly => true, | ||
| } | ||
| ``` | ||
|
|
||
|
|
||
| ## Verifying the configuration | ||
|
|
||
| Confirm Ruby can reach an intercepted host: | ||
|
|
||
| ```console | ||
| /opt/puppetlabs/puppet/bin/ruby -rnet/http -ruri \ | ||
| -e 'Net::HTTP.get(URI("https://forgeapi.puppet.com")); puts "OK"' | ||
| ``` | ||
|
|
||
| A successful response prints `OK`. A certificate verification error means the proxy CA is | ||
| still not trusted by OpenVox's Ruby environment. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This works, but will not survive upgrades that update the public CA bundle. When that happens, integrations will break and someone has to remember to repeat the append.
A better solution should be:
The contents of the
certs/directory should be left alone by an upgrade. There may be some workflows that see changes tocert.pembut miss files added tocerts/afterrehash--- these should be reported as bugs so that we can fix them up.Last I looked, Windows is in a bit of a bind as
openssl rehashwas wrapped in one giant#ifndef WINDOWSin the C source, but maybe things are better in OpenSSL 3.5.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the
packageresource had an option to set an environment variable it would solve the problem. It is enough to set:SSL_CERT_FILE=/path/to/cert.pem