HTTP-Tiny-0.084: After upgrade from v0.083, cpan command fails on Strawberry Perl 5.16.3

[twata1]

Hello,

After I upgraded HTTP::Tiny from version 0.083 to 0.084, the cpan command fails even with the environment variable PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT set to 1.
I usually use cpan and cpanm.
I have cpan version is 2.34 and Mozilla::CA is up to date.

Under Strawberry Perl 5.16.3 (64 bit) on Windows 8.1 (64 bit):

C:\home\pianokey>set PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT=1

C:\home\pianokey>cpanm Mozilla::CA
Mozilla::CA is up to date. (20221114)

C:\home\pianokey>cpan -v
C:\home\pianokey\perl5\bin/cpan script version 1.678, CPAN.pm version 2.34

C:\home\pianokey>cpan SZABGAB/Padre-1.02.tar.gz
CPAN: CPAN::SQLite loaded ok (v0.202)
CPAN: HTTP::Tiny loaded ok (v0.084)
CPAN: Net::SSLeay loaded ok (v1.93_02)
CPAN: IO::Socket::SSL loaded ok (v2.083)
Fetching with HTTP::Tiny:
https://cpan.org/authors/01mailrc.txt.gz
HTTP::Tiny failed with an internal error: SSL connection failed for cpan.org: SSL connect attempt failed error:14090086:
SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Fetching with HTTP::Tiny:
https://cpan.org/modules/02packages.details.txt.gz
HTTP::Tiny failed with an internal error: SSL connection failed for cpan.org: SSL connect attempt failed error:14090086:
SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Fetching with HTTP::Tiny:
https://cpan.org/modules/03modlist.data.gz
HTTP::Tiny failed with an internal error: SSL connection failed for cpan.org: SSL connect attempt failed error:14090086:
SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Database was generated on Thu, 15 Jun 2023 20:38:47 GMT
Updating database file ... Done!Fetching with HTTP::Tiny:
https://cpan.org/authors/id/S/SZ/SZABGAB/Padre-1.02.tar.gz
HTTP::Tiny failed with an internal error: SSL connection failed for cpan.org: SSL connect attempt failed error:14090086:
SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Giving up on 'C:\Strawberry\cpan\sources\authors\id\S\SZ\SZABGAB\Padre-1.02.tar.gz'


C:\home\pianokey>

Thank you,

[twata1]

When I run the attached script as a test, HTTP::Tiny->new->get('https://github.com/') succeeds, but HTTP::Tiny->new->get('https://cpan.org/') fails.

C:\home\pianokey>perl tinytest.pl
HTTP::Tiny (v0.084)
$ENV{PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT} is 1

Get https://github.com/
OK

Get https://cpan.org/
Failed!

C:\home\pianokey>set PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT=0

C:\home\pianokey>perl tinytest.pl
HTTP::Tiny (v0.084)
$ENV{PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT} is 0

Get https://github.com/
OK

Get https://cpan.org/
Failed!

C:\home\pianokey>

tinytest.pl.txt

[noxxi]

The variable is documented as PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT but the code actually uses PERL_HTTP_TINY_INSECURE_BY_DEFAULT, i.e. w/o the "SSL" in the name:

sub _verify_SSL_default {
    my ($self) = @_;
    # Check if insecure default certificate verification behaviour has been
    # changed by the user by setting PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT=1
    return (($ENV{PERL_HTTP_TINY_INSECURE_BY_DEFAULT} || '') eq '1') ? 0 : 1;
}

[stigtsp]

When I run the attached script as a test, HTTP::Tiny->new->get('https://github.com/') succeeds, but HTTP::Tiny->new->get('https://cpan.org/') fails.

Can you include the output of $Mozilla::CA::VERSION as well?

Since your tests fail on cpan.org but work on github.com I'm speculating that it is due to https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/

[twata1]

@stigtsp
I have inclueded the output of $Mozilla::CA::VERSION.

tinytest2.pl.txt

Output:

C:\home\pianokey>set PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT=1

C:\home\pianokey>perl tinytest2.pl
HTTP::Tiny (v0.084)
Mozilla::CA (v20221114)
$ENV{PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT} is 1

Get https://github.com/
OK

Get https://cpan.org/
Failed!

C:\home\pianokey>set PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT=0

C:\home\pianokey>perl tinytest2.pl
HTTP::Tiny (v0.084)
Mozilla::CA (v20221114)
$ENV{PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT} is 0

Get https://github.com/
OK

Get https://cpan.org/
Failed!

C:\home\pianokey>

[noxxi]

Mozilla::CA (v20221114)

This is recent and should work for validating cpan.org, i.e. not disabling certificate validation should be needed. Please run your code with perl -MIO::Socket::SSL=debug4 tinytest2.pl to enable SSL level debugging and include the output here, so that one could narrow down why the certificate validation fails.

[twata1]

@noxxi
The perl -MIO::Socket::SSL=debug4 tinytest2.pl output results were as follows:

C:\home\pianokey>set PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT=1

C:\home\pianokey>perl -MIO::Socket::SSL=debug4 tinytest2.pl
HTTP::Tiny (v0.084)
Mozilla::CA (v20221114)
$ENV{PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT} is 1

Get https://github.com/
DEBUG: .../IO/Socket/SSL.pm:3013: new ctx 45109360
DEBUG: .../IO/Socket/SSL.pm:1613: start handshake
DEBUG: .../IO/Socket/SSL.pm:730: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:774: using SNI with hostname github.com
DEBUG: .../IO/Socket/SSL.pm:815: request OCSP stapling
DEBUG: .../IO/Socket/SSL.pm:831: set socket to non-blocking to enforce timeout=60
DEBUG: .../IO/Socket/SSL.pm:845: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (OUT), TLS handshake, Client hello (1)
DEBUG: .../IO/Socket/SSL.pm:848: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:858: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:868: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:888: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:845: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Server hello (2)
DEBUG: .../IO/Socket/SSL.pm:2904: did not get stapled OCSP response
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Certificate (11)
DEBUG: .../IO/Socket/SSL.pm:2857: ok=1 [2] /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCe
rt Global Root CA
DEBUG: .../IO/Socket/SSL.pm:2857: ok=1 [1] /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA/C=US/O=DigiCert Inc/CN=DigiCert TLS Hybrid ECC SH
A384 2020 CA1
DEBUG: .../IO/Socket/SSL.pm:2857: ok=1 [0] /C=US/O=DigiCert Inc/CN=DigiCert TLS Hybrid ECC SHA384 2020 CA1/C=US/ST=California/L=San Francisco/O=GitHub, Inc./CN=
github.com
DEBUG: .../IO/Socket/SSL.pm:1833: scheme=http cert=46422624
DEBUG: .../IO/Socket/SSL.pm:1843: identity=github.com cn=github.com alt=2 github.com 2 www.github.com
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Server key exchange (12)
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Server finished (14)
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (OUT), TLS handshake, Client key exchange (16)
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1)
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (OUT), TLS handshake, Finished (20)
DEBUG: .../IO/Socket/SSL.pm:848: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:858: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:868: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:888: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:845: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS change cipher, Change cipher spec (1)
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Finished (20)
DEBUG: .../IO/Socket/SSL.pm:848: done Net::SSLeay::connect -> 1
DEBUG: .../IO/Socket/SSL.pm:903: ssl handshake done
DEBUG: .../IO/Socket/SSL.pm:3062: free ctx 45109360 open=45109360
DEBUG: .../IO/Socket/SSL.pm:3066: free ctx 45109360 callback
DEBUG: .../IO/Socket/SSL.pm:3073: OK free ctx 45109360
OK

Get https://cpan.org/
DEBUG: .../IO/Socket/SSL.pm:3013: new ctx 46329072
DEBUG: .../IO/Socket/SSL.pm:1613: start handshake
DEBUG: .../IO/Socket/SSL.pm:730: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:774: using SNI with hostname cpan.org
DEBUG: .../IO/Socket/SSL.pm:815: request OCSP stapling
DEBUG: .../IO/Socket/SSL.pm:831: set socket to non-blocking to enforce timeout=60
DEBUG: .../IO/Socket/SSL.pm:845: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (OUT), TLS handshake, Client hello (1)
DEBUG: .../IO/Socket/SSL.pm:848: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:858: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:868: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:888: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:845: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Server hello (2)
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Certificate (11)
DEBUG: .../IO/Socket/SSL.pm:2857: ok=0 [2] /O=Digital Signature Trust Co./CN=DST Root CA X3/C=US/O=Internet Security Research Group/CN=ISRG Root X1
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (OUT), TLS alert, unknown CA (560)
DEBUG: .../IO/Socket/SSL.pm:848: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:851: SSL connect attempt failed

DEBUG: .../IO/Socket/SSL.pm:851: local error: SSL connect attempt failed error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
DEBUG: .../IO/Socket/SSL.pm:854: fatal SSL error: SSL connect attempt failed error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
DEBUG: .../IO/Socket/SSL.pm:2037: downgrading SSL only, not closing socket
DEBUG: .../IO/Socket/SSL.pm:3062: free ctx 46329072 open=46329072
DEBUG: .../IO/Socket/SSL.pm:3066: free ctx 46329072 callback
DEBUG: .../IO/Socket/SSL.pm:3073: OK free ctx 46329072
Failed!

C:\home\pianokey>set PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT=0

C:\home\pianokey>perl -MIO::Socket::SSL=debug4 tinytest2.pl
HTTP::Tiny (v0.084)
Mozilla::CA (v20221114)
$ENV{PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT} is 0

Get https://github.com/
DEBUG: .../IO/Socket/SSL.pm:3013: new ctx 45895792
DEBUG: .../IO/Socket/SSL.pm:1613: start handshake
DEBUG: .../IO/Socket/SSL.pm:730: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:774: using SNI with hostname github.com
DEBUG: .../IO/Socket/SSL.pm:815: request OCSP stapling
DEBUG: .../IO/Socket/SSL.pm:831: set socket to non-blocking to enforce timeout=60
DEBUG: .../IO/Socket/SSL.pm:845: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (OUT), TLS handshake, Client hello (1)
DEBUG: .../IO/Socket/SSL.pm:848: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:858: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:868: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:888: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:845: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Server hello (2)
DEBUG: .../IO/Socket/SSL.pm:2904: did not get stapled OCSP response
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Certificate (11)
DEBUG: .../IO/Socket/SSL.pm:2857: ok=1 [2] /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCe
rt Global Root CA
DEBUG: .../IO/Socket/SSL.pm:2857: ok=1 [1] /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA/C=US/O=DigiCert Inc/CN=DigiCert TLS Hybrid ECC SH
A384 2020 CA1
DEBUG: .../IO/Socket/SSL.pm:2857: ok=1 [0] /C=US/O=DigiCert Inc/CN=DigiCert TLS Hybrid ECC SHA384 2020 CA1/C=US/ST=California/L=San Francisco/O=GitHub, Inc./CN=
github.com
DEBUG: .../IO/Socket/SSL.pm:1833: scheme=http cert=47209408
DEBUG: .../IO/Socket/SSL.pm:1843: identity=github.com cn=github.com alt=2 github.com 2 www.github.com
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Server key exchange (12)
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Server finished (14)
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (OUT), TLS handshake, Client key exchange (16)
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1)
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (OUT), TLS handshake, Finished (20)
DEBUG: .../IO/Socket/SSL.pm:848: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:858: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:868: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:888: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:845: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS change cipher, Change cipher spec (1)
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Finished (20)
DEBUG: .../IO/Socket/SSL.pm:848: done Net::SSLeay::connect -> 1
DEBUG: .../IO/Socket/SSL.pm:903: ssl handshake done
DEBUG: .../IO/Socket/SSL.pm:3062: free ctx 45895792 open=45895792
DEBUG: .../IO/Socket/SSL.pm:3066: free ctx 45895792 callback
DEBUG: .../IO/Socket/SSL.pm:3073: OK free ctx 45895792
OK

Get https://cpan.org/
DEBUG: .../IO/Socket/SSL.pm:3013: new ctx 47115504
DEBUG: .../IO/Socket/SSL.pm:1613: start handshake
DEBUG: .../IO/Socket/SSL.pm:730: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:774: using SNI with hostname cpan.org
DEBUG: .../IO/Socket/SSL.pm:815: request OCSP stapling
DEBUG: .../IO/Socket/SSL.pm:831: set socket to non-blocking to enforce timeout=60
DEBUG: .../IO/Socket/SSL.pm:845: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (OUT), TLS handshake, Client hello (1)
DEBUG: .../IO/Socket/SSL.pm:848: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:858: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:868: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:888: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:845: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Server hello (2)
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (IN), TLS handshake, Certificate (11)
DEBUG: .../IO/Socket/SSL.pm:2857: ok=0 [2] /O=Digital Signature Trust Co./CN=DST Root CA X3/C=US/O=Internet Security Research Group/CN=ISRG Root X1
DEBUG: .../IO/Socket/SSL.pm:3690: * TLSv1.2 (OUT), TLS alert, unknown CA (560)
DEBUG: .../IO/Socket/SSL.pm:848: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:851: SSL connect attempt failed

DEBUG: .../IO/Socket/SSL.pm:851: local error: SSL connect attempt failed error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
DEBUG: .../IO/Socket/SSL.pm:854: fatal SSL error: SSL connect attempt failed error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
DEBUG: .../IO/Socket/SSL.pm:2037: downgrading SSL only, not closing socket
DEBUG: .../IO/Socket/SSL.pm:3062: free ctx 47115504 open=47115504
DEBUG: .../IO/Socket/SSL.pm:3066: free ctx 47115504 callback
DEBUG: .../IO/Socket/SSL.pm:3073: OK free ctx 47115504
Failed!

C:\home\pianokey>

[noxxi]

Looks like you run into the issue with certificate validation when cross-signing is used: cpan.org sends the following trust chain:

 0 s:CN = cpan.org
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   i:O = Digital Signature Trust Co., CN = DST Root CA X3

The "ISRG Root X1" given as intermediate CA is contained in Mozilla::CA but "DST Root CA X3" as expected root CA is not since it is obsolete for several years now. A proper TLS stack will detect this and stop chain validation at the first CA certificate in the chain which can be backed by the local trust store. The old version of OpenSSL (OpenSSL 1.0.1e) which comes with your old version of Strawberry Perl (released 10 years ago!!!) instead insists on validating the final CA certificate from the server provided chain with the local trust store - which fails.

To provide proper validation while keeping your old distribution you need to explicitly add the missing "DST Root CA X3" to your trust store. It could be downloaded from here. But I really recommend to upgrade your Perl - the OpenSSL version is really old and also does not support modern TLS 1.3.

[twata1]

@stigtsp @noxxi
Thanks for the speculation and advice.

I would probably prefer to use it via HTTP-TLS proxy.
If it were a real job, I would try to upgrade the version of openssl itself (or upgrade the version of programming language).

Thank you,

[twata1]

After upgrading from version 0.084 to 0.086, I have just been able to confirm that the value of the environment variable PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT has an effect.

C:\home\pianokey>perl tinytest2.pl
HTTP::Tiny (v0.086)
Mozilla::CA (v20221114)
$ENV{PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT} is 1

Get https://github.com/
OK

Get https://cpan.org/
OK


C:\home\pianokey>set PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT=0

C:\home\pianokey>perl tinytest2.pl
HTTP::Tiny (v0.086)
Mozilla::CA (v20221114)
$ENV{PERL_HTTP_TINY_SSL_INSECURE_BY_DEFAULT} is 0

Get https://github.com/
OK

Get https://cpan.org/
Failed!

C:\home\pianokey>

Thanks everyone!