12

I have a need to ensure that an SMTP server certificate is signed by a public certificate authority. I would like to use phpseclib or some other trusted library. I believe I can use the root certificates extracted from Firefox.

There are some home-brew approaches here to check cert dates and other metadata, but it does not look like that does any signature checking as such (other than ensuring that OpenSSL does it). In any case, I want to use a library - I want to write as little certificate-handling code as possible, since I am not a cryptographer.

That said, the answers on the above link were still very useful, as it helped me get some code to fetch a certificate from a TLS conversation:

$url = "tcp://{$domain}:{$port}";
$connection_context_option = [
    'ssl' => [
        'capture_peer_cert' => true,
        'verify_peer' => false,
        'verify_peer_name' => false,
        'allow_self_signed' => true,
    ]
];
$connection_context = stream_context_create($connection_context_option);
$connection_client = stream_socket_client($url, $errno, $errstr, 30, STREAM_CLIENT_CONNECT, $connection_context);
stream_set_timeout($connection_client, 2);
fread($connection_client, 10240);
fwrite($connection_client,"HELO alice\r\n");
fread($connection_client, 10240);
fwrite($connection_client, "STARTTLS\r\n");
fread($connection_client, 10240);
$ok = stream_socket_enable_crypto($connection_client, TRUE, STREAM_CRYPTO_METHOD_SSLv23_CLIENT);
if ($ok === false)
{
    return false;
}
$connection_info = stream_context_get_params($connection_client);

openssl_x509_export($info["options"]["ssl"]["peer_certificate"], $pem_encoded);

(Note that I have turned off certificate validation here deliberately. This is because I have no control over what hosts this runs on, and their certificates may be old or misconfigured. Therefore, I wish to fetch the certificate regardless of the verification on the connection I am using, and then verify it myself using a cacert.pem that I will supply.)

That will give me a certificate like this. This one is for Microsoft's Live.com email server at smtp.live.com:587:

-----BEGIN CERTIFICATE-----
MIIG3TCCBcWgAwIBAgIQAtB7LVsRCmgbyWiiw7Sf5jANBgkqhkiG9w0BAQsFADBN
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMScwJQYDVQQDEx5E
aWdpQ2VydCBTSEEyIFNlY3VyZSBTZXJ2ZXIgQ0EwHhcNMTcwOTEzMDAwMDAwWhcN
MTkwOTEzMTIwMDAwWjBqMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3Rv
bjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0
aW9uMRQwEgYDVQQDEwtvdXRsb29rLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEP
ADCCAQoCggEBAIz2tovvgBmK4sOHgpyzCdtXrI0XOujctf6LHMj16wzUnMEatioS
tH0Pz0dKkCr/0yd9qtXbGhD1o6WhFsd7k651K9MZ98+uQ29SzTIAl6y1gkaBbp4h
MFXcE5EpRNHHmK8t2OR7hzmrvvNr6OTYv7BhVCw9pSrQqEFNno0K2TQRhAD9uzrL
OY+rBBVedCXWXH7uhZoZ6joUU7CEA5pPMzKPL1ro+Eorc8vt5FYOC+oAT587+b1M
z+jbZVQlq0qaMkBKRtUIII78MYY0n8DopGqHyzwqWoGySHJNC8256q+MwsZQvvQ3
vmy/rf61h2sg1tU0s7O88Yufxp0LSaMMzZcCAwEAAaOCA5owggOWMB8GA1UdIwQY
MBaAFA+AYRyCMWHVLyjnjUY4tCzhxtniMB0GA1UdDgQWBBT7hLoZ/03rqwcslIc2
0k0z2R+vNTCCAdwGA1UdEQSCAdMwggHPggtvdXRsb29rLmNvbYIWKi5jbG8uZm9v
dHByaW50ZG5zLmNvbYIWKi5ucmIuZm9vdHByaW50ZG5zLmNvbYIgYXR0YWNobWVu
dC5vdXRsb29rLm9mZmljZXBwZS5uZXSCG2F0dGFjaG1lbnQub3V0bG9vay5saXZl
Lm5ldIIdYXR0YWNobWVudC5vdXRsb29rLm9mZmljZS5uZXSCHWNjcy5sb2dpbi5t
aWNyb3NvZnRvbmxpbmUuY29tgiFjY3Mtc2RmLmxvZ2luLm1pY3Jvc29mdG9ubGlu
ZS5jb22CC2hvdG1haWwuY29tgg0qLmhvdG1haWwuY29tggoqLmxpdmUuY29tghZt
YWlsLnNlcnZpY2VzLmxpdmUuY29tgg1vZmZpY2UzNjUuY29tgg8qLm9mZmljZTM2
NS5jb22CFyoub3V0bG9vay5vZmZpY2UzNjUuY29tgg0qLm91dGxvb2suY29tghYq
LmludGVybmFsLm91dGxvb2suY29tggwqLm9mZmljZS5jb22CEm91dGxvb2sub2Zm
aWNlLmNvbYIUc3Vic3RyYXRlLm9mZmljZS5jb22CGHN1YnN0cmF0ZS1zZGYub2Zm
aWNlLmNvbTAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsG
AQUFBwMCMGsGA1UdHwRkMGIwL6AtoCuGKWh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNv
bS9zc2NhLXNoYTItZzEuY3JsMC+gLaArhilodHRwOi8vY3JsNC5kaWdpY2VydC5j
b20vc3NjYS1zaGEyLWcxLmNybDBMBgNVHSAERTBDMDcGCWCGSAGG/WwBATAqMCgG
CCsGAQUFBwIBFhxodHRwczovL3d3dy5kaWdpY2VydC5jb20vQ1BTMAgGBmeBDAEC
AjB8BggrBgEFBQcBAQRwMG4wJAYIKwYBBQUHMAGGGGh0dHA6Ly9vY3NwLmRpZ2lj
ZXJ0LmNvbTBGBggrBgEFBQcwAoY6aHR0cDovL2NhY2VydHMuZGlnaWNlcnQuY29t
L0RpZ2lDZXJ0U0hBMlNlY3VyZVNlcnZlckNBLmNydDAMBgNVHRMBAf8EAjAAMA0G
CSqGSIb3DQEBCwUAA4IBAQA3zjN7I6jTeL+08nhG5eAY0q4pLY40bCQHqONBLSI3
uRmQFUfrQOPYBqLC1QU+J2Z2HcX7YiqE3WAR3ODS9g2BAVXkKOQKNBnr2hKwueOz
qPwyvTyzcIQYUw+SrTX+bfJwYMTmZvtP9S7/pB1jPhrV7YGsD55AI9bGa9cmH7VQ
OiL1p5Qovg5KRsldoZeC04OF/UQIR1fv47VGptsHHGypvSo1JinJFQMXylqLIrUW
lV66p3Ui7pFABGc/Lv7nOyANXfLugBO8MyzydGA4NRGiS2MbGpswPCg154pWausU
M0qaEPsM2o3CSTfxSJQQIyEe+izV3UQqYSyWkNqCCFPN
-----END CERTIFICATE-----

OK, great. So I want to validate this against any public CA. I believe this is a valid certificate, and the chain is correctly verified using this checking service:

Array
(
    [name] => /C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=outlook.com
    [subject] => Array
        (
            [C] => US
            [ST] => Washington
            [L] => Redmond
            [O] => Microsoft Corporation
            [CN] => outlook.com
        )

    [hash] => a3c08ece
    [issuer] => Array
        (
            [C] => US
            [O] => DigiCert Inc
            [CN] => DigiCert SHA2 Secure Server CA
        )

    [version] => 2
    [serialNumber] => 3740952067977374966703603448215281638
    [serialNumberHex] => 02D07B2D5B110A681BC968A2C3B49FE6
    [validFrom] => 170913000000Z
    [validTo] => 190913120000Z
    [validFrom_time_t] => 1505260800
    [validTo_time_t] => 1568376000
    [signatureTypeSN] => RSA-SHA256
    [signatureTypeLN] => sha256WithRSAEncryption
    [signatureTypeNID] => 668
    [purposes] => Array
        (
            [1] => Array
                (
                    [0] => 1
                    [1] => 
                    [2] => sslclient
                )

            [2] => Array
                (
                    [0] => 1
                    [1] => 
                    [2] => sslserver
                )

            [3] => Array
                (
                    [0] => 1
                    [1] => 
                    [2] => nssslserver
                )

            [4] => Array
                (
                    [0] => 
                    [1] => 
                    [2] => smimesign
                )

            [5] => Array
                (
                    [0] => 
                    [1] => 
                    [2] => smimeencrypt
                )

            [6] => Array
                (
                    [0] => 
                    [1] => 
                    [2] => crlsign
                )

            [7] => Array
                (
                    [0] => 1
                    [1] => 1
                    [2] => any
                )

            [8] => Array
                (
                    [0] => 1
                    [1] => 
                    [2] => ocsphelper
                )

            [9] => Array
                (
                    [0] => 
                    [1] => 
                    [2] => timestampsign
                )

        )

    [extensions] => Array
        (
            [authorityKeyIdentifier] => keyid:0F:80:61:1C:82:31:61:D5:2F:28:E7:8D:46:38:B4:2C:E1:C6:D9:E2

            [subjectKeyIdentifier] => FB:84:BA:19:FF:4D:EB:AB:07:2C:94:87:36:D2:4D:33:D9:1F:AF:35
            [subjectAltName] => DNS:outlook.com, DNS:*.clo.footprintdns.com, DNS:*.nrb.footprintdns.com, DNS:attachment.outlook.officeppe.net, DNS:attachment.outlook.live.net, DNS:attachment.outlook.office.net, DNS:ccs.login.microsoftonline.com, DNS:ccs-sdf.login.microsoftonline.com, DNS:hotmail.com, DNS:*.hotmail.com, DNS:*.live.com, DNS:mail.services.live.com, DNS:office365.com, DNS:*.office365.com, DNS:*.outlook.office365.com, DNS:*.outlook.com, DNS:*.internal.outlook.com, DNS:*.office.com, DNS:outlook.office.com, DNS:substrate.office.com, DNS:substrate-sdf.office.com
            [keyUsage] => Digital Signature, Key Encipherment
            [extendedKeyUsage] => TLS Web Server Authentication, TLS Web Client Authentication
            [crlDistributionPoints] => 
Full Name:
  URI:http://crl3.digicert.com/ssca-sha2-g1.crl

Full Name:
  URI:http://crl4.digicert.com/ssca-sha2-g1.crl

            [certificatePolicies] => Policy: 2.16.840.1.114412.1.1
  CPS: https://www.digicert.com/CPS
Policy: 2.23.140.1.2.2

            [authorityInfoAccess] => OCSP - URI:http://ocsp.digicert.com
CA Issuers - URI:http://cacerts.digicert.com/DigiCertSHA2SecureServerCA.crt

            [basicConstraints] => CA:FALSE
        )

)

Here is how I am trying to validate the sig in phpseclib:

$x509 = new \phpseclib\File\X509();

// From the Mozilla bundle (getPublicCaCerts splits them with a regex)
$splitCerts = getPublicCaCerts(file_get_contents('cacert.pem'));

// Load the certs separately
$caStatus = true;
foreach ($splitCerts as $caCert)
{
    $caStatus = $caStatus && $x509->loadCA($caCert);
}
// $caStatus is now true, so all good here

$certData = $x509->loadX509($pem_encoded); // From the TLS server
$valid = $x509->validateSignature();
// $valid is now false

This returns false, which is not what I expect. I wonder if I have got the input formats correct? The loading of the CAs and the cert under test seem to return good values. Unfortunately, the phpseclib docs are a bit light on examples, and I've not found much elsewhere on the web.

Aside: I have a vague suspicion that this library could help me, assuming it has the feature to verify a certificate. However, I think it is trying to do to much for my case - I want my software to run on shared hosting, and auto-downloading feels like another moving part that might fail. I would rather deploy my own package, supply the public CA information as a (large) parameter, and run the validation test in situ. phpseclib is probably perfect for that, as long as I can figure out the input formats!

Possible reason: phpseclib can't find a matching cert to test

I have narrowed the problem down to a search loop in phpseclib's validator. On L2156, we have this code:

case !defined('FILE_X509_IGNORE_TYPE') && $this->currentCert['tbsCertificate']['issuer'] === $ca['tbsCertificate']['subject']:

The constant is indeed undefined, so the test really is whether a CA can match on the right cert particulars. The cert has this meta-data:

id-at-countryName = US
id-at-organizationName = DigiCert Inc
id-at-organizationalUnitName = www.digicert.com
id-at-commonName = DigiCert SHA2 High Assurance Server CA

And for all the current certs that would otherwise match, I have only these values in the latest cert bundle (i.e. all of the below would match if it was not for the common name DigiCert SHA2 High Assurance Server CA not being found):

id-at-commonName = DigiCert Assured ID Root CA

id-at-commonName = DigiCert High Assurance EV Root CA

id-at-commonName = DigiCert Assured ID Root G2

id-at-commonName = DigiCert Assured ID Root G3

id-at-commonName = DigiCert Global Root G2

id-at-commonName = DigiCert Global Root G3

id-at-commonName = DigiCert Trusted Root G4

Thus, the system does not even get at far as a digital signature check, since it cannot find the CA corresponding to this cert. What am I missing? This simple task should be a lot easier than this!

Possible reason: the Mozilla bundle is web server certs only

I have speculated that mail server certificates are not in the Mozilla bundle because a web browser would have no need for them. I would assume though that the certs on my GNU/Linux Mint install would be up-to-date and suitable for the purpose, since an operating system should be able to verify certs used in mail servers.

I therefore tried this code, which loads all the system certs into phpseclib:

$certLocations = openssl_get_cert_locations();
$dir = $certLocations['default_cert_dir'];
$glob = $dir . '/*';
echo "Finding certs: " . $dir . "\n";

$x509 = new \phpseclib\File\X509();

foreach (glob($glob) as $certPath)
{
    // Change this so it is recursive?
    if (is_file($certPath))
    {
        $ok = $x509->loadCA(file_get_contents($certPath));
        if (!$ok)
        {
            echo sprintf("CA cert `%s` is invalid\n", $certPath);
        }
    }
}

// The 'getCertToTest' func just gets the live.com cert as a string
$data = $x509->loadX509(getCertToTest());
if (!$data)
{
    echo "Cert is invalid\n";
    exit();
}

$valid = $x509->validateSignature();
echo sprintf("Validation: %s\n", $valid ? 'Yes' : 'No');

Unfortunately this fails as well.

Confirm that my system certs are OK using openssl

I have issued this command on my system, and the remote TLS cert is verified OK. I don't know the phpseclib code well, but it doesn't look like it is doing any chaining, which is evidently necessary.

openssl s_client -connect smtp.live.com:25 -starttls smtp
CONNECTED(00000003)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert Cloud Services CA-1
verify return:1
depth=0 C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, CN = outlook.com
verify return:1
---
Certificate chain
 0 s:/C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=outlook.com
   i:/C=US/O=DigiCert Inc/CN=DigiCert Cloud Services CA-1
 1 s:/C=US/O=DigiCert Inc/CN=DigiCert Cloud Services CA-1
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIG/jCCBeagAwIBAgIQDs2Q7J6KkeHe1d6ecU8P9DANBgkqhkiG9w0BAQsFADBL
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMSUwIwYDVQQDExxE
aWdpQ2VydCBDbG91ZCBTZXJ2aWNlcyBDQS0xMB4XDTE3MDkxMzAwMDAwMFoXDTE4
MDkxMzEyMDAwMFowajELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24x
EDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlv
(snipped, see other code block)
nGhseM2tJfwa2HMwUpuuo5029u4Dd40qvD0cMz33cOvBLRGkTPbXCFw24ZBdQrkt
SC5TAWzHFyT2tLC17LeSb7d0g+fuj41L6y4a9och8cPiv9IAP4sftzYupO99h4qg
7UXP7o3AOOGqrPS3INhO4068Z63indstanIHYM0IUHa3A2xrcz7ZbEuw1HiGH/Ba
HMz/gTSd2c0BXNiPeM7gdOK3
-----END CERTIFICATE-----
subject=/C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=outlook.com
issuer=/C=US/O=DigiCert Inc/CN=DigiCert Cloud Services CA-1
---
No client certificate CA names sent
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA256:RSA+SHA384:RSA+SHA1:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA1:DSA+SHA1:RSA+SHA512:ECDSA+SHA512
Shared Requested Signature Algorithms: RSA+SHA256:RSA+SHA384:RSA+SHA1:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA1:DSA+SHA1:RSA+SHA512:ECDSA+SHA512
Peer signing digest: SHA1
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3831 bytes and written 478 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: C11A0000050CD144CB5C49DD873D2C911F7CDDECFE18001F70FE0427C88B52F7
    Session-ID-ctx: 
    Master-Key: 5F4EC0B1198CF0A16D19F758E6A0961ED227FCEBD7EF96D4D6A7470E3F9B0453A2A06AC0C1691C31A1CA4B73209B38DE
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1519322480
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
250 SMTPUTF8

I may drop phpseclib in favour of the binary command, but I would be relying on system/exec etc, which may not be available. Still, working sometimes is better than not working always!

Summary

Despite extensive work, I have reached a dead end on this. I will summarise here what I am wanting to do.

I want to use PHP to verify mail server SSL certificates against known public CAs. I don't know if the Mozilla certificates are appropriate to use for this, or whether I need to obtain them from elsewhere. I have found that my Linux Mint development machine has certificates that will verify the example mail server above.

The trivial strategy here is to use PHP 5.6+ and ensure all verification options are enabled in the stream context (though ideally, I wish to support 5.5 also). However, I want to do the proof myself, either using openssl_ functions or a library such as phpseclib, so I can see why a given cert is valid (or not). The openssl binary does this (as shown above) and it does so presumably using something very similar to PHP's openssl calls, but I don't know how it does so. For example, does the openssl binary use cert chain information to do this?

Another approach would be to read some information from a valid SSL session, but I cannot find anything in the manual to do that either.

halfer
  • 18,701
  • 13
  • 79
  • 158
  • Right, having just dictated this to the rubber duck, I have noticed that `$x509->loadCA($certs)` is returning `false`, which means my public CA certs are indeed not recognised. I have a sneaking suspicion I have to parse the Mozilla extract and supply each `BEGIN ... END` to this method separately. Answers/comments still welcome, but I will update this in due course! – halfer Feb 22 '18 at 13:10
  • The original developer of phpseclib actually did this a few years ago at http://www.frostjedi.com/phpbb3/viewtopic.php?f=46&t=30786 . The source code is in that link. That was done a number of years ago tho - it's possible the number of CA's now using ECDSA has gone up idk. – neubert Feb 22 '18 at 13:49
  • Thanks @neubert. That confirms that I was right in my earlier comment - the CA certs need to be split and added separately. I am now getting `true` for when each one is loaded. However, the validation method is still coming back false. There are a lot of clauses in that, so I may have to do some digging! – halfer Feb 22 '18 at 13:55
  • I suspect a lot more certs are using ECDSA than did back when the frostjedi.com post was made. phpseclib 1.0 / 2.0 only support RSA. It's my understanding ECDSA support is being worked on for the master branch but that's still a work in progress. That said, you could run some stats like the frostjedi.com poster did to find out for sure. – neubert Feb 22 '18 at 17:04
  • @neubert: my latest research indicates that it is failing because it cannot find a matching `id-at-commonName` between the cert under test and my CA list. Given that I am testing a mail server cert and the list of CAs comes from Mozilla, I wonder if my CA list is in fact "web server only"? – halfer Feb 22 '18 at 17:05
  • 1
    That's a distinct possibility. I can try my own hand at it but won't be able to do so until the weekend. – neubert Feb 22 '18 at 17:06
  • Thanks @neubert. I will of course update here if I find any new info. – halfer Feb 22 '18 at 17:08
  • @neubert: I've updated this to as far as I have got. I can do the test I want using the `openssl` binary via `exec()`, but in my little collection of four real-world shared hosting test accounts, two had the system functions disabled. So I am hesitant to implement this with `exec` when using the openssl module would be more resilient (and that module is a system requirement anyway). If it tempts you into doing some investigation, I am happy to put a juicy bounty on this! – halfer Feb 23 '18 at 20:45
  • I'd like to find out, in particular, how the `openssl` command output does the cert chaining. Perhaps one approach is to do two `openssl_` PHP calls: one with verification off, to obtain the cert for cases where it is self-signed (and other invalid cases), and then once with verification on, to test whether it is verifiable. Perhaps in the latter case, there is some SSL session info available that can be retrieved, which describes the chain? – halfer Feb 23 '18 at 20:48
  • If you want to do chaining with phpseclib you need to import each of the certs in the chain as a CA cert. ie. `loadCA()` should be seen more as `loadTrustedCerts()`. CA certs are auto-trusted as are certs signed by the CA certs. If you have an intermediate cert that's signing your end cert and that intermediate cert is signed by a CA then that intermediate cert should be loaded with `loadCA()`. – neubert Feb 23 '18 at 22:25
  • @neubert: well, I _think_ I want chaining, but half the problem is my knowledge of public CAs is not extensive, and I don't know what I need `:o)`. Essentially, I can tell that the certs on my machine are enough to trust the `live.com` certificate (as confirmed by the `openssl` binary) but I am struggling to find out how to do this using the `openssl` PHP extension. Whether this necessitates chaining can be part of my question! – halfer Feb 23 '18 at 22:29
  • It could be argued that if I remove all the unsafe stuff (e.g. `verify_peer`, `verify_peer_name`, etc) _and_ I am on PHP 5.6+ then I "know" the remote cert is trusted by the server (since the connection will be aborted if the connection is unverified). However, if I am on PHP 5.5 then I believe the status is "unchecked", and I'd be more comfortable if I can verify this properly using the extension, rather than hoping it is done for me. – halfer Feb 23 '18 at 22:32

3 Answers3

3

I was able to get it to verify thusly:

<?php
include('File/X509.php');

$certs = file_get_contents('cacert.pem');
$certs = preg_split('#==(?:=)+#', $certs);
foreach ($certs as &$cert) {
   $cert = trim(preg_replace('#-----END CERTIFICATE-----.+#s', '-----END CERTIFICATE-----', $cert));
}
unset($cert);
array_shift($certs);

$x509 = new File_X509();

foreach ($certs as $i => $cert) {
   $x509->loadCA($cert);
}

$test = file_get_contents('test.cer');

$x509->loadX509($test);
$opts = $x509->getExtension('id-pe-authorityInfoAccess');
foreach ($opts as $opt) {
    if ($opt['accessMethod'] == 'id-ad-caIssuers') {
        $url = $opt['accessLocation']['uniformResourceIdentifier'];
        break;
    }
}

$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$intermediate = curl_exec($ch);

$x509->loadX509($intermediate);

if (!$x509->validateSignature()) {
    exit('validation failed');
}

$x509->loadCA($intermediate);

$x509->loadX509($test);

echo $x509->validateSignature() ?
    'good' :
    'bad';

Note the $test = file_get_contents('test.cer'); bit. That's where I loaded your cert. If I commented out $x509->loadCA($intermediate); the cert didn't validate. If I leave it in it does validate.

edit:

This branch does this automatically:

https://github.com/terrafrost/phpseclib/tree/authority-info-access-1.0

Unit tests still need to be added however it's not in the 2.0 or master branches yet either. I'll try to do work on that this weekend.

Example of how to use:

<?php

include('File/X509.php');

$certs = file_get_contents('cacert.pem');
$certs = preg_split('#==(?:=)+#', $certs);
foreach ($certs as &$cert) {
   $cert = trim(preg_replace('#-----END CERTIFICATE-----.+#s', '-----END CERTIFICATE-----', $cert));
}
unset($cert);
array_shift($certs);

$x509 = new File_X509();

foreach ($certs as $i => $cert) {
   $x509->loadCA($cert);
}

$test = file_get_contents('test.cer');

$x509->loadX509($test);
//$x509->setRecurLimit(0);

echo $x509->validateSignature() ?
    'good' :
    'bad';
neubert
  • 14,208
  • 21
  • 90
  • 172
  • This looks marvellous, thanks - will try it as soon as I can. So it has to do HTTP fetches from the CAs themselves in order to verify the chain? – halfer Feb 26 '18 at 17:48
  • 2
    @halfer - Yah - the cert you're trying to verify isn't signed directly by the CA - it's signed by an intermediate cert. – neubert Feb 26 '18 at 17:50
  • Great. Is there a possibility one would have to do this is a loop, involving certs signed to an arbitrary depth? Yes please to being simplified by phpseclib! `:-)` – halfer Feb 26 '18 at 17:54
  • @halfer - it looks like that is indeed a possibility. https://crypto.stackexchange.com/a/55974/4520 talks more about this. Anyway, I'll work on getting it into phpseclib. Could take a few days tho. Might not even be done before the bounty is up but I'll see what I can do! – neubert Feb 26 '18 at 20:01
  • 1
    Cool stuff. I will see if I can get some examples in order to write some tests. No rush on getting it into phpseclib - the bounty is yours in a week unless someone adds another a better answer, which I would imagine is unlikely! If this works for me, it will get me unblocked nicely. – halfer Feb 26 '18 at 20:11
  • 1
    I got this to work on my end. As per the warning on the link you supply, it looks like it's worth being careful about what links the cert causes a server-side system to follow (or indeed it could force it to follow them in an infinite loop). I am wondering whether to operate a whitelist too, for maximum paranoia. – halfer Feb 27 '18 at 14:16
  • @halfer - here's a branch for 1.0 that does this automatically: https://github.com/terrafrost/phpseclib/tree/authority-info-access-1.0 Unit tests still need to be added however and I want to get it into the 2.0 / master branches as well. I've edited my post to include examples. – neubert Mar 02 '18 at 13:28
  • That sounds good @neubert. However, just to stop you doing any work that is unnecessary, have you seen the other answer? It turns out that the remote server can be interrogated using `capture_peer_cert_chain` rather than `capture_peer_cert`, and it retrieves an array of all the certs in a chain. I'd be happy to hear that your recent good work is useful for phpseclib anyway, but if I can get a remote server to provide all the certs it needs, that does sound like a compact way to do it. – halfer Mar 02 '18 at 16:09
  • @halfer - `capture_peer_cert_chain` looks like a great solution if you're using OpenSSL but it still seems like a very useful feature for phpseclib. – neubert Mar 02 '18 at 16:32
3

The certificate is signed by an intermediate, which in this case is DigiCert SHA2 Secure Server CA. Intermediate certificates are not present in a root certificate list. Whatever library you're using, I believe you have to explicitly provide valid intermediate certificates for the validation process.

Here's an example using sop/x509 library.

// certificate from smtp.live.com
$cert = Certificate::fromPEM(PEM::fromString($certdata));
// list of trust anchors from https://curl.haxx.se/ca/cacert.pem
$trusted = CertificateBundle::fromPEMBundle(PEMBundle::fromFile('cacert.pem'));
// intermediate certificate from
// https://www.digicert.com/CACerts/DigiCertSHA2SecureServerCA.crt
$intermediates = new CertificateBundle(
    Certificate::fromDER(file_get_contents('DigiCertSHA2SecureServerCA.crt')));
// build certification path
$path_builder = new CertificationPathBuilder($trusted);
$certification_path = $path_builder->shortestPathToTarget($cert, $intermediates);
// validate certification path
$result = $certification_path->validate(PathValidationConfig::defaultConfig());
// failure would throw an exception
echo "Validation successful\n";

This does signature validation and some basic checks per RFC 5280. It does not verify that CN or SANs match the destination domain.

Disclaimer! I'm the author of said library. It's not battle-proven and thus I'm afraid it won't fall into your "some other trusted library" category. Feel free to experiment with it however :).

halfer
  • 18,701
  • 13
  • 79
  • 158
Joe
  • 1,488
  • 9
  • 9
  • This looks very good, thanks! On the other answer, I raised the possibility of there being a verification chain of arbitrary length. I wonder, is this something you could do in a utility class - keep on loading new certs, up to a reasonable maximum, until there are no more certs to load? – halfer Mar 02 '18 at 10:25
  • The quality of the docs, testing and CI here would not necessarily lead me to insist on high levels of usage before using this library, since they are good confidence measures in themselves. However, it understandably requires PHP7, and for the time being I'm still supporting PHP 5.5, as that's the current minimum of my other dependencies. I may bump that up soon however! – halfer Mar 02 '18 at 10:31
  • [0.5.0](https://github.com/sop/x509/tree/0.5.0) version is targeted for PHP 5.6, but that doesn't help you either. For the AIA extension and downloading certificates without user intervention, that sounds a bit dubious in my opinion, but then again is it any worse than to interpret certificates given by the server on TLS handshake? – Joe Mar 02 '18 at 10:53
  • Ah, that's an interesting note: why do you feel auto-fetching links is dubious? Is that for the reasons I commented on the other answer, which is that a rogue cert could try to send an automated script to fetch from localhost, or `file://` protocols, or other security risks? – halfer Mar 02 '18 at 10:55
  • (FWIW, I assume the `openssl` binary call I illustrated in my question would have transparently downloaded the intermediate cert, and thus perhaps it would indeed download a chain of them if necessary? Not sure, but it did not declare that it was downloading the one!). – halfer Mar 02 '18 at 10:57
  • You can get all certs that the server provides by using `capture_peer_cert_chain` instead of `capture_peer_cert` option. I'm not sure but i doubt that openssl downloads AIA pointers. Far as i understand TLS protocol should provide necessary intermediates if the server is configured correctly. Handling AIA by default just seems quite error prone. Actually RFC also dictates that the extension must be non-critical. – Joe Mar 02 '18 at 11:16
  • Aha! I will try `capture_peer_cert_chain` later today - I had not spotted that [in the docs](https://secure.php.net/manual/en/context.ssl.php). If I can get openssl to do the necessary downloading for me, in a safe manner, that would be excellent. I will let you know how I get on. – halfer Mar 02 '18 at 11:24
  • Update: I have added a solution involving your excellent advice, and it works very well. – halfer Mar 02 '18 at 21:26
3

It turns out I can fetch the whole of the certificate chain from the remote server - I have had to go through various false leads and dodgy assumptions to get to this point! Credit to Joe who pointed out, in the comments, that the context option capture_peer_cert only gets the certificate cert without any chain certificates that would complete the validation path to a public CA; to do that, one needs capture_peer_cert_chain.

Here is some code to do that:

$url = "tcp://{$domain}:{$port}";
$connection_context_option = [
    'ssl' => [
        'capture_peer_cert_chain' => true,
        'verify_peer' => false,
        'verify_peer_name' => false,
        'allow_self_signed' => true,
    ]
];
$connection_context = stream_context_create($connection_context_option);
$connection_client = stream_socket_client($url, $errno, $errstr, 30, STREAM_CLIENT_CONNECT, $connection_context);
// timeout fread after 2s
stream_set_timeout($connection_client, 2);
fread($connection_client, 10240);
fwrite($connection_client,"HELO alice\r\n");
// let the server introduce it self before sending command
fread($connection_client, 10240);
// send STARTTLS command
fwrite($connection_client, "STARTTLS\r\n");
// wait for server to say its ready, before switching
fread($connection_client, 10240);
// Switching to SSL/TLS
$ok = stream_socket_enable_crypto($connection_client, TRUE, STREAM_CRYPTO_METHOD_SSLv23_CLIENT);
if ($ok === false)
{
    return false;
}

$chainInfo = stream_context_get_params($connection_client);

Then we can extract all the certificates using OpenSSL:

if (isset($chainInfo["options"]["ssl"]["peer_certificate_chain"]) && is_array($chainInfo["options"]["ssl"]["peer_certificate_chain"]))
{
    $verboseChainCerts = [];
    foreach ($chainInfo["options"]["ssl"]["peer_certificate_chain"] as $ord => $intermediate)
    {
        $chainCertOk = openssl_x509_export($intermediate, $verboseChainCerts[$ord]);
        if (!$chainCertOk)
        {
            $verboseChainCerts[$ord] = 'Cannot read chain info';
        }
    }

    $chainValid = checkChainAutomatically($x509Chain, $verboseChainCerts);
}

Finally, the function to do the check is here. You should assume that a good set of public certificates are loaded already, as per the question:

function checkChainAutomatically(X509 $x509, array $encodedCerts)
{
    // Set this to true as long as the loop will run
    $verified = (bool) $encodedCerts;

    // The certs should be tested in reverse order
    foreach (array_reverse($encodedCerts) as $certText)
    {
        $cert = $x509->loadX509($certText);
        $ok = $x509->validateSignature();
        if ($ok)
        {
            $x509->loadCA($cert);
        }
        $verified = $verified && $ok;
    }

    return $verified;
}

I tried verifying them in forward order, but the first one failed. I thus reversed the order, and they all succeeded. I have no idea whether certs are provided in chain order, so a very solid approach would be to loop through with two nested loops, adding any valid certs as a CA, and then continuing on the outer loop. This can be done until all certs in the list are confirmed as having a validated signature.

halfer
  • 18,701
  • 13
  • 79
  • 158
  • TLS gives you the certificate chain with [target certificate first](https://tools.ietf.org/html/rfc5246#section-7.4.2). Chain validation however must be performed in reverse order, so that [root CA is validated first](https://tools.ietf.org/html/rfc5280#section-6.1). This is because root CA contains the public key used to verify intermediate certificate, which in turn contains public key used to verify target certificate. – Joe Mar 05 '18 at 11:30
  • Thanks for confirming, @Joe. I had wondered if it was feasible for three or more certs to be delivered in a random order, e.g. with the first to verify in the middle. If this should not happen, it simplifies the chain verification code I have to write on my side. – halfer Mar 05 '18 at 11:39
  • Specification states that _"Each following certificate MUST directly certify the one preceding it"_, so you can expect the chain to always be in order. One thing to consider is that the root certificate may or may not be in the chain. In my experience it's generally omitted. But in either case you must check that the intermediate is signed by a CA you trust. – Joe Mar 05 '18 at 11:55
  • Thanks @Joe. Yes, my code does that - the certs I used were from the Firefox bundle, and then the only ones from the server I have treated as CA certs are ones that can be verified by the bundle. If an intermediate does not verify, it is not treated as a CA cert, and so anything relying on that (including the server cert itself) will fail too. – halfer Mar 05 '18 at 12:19
  • You forgot to validate the dates of the certificates. – Pepijn Mar 09 '18 at 12:35
  • Good point @Pepijn, thanks. I haven't implemented this code yet, but I suspect that when I do, I'll ask one fetch to respect cert validation (so the date is checked for me) and then another fetch without the validation (so I can fetch the cert even if it does not validate). – halfer Mar 09 '18 at 12:38