2

I'm currently tinkering on what is effectively a chat server. Since I do not want to expose my users too much, I added TLS encryption to it using LibreSSL's fork of the OpenSSL library. The rest of the code appears to work fine, but I think I am not doing the certificates correctly.

I have a private/public certificate on the server which should be used not only to encrypt the communication, but also to ensure the server is really who the client wanted to talk to. And that's the part I can't figure out:

  1. How do I give the server's public key to the client? It needs it to verify that it's talking to the right server. Or should I be doing something else, maybe involving the root CA's certificate? Is there API to provide that? I can package the public key with the executable as a .pem, but I can't find the API to tell OpenSSL about a public server key to use for client requests, or the root CA.

  2. How do I get the system's certificate for the client? Right now I just created one in a .pem file, but I don't really want to have to build a new download with a unique certificate for every user downloading the client. Surely there's a way to get "the current user's certificate" or auto-generate one for this use via some OpenSSL API?

If anyone could point me at the right API to use, that'd be great! I'd also take clues, links to similar questions on SO, tutorials, pointers at books aimed at OpenSSL-crypto-beginners, answers or sample code.

Currently, I'm using both SSL_CTX_use_certificate_file() and SSL_CTX_use_PrivateKey_file() to set certificates in both the client and the server program. You can see the code in the above linked Github chat server repository, in eleven_session.cpp

uliwitness
  • 7,806
  • 31
  • 50
  • Your question is very broad and contains programming issues (how to set certificate to use) and design issues. I would suggest you look for the programming issues into the source code of s_server and s_client in OpenSSL. For the design issues you better ask at security.stackexchange.com. – Steffen Ullrich Nov 20 '14 at 10:16
  • @SteffenUllrich How is it broad? I'm asking which API calls I should be using to perform this task. The rest is just "making sure I'm not barking up the wrong tree". But thanks for the pointer, I'll have another look at s_server and s_client, maybe I'll understand them on a second read, now that I've done more with LibreSSL. OK, I rewrote one paragraph a bit, is this clearer? – uliwitness Nov 20 '14 at 10:19
  • 1
    "My security appears to work fine." – Kerrek SB Nov 20 '14 at 10:20
  • @KerrekSB Yeah, it's intentionally phrased that way. :-D I know it's not right, but it runs and the *rest* of the code does what it did before I added TLS. :-) – uliwitness Nov 20 '14 at 10:22
  • @KerrekSB There, is it better now? :-) – uliwitness Nov 20 '14 at 10:25
  • @uliwitness: No no, I liked it -- it told me precisely everything I needed to know :-) – Kerrek SB Nov 20 '14 at 10:26
  • 1
    @uliwitness: question#2 about the deployment is less a programming question but more a question about how to design the system. A better place for this would be security.stackexchange.com. And I felt your question is too broad because it is not restricted to a single specific programming question but includes this deployment problem, which should be a separate question. – Steffen Ullrich Nov 20 '14 at 10:33
  • Is the current phrasing better? I wanted to avoid certain terms that have a different meaning in a security context. I guess I managed to overshoot and pick the word "deployment" that invites more misunderstanding. – uliwitness Nov 20 '14 at 11:28
  • @SteffenUllrich Looks like what I want to do is what you mention in this answer on security.so: http://security.stackexchange.com/a/58720 but you don't mention how I'd actually match the private certificate against the server's. Any pointers? – uliwitness Nov 20 '14 at 20:50
  • @uliwitness: your wording is unclear, but I guess you want to roll out your own CA, ship each client with its own client certificate and verify these when the client connects to the server? And your question is how to verify these client certificates inside the server? This works the same as verifying the server cert at the client, e.g. specify the CA it should use to verify the certificate and set the verification mode to verify the peer. – Steffen Ullrich Nov 20 '14 at 21:04
  • @SteffenUllrich Oh no, I mis-typed. I want to match the *public* certificate against the server's. I.e. I want to ship clients that can verify that the server they're talking to is really my server, by looking at its certificate and comparing it to e.g. a copy of the server's public certificate that I include with each client. – uliwitness Nov 20 '14 at 21:25
  • 1
    @uliwitness: I think you need to look at SSL_get_peer_certificate then and use X509_cmp to compare against the stored version. Or you have access to the certificate in the verify callback. I really recommend to dig through the source code because the documentation of OpenSSL (and LibreSSL probably too) is sparse and sometimes wrong. – Steffen Ullrich Nov 20 '14 at 21:48
  • @SteffenUllrich Thanks, `SSL_get_peer_certificate` and `X509_cmp` were the missing links. This would all be much easier if 90% of the Google results weren't for the command line tool, or if "C" had a more unique name :-) – uliwitness Nov 21 '14 at 08:36

2 Answers2

1

1: So, fist of all, you don't need to give the server's public key to the client, because the OpenSSL API will do it for you. You create a socket using the regular socket function and then transfer it to OpenSSL, and this socket is the identification for every client that connects to the server. Then you use this SSL Socket for example in the socket write function SSL_write to send encrypted text to the client.

2: When I started i started using encrypted sockets, I also had a lot of problems finding the right way to create a working .pemfile, so here's my try how I've done it. You have to use linux terminal and you have to install OpenSSL using sudo apt-get install openssl:

openssl genrsa -des3 -out self-ssl.key 2048
openssl req -new -key self-ssl.key -out self-ssl.csr
cp -v self-ssl.{key,original}
openssl rsa -in self-ssl.original -out self-ssl.key
rm -v self-ssl.original
openssl x509 -req -days 3650 -in self-ssl.csr -signkey self-ssl.key -out self-ssl.crt
cat self-ssl.crt self-ssl.key > server.pem

This file is then used like you've already found out with these 2 functions: SSL_CTX_use_certificate_file and SSL_CTX_use_PrivateKey_file

Try it with that .pemfile, if it still doesn't work, write it into comments.

Hope this will help you.

schacker22
  • 399
  • 2
  • 4
  • 12
  • Re 1: So how does OpenSSL in the client know it is talking to *my* server, and not to any other server that has a valid SSL certificate if I don't give it a certificate for my server? – uliwitness Nov 20 '14 at 20:26
  • Re 2: I have a valid certificate for the server, that's not the problem. What wasn't clear to me was that I do not need to give the client a certificate. I've removed that code and it all works fine, so really all I need is a solution for #1, verifying that I'm talking to the *right* server. – uliwitness Nov 20 '14 at 20:31
  • It seems what I want to do is what @SteffenUllrich mentions here: http://security.stackexchange.com/a/58720 but sadly he doesn't mention how to fingerprint a certificate & match against the server. – uliwitness Nov 20 '14 at 20:52
  • @uliwitness Re 1: I know that you've already made it, but anyway I want to explain it: On the server you `bind` a ip address, normally the localhost. And then you connect to this server from the client with the ip address of the server. So you can't talk to any other server, because you decide to witch server you connect with the ip address. By the way, the client connects to the server with the `connect` function – schacker22 Nov 21 '14 at 10:00
  • But I *can* talk to another server. Anyone in control of the router my computer is on (e.g. admin of the hotel wifi) can re-direct an IP address to another machine. I wouldn't even know that I'm talking to the wrong computer. That's why I *must* check the certificate. The connection to that other server may be encrypted, too, but the receiver is not who I think it is. – uliwitness Nov 21 '14 at 11:00
  • @uliwitness What I would do is, send a specific command to the server after connecting. And I think you have access to your server, so you you test this command, and if it's right, then he responds to the client. If there is no response from the server with the right text, you can be sure that you're connected to the wrong server. – schacker22 Nov 21 '14 at 13:51
  • Checking the certificate is much more secure. That's what certificates are for. Having such a passphrase is just security by obscurity. A determined attacker can figure that out by reverse-engineering a copy of the client. But they can't make a server correctly authorize with my server's private key. – uliwitness Nov 21 '14 at 14:42
1

Thanks to SteffenUllrich and schacker22 for helping me with this issue. Here's the solution I cobbled together with their guidance:

After connecting, the client uses BIO_new(BIO_s_file()), BIO_read_filename() and PEM_read_bio_X509_AUX() to load the public certificate of the server from a PEM file, then compares it to the certificate returned by SSL_get_peer_certificate() using X509_cmp(). If that doesn't give 0, the client aborts the connection because it is not talking to the correct server.

Only the server gets a certificate, which I set using SSL_CTX_use_certificate_file() (public) and SSL_CTX_use_PrivateKey_file() to point to the same PEM file containing both the public and private key.

I decided not to use the verify callback, as it would replace the existing certificate verification, so I'd have to do that part as well. The load_cert() function in LibreSSL's apps.c file was a great help in figuring out how to load the certificate.

If you're interested in seeing the final source code, it's publicly available from my Git repository for the eleven chat server and client.

This is my first time working with SSL sockets, so if you spot anything wrong here, please let me know. The whole point of this exercise is to learn how to do it right.

uliwitness
  • 7,806
  • 31
  • 50