How to get Python requests to trust a self signed SSL certificate?

Question

import requests
data = {'foo':'bar'}
url = 'https://foo.com/bar'
r = requests.post(url, data=data)

If the URL uses a self signed certificate, this fails with

requests.exceptions.SSLError: [Errno 1] _ssl.c:507: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

I know that I can pass False to the verify parameter, like this:

r = requests.post(url, data=data, verify=False)

However, what I would like to do is point requests to a copy of the public key on disk and tell it to trust that certificate.

http://stackoverflow.com/questions/10667960/python-requests-throwing-up-sslerror?rq=1 — Ajay, May 22 '15 at 21:08

score 72 · Accepted Answer · answered May 22 '15 at 21:12

72

try:

r = requests.post(url, data=data, verify='/path/to/public_key.pem')

answered May 22 '15 at 21:12

krock

26,870
12
71
83

1

Can you do the same and use client certificates at the same time? I'm getting problems with this. – user1156544 Jun 14 '19 at 14:46
14

Note that the .pem file you pass must include the server's certificate **and any intermediate certificates**. I lost a few hours trying to figure out why it didn't work after adding the server's cert. – ChrisBob Oct 15 '19 at 12:39
I added self signed certificate.pem, and it worked. – H S Rathore Nov 12 '19 at 04:38
10

This technique didn't work for me. I used `ssl.get_server_certificate` to download a certificate for `(self-signed.badssl.com, 443)`, saved that certificate to `cert.pem`, and then ran `requests.get('https://self-signed.badssl.com/', verify='cert.pem')` and it still failed with an SSL error (that certificate is self-signed). – Jason R. Coombs Nov 21 '19 at 03:31
@ChrisBob I can't thank you enough. Your comment is much more valuable than the accepted answers to the many questions about this (which only repeat what is in requests documentation). After pulling my hair for hours, your comment put me on the right direction... – Gaëtan de Menten Jan 15 '21 at 09:19
2

For reference (possibly for my future self), I had to download the certicicate as a .pem file by clicking on the lock icon in Firefox > Show Connection details > More information > View certificate > Download **"PEM (chain)"**. The (chain) is the important part that I was missing as the alternative "PEM (cert)" will **not** work with requests. – Gaëtan de Menten Jan 15 '21 at 09:28

Dr. Jan-Philip Gehrcke · Answer 2 · 2020-03-09T12:03:05.743

44

With the verify parameter you can provide a custom certificate authority bundle

requests.get(url, verify=path_to_bundle_file)

From the docs:

You can pass verify the path to a CA_BUNDLE file with certificates of trusted CAs. This list of trusted CAs can also be specified through the REQUESTS_CA_BUNDLE environment variable.

edited Mar 09 '20 at 12:03

answered May 22 '15 at 21:14

Dr. Jan-Philip Gehrcke

26,261
10
71
115

score 28 · Answer 3 · answered May 12 '17 at 10:20

28

The easiest is to export the variable REQUESTS_CA_BUNDLE that points to your private certificate authority, or a specific certificate bundle. On the command line you can do that as follows:

export REQUESTS_CA_BUNDLE=/path/to/your/certificate.pem
python script.py

If you have your certificate authority and you don't want to type the export each time you can add the REQUESTS_CA_BUNDLE to your ~/.bash_profile as follows:

echo "export REQUESTS_CA_BUNDLE=/path/to/your/certificate.pem" >> ~/.bash_profile ; source ~/.bash_profile

answered May 12 '17 at 10:20

Mike N

4,877
3
21
19

The environment variable was what I needed to get PyCharm to work with the certificates stored in the OpenSSL cert file. – Brady Aug 30 '19 at 12:04
I have a self-signed certificate in the chain. This solution solved my problem with boto3 library. – Ilkin Oct 12 '20 at 11:53

score 7 · Answer 4 · answered Jun 08 '17 at 15:55

7

Case where multiple certificates are needed was solved as follows: Concatenate the multiple root pem files, myCert-A-Root.pem and myCert-B-Root.pem, to a file. Then set the requests REQUESTS_CA_BUNDLE var to that file in my ./.bash_profile.

$ cp myCert-A-Root.pem ca_roots.pem
$ cat myCert-B-Root.pem >> ca_roots.pem
$ echo "export REQUESTS_CA_BUNDLE=~/PATH_TO/CA_CHAIN/ca_roots.pem" >> ~/.bash_profile ; source ~/.bash_profile

answered Jun 08 '17 at 15:55

Halbert Stone

71
1
2

That was my "ahhh" moment of the day... Thanks a lot... With this hint I got my self signed jira certificate to work... ;-) I know there are maybe hundrets of sites and answers who describe this, but I found yours, so you get my credit for helping me solve my probelm... d – alexrjs Mar 13 '19 at 09:04

score 6 · Answer 5 · answered Sep 21 '17 at 07:21

6

Setting export SSL_CERT_FILE=/path/file.crt should do the job.

answered Sep 21 '17 at 07:21

gizzmole

1,019
12
22

Thanks. Works for me (Whereas `REQUESTS_CA_BUNDLE` variable has no effect in my case). – Pascal H. Dec 03 '19 at 11:46
where to run this command? – Jing He May 16 '21 at 12:08
in your terminal – gizzmole May 17 '21 at 18:13

score 0 · Answer 6 · answered Jun 29 '17 at 02:39

Incase anyone happens to land here (like I did) looking to add a CA (in my case Charles Proxy) for httplib2, it looks like you can append it to the cacerts.txt file included with the python package.

For example:

cat ~/Desktop/charles-ssl-proxying-certificate.pem >> /usr/local/google-cloud-sdk/lib/third_party/httplib2/cacerts.txt

The environment variables referenced in other solutions appear to be requests-specific and were not picked up by httplib2 in my testing.

score 0 · Answer 7 · edited Sep 29 '17 at 15:32

0

You may try:

settings = s.merge_environment_settings(prepped.url, None, None, None, None)

You can read more here: http://docs.python-requests.org/en/master/user/advanced/

edited Sep 29 '17 at 15:32

Peter Tretyakov

3,094
6
33
46

answered Sep 29 '17 at 14:40

gan

11
1

Oh yeah that old readthedocs maze link as of today : https://requests.readthedocs.io/en/latest/user/advanced/ – Boop Sep 10 '20 at 07:57

How to get Python requests to trust a self signed SSL certificate?

7 Answers7

Linked