Properly catch boto3 Errors

Question

I am developing a django app which communicates with several Amazon Web Services.

So far I am having trouble dealing with and catching exceptions thrown by the boto3 client. What I am doing seems unnecessarily tedious:

Example:

client = boto3.client('sns')
client.create_platform_endpoint(PlatformApplicationArn=SNS_APP_ARN, Token=token)

this might throw an botocore.errorfactory.InvalidParameterException if e.g. the token is bad.

client.get_endpoint_attributes(EndpointArn=endpoint_arn)

might throw an botocore.errorfactory.NotFoundException.

First, I can't find these Errors anywhere in code, so they are probably generated somewhere. Bottom line: I can't import it and catch it as usual.

Second, I found one way to catch the error here using:

try:
    # boto3 stuff
except botocore.exceptions.ClientError as e:
    if e.response['Error']['Code'] == 'NotFound':
        # handle exception
    else:
        raise e

But I have to remove the Exception part of the error name. Seems very random and I have no clue whether I would remove the Error in botocore.exceptions.ParamValidationError if I wanted to catch that one. So it's hard to generalize.

Another way to catch the error is using the boto3 client object I got:

try:
    # boto3 stuff
except client.exceptions.NotFoundException as e:
    # handle exception

This seems the cleanest way so far. But I don't always have the boto3 client object at hand where I want to catch the error. Also I am still only trying things out, so it's mostly guess work.

Does anybody know how boto3 errors are supposed to be handled?

Or can point me towards some coherent documentation which mentions the errors above? Thanks

https://github.com/boto/boto3/blob/develop/boto3/exceptions.py#L19 — hjpotter92, Sep 12 '17 at 11:03
This is the first place I looked, please read my first point again. The errors don't show up anywhere in code! — David Schumann, Sep 12 '17 at 11:19
There is existing discussion on very similar subject, may be useful: https://stackoverflow.com/questions/33068055/boto3-python-and-how-to-handle-errors — R. Simac, Apr 04 '18 at 16:23

ben author · Accepted Answer · 2018-04-02T21:16:58.760

14

You've summarized the situation well. The old boto had a simple hardcoded approach to supporting AWS APIs. boto3, in what appears to be an attempt to reduce the overhead of keeping Python client synced with evolving features on the various apis, has been more squishy around exceptions, so the ClientError approach you outlined above used to be the canonical way.

In 2017 they introduced the second mechanism you highlight: 'modeled' exceptions available on the client.

I am not familiar with SNS but in my experience with other AWS products, the ClientError naming matches up with the HTTP apis, which tend to be well documented. So I would start here: https://docs.aws.amazon.com/sns/latest/api/Welcome.html

It looks like the new-style modeled exceptions are generated from service definition files that live in botocore module. I can't find any documentation about it, but go browse around the AWS service models in https://github.com/boto/botocore/tree/master/botocore/data.

Also, it's good to know that if you are not (in contrast to OP's code) dealing directly with the low-level client, but instead are using a high-level AWS ServiceResource object, a low-level client is still easily available at my_service_resource.meta.client so you can handle exceptions like this:

try:
    my_service_resource.do_stuff()
except my_service_resource.meta.client.exceptions.NotFoundException as e:
    # handle exception

edited Apr 02 '18 at 21:16

answered Mar 23 '18 at 18:52

ben author

2,675
2
23
43

Thank you, that documentation link was exactly what I needed! So it seems that the first option is better, as it uses the documented exception names. `NotFound` exception is documented, `NotFoundException` is not. I guess this is part of the hardcoding / keeping in sync issue you mentioned? Because if not, I do not understand where that exception name originates from as it does not appear anywhere in the API docs. – David Schumann Mar 24 '18 at 21:08
1

I decided to go looking. Turns out there is a kind of service discovery mechanism going on, based on definitions stored in a declarative way within botocore. Check out https://github.com/boto/botocore/tree/master/botocore/data. I'll update the answer to reflect this discovery. – ben author Mar 26 '18 at 20:31
1

I don't have exceptions under meta, they are on the client directly so `my_server_resource.exceptions.NotFoundException` – Aaron McMillin Apr 02 '18 at 15:51
2

@AaronMcMillin boto exposes two kinds of clients. One is the low-level Client object, as the OP is using and as you too are using. You get one of these (let's use s3 as an example) by calling `boto3.client('s3')`. There is also a higher level 'Service Resource' object, which is also reasonable to call a client in the generic sense of the word. You'd get one like so: `s3 = boto3.resource('s3')`. Boto docs tend to point users towards the resource object for simple use cases, so I thought it was worth calling out the difference. – ben author Apr 02 '18 at 21:12
@benauthor do we need to check statuscode in response after do_stuff? like `if resp["ResponseMetadata"].get("HTTPStatusCode") != 200` if I am capturing response as well `resp = my_service_resource.do_stuff() ` – himanshu219 Nov 18 '18 at 04:39

score 4 · Answer 2 · answered Sep 23 '19 at 12:07

4

Use Boto3 exceptions: https://www.programcreek.com/python/example/97944/boto3.exceptions

client = boto3.client('logs')
try:
   client.create_log_group(logGroupName=LOG_GROUP)
except client.exceptions.ResourceAlreadyExistsException:
   pass

answered Sep 23 '19 at 12:07

sarath kumar

755
7
4

Properly catch boto3 Errors

2 Answers2