8

I have a Python 3.6 - Flask application deployed onto AWS Lambda using Zappa, in which I have an asynchronous task execution function defined using @Task as discussed here

However, I find that the function call still times out at 30 seconds as against the 5 minute timeout that AWS Lambda enforces for non-API calls. I even checked the timeout in my Lambda settings and it is set to 5 minutes.

The way I discovered this is when the lambda's debug output started repeating without a request - something that happens because the lamba is called 2 more times because of either an error or timeout (as per the AWS Lambda documentation).

Can anyone help me with getting this resolved?

[EDIT : The lambda function is also not part of any VPC and is set to be accessible from the internet.]

Here are the logs below. Basically, the countdown is a sleep timer counting to 20 seconds, followed by a @task call to application.reviv_assign_responder, but as we see, there is no outpust past 'NEAREST RESPONDER' and the countdown starts again, indicating that the function has timed out and has been called again by (AWS') design.

Log output in Pastebin : https://pastebin.com/VEbdCALg

Second incident - https://pastebin.com/ScNhbMcn

As we can see in the second log, it clearly states:

[1515842321866] wait_one_and_notify : 30 : 26 [1515842322867] wait_one_and_notify : 30 : 27 [1515842323868] wait_one_and_notify : 30 : 28 [1515842324865] 2018-01-13T11:18:44.865Z 72a8d34a-f853-11e7-ac2f-dd12a3d35bcb Task timed out after 30.03 seconds

kilokahn
  • 901
  • 12
  • 32

2 Answers2

5

You can check the default settings that Zappa applies to all your lambda functions here, and you will see that by default timeout_seconds is set-up to 30 seconds, This will apply over the default Lambda setup in AWS Console, because by default this is 3 seconds (you can check this limit in AWS Lambda FAQ.

For your @Task you must increase/setup your timeout_seconds in your zappa_settings.(json|yaml) file and redeploy this, You can put 5 mins (5*60==300 seconds) but this increase will be for all your functions defined in your virtualenv deployed with zappa.

You can check more details exposed in this issue in Zappa repo.

Yonsy Solis
  • 804
  • 9
  • 14
  • 1
    Hi, I tried this and it doesn't seem to work. My lambda hits a dead end at 30 seconds. This is most probably because the request is routed through API Gateway, which has a hard limit of 30, and this is inherited by the lambda. – kilokahn Jun 26 '18 at 20:29
  • 2
    { "dev": { "app_function": "application.application", "aws_region": "ap-southeast-1", "profile_name": "default", "project_name": "aws", "runtime": "python3.6", "s3_bucket": "zappa-xxxxxxx", "route53_enabled" : "false", "domain" : "alpha.reviv.in", "timeout_seconds": 90, "certificate_arn" : "arn:aws:acm:us-east-1:xxxx:certificate/xx-xx-xx-xx-xxxx" } } – kilokahn Jun 26 '18 at 20:32
  • @kilokahn are you calling your lambda function attached to an API Gateway ? if is not needed, you can decouple this. – Yonsy Solis Jun 27 '18 at 20:46
  • Hi, Unfortunately, I need this lambda to be triggered from within my API, based on event triggers (such as a specific API call) - zappa's @task allows me to do this from within my application (@task_sns has a bug in it that hasn't yet been fixed). I have intentionally set this to be so because I want to control who calls the lambda and when. – kilokahn Jun 28 '18 at 00:35
5

The timeout_seconds parameter in Zappa is misleading. That is, it does limit the timeout of the Lambda function, but the requests are served through CloudFront, which has a default timeout of 30 seconds. To verify that, try lowering the timeout_seconds to 20 - it will correctly timeout in 20 seconds. However past 30 there is no effect because of CloudFront limitation.

The default timeout is 30 seconds. You can change the value to be from 4 to 60 seconds. If you need a timeout value outside that range, request a change to the limit.

In other words, there is nothing you can do in either Zappa or Lambda to fix this, because the problem lies elsewhere (CloudFront).

I haven't tried it myself, but you might be able to up the limit by creating the cloudfront distribution in front of Lambda, though it seems you are still limited by max. 60s (unless you request more through AWS support, as indicated in the previous link).

johndodo
  • 13,994
  • 13
  • 81
  • 105
  • Thanks for the answer @Johndodo, but for those who don't know how (myself included) could you please also share the specifics (or links to resources) to show how to create the cloudfront distribution in front of lambda - I let zappa do all the heavy lifting for me and am clueless on how to do this. – kilokahn Nov 06 '19 at 08:29
  • I would love to help you, but I haven't gone that way myself. The point was more that you are not hitting a limitation of Lambda, but rather CloudFront, so it needs to be solved there (or, ideally, you should avoid such long response times - if you can). I have edited the answer to clarify that. – johndodo Nov 07 '19 at 17:06