Google App Engine Task Deadline

Question

is there a way to handle any soft deadline while executing a task ? The DeadlineExceededError is thrown after 10 minutes of execution and I am given few seconds after that to do some stuff. I want to clean up somethings before the task dies and create a new task. This may take few seconds. Is there a way to do this by capturing any exception like around 9 mins. I know I can manually throw an exception after 9 mins. But can this be done automatically by GAE ?

class FillMtxHandler():

def post(self,index,user,seqlen):

    try :           
        FillMtx(index,user,seqlen)

    except DeadlineExceededError:

        deferred.defer(self.post,index,user,seqlen)

The above is my code. index is a list and starts from 0. It will be incremented inside FillMtx. Once a deadline exceeded error is thrown, i want to continue from where index was incremented last. I get the following error

   The API call taskqueue.BulkAdd() was explicitly cancelled.
Traceback (most recent call last):
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 517, in __call__
    handler.post(*groups)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 258, in post
    run(self.request.body)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 124, in run
    return func(*args, **kwds)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 166, in invoke_member
    return getattr(obj, membername)(*args, **kwargs)
  File "/base/data/home/apps/0xxopdp/3.347813391084738922/fillmtx.py", line 204, in post
    deferred.defer(self.post,index,user,seqlen)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 241, in defer
    return task.add(queue, transactional=transactional)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/taskqueue/taskqueue.py", line 688, in add
    return Queue(queue_name).add(self, transactional=transactional)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/taskqueue/taskqueue.py", line 744, in add
    self.__AddTasks(tasks, transactional)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/taskqueue/taskqueue.py", line 770, in __AddTasks
    apiproxy_stub_map.MakeSyncCall('taskqueue', 'BulkAdd', request, response)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 86, in MakeSyncCall
    return stubmap.MakeSyncCall(service, call, request, response)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 286, in MakeSyncCall
    rpc.CheckSuccess()
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_rpc.py", line 126, in CheckSuccess
    raise self.exception
CancelledError: The API call taskqueue.BulkAdd() was explicitly cancelled.

I find that a new task has been created and queued up. But why does GAE throw this error still?

score 4 · Answer 1 · edited Apr 26 '16 at 07:42

4

You don't have to raise an exception after 9 minutes; when the soft deadline exception raises, you have enough time to add a clean-up task to the Task Queue via deferred.

from google.appengine.ext import deferred
...
try:
  # Do your stuff
except DeadlineExceededError:
  deferred.defer(do_your_cleanup, ..)

In this way you have 10 minutes to do any clean-up stuff you need in your app.

edited Apr 26 '16 at 07:42

Cyril N.

35,568
34
119
213

answered Jan 22 '11 at 10:54

systempuntoout

65,753
43
162
239

how can i be sure that the task will be added to the task queue before the hard limit is hit ? i get the "CancelledError: The API call taskqueue.BulkAdd() was explicitly cancelled" even though the task has been added – Sam Jan 22 '11 at 15:00
If you run out of grace time the CancelledError kicks in. – systempuntoout Jan 22 '11 at 17:37
deferred.defer and taskqueue.add behave the same way here. In both cases, i get the "Cancelled Error" because the process of adding the task to the queue was not completed before the hard time limit. http://code.google.com/appengine/articles/deferred.html how is he able to update the db and create a task before the grace period ? – Sam Jan 23 '11 at 01:40
I have modified the question. i have added my code and error log – Sam Jan 23 '11 at 10:04

score 4 · Accepted Answer · answered Jan 24 '11 at 00:48

4

You can't control when you get the soft deadline exceeded error. Instead, you should use your own timer (take the wallclock time when you start, and compare it to the current time on each trip around your main loop), and abort when you're near enough the limit that you want to stop.

answered Jan 24 '11 at 00:48

Nick Johnson

98,961
16
125
196

Thanks for the reply. In your post about deferred, you add a new task after catching a DeadlineExceededError. Can we be sure that the task will be definitely added to the queue before the hard deadline error ? – Sam Jan 24 '11 at 02:04
@Sam It should generally be added, but it's possible that it won't be, as you've observed. If it doesn't, though, your task will be requeued (since it didn't exit cleanly); your code can then check if it's already done the work, and skip straight to the cleanup phase if it did. – Nick Johnson Jan 24 '11 at 02:15

Google App Engine Task Deadline

2 Answers2