GAE app raising DeadlineExceededError locally and DownloadError deployed

Question

This is very weird. The title says most of it, my code should say the rest. Here's my main.py file:

from google.appengine.api import urlfetch
import webapp2
import jinja2

import json
import os

jinja_environment = jinja2.Environment(
    loader=jinja2.FileSystemLoader(os.path.dirname(__file__)))


class MainPage(webapp2.RequestHandler):
    def get(self):
        response = urlfetch.fetch("http://localhost:8080/api/helloworld?name=totty", method=urlfetch.GET)
        if response.status_code == 200:
            result = json.loads(response.content)

        template_values = {'response': result['msg']}
        template = jinja_environment.get_template('index.html')
        self.response.out.write(template.render(template_values))


app = webapp2.WSGIApplication(
    [('/', MainPage)],
    debug=True)

Here's my api.py file:

import webapp2
import json

class HelloWorld(webapp2.RequestHandler):
    def get(self):
        name = self.request.get('name') or 'world'
        msg = "Hello {}!".format(name)
        payload = json.dumps({'msg': msg})
        # payload = json.dumps({'dir': str(dir(self.request)), 'body': str(self.request.body), 'name': str(self.request.arguments())})

        self.response.headers['Content-Type'] = 'application/json'
        self.response.write(payload)

app = webapp2.WSGIApplication(
    [('/api/helloworld', HelloWorld)],
    debug=True)

And in case my app.yaml file would help:

application: stacksort
version: 1
runtime: python27
api_version: 1
threadsafe: true

handlers:
- url: /api/.*
  script: api.app
- url: /.*
  script: main.app

libraries:
- name: webapp2
  version: latest
- name: jinja2
  version: latest

Nothing changes even if I add deadline=30 to my urlfetch call. I tested the API using httpie and JQuery and it works perfectly fine and returns in under 5 seconds.

I looked at the other questions, but I'm still stumbling into the dark. Help, tips or refactoring would be appreciated.

I plan to add calls to the StackEchange Search API, so I suspect the problem might come in that time also. If there's a better way to do this, please tell. Thanks.

score 1 · Answer 1 · answered Apr 05 '13 at 07:24

1

You are trying to fetch a URL to and from your application which is strongly discouraged on Google App Engine.

Locally you won't be able to call the development server because it serves only one request at a time. Multi-threading is not observed.

Note: the new experimental development server is now able to handle multiple requests at a time.

Multithreaded serving for better performance for complex applications and more correct semantics e.g. accessing your own application through urlfetch no longer deadlocks.

On production environment, GAE prevents the fetch service to call the same application.

To prevent an app from causing an endless recursion of requests, a request handler is not allowed to fetch its own URL. It is still possible to cause an endless recursion with other means, so exercise caution if your app can be made to fetch requests for URLs supplied by the user.

answered Apr 05 '13 at 07:24

Tony Baguette

585
5
13

What do you suggest I do then? – Yatharth Agarwal Apr 05 '13 at 07:26
I'm not sure what you are trying to achieve. I think you should separate the API request handler and its implementation counterparts. Then you can call the api implementation directly.By the way, you won't have any problem fetching the StackExchange API. – Tony Baguette Apr 05 '13 at 07:37
I just want one part of the code to use the SE API and process it and then pass it on to some other interface part. I want to separate the UI from the processing as I want to have an AJAX-like interface. – Yatharth Agarwal Apr 05 '13 at 08:28
I didn't get your comment about separating the request handler from the implementation. Are you talking about having one Handler do everything and use helper functions? Or have an abstract handler class and inherit fomr it? – Yatharth Agarwal Apr 05 '13 at 08:29
If you want to invoke SE API then it is the good approach and it should work. – Tony Baguette Apr 05 '13 at 11:11

score 0 · Answer 2 · answered Apr 05 '13 at 07:07

While upgrading the SDK, I noticed this addition to the DevServer page:

Note: dev_appserver.py can only serve one request at a time. If your application makes URL fetch requests to itself while processing a request, these requests will fail when using the development web server. (They will not fail when running on App Engine.) To test such requests, you can run a second instance of dev_appserver.py on a different port, then code your application to use the other server when making requests to itself.

So I guess that solves my problem (or at least gives a satisfactory explanation for it).

GAE app raising DeadlineExceededError locally and DownloadError deployed

2 Answers2