3

I need to create a project that has a web frontend to manage synchronous task execution (ala fabric), async tasks (AMQP), and long-polling/ajax for tabular viewing of results and queues/large, frequently changing datasets (think tail -f syslog). I have an existing Python codebase for a lot of the implementation-specific stuff.

After looking at a bunch of existing frameworks, the obvious answer appears to be Django+Celery. However, I do not want to "learn Django", nor do I need 95% of it's functionality. I just need simple auth, maybe sqlalchemy, easy ajax, amqp, xmlrpc would be helpful.

I would consider using Mongrel2, but I have a strong preference for RabbitMQ over 0MQ (for a few implementation-specific reasons).

I originally spent a great deal of time learning Twisted, and ended up getting a few hundred useful LOC out of it, but I found that I was twisting (lol) too much of my platform code to fit it's callback model. It actually 'fit the bill' very well (except with it's own amqp implementation), but it was so frustrating, and I went through so many iterations of code (one for each 'twisted ahah moment'), that it's 100% out.

Can somebody please help me wade through the mire? Tornado? Pylons? repoze? Pyramid? Flask? Bottle? CherryPy? Web2py? Paster/Webob? Anything else@ http://wiki.python.org/moin/WebFrameworks?

Edit:

To be clear, integration with RabbitMQ (or another amqp provider) is of the utmost importance, and is really the crux of problem.

mikewaters
  • 3,314
  • 3
  • 26
  • 21
  • Twisted is the most suitable of all the options you've mentioned. This kind of concurrent management is just what it's intended for. It seems strange that you disqualify it *because* you've learned how to use it. – Jean-Paul Calderone May 26 '11 at 20:18
  • I was hoping you'd miss this question, JP ;) Your tutorials have been instrumental in my learning twisted, and improved my python chops considerably. I just got terribly frustrated, and by the time I figured out @inlineCallbacks/yield, I had been writing code for weeks that had to be tossed. I just closed Eclipse and went drinking. Also, the whole divmod issue scared me, as far as the longevity of twisted is concerned; I don't need to use the latest-and-greatest [g]event[let] async lib, but do I need to be sure that my significant time investment won't be wasted. What do you think? – mikewaters May 26 '11 at 20:38

3 Answers3

2

I don't have a full vision of python web frameworks but just want to share my point of view on 2 of them :

  • Bottle is light and works fine. If you want something easy to learn and easy to use that may be the right choice. I used it for quite simple front-end apps running locally and i liked it very much.

  • Tornado seems to me as a very good non-blocking server for real-time web app. Combined with tornadio it makes ajax-long-polling quite easy. However, it may be a little harder to learn than Bottle. I would recommend to have a look to the chat app in the example folder of tornadio.

I hope it helps

luc
  • 37,543
  • 21
  • 117
  • 168
1

If you are going to use AMQP long term then I would steer clear of Celery because they use AMQP in a wierd way that suggests the developers did not understand the AMQP model.

bottle is a nice framework for knocking together RESTful apps (I use it to create mock servers for testing) and if you already have the code that does the real work, you may be surprised at how short a bottle app can be.

I'm currently building Python apps using RabbitMQ and using amqplib by way of kombu. I originally chose kombu in case I wanted to swap libraries and use pika or something else, but now I wish that I had just gone with amqplib and built a proper Pythonic AMQP model on top of that.

Do spend some time on the RabbitMQ site reading some of the blogs and slide presentations on AMQP before you get too deep into coding or you won't really understand the AMQP model and will make things harder for yourself.

Please don't use xmlrpc unless you have to talk to other apps. Bottle makes simple RESTful apps so simple, that XMLRPC is just uneccessary complexity.

Michael Dillon
  • 30,332
  • 5
  • 65
  • 99
-1

A couple of suggestions.

CherryPy is a great low level framework. It doesn't provide a lot of functionality, but it provide a very easy system for mapping http requests to function calls.

web.py is another extremely lightweight and easy to use framework. It is more comprehensive than CherryPy, including templates and other features.

Plain wsgi is not a bad choice if your needs are extremely simple. It is a little more complicated to do simple stuff than CherryPy or Web.py. WSGI is the lowest common denominator, these days most web frameworks are built on top of it.

mikerobi
  • 18,957
  • 5
  • 43
  • 42