83

I'm increasingly hearing that Python's Twisted framework rocks and other frameworks pale in comparison.

Can anybody shed some light on this and possibly compare Twisted with other network programming frameworks.

Anton Gogolev
  • 107,051
  • 37
  • 191
  • 278
  • 2
    There doesn't seem to be anything [out there](http://wiki.python.org/moin/UsefulModules#Networking) that even remotely compares to Twisted, so tis question is a bit hard to answer. What alternatives are you considering? – Sven Marnach Mar 28 '11 at 12:02
  • 1
    @Sven Not even being a Python developer, I'm just curious as to what makes Twisted as great as it is declared to be. – Anton Gogolev Mar 28 '11 at 12:07
  • 2
    @Sven Marnach: It depends on a problem domain e.g., [`gevent`](http://www.gevent.org/) is an alternative to Twisted for the "fortune teller" application http://blip.tv/file/4883016 – jfs Mar 28 '11 at 12:40

2 Answers2

133

There are a lot of different aspects of Twisted that you might find cool.

Twisted includes lots and lots of protocol implementations, meaning that more likely than not there will be an API you can use to talk to some remote system (either client or server in most cases) - be it HTTP, FTP, SMTP, POP3, IMAP4, DNS, IRC, MSN, OSCAR, XMPP/Jabber, telnet, SSH, SSL, NNTP, or one of the really obscure protocols like Finger, or ident, or one of the lower level protocol-building-protocols like DJB's netstrings, simple line-oriented protocols, or even one of Twisted's custom protocols like Perspective Broker (PB) or Asynchronous Messaging Protocol (AMP).

Another cool thing about Twisted is that on top of these low-level protocol implementations, you'll often find an abstraction that's somewhat easier to use. For example, when writing an HTTP server, Twisted Web provides a "Resource" abstraction which lets you construct URL hierarchies out of Python objects to define how requests will be responded to.

All of this is tied together with cooperating APIs, mainly due to the fact that none of this functionality is implemented by blocking on the network, so you don't need to start a thread for every operation you want to do. This contributes to the scalability that people often attribute to Twisted (although it is the kind of scalability that only involves a single computer, not the kind of scalability that lets your application grow to use a whole cluster of hosts) because Twisted can handle thousands of connections in a single thread, which tends to work better than having thousands of threads, each for a single connection.

Avoiding threading is also beneficial for testing and debugging (and hence reliability in general). Since there is no pre-emptive context switching in a typical Twisted-based program, you generally don't need to worry about locking. Race conditions that depend on the order of different network events happening can easily be unit tested by simulating those network events (whereas simulating a context switch isn't a feature provided by most (any?) threading libraries).

Twisted is also really, really concerned with quality. So you'll rarely find regressions in a Twisted release, and most of the APIs just work, even if you aren't using them in the common way (because we try to test all the ways you might use them, not just the common way). This is particularly true for all of the code added to Twisted (or modified) in the last 3 or 4 years, since 100% line coverage has been a minimum testing requirement since then.

Another often overlooked strength of Twisted is its ten years of figuring out different platform quirks. There are lots of undocumented socket errors on different platforms and it's really hard to learn that they even exist, let alone handle them. Twisted has gradually covered more and more of these, and it's pretty good about it at this point. Younger projects don't have this experience, so they miss obscure failure modes that will probably only happen to users of any project you release, not to you.

All that say, what I find coolest about Twisted is that it's a pretty boring library that lets me ignore a lot of really boring problems and just focus on the interesting and fun things. :)

Glyph
  • 29,525
  • 9
  • 79
  • 112
Jean-Paul Calderone
  • 45,622
  • 5
  • 85
  • 111
  • 1
    and looks poorly documented from the link you gave about Resource abstraction. But thanks for the great reply, upvoting. – vinipsmaker Mar 06 '14 at 02:03
  • One good point is, explaining about "undocumented socket errors". Nice explanation. Thanks. – Haranadh Mar 17 '17 at 08:42
  • I’ve been using Twisted and Django for a while now and I am getting to that point where I feel I can replace Django with Twisted. Sounds weird but I just need a Front-end application and I’d define all my protocols on Twisted and use some ORM to manage my database. However, I see twisted as the framework that is not trying to box you into being able to do just one thing. – Durodola Opemipo Jul 12 '19 at 18:04
9

Well it's probably according to taste.

Twisted allows you to easily create event driven network servers/clients, without really worrying about everything that goes into accomplishing this. And thanks to the MIT License, Twisted can be used almost anywhere. But I haven't done any benchmarking so I have no idea how it scales, but I'm guessing quite good.

Another plus would be the Twisted Projects, with which you can quickly see how to implement most of the server/services that you would want to.

Twisted also has some great documentation, when I started with it a couple of weeks ago I was able to quickly get a working prototype.

Quite new to the python scene please correct me if i'm in the wrong.

Johann du Toit
  • 2,521
  • 2
  • 13
  • 29
  • 8
    One of the big advantages of starting with Twisted is that there are a whole host of scalability problems that simply never come up due to the way Twisted itself is designed. It lets you get on with discovering new and creative ways to shoot yourself in the foot, instead of first rediscovering all the mundane ways to do so when building a networked application :) – ncoghlan Mar 28 '11 at 12:44
  • Yes as a noobie myself with working with Twisted, that's what I quickly discovered. And I love how the architecture reminds me a lot of Netty [ http://www.jboss.org/netty ] which I used quite extensively so picking up the way of thinking was very fast. – Johann du Toit Mar 28 '11 at 12:50