Combining trio and flask

Question

I'm trying to make an HTTP API that can create and destroy concurrent tasks that open TCP connections to remote servers streaming ~15-second data. I'll have to figure out how to handle the data later. For now I just print it.

In the example below, I can create multiple TCP connections by navigating to http://192.168.1.1:5000/addconnection.

Questions:

1) Is this approach reasonable? I think Flask may be creating a new thread for each /addconnection request. I'm not sure what performance limits I'll hit doing that.

2) Is it possible to keep track of each connection? I'd like to implement /listconnections and /removeconnections.

3) Is there a more Pythonic way to do this? I've read a little about Celery, but I don't really understand it very well yet. Perhaps there are other already existing tools for handling similar problems.

import trio
from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello World!"


@app.route("/addconnection")
def addconnection():

    async def receiver(client_stream):
        print("Receiver: started!")
        while True:
            data = await client_stream.receive_some(16800)
            print("Received Data: {}".format(data))

    async def parent():
        async with trio.open_nursery() as nursery:
            client_stream = await trio.open_tcp_stream('192.168.1.1', 1234)
            nursery.start_soon(receiver, client_stream)

    trio.run(parent)

pgjones · Accepted Answer · 2019-03-08T19:58:42.383

1) You will create a new event loop for each /addconnection request which will block the Flask runtime. This will likely limit you to a single request per thread.

2) Yes, in the simplest case you can store them in a global set, see connections below.

3) I'm the author of Quart-Trio, which I think is a better way. Quart is the Flask API re-implemented with async/await (which solves most of 1)). Quart-Trio is an extension to use Trio rather than asyncio for Quart.

Roughly (and I've not tested this) your code becomes,

import trio
from quart_trio import QuartTrio

connections = set()

app = QuartTrio(__name__)

@app.route("/")
async def hello():
    return "Hello World!"


@app.route("/addconnection")
async def addconnection():

    async def receiver(client_stream):
        print("Receiver: started!")
        while True:
            data = await client_stream.receive_some(16800)
            print("Received Data: {}".format(data))

    async def parent():
        async with trio.open_nursery() as nursery:
            client_stream = await trio.open_tcp_stream('192.168.1.1', 1234)
            connections.add(client_stream)
            nursery.start_soon(receiver, client_stream)
        connections.remove(client_stream)

    app.nursery.start_soon(parent)
    return "Connection Created"

if __name__ == "__main__":
    # Allows this to run and serve via python script.py
    # For production use `hypercorn -k trio script:app`
    app.run()

When using quart_trio, how do I go about reading data from a post request? — hankivstmb, Mar 14 '19 at 22:07
Same as in Quart, see https://pgjones.gitlab.io/quart/request_body.html — pgjones, Mar 15 '19 at 23:04
Thanks, I was using: `from quart_trio import QuartTrio, request` and I should have been using `from quart_trio import QuartTrio; from quart import request`. The former was giving me an error. — hankivstmb, Mar 18 '19 at 16:41

Combining trio and flask

1 Answers1