A clean API for making a function call threaded in python

Question

I would like to call a function in a thread. Calling it with the conventional API looks like:

from threading import Thread
import numpy as np
a = np.random.rand(int(1e8),1)

Thread(target=np.savez_compressed, args=('/tmp/values.a', dict(a=a))).start()

I was wondering if there is a pythonic was of making this threaded call with a cleaner API, without defining a function which is specific for np.savez_compressed.

E.g. something in the style of (pseudo-code):

@make_threaded
np.savez_compressed('/tmp/values.a', dict(a=a))

Unfortunately decorators can only be applied to function definitions, so the pseudo-code above is not legal.

EDIT: I am not looking specifically for a decorator API. Rather, a cleaner way to make a function call threaded

Decorators don't work like that. They go on top of functions or classes - not function calls — rdas, Mar 01 '20 at 10:01
I’m sure that’s possible. Have you tried anything yet? Have you ever written a decorator yourself? — quamrana, Mar 01 '20 at 10:03
I am not looking specifically for a decorator API. Rather, a cleaner way to make a function call threaded — Yuval Atzmon, Mar 01 '20 at 10:05
@quamrana I looked on decorators and context-managers but I could not find anything that is satisfying. — Yuval Atzmon, Mar 01 '20 at 10:07
What do you mean with "clean"? Why isn't the first snippet "clean"? — BlackBear, Mar 01 '20 at 10:16
Are you aware of [concurrent.futures](https://docs.python.org/3/library/concurrent.futures.html)? — MisterMiyagi, Mar 01 '20 at 11:10

MisterMiyagi · Accepted Answer · 2020-03-01T11:26:37.250

The concurrent.futures module provides a more high-level API for using threads or processes for individual operations.

from concurrent.futures import ThreadPoolExecutor

executor = ThreadPoolExecutor()
executor.submit(np.savez_compressed, '/tmp/values.a', dict(a=a))

If you don't want the entire Executor API, you can define your own helper to run a function in a thread.

def threaded(call, *args, **kwargs):
    """Execute ``call(*args, **kwargs)`` in a thread"""
    thread = threading.Thread(target=call, args=args, kwargs=kwargs)
    thread.start()
    return thread

threaded(np.savez_compressed, '/tmp/values.a', dict(a=a))

score 0 · Answer 2 · answered Mar 02 '20 at 12:05

OP here:

Another solution I found was by using decorators with the decorators "classic" API:

from threading import Thread

call_threaded(np.savez_compressed)('/tmp/values.a', dict(a=a))

# https://stackoverflow.com/a/19846691/2476373
def call_threaded(fn):        
    def wrapper(*args, **kwargs):
        thread = Thread(target=fn, args=args, kwargs=kwargs)
        thread.start()
        return thread
    return wrapper

A clean API for making a function call threaded in python

2 Answers2