Python Threading inside a class

Question

I recently started with python's threading module. After some trial and error I managed to get basic threading working using the following sample code given in most tutorials.

class SomeThread(threading.Thread):
    def __init__(self, count):
        threading.Thread.__init__(self)

    def run(self):
        print "Do something"

My problem is: I have a Class that has class variables and a function that I want to be run in a separate thread. However the function uses class variables and also writes to class variables. Like so:

class MyClass:
    somevar = 'someval'

    def func_to_be_threaded(self):
        # Uses other class functions
        # Do something with class variables

So how would I essentially 'put the thread class in MyClass'. So that if MyClass().func_to_threaded() is called it would run in a thread.

So each call to the class method `funct_to_be_threaded` should launch in a new thread? — aychedee, Nov 07 '13 at 20:56
Also, this is the point where you have to remember that some operations are not thread safe. For instance writing to a file is not thread safe. Printing to stdout is not thread safe. What this means is that unexpected things can happen. Like the contents of the file might be out of order or the output to stdout might be interspersed with other bits. — aychedee, Nov 07 '13 at 21:02
@aychedee Hi thank you for the useful information on thread safety. I was just wondering: I know that output to a file is not thread safe but how about reading? — Niels, Nov 07 '13 at 23:30
Reading from a file is fine as long as you aren't also writing to it from another thread. If you are then the output won't be deterministic. Moving a file is atomic. So some people will write to a file, then move it into place. Of course this risks overwriting any new content in the location. — aychedee, Nov 08 '13 at 11:32
The request to "run calls to `MyClass().func_to_threaded()` in its own thread" is -- generally -- the wrong way to think about threads... UNLESS you mean "run each call to `MyClass().func_to_threaded()` in its own thread EACH TIME". For example, you CAN'T *call* into a thread once it is started. You CAN pass input/output in various ways (globals, queues, sockets) if you design the thread to read from/write to those structures. Remember: you DON'T assign objects to threads; a single object might have concurrent threads of execution in MULTIPLE different threads. — Dan H, Mar 09 '16 at 14:43

freakish · Accepted Answer · 2016-10-17T13:08:42.413

If I understand correctly you want to run a function in a separate thread? There are several ways to do that. But basically you wrap your function like this:

class MyClass:
    somevar = 'someval'

    def _func_to_be_threaded(self):
        # main body

    def func_to_be_threaded(self):
        threading.Thread(target=self._func_to_be_threaded).start()

It can be shortened with a decorator:

def threaded(fn):
    def wrapper(*args, **kwargs):
        threading.Thread(target=fn, args=args, kwargs=kwargs).start()
    return wrapper

class MyClass:
    somevar = 'someval'

    @threaded
    def func_to_be_threaded(self):
        # main body

Edit Updated version with a handle:

def threaded(fn):
    def wrapper(*args, **kwargs):
        thread = threading.Thread(target=fn, args=args, kwargs=kwargs)
        thread.start()
        return thread
    return wrapper

class MyClass:
    somevar = 'someval'

    @threaded
    def func_to_be_threaded(self):
        print 'xyz'

This can be used as follows:

>>> my_obj = MyClass()
>>> handle = my_obj.func_to_be_threaded()
>>> handle.join()

Now it is possible to extend it even more if you wish to return a value from the function. Consider this:

from threading import Thread
from concurrent.futures import Future

def call_with_future(fn, future, args, kwargs):
    try:
        result = fn(*args, **kwargs)
        future.set_result(result)
    except Exception as exc:
        future.set_exception(exc)

def threaded(fn):
    def wrapper(*args, **kwargs):
        future = Future()
        Thread(target=call_with_future, args=(fn, future, args, kwargs)).start()
        return future
    return wrapper


class MyClass:
    @threaded
    def get_my_value(self):
        return 1

>>> my_obj = MyClass()
>>> fut = my_obj.get_my_value()  # this will run in a separate thread
>>> fut.result()  # will block until result is computed
1

If you don't have concurrent.futures.Future class (because for example you are using Python2.7 or older) then you can use this simplified implementation:

from threading import Event

class Future(object):
    def __init__(self):
        self._ev = Event()

    def set_result(self, result):
        self._result = result
        self._ev.set()

    def set_exception(self, exc):
        self._exc = exc
        self._ev.set()

    def result(self):
        self._ev.wait()
        if hasattr(self, '_exc'):
            raise self._exc
        return self._result

I advice reading through concurrent.futures module since it has a lot of neat tools. For example Thread class should be replaced with a ThreadPoolExecutor instance to limit concurrency (e.g. you don't want to spam 10k threads). Also with ThreadPoolExecutor the code is even simplier (and less error prone):

from concurrent.futures import ThreadPoolExecutor

tp = ThreadPoolExecutor(10)  # max 10 threads

def threaded(fn):
    def wrapper(*args, **kwargs):
        return tp.submit(fn, *args, **kwargs)  # returns Future object
    return wrapper

Just remember you have to tp.shutdown() after you're done with all parallel work.

@jeremyjjbrown You don't, I mean not with this code. There was no such requirement. — freakish, Mar 24 '15 at 07:43
Is there a version of the annotation that allows it? This would be very cool if you could. — jeremyjjbrown, Mar 27 '15 at 15:19
@jeremyjjbrown I'm very sorry for this extremely late response. Somehow I've missed your comment and I've just noticed it by going through my old posts. My bad. :( Anyway I'm giving the extended implementation. — freakish, Oct 17 '16 at 12:42

score 6 · Answer 2 · answered Nov 07 '13 at 20:57

You can pass class instance to the thread:

class SomeThread(threading.Thread):
    def __init__(self, count, instance):
        threading.Thread.__init__(self)
        self.instance = instance

    def run(self):
        print "Do something"
        self.instance.some_param = data
        self.instance.some_function()

score -7 · Answer 3 · answered Nov 07 '13 at 21:03

-7

I'm fairly certain that you can't make a single function threaded.

The whole class will be threaded (sort of). When you instantiate the object, its __init__ will be called on another thread, and then when you call start() on that object, its run() will be called once, on another thread.

So, if you have a TASK that needs to be on its own thread (disc IO, socket listening, etc), then you need a class to handle that task.

@ndpu's answer solves your scope/access problems.

answered Nov 07 '13 at 21:03

Matt

717
5
22

This answer is wrong on both points: You CAN put a single function in a thread; you CAN'T "put a class in a thread". Well, not unless a given thread has the only references to it. In other words, threading is NOT about "assigning objects" to "threads"... it is about having multiple [quasi-]simultaneous "threads of execution", possibly on the "same object at the same time". – Dan H Mar 08 '16 at 02:32

Python Threading inside a class

3 Answers3

Linked