I have an application which polls a bunch of servers every few minutes. To do this, it spawns one thread per server to poll (15 servers) and writes back the data to an object:
import requests
class ServerResults(object):
def __init__(self):
self.results = []
def add_server(some_argument):
self.results.append(some_argument)
servers = ['1.1.1.1', '1.1.1.2']
results = ServerResults()
for s in servers:
t = CallThreads(poll_server, s, results)
t.daemon = True
t.start()
def poll_server(server, results):
response = requests.get(server, timeout=10)
results.add_server(response.status_code);
The CallThreads
class is a helper function to call a function (in this case poll_server()
with arguments (in this case s
and results
), you can see the source at my Github repo of Python utility functions. Most of the time this works fine, however sometimes a thread intermittently hangs. I'm not sure why, since I am using a timeout on the GET request. In any case, if the thread hangs then the hung threads build up over the course of hours or days, and then Python crashes:
File "/usr/lib/python2.7/threading.py", line 495, in start
_start_new_thread(self.__bootstrap, ())
thread.error: can't start new thread
Exception in thread Thread-575 (most likely raised during interpreter shutdown)
Exception in thread Thread-1671 (most likely raised during interpreter shutdown)
Exception in thread Thread-831 (most likely raised during interpreter shutdown)
How might I deal with this? There seems to be no way to kill a blocking thread in Python. This application needs to run on a Raspberry Pi, so large libraries such as twisted won't fit, in fact I need to get rid of the requests library as well!