I tried to use BaseManager and register my customized class to make it happy, and get the problem about nested class just as Tom had mentioned above.
I think the main reason is irrelevant to the nested class as said, yet the communication mechanism that python take in low level. The reason is python use some socket-alike communication mechanism to synchronize the modification of customized class within a server process in low level. I think it encapsulate some rpc methods, make it just transparent to the user as if they called the local methods of a nested class object.
So, when you want to modify, retrieve your self-defined objects or some third-party objects, you should define some interfaces within your processes to communicate to it rather than directly get or set values.
Yet when operating the multi-nested objects in the nested objects, one can ignore the issues mentioned above, just as what you do in your common routine because your nested objects in the registered class is not a proxy objects any longer, on which the operation will not go through the socket-alike communication routine again and is localized.
Here is the workable code I wrote to solve the problem.
from multiprocessing import Process, Manager, Lock
from multiprocessing.managers import BaseManager
import numpy as np
class NestedObj(object):
def __init__(self):
self.val = 1
class CustomObj(object):
def __init__(self, numpy_obj):
self.numpy_obj = numpy_obj
self.nested_obj = NestedObj()
def set_value(self, p, q, v):
self.numpy_obj[p, q] = v
def get_obj(self):
return self.numpy_obj
def get_nested_obj(self):
return self.nested_obj.val
class CustomProcess(Process):
def __init__(self, obj, p, q, v):
super(CustomProcess, self).__init__()
self.obj = obj
self.index = p, q
self.v = v
def run(self):
self.obj.set_value(*self.index, self.v)
if __name__=="__main__":
BaseManager.register('CustomObj', CustomObj)
manager = BaseManager()
manager.start()
data = [[0 for x in range(10)] for y in range(10)]
matrix = np.matrix(data)
custom_obj = manager.CustomObj(matrix)
print(custom_obj.get_obj())
process_list = []
for p in range(10):
for q in range(10):
proc = CustomProcess(custom_obj, p, q, 10*p+q)
process_list.append(proc)
for x in range(100):
process_list[x].start()
for x in range(100):
process_list[x].join()
print(custom_obj.get_obj())
print(custom_obj.get_nested_obj())