0

So I have a main python script which internally calls 25 other scripts. All these 25 scripts access a common folder and it just reads information from different binary files in that folder.

I want these 25 scripts to equally get distributed to all the cores. Want to achieve parallelism

It is working fine with pool.apply but when I use pool.apply_async with get(), I am not getting the correct information. Only partial info.

BTW these scripts are used to parse memory dumps.

from multiprocessing.pool import ThreadPool as Pool
pool_size=8
pool = Pool(pool_size)
x2=[pool.apply_async(func_script, (i,p,)) for i,p in 
enumerate(scripts_to_run)]
output = [z.get() for z in x2] 
print(output)

so this func_scripts will call the other 25 scripts

why is this working with pool.apply ? what's the use of pool.apply anyways ? Also how can I make it work with pool.async()

Harsh Agarwal
  • 17
  • 5
  • There is a typo in last line: it should be `output`, you have given `output2` – ViKiG May 30 '19 at 12:33
  • `Pool.apply()` is generally avoided - it is a holdover from python2's first-class `apply()` function. `Pool.map()` is the typical implementation since it has the added benefit of preserving order. https://stackoverflow.com/questions/8533318/multiprocessing-pool-when-to-use-apply-apply-async-or-map – Jay May 30 '19 at 12:44
  • But it still doesn't work – Harsh Agarwal May 31 '19 at 12:38

0 Answers0