0

I have a time-consuming function in my project with arguments x, y, and z where z iterates over a big NumPy array Z. I have to call this function lots of times in my project. The first designs that I have used was list comprehension:

output = np.array([my_function(z, x, y) for z in Z])

and the second was map function:

output = np.array(list(map(my_function, Z, repeat(x, len(Z)), repeat(y, len(Z)))))

Unfortunately, I didn’t get much speed up and my code is too slow yet. Is there a more efficient way to do this job?

Samin
  • 1
  • 1
  • 2
    `where z in Z` do you mean `for z in Z`? – yatu Sep 21 '20 at 12:28
  • If the function has no side effects, e.g. the function calls are independent from each other, your could split up the work and use multiprocessing. – Wups Sep 21 '20 at 12:45
  • Thanks yatu. I made it correct now. – Samin Sep 21 '20 at 12:46
  • Hello! Welcome to StackOverflow. Does this answer your question? [Most efficient way to map function over numpy array](https://stackoverflow.com/questions/35215161/most-efficient-way-to-map-function-over-numpy-array) – Stef Sep 21 '20 at 13:34
  • Thanks @Stef. I've seen this link before, and it didn't help me. – Samin Sep 21 '20 at 13:47
  • Why does it not help you? Can you explain in a bit more detail? It appears your question and the question at that link are almost exactly the same, so if you couldn't find a satisfying answer there, how can we give a better here? – Stef Sep 21 '20 at 13:51
  • What `my_function` is supposed to do? – Jérôme Richard Sep 21 '20 at 14:56
  • As the link shows, without modification to `my_function` the list comprehension is about the best you can do. Especially when the function is complex, the spent calling and running it dominates the overall time. The iteration mechanism is a minor part of the overall time. – hpaulj Sep 21 '20 at 15:08
  • @Jérôme Richard - my_function contains two loops over two other functions f_1 and f_2. These two functions are composed of many nested if, elif, and else statements. That makes it impossible to use even GPU to get speed up. – Samin Sep 21 '20 at 16:51
  • 1
    Maybe it is possible to vectorize your function so that you just can pass the big array `Z` and the operations are performed not on the elements but on whole arrays. Please show us `f_1` and `f_2` so that we can help – scleronomic Sep 21 '20 at 18:10

0 Answers0