2

I am working on matrix multiplications in NumPy using np.dot(). As the data set is very large, I would like to reduce the overall run time as far as possible - i.e. perform as little as possible np.dot() products.

Specifically, I need to calculate the overall matrix product as well as the associated flow from each element of my values vector. Is there a way in NumPy to calculate all of this together in one or two np.dot() products? In the code below, is there a way to reduce the number of np.dot() products and still get the same output?

import pandas as pd
import numpy as np

vector = pd.DataFrame([1, 2, 3],
                      ['A', 'B', 'C'], ["Values"])

matrix = pd.DataFrame([[0.5, 0.4, 0.1],
                       [0.2, 0.6, 0.2],
                       [0.1, 0.3, 0.6]],
                      index = ['A', 'B', 'C'], columns = ['A', 'B', 'C'])

# Can the number of matrix multiplications in this part be reduced?
overall = np.dot(vector.T, matrix)
from_A = np.dot(vector.T * [1,0,0], matrix)
from_B = np.dot(vector.T * [0,1,0], matrix)
from_C = np.dot(vector.T * [0,0,1], matrix)

print("Overall:", overall)
print("From A:", from_A)
print("From B:", from_B)
print("From C:", from_C)
Divakar
  • 204,109
  • 15
  • 192
  • 292
Andreas
  • 173
  • 9

2 Answers2

2

If the vectors you use to select the row are indeed the unit vectors, you are much better off not doing matrix multiplication at all for from_A, from_B, from_C. Matrix multiplication requires a lot more addition and multiplications than you need to just multiply each row of the matrix by it's corresponding entry in the vector:

from_ABC = matrix.values * vector.values

You will only need a single call to np.dot to get overall.

Mad Physicist
  • 76,709
  • 19
  • 122
  • 186
  • thanks. Very helpful to know that np.dot is much slower than a simple product using *. – Andreas Oct 31 '16 at 17:15
  • It's just a matter of quantity of operations. My product broadcasts so that there are just matrix.size products and no sums. The dot has vector.size * matrix.size products and matrix.shape[0] sums. – Mad Physicist Oct 31 '16 at 17:20
  • I just assumed OP had a minimal sample case with the scaling values, but it seems those are the actual values OP is working with. So, this one's good! – Divakar Oct 31 '16 at 17:40
  • @Divakar. I thought so too at first, but the names `from_A`, etc. made me think that they are just row selectors. – Mad Physicist Oct 31 '16 at 17:46
1

You could define a 3 x 3 shaped 2D array of those scaling values and perform matrix-multiplication, like so -

scale = np.array([[1,0,0],[0,1,0],[0,0,1]])
from_ABC = np.dot(vector.values.ravel()*scale,matrix)

Sample run -

In [901]: from_A
Out[901]: array([[ 0.5,  0.4,  0.1]])

In [902]: from_B
Out[902]: array([[ 0.9,  1.6,  0.5]])

In [903]: from_C
Out[903]: array([[ 0.8,  1.3,  1.9]])

In [904]: from_ABC
Out[904]: 
array([[ 0.5,  0.4,  0.1],
       [ 0.9,  1.6,  0.5],
       [ 0.8,  1.3,  1.9]])

Here's an alternative with np.einsum to do all those in one step -

np.einsum('ij,ji,ik->jk',vector.values,scale,matrix)

Sample run -

In [915]: np.einsum('ij,ji,ik->jk',vector.values,scale,matrix)
Out[915]: 
array([[ 0.5,  0.4,  0.1],
       [ 0.9,  1.6,  0.5],
       [ 0.8,  1.3,  1.9]])
Divakar
  • 204,109
  • 15
  • 192
  • 292