0

I am trying to understand the following code from https://github.com/rezazad68/BCDU-Net/blob/master/Retina%20Blood%20Vessel%20Segmentation/evaluate.py:

patches_imgs_test = np.einsum('klij->kijl', patches_imgs_test)

and also the following:

predictions = np.einsum('kijl->klij', predictions)

I tried looking up the einsum operands klij->kijl and kijl->klij but lady luck has yet to be on my side. The closest I got (probably) are the following which do not explain cases with '4 charcters':

https://docs.scipy.org/doc/numpy/reference/generated/numpy.einsum.html Understanding NumPy's einsum

My intuition is that its just rotations of the images based on how the characters are shifting. Am I right or close on this? Some insights will be appreciated!

P.S. The numpy einsum documentation is killing me..

Stoner
  • 706
  • 1
  • 8
  • 22
  • `subscripts : str Specifies the subscripts for summation as comma separated list of subscript labels. An implicit (classical Einstein summation) calculation is performed unless the explicit indicator ‘->’ is included as well as subscript labels of the precise output form.` So explicit calculation from one subscripts to others. In the examples underneath there's transposition written as `ij->ji`. So yes, this will also be some rotation/reordering. You could test it with 3 or 4 and ordered elements and see for yourself. :) – h4z3 Jan 20 '20 at 10:35
  • 1
    Do you have problems with the einstein notation itself? https://en.wikipedia.org/wiki/Einstein_notation You could also rewrite this to 4 nested loops with `patches_imgs_test_out[k,i,j,l]+=patches_imgs_test[k,l,i,j]` in the inner loop (patches_imgs_test_out is initialized with zeros) if this is easier to understand. – max9111 Jan 20 '20 at 10:40
  • @h4z3 Thank you for the clarifications! I'll try out some test cases to paint a better picture :) – Stoner Jan 20 '20 at 11:30
  • @max9111 Yes unfortunately.. `einsum` is relatively new to me and I actually only first encountered it yesterday. Have been pondering over it since.. In any case, thanks for the clarifications! I'll check out the link to get a deeper understanding :) – Stoner Jan 20 '20 at 11:32

1 Answers1

1

The provided einsum statement is equivalent to (using np.moveaxis):

 patches_imgs_test  = np.moveaxis(patches_imgs_test, 1, -1)

followed by:

predictions = np.moveaxis(predictions, -1, 1)

Basically, moving the second axis to the end, and then putting it back in the results.

In this case, it's patches of pictures being dumped into a neural network. The second and fourth axes are the actual patches, so the code puts them at the end before passing to the NN, while the first and third axes are location data.

Daniel F
  • 11,893
  • 1
  • 21
  • 50
  • Thanks for this! With the clarifications by @max9111 and @h4z3 as well, I kinda get it now. Also, perhaps a little off-topic, is it possible that the reason why the author of that code did the above steps is because the neural network only takes in that particular format of the output of `einsum`? – Stoner Jan 20 '20 at 11:34
  • No, more likely the author was just more comfortable with Einstein Summation Notation than programming. `moveaxis` gives exactly the same format of output, but `einsum` is more readable for a scientist with a deeper background in tensor algebra than computer science. – Daniel F Jan 20 '20 at 12:08
  • 1
    Also as an aside, since the second axis is pushed to last, all of the patches fed to the NN are flipped along the diagonal compared to the original images. This may be how the network was trained, but if not, and the output is not as expected you might want to try `patches_imgs_test = np.einsum('klij->kilj', patches_imgs_test)` and `predictions = np.einsum(kilj'->klij', predictions)` – Daniel F Jan 20 '20 at 12:11
  • Thank you for the feedback, really appreciate it! I'll try that out once I get the code running proper on my terminal. :) – Stoner Jan 20 '20 at 12:54