I do a lot of simulations, for which I often need to minimise complicated user-defined functions, for which I generally use numpy
and scipy.optimize.minimize()
. However, the problem with this is that I need to explicitly write down a gradient function, which can sometimes be very difficult/impossible, to find. And for large dimensional vectors, the numerical derivatives calculated by scipy
are prohibitively expensive.
So, I am trying to switch to Tensorflow
or PyTorch
to take advantage both of their automatic differentiation capabilities, and to be able to exploit GPUs freely. Let me give an explicit example of a function who derivative is somewhat complicated to write down (would required a lot of chain rule), and thus seems ripe for Tensorflow
or PyTorch
-- calculating the dihedral angle between the two triangles formed by four points in 3d space:
def dihedralAngle(xyz):
## calculate dihedral angle between 4 nodes
p1, p2, p3, p4 = 0, 1, 2, 3
## get unit normal vectors
N1 = np.cross(xyz[p1]-xyz[p3] , xyz[p2]-xyz[p3])
N2 = - np.cross(xyz[p1]-xyz[p4] , xyz[p2]-xyz[p4])
n1, n2 = N1 / np.linalg.norm(N1), N2 / np.linalg.norm(N2)
angle = np.arccos(np.dot(n1, n2))
return angle
xyz1 = np.array([[0.2 , 0. , 0. ],
[0.198358 , 0.02557543, 0. ],
[0.19345897, 0.05073092, 0. ],
[0.18538335, 0.0750534 , 0. ]]) # or any (4,3) ndarray
print(dihedralAngle(xyz1)) >> 3.141
I could easily minimise this using scipy.optimize.minimize()
, and I should get 0. For such a small function, I don't really the need a gradient (explicit or numerical). However, if I wish to iterate over many, many nodes, and minimise some function that depends on all the dihedral angles, then the overhead is much higher?
My questions then --
- How would I implement this minimisation problem using
TensorFlow
orPyTorch
? Both for a single dihedral angle, and for a list of such angles (i.e. we need to account for looping over lists). - Also, could I just get the gradient using automatic differentiation, to plug back into
scipy.optimize.minimize()
if desired? For example,scipy.optimize.minimize()
allows easily for bounds and constraints, something that I haven't noticed in the Tensorflow or PyToch optimisation modules.