-2

Derivative of sigmoid function $\sigma (x) = \frac{1}{1+e^{-x}}$

but:

derive wrt θ1 and not wrt z=∑θixi

show that: $$ \frac{\partial \sigma(z)}{\partial \theta_1} = \sigma(z)(1-\sigma(z)) \cdot x_1 $$ with: $$ z = \theta_0 x_0 + \theta_1 x_1 $$

Note that in general (because of symmetry) holds:

$$ z = \theta_0 x_0 + \theta_1 x_1 + \dots $$

$$ \frac{\partial \sigma(z)}{\partial \theta_j} = \sigma(z)(1-\sigma(z)) \cdot x_j $$

How should I do this?

LarryTran
  • 1
  • 1

1 Answers1

0

Using $\sigma(z)=\frac{1}{1+e^{-(\theta_0 x_0 + \theta_1 x_1)}}$ $$ \partial_{\theta_0}\sigma(z) = \frac{e^{-(\theta_0 x_0 + \theta_1 x_1)}\partial_{\theta_0}z}{(1+e^{-(\theta_0 x_0 + \theta_1 x_1)})^2} = x_1\frac{1}{1+e^{-(\theta_0 x_0 + \theta_1 x_1)}}\frac{e^{-(\theta_0 x_0 + \theta_1 x_1)}}{1+e^{-(\theta_0 x_0 + \theta_1 x_1)}} = x_1\sigma(z)\frac{e^{-(\theta_0 x_0 + \theta_1 x_1)}}{1+e^{-(\theta_0 x_0 + \theta_1 x_1)}} $$ Now use $$\frac{e^u}{e^u+1} = \frac{e^u+1-1}{e^u+1} = 1-\frac{1}{e^u+1}$$

denklo
  • 771
  • 3
  • 11