I am trying to implement the logistic regression learning algorithm in Python
.
The hypothesis function I implemented is:
def hypothesis(params, features):
z = sum(p * f for p, f in zip(params, features))
return 1 / (1 + math.e ** -z)
The dataset I use for testing is from UCI Machine Learning Repository, it contains data like these (The first column is the target, the other columns are selected features):
1,14.23,1.71,2.43,15.6,127,2.8,3.06,.28,2.29,5.64,1.04,3.92,1065
3,13.2,1.78,2.14,11.2,100,2.65,2.76,.26,1.28,4.38,1.05,3.4,1050
2,13.16,2.36,2.67,18.6,101,2.8,3.24,.3,2.81,5.68,1.03,3.17,1185
1,14.37,1.95,2.5,16.8,113,3.85,3.49,.24,2.18,7.8,.86,3.45,1480
As you can see, some features are very large compared to the others.
So the z
in my hypothesis
function is too large as a power of e
.
In fact, for larger values, the z
s are all 0.0
.
And if I change numerator to 3, OverflowError
is raised.
Maybe I should have normalized the data before I feed them to my program, any idea how can I do this?