I wanted to play with neural networks and had the idea to implement a very simple one in php. Although I had no experience with this subject, i wanted to manage to run it.
I want to create the following neural net:
One Input with the values of 1 or 2 , one hiddenlayer with 2 neurons, one output. If the input=1, output=0; if input=2, output=1.
- ##############--Hidden 1 ---->
--> input (1, 2) --> ###########----> Output (0, 1)
###############-- Hidden 2 ------>
The Problem I have is that my program seems to converge the output to 0 or 1 for either input. I feel that I have something wrong in the calculation of the needed change-difference of the weights and biases.
Here is my code (the weights and biases are stored in an sql-database. It also gets its training data from there):
- Code: Select all
$learningrate=0.2;
if(isset($_POST["submitTRAIN"]))
{
$result=mysql_query("SELECT * FROM traindata") or die (mysql_error());
while($row=mysql_fetch_assoc($result))
{
$result2=mysql_query("SELECT * FROM network232 WHERE ID='1'") or die (mysql_error());
$row2=mysql_fetch_assoc($result2) or die (mysql_error());
$i1=$row["i1"]; // The input from the training dataset
$o1=$row["o1"]; //The desired output. If the input is 1, output should be 0. if input is 2, output should be 1.
$w1=$row2["w1"]; // weight 1 (input to hidden1)
$w2=$row2["w2"]; // weight 2 (input to hidden2)
$w3=$row2["w3"]; // weight 3 (hidden1 to output )
$w4=$row2["w4"]; // weight 4 (hidden2 to output)
$b1=$row2["b1"]; // bias hidden1
$b2=$row2["b2"]; // bias hidden2
$b3=$row2["b3"];// bias output
$oh1= 1 / (1 + exp((-1)*($w1*$i1+$b1))); // output of hidden1
$oh2= 1 / (1 + exp((-1)*($w2*$i1+$b2))); // output of hidden2
$oo1= 1 / (1 + exp((-1)*($w3*$oh1+$w4*$oh2+$b3))); // final output of output
$do1=($o1-$oo1)*$oo1*(1-$oo1); // delta of output
$dh1=$oh1*(1-$oh1)*($w3*$do1); // delta of hidden1
$dh2=$oh2*(1-$oh2)*($w4*$do1); // delta of hidden2
$addw1=(-1)*$learningrate*$dh1*$i1; // whats needed to be added to weight 1
$addw2=(-1)*$learningrate*$dh2*$i1;
$addw3=(-1)*$learningrate*$do1*$oh1;
$addw4=(-1)*$learningrate*$do1*$oh2;
$addb1=(-1)*$learningrate*$dh1; // whats needed to be added to bias 1
$addb2=(-1)*$learningrate*$dh2;
$addb3=(-1)*$learningrate*$do1;
$neuw1=$w1+$addw1; // new weight 1
$neuw2=$w2+$addw2;
$neuw3=$w3+$addw3;
$neuw4=$w4+$addw4;
$neub1=$b1+$addb1; // new bias 1
$neub2=$b2+$addb2;
$neub3=$b3+$addb3;
echo $o1 . " " . $oo1 . " <br>";
$result3=mysql_query("UPDATE network232 SET w1='$neuw1', w2='$neuw2', w3='$neuw3', w4='$neuw4', b1='$neub1', b2='$neub2', b3='$neub3'") or die (mysql_error());
}
}
This code converges all results towards 1. If i delete all minus-signs from the following example:
turn $addw1=(-1)*$learningrate*$dh1*$i1; into $addw1=*$learningrate*$dh1*$i1;
... then it will converge to 0 for all inputs.
I know that I am very new to this field and it is most likely a little stupid error I am making. it would be very helpful if you could find it for me, because i've been trying very hard for the last couple of days.
Thank you!
