Network creation doubt

[isasi]

Hi all,

Well first of all, congratulations for your job. I found it a few days ago and it is awesome.

As I said, I've been working with it a couple of days and I have a ¿newby? doubt about how to create a network. The network is created calling ActivationNetwork constructor. And its parameters are:

- ActivationFunction
- Number of inputs
- How many neurons per layer


Well, my problem is that I want to implement a matlab network in my C# application. The matlab network has 7 inputs, 1 output and 2 layers (input layer + 1 hidden layer with one neuron). The output is 1 o 0.

InputData (7) -> [input layer] -> [hiden layer] -> [outpùt layer] -> OutputData (1)


The goal is to classify the data into 2 different classes.


Matlab allows you to use different fuction for each layer. ¿Is it possible with your framework? I mean, in matlab code, I have a "tansig" (bipolar sigmoid) for the first layer an "purelin" (linear function) for the second. Is it possible to implement something like that (bipolar sigmoide + threshold for example)?

For matlab, it is a 2 layer network... but I don't know how many "layers" and neurons per layer I have to create ¿is the input layer considered in the last input parameter? How do I call the constructor? (thats my mean problem )

network = new ActivationNetwork((IActivationFunction)new BipolarSigmoidFunction(2), 7, 1, 1);

or

network = new ActivationNetwork((IActivationFunction)new BipolarSigmoidFunction(2), 7, 7, 1); (I don't know if the first layer is the input layer or is the first layer after input...)



Another question is about the input data range. The functions (bipolar sigmoide) have a [-1, 1] range... ¿can the data has any value? or must be adjust to [-1,1] range? I can calculate the max/min value for each input with the taining dataset (train and test, and as far as i see, they are near these max/min values) but it could not be the maximun value for the input data... so although I adjust the network with the max/min values for the training dataset it would be possible that some data values have value under -1 or over 1. Could be it a problem?


Any ideas? Thanks in advance for your help.

Regards

[isasi]

More doubts...

I think I got my network. I finally used

network = new ActivationNetwork((IActivationFunction)new BipolarSigmoidFunction(2), 7, 1, 1);

I got it debugging the network values until i see the numer of layers, neurons and weights I was looking for

I think it is ok, but if someone see something wrong, please tell me...


But now I have a new doubt. I want to access to the weight values for each neuron and layer but I cannot read it. They are protected values and I can't find how to access them. ¿Is it possible? Can anyone tell me how? (I was reading the help files but i can't fin anything...)


I also was looking for the Bias values, but there is nothing about them in the docs ¿Something about them?


Thanks in advance !!

[andrew.kirillov]

Hello,

Well, my problem is that I want to implement a matlab network in my C# application. The matlab network has 7 inputs, 1 output and 2 layers (input layer + 1 hidden layer with one neuron). The output is 1 o 0.

In AForge.NET Framework there it is not required to have any special input layer. Input layers are required in those applications, where each neuron in input layer is connected to single input. AForge.NET Framework uses another common approach - it connects all inputs to all neurons in the first layer. So each neuron in the first layer has the same number of inputs, as number of network's inputs.

I believe in your case the correct constructor should be:

Code: Select all

network = new ActivationNetwork((IActivationFunction)new BipolarSigmoidFunction(2), 7, 1, 1);


, which means a network with 7 inputs, 2 layers, 1 neuron in the first layer (each neuron there has 7 inputs), 1 neuron in output layer.

The goal is to classify the data into 2 different classes.

If these classes are linear separable, then you can solve the task with one neuron at all - network with one layer, which has one neuron. If the data is not linear separable, then you need to use at least two layers as you do, but you need to have more neurons in the first layer than one.

Matlab allows you to use different fuction for each layer. ¿Is it possible with your framework?

Not in the current version yet. For now single activation function is used for entire network. But it is easy to add in the future.

The functions (bipolar sigmoide) have a [-1, 1] range... ¿can the data has any value? or must be adjust to [-1,1] range?

It is preferred, but may be in wider range (don't make it too wide). But output should be for sure in [-1, 1] range, otherwise network will not learn.

I want to access to the weight values for each neuron and layer but I cannot read it. They are protected values and I can't find how to access them. ¿Is it possible?

Easy ... Take a look here, here and here. See the code ...

Code: Select all

// access 0-th layer, 0-th neuron, 0-th weight
double a = network[0][0][0];
// access i-th layer, j-th neuron, k-th weight
double b = network[i][j][k];


[isasi]

Thanks for your help Andrew.

Yesterday I saw the Item property in the help file but I don't know why I was looking for the Item method... instead of use the [] operator... I think I missed that when I read the file...


Anyway, thanks for the quik help.


I'm also looking for the bias values of the networks. I think they are called threshold, aren't they? Is it possible to access them directly? One way could be subtractring to the neuron output the sum of its (weights*input) (just the opposite to calculate the output...) but maybe there is something easier...


Again, thanks for the help.


Regards,

[andrew.kirillov]

I'm also looking for the bias values of the networks. I think they are called threshold, aren't they? Is it possible to access them directly?

In the case of activation neuron, you may use its Threshold property.

[isasi]

Thank you. That was what I was looking for.


Regards

[isasi]

Hi again,


A new doubt about how to calculate the output of the network (in my case, to determine to which clases the input belongs: 1 or -1)

as far as I know, the output is as folows:

stept 1:
sum of (input[i]*network_weight[i])

stept 2:
add threshold value of layer 1 to the value obtained in stept 1

stept 3:
calculate the sigmoide value of the value from stept 2

Code: Select all

Bipolar Sigmoide
                              2
            f(x) = ------------------ - 1
                    1 + exp(-alpha * x)
where alpha=2

stept 4:
multiply the value from stept 4 by weight of layer 2 (as I only have 1 neuron in layer 2)

stept 5:
add the threshold value of layer 2 to the result of stept 4

stept 6:
calculate the sigmoide of the stept 5 value


but I always get 1 when I compute the output manually while with the network function "Compute()" I get values of 1 and -1 getting a classification rate over 95%


Any idea?


Thanks in advance,

[andrew.kirillov]

Hello,

Here is copy-paste from framework's sources:

Code: Select all

// initial sum value
double sum = 0.0;

// compute weighted sum of inputs
for ( int i = 0; i < inputsCount; i++ )
{
    sum += weights[i] * input[i];
}
sum += threshold;

// local variable to avoid mutlithreaded conflicts
double output = function.Function( sum );


So, just recheck your calculations again and make sure that input for the first layer is actually network's input, but input for each next layer is output of previous layer.

[isasi]

I think I discovered what is happening in my code. The code is OK, but the error comes from the data input normalization to [-1,1] range.

I have 2 applications, one to train the network and another one to use the results (thats why I wanted to extract the network values). In the first one I make the range correction but I don't in the second one because I don't know the max/min values.

I have to do some test with those values, but I think that the problem is there.


Thanks for the help !!


Regards,

[isasi]

Definitely it was an input data format error... The code works OK

Regards,