Training a Neural Network: Step-by-Step Tutorial

[squaremeplz]

Homework Statement

I am trying to train a neural network using the following training set:

(4,6)
(9,10)
(1,2)
(6,2)

So, if I enter input 4, it should give me 6. If I enter 9, output should be 10.
If I enter 3.5, it should approximate an output based on the training examples.

Homework Equations

weight update rule:

w_new = w_old + (-learning rate) * (error) * (input)

The Attempt at a Solution

So, let's say I randomly initialize the weights and get w1 = 3 and w0 = 4

For the first example,

(4,6)

The input will be (4,1) because of the intercept.

4*3 + 4*1 = 16

The difference between the target value and the actual output is 10.

Now, I just need help with the weight update rule because I am not sure how to proceed.

For example:

Can I subtract 4 from the error because of the intercept to get error = 6
learning rate = -.1
w_new = 3 + (-.1)*(6)*(4)
w_new = .6

Then to test this I get:
4*.6 + 4*1 = 6.4

Any help greatly appreciated!

 

[mighty2000]

your first step in responding to this forum post would be to clarify what type of neural network is being used for this task. Is it a single-layer perceptron or a multi-layer neural network? This will help determine the appropriate weight update rule to use.

Assuming it is a single-layer perceptron, the weight update rule would be:

w_new = w_old + (-learning rate) * (error) * (input)

where w_new is the new weight, w_old is the old weight, learning rate is a parameter that determines the size of the weight update, error is the difference between the target value and the actual output, and input is the input value.

In this case, you would not subtract the intercept from the error, as the intercept is not included in the weight update rule. So for the first example, the weight update would be:

w_new = 3 + (-0.1) * (10-16) * 4 = 2.4

To test this, you would plug in the new weight (2.4) into the equation and see if it gives a more accurate output.

It is important to note that for a single-layer perceptron, this weight update rule only works for linearly separable data. If the data is not linearly separable, then a multi-layer neural network would be needed and a different weight update rule would be used.